Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

3398 - FRA Reports - search and upload #3445

Open
wants to merge 63 commits into
base: develop
Choose a base branch
from
Open

Conversation

jtimpe
Copy link

@jtimpe jtimpe commented Jan 28, 2025

Summary of Changes

Pull request closes #3398

How to Test

cd tdrs-backend && docker-compose up --build
cd tdrs-frontend && docker-compose up --build
  1. Open http://localhost:3000/ and sign in.
  2. Add {"fra_reports": true} to user's feature flags to enable the page. Navigate to the FRA Reports page
  3. Make sure the search form functionality works - all fields are required and should show errors. Missing STT error for non-admins with no stt in user profile
  4. Make sure the upload form functionality works - file type validation, encoding validation (TODO), uploads should succeed
  5. Make sure the modal functionality works - attach a file but don't upload, change search params and click search
  6. Make sure feature flag functionality works - change fra_reports flag to various values. should work only when === true

Deliverables

More details on how deliverables herein are assessed included here.

Deliverable 1: Accepted Features

Checklist of ACs:

  • Users can select new FRA report types and perform a search.
  • Users can upload FRA sections for the selected fiscal quarter.
  • Users will not yet receive error reports for the upload action.
  • Testing Checklist has been run and all tests pass
  • lfrohlich and/or adpennington confirmed that ACs are met.

Deliverable 2: Tested Code

  • Are all areas of code introduced in this PR meaningfully tested?
    • If this PR introduces backend code changes, are they meaningfully tested?
    • If this PR introduces frontend code changes, are they meaningfully tested?
  • Are code coverage minimums met?
    • Frontend coverage: [insert coverage %] (see CodeCov Report comment in PR)
    • Backend coverage: [insert coverage %] (see CodeCov Report comment in PR)

Deliverable 3: Properly Styled Code

  • Are backend code style checks passing on CircleCI?
  • Are frontend code style checks passing on CircleCI?
  • Are code maintainability principles being followed?

Deliverable 4: Accessible

  • Does this PR complete the epic?
  • Are links included to any other gov-approved PRs associated with epic?
  • Does PR include documentation for Raft's a11y review?
  • Did automated and manual testing with iamjolly and ttran-hub using Accessibility Insights reveal any errors introduced in this PR?

Deliverable 5: Deployed

  • Was the code successfully deployed via automated CircleCI process to development on Cloud.gov?

Deliverable 6: Documented

  • Does this PR provide background for why coding decisions were made?
  • If this PR introduces backend code, is that code easy to understand and sufficiently documented, both inline and overall?
  • If this PR introduces frontend code, is that code easy to understand and sufficiently documented, both inline and overall?
  • If this PR introduces dependencies, are their licenses documented?
  • Can reviewer explain and take ownership of these elements presented in this code review?

Deliverable 7: Secure

  • Does the OWASP Scan pass on CircleCI?
  • Do manual code review and manual testing detect any new security issues?
  • If new issues detected, is investigation and/or remediation plan documented?

Deliverable 8: User Research

Research product(s) clearly articulate(s):

  • the purpose of the research
  • methods used to conduct the research
  • who participated in the research
  • what was tested and how
  • impact of research on TDP
  • (if applicable) final design mockups produced for TDP development

@jtimpe jtimpe self-assigned this Jan 28, 2025
@jtimpe jtimpe added the raft review This issue is ready for raft review label Jan 29, 2025
Copy link

codecov bot commented Jan 29, 2025

Codecov Report

Attention: Patch coverage is 91.61290% with 26 lines in your changes missing coverage. Please review.

Project coverage is 90.70%. Comparing base (f8353dd) to head (2fab63c).
Report is 2 commits behind head on develop.

Files with missing lines Patch % Lines
...drs-frontend/src/components/Reports/FRAReports.jsx 89.26% 13 Missing and 9 partials ⚠️
...src/components/PermissionGuard/PermissionGuard.jsx 57.14% 1 Missing and 2 partials ⚠️
tdrs-frontend/src/components/Form/RadioSelect.jsx 75.00% 1 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff             @@
##           develop    #3445      +/-   ##
===========================================
- Coverage    91.15%   90.70%   -0.46%     
===========================================
  Files          308      318      +10     
  Lines         8832     9189     +357     
  Branches       670      715      +45     
===========================================
+ Hits          8051     8335     +284     
- Misses         654      717      +63     
- Partials       127      137      +10     
Flag Coverage Δ
dev-backend 90.50% <100.00%> (-0.51%) ⬇️
dev-frontend 91.86% <90.87%> (-0.33%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
...ta_files/migrations/0018_alter_datafile_section.py 100.00% <100.00%> (ø)
tdrs-backend/tdpservice/data_files/models.py 87.64% <100.00%> (ø)
tdrs-backend/tdpservice/data_files/serializers.py 96.82% <100.00%> (+0.27%) ⬆️
tdrs-backend/tdpservice/data_files/validators.py 96.42% <100.00%> (+0.27%) ⬆️
.../users/migrations/0044_alter_user_feature_flags.py 100.00% <100.00%> (ø)
tdrs-backend/tdpservice/users/models.py 92.45% <100.00%> (ø)
tdrs-backend/tdpservice/users/serializers.py 90.19% <ø> (ø)
tdrs-frontend/src/actions/fraReports.js 100.00% <100.00%> (ø)
tdrs-frontend/src/actions/reports.js 100.00% <ø> (ø)
...-frontend/src/components/FileUpload/FileUpload.jsx 88.88% <ø> (-0.18%) ⬇️
... and 14 more

... and 1 file with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9f452a3...2fab63c. Read the comment docs.

ref={errorsRef}
tabIndex="-1"
>
There {errorsCount === 1 ? 'is' : 'are'} {form.errors} error(s) in
Copy link

@elipe17 elipe17 Feb 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I noticed the form error count doesn't update as errors are resolved which is different than the datafiles page.
Screenshot 2025-02-05 at 9 48 03 AM

'We can’t process that file format. Please provide a plain text file.'

const INVALID_EXT_ERROR =
'Invalid extension. Accepted file types are: .txt, .ms##, .ts##, or .ts###.'
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think FRA is only supposed to be .csv and .xlsx. We should verify

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes just those @jtimpe @elipe17

return (
<>
<div className={classNames({ 'border-bottom': isUploadReportToggled })}>
<SearchForm
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you cause a form error in the UploadForm it does not get reset if you re-search using the SearchForm.

Screenshot 2025-02-05 at 10 12 33 AM

<fieldset className="usa-fieldset">
<legend className="usa-label text-bold">File Type</legend>

{options.map(({ label, value }, index) => (
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We use this pattern for radio selecters in the request access page too. Should make the radio controls a generic "use everywhere" component?

Copy link
Author

@jtimpe jtimpe Feb 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is captured by another ticket - #3015 - but was a simple enough change (that we keep kicking around because it's low priority) so i went for it. I also made a DropdownSelect component that handles the year/quarter inputs, since those looked very similar. I tried to structure them in a way that we could go implement a Form higher-order component (or context component) and use these components within it.

one caveat is that each input still needs to be wrapped in a div to add the correct styling. e.g.,

<div
    className={classNames('usa-form-group maxw-mobile margin-top-4', {
      'usa-form-group--error': !valid,
    })}
>
  <DropdownSelect ... />
</div>

This is partly necessary (form-group) partly stylistic (margin-top-4) and partly-logical (usa-form-group--error if invalid), so it didn't fit neatly into either the reusable input component or the form-specific component. To solve that, I added a classes prop where the form-specific (such as margins) styling can be added conditionally from the wrapping component, if desired. I think we use margin-top-4 everywhere, but this adds flexibility should we decide to change something in the future.

The other bit of weirdness is the default/disabled/hidden first option in DropdownSelect. I decided to key off value === '' to decide if it's the disabled option, rather than add a disabled: true key to the options array or passing in a separate disabledOption="xyz" prop to the component.

Open to suggestions on all of this! I realize both those design decisions may be confusing to read and implement

@jtimpe jtimpe requested a review from elipe17 February 18, 2025 14:52
@raftmsohani
Copy link

I am getting circular import error.

@raftmsohani
Copy link

I am getting circular import error.

It is fixed now

@raftmsohani
Copy link

Screenshot 2025-02-18 at 10 12 52 AM

options for FRA file types are grayed out

@jtimpe
Copy link
Author

jtimpe commented Feb 18, 2025

Screenshot 2025-02-18 at 10 12 52 AM

options for FRA file types are grayed out

This is correct per the ticket AC. covered here - #3445 (comment)

@@ -14,7 +13,11 @@ import {
} from '../../actions/reports'
import Button from '../Button'
import createFileInputErrorState from '../../utils/createFileInputErrorState'
import { handlePreview, getTargetClassName } from './utils'

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'Cancel' button in the file submit form: makes the form to disappear. I think it should only clear the form and not delete it.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@reitermb @victoriaatraft hiding the form on Cancel is consistent with the other data files page, but Mo brings up a good point. thoughts?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm good to move to clearing the upload portion of the form instead of hiding it. Open to doing that here or in a follow-on as long as we make sure to ticket/implement for both the TANF/SSP and FRA sides.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I started to change the form cancel behaviors, but I forgot about a bug that prevented it from actually working

uswds/uswds#4848

the uswds file input doesn't automatically clear the preview state when the file is set to null. it doesn't have a clear function exposed at all, so the only way to get the preview to disappear (even after you've cleared the file itself) is to un-render and re-render the file input. We could hide and re-show the form, but i thought the jumpiness would be poor user experience. i'll leave it the way it is for now

@jtimpe jtimpe requested a review from raftmsohani February 18, 2025 16:49
Copy link

@raftmsohani raftmsohani left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

everything works as intended, I only had a nit suggestion, other than that it is approved

'file': create_temporary_file(fake_file, 'report.csv'),
"original_filename": 'report.csv',
"slug": str(uuid.uuid4()),
"extension": "txt",
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this be "csv"?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, should the section and group be different?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated! good catch

} from '../../selectors/auth'

const isAllowed = (
{ permissions, isApproved },
{ permissions, isApproved, featureFlags },
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this have an override to all Sys admin and maybe developer groups to see everything? I was a little confused when I logged in as a Sys admin and could not see the FRA tab by default.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could certainly be follow on work. Or we can just specifiy that feature flags can be mutually exclusive from group assignments.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good point. i personally prefer the explicit/mutually-exclusive approach but i'll let @ADPennington make the final call

Copy link

@elipe17 elipe17 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couple of non blocking questions. Nice work sir, there is a lot here!

@lhuxraft
Copy link
Collaborator

@reitermb looking into header focus, then will move to QASP

@jtimpe
Copy link
Author

jtimpe commented Feb 19, 2025

@reitermb looking into header focus, then will move to QASP

@reitermb noticed that the header focus behavior was broken on the original data files page as well. we're unsure of how that was introduced, so spun up #3480 to cover research and fix.

@jtimpe jtimpe added QASP Review and removed raft review This issue is ready for raft review a11y-review PR is ready for accessibility review labels Feb 19, 2025
@jtimpe jtimpe requested a review from ADPennington February 19, 2025 22:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Implement FRA report selection and upload interface
6 participants