Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

3398 - FRA Reports - search and upload #3445

Open
wants to merge 55 commits into
base: develop
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 49 commits
Commits
Show all changes
55 commits
Select commit Hold shift + click to select a range
bca4311
boilerplate + nav (with wrong permissions)
jtimpe Jan 6, 2025
96b469b
search form components + state
jtimpe Jan 9, 2025
c8ad478
search + upload form components
jtimpe Jan 17, 2025
7996526
redux, frontend api setup for fra reports
jtimpe Jan 21, 2025
7025dab
add error modal
jtimpe Jan 22, 2025
bfd1dc1
fix sttList and class names
jtimpe Jan 22, 2025
c6694b0
fix vars
jtimpe Jan 22, 2025
71e1e6e
reorg + fix unreferenced var
jtimpe Jan 23, 2025
8c36164
search form tests
jtimpe Jan 23, 2025
e25ca72
temp comment two tests
jtimpe Jan 28, 2025
dedee58
fix aria
jtimpe Jan 28, 2025
e2eb9ac
Merge branch 'develop' into 3398-fra-frontend
jtimpe Jan 28, 2025
902ca31
fix test
jtimpe Jan 28, 2025
f2f42d6
fix tests and stub future work
jtimpe Jan 28, 2025
dc4b650
add extra test cov
jtimpe Jan 29, 2025
ba40b4e
Merge branch 'develop' into 3398-fra-frontend
jtimpe Jan 29, 2025
a8e4672
Merge branch 'develop' into 3398-fra-frontend
jtimpe Feb 6, 2025
37f8a4a
implement upload api
jtimpe Feb 6, 2025
691d413
handle upload errors and no file selected
jtimpe Feb 6, 2025
9dfad44
fix test
jtimpe Feb 6, 2025
16da0a2
update errors count on input change
jtimpe Feb 7, 2025
556d89f
fix test
jtimpe Feb 7, 2025
5254494
Merge branch 'develop' into 3398-fra-frontend
jtimpe Feb 7, 2025
5816e4f
create + implement reusable form input components
jtimpe Feb 7, 2025
1d347f9
implement conditional classNames
jtimpe Feb 7, 2025
cd9f375
remove comments and console.logs
jtimpe Feb 7, 2025
40e5f0a
Merge branch 'develop' into 3398-fra-frontend
jtimpe Feb 7, 2025
b808d26
unused import
jtimpe Feb 7, 2025
91ce6f3
additional tests
jtimpe Feb 7, 2025
72e90a5
implement fra reports feature flag
jtimpe Feb 7, 2025
97271f6
implement file encoding check
jtimpe Feb 7, 2025
641f999
return
jtimpe Feb 7, 2025
0b0be2f
show message on no file selected
jtimpe Feb 11, 2025
16fc35d
clear alert on search
jtimpe Feb 11, 2025
6b3a133
add backend feat flag check
jtimpe Feb 11, 2025
84dd282
fix tests
jtimpe Feb 11, 2025
fe16f4e
fix test
jtimpe Feb 11, 2025
c4eb8e4
change page title
jtimpe Feb 11, 2025
b46b1f6
change Data Files -> TANF Data Files
jtimpe Feb 11, 2025
60998a7
Merge branch 'develop' into 3398-fra-frontend
jtimpe Feb 11, 2025
e538493
fix tests
jtimpe Feb 11, 2025
0398d05
disable other report types
jtimpe Feb 11, 2025
6862582
add report description
jtimpe Feb 11, 2025
a71d996
fix tests
jtimpe Feb 11, 2025
d6db5ff
add cov
jtimpe Feb 11, 2025
903134a
for -> of
jtimpe Feb 11, 2025
7f8d4b2
test
jtimpe Feb 11, 2025
39e8488
focus header
jtimpe Feb 12, 2025
2795258
add cov
jtimpe Feb 12, 2025
cf82ab6
change allowed extensions
jtimpe Feb 13, 2025
630cc1e
lint
jtimpe Feb 13, 2025
bedb71d
fix file encoding
jtimpe Feb 14, 2025
8d81d34
test
jtimpe Feb 14, 2025
81c5130
tests
jtimpe Feb 14, 2025
247a477
code style
jtimpe Feb 14, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# Generated by Django 3.2.15 on 2025-02-11 22:04

from django.db import migrations, models


class Migration(migrations.Migration):

dependencies = [
('data_files', '0017_alter_datafile_section'),
]

operations = [
migrations.AlterField(
model_name='datafile',
name='section',
field=models.CharField(choices=[('Tribal Closed Case Data', 'Tribal Closed Case Data'), ('Tribal Active Case Data', 'Tribal Active Case Data'), ('Tribal Aggregate Data', 'Tribal Aggregate Data'), ('Tribal Stratum Data', 'Tribal Stratum Data'), ('SSP Aggregate Data', 'Ssp Aggregate Data'), ('SSP Closed Case Data', 'Ssp Closed Case Data'), ('SSP Active Case Data', 'Ssp Active Case Data'), ('SSP Stratum Data', 'Ssp Stratum Data'), ('Active Case Data', 'Active Case Data'), ('Closed Case Data', 'Closed Case Data'), ('Aggregate Data', 'Aggregate Data'), ('Stratum Data', 'Stratum Data'), ('Work Outcomes of TANF Exiters', 'Fra Work Outcome Tanf Exiters'), ('Secondary School Attainment', 'Fra Secondry School Attainment'), ('Supplemental Work Outcomes', 'Fra Supplement Work Outcomes')], max_length=32),
),
]
2 changes: 1 addition & 1 deletion tdrs-backend/tdpservice/data_files/models.py
Original file line number Diff line number Diff line change
Expand Up @@ -119,7 +119,7 @@ class Section(models.TextChoices):
AGGREGATE_DATA = "Aggregate Data"
STRATUM_DATA = "Stratum Data"

FRA_WORK_OUTCOME_TANF_EXITERS = "Work Outcomes for TANF Exiters"
FRA_WORK_OUTCOME_TANF_EXITERS = "Work Outcomes of TANF Exiters"
FRA_SECONDRY_SCHOOL_ATTAINMENT = "Secondary School Attainment"
FRA_SUPPLEMENT_WORK_OUTCOMES = "Supplemental Work Outcomes"

Expand Down
4 changes: 3 additions & 1 deletion tdrs-backend/tdpservice/data_files/serializers.py
Original file line number Diff line number Diff line change
Expand Up @@ -118,5 +118,7 @@ def validate_file(self, file):
def validate_section(self, section):
jtimpe marked this conversation as resolved.
Show resolved Hide resolved
"""Validate the section field."""
if DataFile.Section.is_fra(section):
raise serializers.ValidationError("Section cannot be FRA")
user = self.context.get('user')
if not user.has_fra_access:
raise serializers.ValidationError("Section cannot be FRA")
return section
10 changes: 9 additions & 1 deletion tdrs-backend/tdpservice/data_files/test/test_api.py
Original file line number Diff line number Diff line change
Expand Up @@ -228,12 +228,20 @@ def test_create_data_file_file_entry(self, api_client, data_file_data, user):
self.assert_data_file_created(response)
self.assert_data_file_exists(data_file_data, 1, user)

def test_create_data_file_fra(self, api_client, data_file_data, user):
def test_create_data_file_fra_no_feat_flag(self, api_client, data_file_data, user):
"""Test ability to create data file metadata registry."""
response = self.post_data_file_fra(api_client, data_file_data)
assert response.data == {'section': [ErrorDetail(string='Section cannot be FRA', code='invalid')]}
self.assert_data_file_error(response)

def test_create_data_file_fra_with_feat_flag(self, api_client, data_file_data, user):
"""Test ability to create data file metadata registry."""
user.feature_flags = {"fra_reports": True}
user.save()
response = self.post_data_file_fra(api_client, data_file_data)
self.assert_data_file_created(response)
self.assert_data_file_exists(data_file_data, 1, user)

def test_data_file_file_version_increment(
self,
api_client,
Expand Down
2 changes: 1 addition & 1 deletion tdrs-backend/tdpservice/data_files/test/test_models.py
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,7 @@ def test_data_files_filename_is_expected(user):
('SSP Closed Case Data', 'SSP'),
('Active Case Data', 'TAN'),
('Aggregate Data', 'TAN'),
('Work Outcomes for TANF Exiters', 'FRA'),
('Work Outcomes of TANF Exiters', 'FRA'),
('Secondary School Attainment', 'FRA'),
('Supplemental Work Outcomes', 'FRA')
])
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# Generated by Django 3.2.15 on 2025-02-11 13:40

from django.db import migrations, models


class Migration(migrations.Migration):

dependencies = [
('users', '0042_user_feature_flags'),
]

operations = [
migrations.AlterField(
model_name='user',
name='feature_flags',
field=models.JSONField(blank=True, default=dict, help_text='Feature flags for this user. This is a JSON field that can be used to store key-value pairs. E.g: {"fra_reports": true}'),
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i accidentally changed the feature flag from fra_access to fra_reports. just updated the help text here to be consistent, but it created a whole new migration. I named it 0044 instead of the auto-generated 0043to avoid conflicts because #3440 created 0043

),
]
4 changes: 2 additions & 2 deletions tdrs-backend/tdpservice/users/models.py
Original file line number Diff line number Diff line change
Expand Up @@ -118,14 +118,14 @@ class Meta:
feature_flags = models.JSONField(
default=dict,
help_text='Feature flags for this user. This is a JSON field that can be used to store key-value pairs. ' +
'E.g: {"fra_access": true}',
'E.g: {"fra_reports": true}',
blank=True,
)

@property
def has_fra_access(self):
"""Return whether or not the user has FRA access."""
return self.feature_flags.get('fra_access', False)
return self.feature_flags.get('fra_reports', False)

def __str__(self):
"""Return the username as the string representation of the object."""
Expand Down
4 changes: 3 additions & 1 deletion tdrs-backend/tdpservice/users/serializers.py
Original file line number Diff line number Diff line change
Expand Up @@ -115,7 +115,8 @@ class Meta:
'date_joined',
'access_request',
'access_requested_date',
'account_approval_status'
'account_approval_status',
'feature_flags',
]
read_only_fields = (
'id',
Expand All @@ -131,6 +132,7 @@ class Meta:
'access_request',
'access_requested_date',
'account_approval_status',
'feature_flags',
)

"""Enforce first and last name to be in API call and not empty"""
Expand Down
4 changes: 2 additions & 2 deletions tdrs-backend/tdpservice/users/test/test_models.py
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ def test_user_with_fra_access(client, admin_user, stt):
"""Test that a user with FRA access can only have an STT."""
admin_user.stt = stt
admin_user.is_superuser = True
admin_user.feature_flags = {"fra_access": False}
admin_user.feature_flags = {"fra_reports": False}

admin_user.clean()
admin_user.save()
Expand All @@ -94,7 +94,7 @@ def test_user_with_fra_access(client, admin_user, stt):
response = client.get(f"/admin/data_files/datafile/{datafile.id}/change/")
assert response.status_code == 302

admin_user.feature_flags = {"fra_access": True}
admin_user.feature_flags = {"fra_reports": True}
admin_user.save()

response = client.get(f"/admin/data_files/datafile/{datafile.id}/change/")
Expand Down
112 changes: 112 additions & 0 deletions tdrs-frontend/src/actions/fraReports.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,112 @@
import { v4 as uuidv4 } from 'uuid'
import axios from 'axios'
import axiosInstance from '../axios-instance'
import { objectToUrlParams } from '../utils/stringUtils'

const BACKEND_URL = process.env.REACT_APP_BACKEND_URL

export const SET_IS_LOADING_SUBMISSION_HISTORY =
'SET_IS_LOADING_SUBMISSION_HISTORY'
export const SET_FRA_SUBMISSION_HISTORY = 'SET_FRA_SUBMISSION_HISTORY'
export const SET_IS_UPLOADING_FRA_REPORT = 'SET_IS_UPLOADING_FRA_REPORT'

export const getFraSubmissionHistory =
(
{ stt, reportType, fiscalQuarter, fiscalYear },
onSuccess = () => null,
onError = () => null
) =>
async (dispatch) => {
dispatch({
type: SET_IS_LOADING_SUBMISSION_HISTORY,
payload: { isLoadingSubmissionHistory: true },
})

try {
const requestParams = {
stt: stt.id,
file_type: reportType,
year: fiscalYear,
quarter: fiscalQuarter,
}

const response = await axios.get(
`${BACKEND_URL}/data_files/?${objectToUrlParams(requestParams)}`,
{
responseType: 'json',
}
)

dispatch({
type: SET_FRA_SUBMISSION_HISTORY,
payload: { submissionHistory: response?.data },
})

onSuccess()
} catch (error) {
onError(error)
}

dispatch({
type: SET_IS_LOADING_SUBMISSION_HISTORY,
payload: { isLoadingSubmissionHistory: false },
})
}

export const uploadFraReport =
(
{ stt, reportType, fiscalQuarter, fiscalYear, file, user },
onSuccess = () => null,
onError = () => null
) =>
async (dispatch) => {
dispatch({
type: SET_IS_UPLOADING_FRA_REPORT,
payload: { isUploadingFraReport: true },
})

const formData = new FormData()
const fraReportData = {
file: file,
original_filename: file.name,
slug: uuidv4(),
user: user.id,
section: reportType,
year: fiscalYear,
stt: stt.id,
quarter: fiscalQuarter,
ssp: false,
}
for (const [key, value] of Object.entries(fraReportData)) {
formData.append(key, value)
}

try {
const response = await axiosInstance.post(
`${process.env.REACT_APP_BACKEND_URL}/data_files/`,
formData,
{
headers: { 'Content-Type': 'multipart/form-data' },
withCredentials: true,
}
)

// dispatch(
// getFraSubmissionHistory({
// stt,
// reportType,
// fiscalQuarter,
// fiscalYear,
// })
// )
// or, dispatch the state update if response from upload can contain updated submission history
onSuccess()
} catch (error) {
onError(error)
}

dispatch({
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we stick this in a finally block just for safety? Looks like onError just returns null for the moment so it might not be necessary.

type: SET_IS_UPLOADING_FRA_REPORT,
payload: { isUploadingFraReport: false },
})
}
Loading