Skip to content

Conversation

@dmitrytrager
Copy link
Collaborator

What Issue Does This PR Cover, If Any?

Resolves #420 The best issue number!

What Changed? And Why Did It Change?

  • Allow FileUploadJob to receive language or provider
  • Add 5 CSV files to the set of provider files to send
  • Loop over provider files in content processor so every file still gets its own job
  • Update tests for job and processor

How Has This Been Tested?

Please Provide Screenshots

Additional Comments

@dmitrytrager dmitrytrager marked this pull request as ready for review September 16, 2025 12:39
# Consider removing concurrency limits due to SolidQueue blocking issues
# or use a more specific key to avoid blocking all jobs for a language
limits_concurrency to: 3, key: ->(_language_id, content_id, _content_type) { "hard-limit" }
limits_concurrency to: 3, key: ->(_language_id, _content_id, _content_type) { "hard-limit" }
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wouldn't we keep the param names in parallel with the perform params? _language_id, _file_id, _provider_id?

To be clear, I don't fully grasp how the concurrency keys work.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When I run locally I get this error.
image

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This param is unused. However, number of parameters here should match the number of parameters when we call the job. I am trying to replace this with just *args

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm pushing this branch to staging so Manoj can review the files.

@dmitrytrager dmitrytrager force-pushed the provider-specific-csv-files-420 branch from a11f8e8 to 47a6a0b Compare September 21, 2025 13:13
@dcollie2
Copy link
Collaborator

Manoj said the file formats are working but he has some changes to the naming convention. He's going to get that to me later this week.

@dcollie2
Copy link
Collaborator

Manoj said the file formats are working but he has some changes to the naming convention. He's going to get that to me later this week.

Still waiting for that from Manoj.

@dcollie2
Copy link
Collaborator

The answer was that they want to require the providers to have a file name prefix and use that as the prefix for these files. I have some concerns I'd like to discuss.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Provider-specific CSV Files

2 participants