backpopulate files by resource id #2174

abeglova · 2025-04-02T15:29:18Z

What are the relevant tickets?

closes https://github.com/mitodl/hq/issues/7021

Description (What does it do?)

This pr updates the backpopulate files management commands for edx platforms to accept learning_resource_id to make it possible to update content files for a single learning resource

How can this be tested?

Go to http://api.open.odl.local:8063/api/v1/learning_resources/ and find an course on one of the edx platforms

Go to
http://api.open.odl.local:8063/api/v1/contentfiles/?resource_id= to check if the learning resource has content files

if you find some content files you can run

from learning_resources.models import *
ContentFile.objects.filter(run__learning_resource_id=<resource id>).update(content='i like dogs')

run

docker-compose run -u root --rm web ./manage.py <appropriate edx platform backpopulate job> --overwrite --resource_id = <resource id>

The job should finish relatively quickly
Go back to
http://api.open.odl.local:8063/api/v1/contentfiles/?resource_id= and verify that the content files exist and have the correct content

Run a file backpopulate command without the learning_resource flag and verify that it works normally. You can kill the job prematurely since it can take a while if you haven't already backpopulated the content files for that platform or if you use the --overwrite flag

shanbady

Not sure exactly whats happening buy when i try running python manage.py backpopulate_mitxonline_files --overwrite --learning_resource_id 8077 it does not appear to be updating the content after I changed it. i also tried deleting the contentfile object altogether to see if it would come back but it didnt. the celery task did seem to only process one item though just looking at the runtime

shanbady · 2025-04-03T18:32:17Z

learning_resources/management/commands/backpopulate_mit_edx_files.py

@@ -28,11 +28,20 @@ def add_arguments(self, parser):
            help="Overwrite any existing records",
        )

+        parser.add_argument(
+            "--learning_resource_id",
+            dest="learning_resource_id",


it might be good to have this parameter accept a list of comma separated ids like we do with embeddings and summary generation commands. something like --resource-ids 1,234,422

github-actions · 2025-04-09T20:30:53Z

OpenAPI Changes

Show/hide 18 changes: 12 error, 0 warning, 6 info

18 changes: 12 error, 0 warning, 6 info
error	[response-required-property-removed] at head/openapi/specs/v0.yaml	
	in API GET /api/v0/users/
		removed the required property '/items/is_authenticated' from the response with the '200' status

error	[request-property-became-required] at head/openapi/specs/v0.yaml	
	in API POST /api/v0/users/
		the request property 'profile' became required

error	[request-property-became-required] at head/openapi/specs/v0.yaml	
	in API POST /api/v0/users/
		the request property 'profile' became required

error	[request-property-became-required] at head/openapi/specs/v0.yaml	
	in API POST /api/v0/users/
		the request property 'profile' became required

error	[response-required-property-removed] at head/openapi/specs/v0.yaml	
	in API POST /api/v0/users/
		removed the required property 'is_authenticated' from the response with the '201' status

error	[response-required-property-removed] at head/openapi/specs/v0.yaml	
	in API GET /api/v0/users/me/
		removed the required property 'is_authenticated' from the response with the '200' status

error	[response-required-property-removed] at head/openapi/specs/v0.yaml	
	in API GET /api/v0/users/{username}/
		removed the required property 'is_authenticated' from the response with the '200' status

error	[response-required-property-removed] at head/openapi/specs/v0.yaml	
	in API PATCH /api/v0/users/{username}/
		removed the required property 'is_authenticated' from the response with the '200' status

error	[request-property-became-required] at head/openapi/specs/v0.yaml	
	in API PUT /api/v0/users/{username}/
		the request property 'profile' became required

error	[request-property-became-required] at head/openapi/specs/v0.yaml	
	in API PUT /api/v0/users/{username}/
		the request property 'profile' became required

error	[request-property-became-required] at head/openapi/specs/v0.yaml	
	in API PUT /api/v0/users/{username}/
		the request property 'profile' became required

error	[response-required-property-removed] at head/openapi/specs/v0.yaml	
	in API PUT /api/v0/users/{username}/
		removed the required property 'is_authenticated' from the response with the '200' status

info	[response-property-became-required] at head/openapi/specs/v0.yaml	
	in API GET /api/v0/users/
		the response property '/items/profile' became required for the status '200'

info	[response-property-became-required] at head/openapi/specs/v0.yaml	
	in API POST /api/v0/users/
		the response property 'profile' became required for the status '201'

info	[response-property-became-required] at head/openapi/specs/v0.yaml	
	in API GET /api/v0/users/me/
		the response property 'profile' became required for the status '200'

info	[response-property-became-required] at head/openapi/specs/v0.yaml	
	in API GET /api/v0/users/{username}/
		the response property 'profile' became required for the status '200'

info	[response-property-became-required] at head/openapi/specs/v0.yaml	
	in API PATCH /api/v0/users/{username}/
		the response property 'profile' became required for the status '200'

info	[response-property-became-required] at head/openapi/specs/v0.yaml	
	in API PUT /api/v0/users/{username}/
		the response property 'profile' became required for the status '200'

github-actions · 2025-04-09T20:56:41Z

OpenAPI Changes

Show/hide 18 changes: 12 error, 0 warning, 6 info

18 changes: 12 error, 0 warning, 6 info
error	[response-required-property-removed] at head/openapi/specs/v0.yaml	
	in API GET /api/v0/users/
		removed the required property '/items/is_authenticated' from the response with the '200' status

error	[request-property-became-required] at head/openapi/specs/v0.yaml	
	in API POST /api/v0/users/
		the request property 'profile' became required

error	[request-property-became-required] at head/openapi/specs/v0.yaml	
	in API POST /api/v0/users/
		the request property 'profile' became required

error	[request-property-became-required] at head/openapi/specs/v0.yaml	
	in API POST /api/v0/users/
		the request property 'profile' became required

error	[response-required-property-removed] at head/openapi/specs/v0.yaml	
	in API POST /api/v0/users/
		removed the required property 'is_authenticated' from the response with the '201' status

error	[response-required-property-removed] at head/openapi/specs/v0.yaml	
	in API GET /api/v0/users/me/
		removed the required property 'is_authenticated' from the response with the '200' status

error	[response-required-property-removed] at head/openapi/specs/v0.yaml	
	in API GET /api/v0/users/{username}/
		removed the required property 'is_authenticated' from the response with the '200' status

error	[response-required-property-removed] at head/openapi/specs/v0.yaml	
	in API PATCH /api/v0/users/{username}/
		removed the required property 'is_authenticated' from the response with the '200' status

error	[request-property-became-required] at head/openapi/specs/v0.yaml	
	in API PUT /api/v0/users/{username}/
		the request property 'profile' became required

error	[request-property-became-required] at head/openapi/specs/v0.yaml	
	in API PUT /api/v0/users/{username}/
		the request property 'profile' became required

error	[request-property-became-required] at head/openapi/specs/v0.yaml	
	in API PUT /api/v0/users/{username}/
		the request property 'profile' became required

error	[response-required-property-removed] at head/openapi/specs/v0.yaml	
	in API PUT /api/v0/users/{username}/
		removed the required property 'is_authenticated' from the response with the '200' status

info	[response-property-became-required] at head/openapi/specs/v0.yaml	
	in API GET /api/v0/users/
		the response property '/items/profile' became required for the status '200'

info	[response-property-became-required] at head/openapi/specs/v0.yaml	
	in API POST /api/v0/users/
		the response property 'profile' became required for the status '201'

info	[response-property-became-required] at head/openapi/specs/v0.yaml	
	in API GET /api/v0/users/me/
		the response property 'profile' became required for the status '200'

info	[response-property-became-required] at head/openapi/specs/v0.yaml	
	in API GET /api/v0/users/{username}/
		the response property 'profile' became required for the status '200'

info	[response-property-became-required] at head/openapi/specs/v0.yaml	
	in API PATCH /api/v0/users/{username}/
		the response property 'profile' became required for the status '200'

info	[response-property-became-required] at head/openapi/specs/v0.yaml	
	in API PUT /api/v0/users/{username}/
		the response property 'profile' became required for the status '200'

github-actions · 2025-04-09T21:04:40Z

OpenAPI Changes

Show/hide 18 changes: 12 error, 0 warning, 6 info

18 changes: 12 error, 0 warning, 6 info
error	[response-required-property-removed] at head/openapi/specs/v0.yaml	
	in API GET /api/v0/users/
		removed the required property '/items/is_authenticated' from the response with the '200' status

error	[request-property-became-required] at head/openapi/specs/v0.yaml	
	in API POST /api/v0/users/
		the request property 'profile' became required

error	[request-property-became-required] at head/openapi/specs/v0.yaml	
	in API POST /api/v0/users/
		the request property 'profile' became required

error	[request-property-became-required] at head/openapi/specs/v0.yaml	
	in API POST /api/v0/users/
		the request property 'profile' became required

error	[response-required-property-removed] at head/openapi/specs/v0.yaml	
	in API POST /api/v0/users/
		removed the required property 'is_authenticated' from the response with the '201' status

error	[response-required-property-removed] at head/openapi/specs/v0.yaml	
	in API GET /api/v0/users/me/
		removed the required property 'is_authenticated' from the response with the '200' status

error	[response-required-property-removed] at head/openapi/specs/v0.yaml	
	in API GET /api/v0/users/{username}/
		removed the required property 'is_authenticated' from the response with the '200' status

error	[response-required-property-removed] at head/openapi/specs/v0.yaml	
	in API PATCH /api/v0/users/{username}/
		removed the required property 'is_authenticated' from the response with the '200' status

error	[request-property-became-required] at head/openapi/specs/v0.yaml	
	in API PUT /api/v0/users/{username}/
		the request property 'profile' became required

error	[request-property-became-required] at head/openapi/specs/v0.yaml	
	in API PUT /api/v0/users/{username}/
		the request property 'profile' became required

error	[request-property-became-required] at head/openapi/specs/v0.yaml	
	in API PUT /api/v0/users/{username}/
		the request property 'profile' became required

error	[response-required-property-removed] at head/openapi/specs/v0.yaml	
	in API PUT /api/v0/users/{username}/
		removed the required property 'is_authenticated' from the response with the '200' status

info	[response-property-became-required] at head/openapi/specs/v0.yaml	
	in API GET /api/v0/users/
		the response property '/items/profile' became required for the status '200'

info	[response-property-became-required] at head/openapi/specs/v0.yaml	
	in API POST /api/v0/users/
		the response property 'profile' became required for the status '201'

info	[response-property-became-required] at head/openapi/specs/v0.yaml	
	in API GET /api/v0/users/me/
		the response property 'profile' became required for the status '200'

info	[response-property-became-required] at head/openapi/specs/v0.yaml	
	in API GET /api/v0/users/{username}/
		the response property 'profile' became required for the status '200'

info	[response-property-became-required] at head/openapi/specs/v0.yaml	
	in API PATCH /api/v0/users/{username}/
		the response property 'profile' became required for the status '200'

info	[response-property-became-required] at head/openapi/specs/v0.yaml	
	in API PUT /api/v0/users/{username}/
		the response property 'profile' became required for the status '200'

github-actions · 2025-04-10T13:20:25Z

OpenAPI Changes

Show/hide No detectable change.

shanbady

👍

github-actions · 2025-04-11T16:01:45Z

OpenAPI Changes

Show/hide No detectable change.

abeglova marked this pull request as ready for review April 2, 2025 16:18

abeglova added the Needs Review An open Pull Request that is ready for review label Apr 2, 2025

shanbady self-assigned this Apr 3, 2025

shanbady requested changes Apr 3, 2025

View reviewed changes

shanbady assigned abeglova Apr 3, 2025

shanbady added the Waiting on author label Apr 3, 2025

abeglova force-pushed the ab/backpopulate-files-by-run branch from 150134e to 76b063f Compare April 10, 2025 13:20

shanbady approved these changes Apr 10, 2025

View reviewed changes

backpopulate edx files by learning_resource_id

3a52ed8

abeglova force-pushed the ab/backpopulate-files-by-run branch from 76b063f to 3a52ed8 Compare April 11, 2025 16:01

abeglova merged commit 70b73c4 into main Apr 11, 2025
12 checks passed

abeglova deleted the ab/backpopulate-files-by-run branch April 11, 2025 16:16

This was referenced Apr 29, 2025

Release 0.32.0 #2218

Closed

Release 0.32.0 #2227

Closed

Release 0.32.0 #2228

Closed

Release 0.31.2 #2229

Closed

This was referenced May 9, 2025

Release 0.31.2 #2243

Closed

Release 0.31.3 #2244

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

backpopulate files by resource id #2174

backpopulate files by resource id #2174

Uh oh!

abeglova commented Apr 2, 2025 •

edited

Loading

Uh oh!

shanbady left a comment

Uh oh!

shanbady Apr 3, 2025

Uh oh!

github-actions bot commented Apr 9, 2025

Uh oh!

github-actions bot commented Apr 9, 2025

Uh oh!

github-actions bot commented Apr 9, 2025

Uh oh!

github-actions bot commented Apr 10, 2025

Uh oh!

shanbady left a comment

Uh oh!

github-actions bot commented Apr 11, 2025

Uh oh!

Uh oh!

Uh oh!

backpopulate files by resource id #2174

backpopulate files by resource id #2174

Uh oh!

Conversation

abeglova commented Apr 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What are the relevant tickets?

Description (What does it do?)

How can this be tested?

Uh oh!

shanbady left a comment

Choose a reason for hiding this comment

Uh oh!

shanbady Apr 3, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Apr 9, 2025

OpenAPI Changes

Uh oh!

github-actions bot commented Apr 9, 2025

OpenAPI Changes

Uh oh!

github-actions bot commented Apr 9, 2025

OpenAPI Changes

Uh oh!

github-actions bot commented Apr 10, 2025

OpenAPI Changes

Uh oh!

shanbady left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Apr 11, 2025

OpenAPI Changes

Uh oh!

Uh oh!

Uh oh!

abeglova commented Apr 2, 2025 •

edited

Loading