Skip to content

Restore missing symlink to python in docker image (SCP-4547) #299

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Mar 31, 2023

Conversation

jlchang
Copy link
Contributor

@jlchang jlchang commented Mar 31, 2023

Minor updates to Dockerfile and requirements.txt to fix broken ingest docker image:

  • Restore missing symlink to python
  • Update pymongo for python 3.10 breaking change (references: PEP-0353 and bpo-40943)

This docker image update may be more easily evaluated by checking that portal core integration tests pass. For those who wish to test at this stage, the following instructions are for manual testing:

To test, follow README instructions for Testing in Docker locally Steps 1,2 & 4

Once in the docker container, set up to bypass the mongo database requirement:
export BYPASS_MONGO_WRITES='yes'

Test ingest of cluster and metadata files should succeed (despite unregistered study-id and study-file-id provided):

  1. python ingest_pipeline.py --study-id 5d276a50421aa9117c982845 --study-file-id 5dd5ae25421aa910a723a337 ingest_cell_metadata --cell-metadata-file ../tests/data/annotation/metadata/convention/valid_no_array_v2.0.0.txt --study-accession SCP123 --ingest-cell-metadata
  2. python ingest_pipeline.py --study-id 5d276a50421aa9117c982845 --study-file-id 5dd5ae25421aa910a723a337 ingest_cluster --cluster-file ../tests/data/cluster_example.txt --ingest-cluster --name cluster1 --domain-ranges "{'x':[-1, 1], 'y':[-1, 1], 'z':[-1, 1]}"

Test ingest of expression matrix file will fail (due to unregistered study-id and study-file-id provided) with:

ValueError: Study ID is not registered with a study. Please provide a valid study ID.
Due to lack of database bypass mechanism for expression matrix ingests.

for reference, expression matrix ingest command:
python ingest_pipeline.py --study-id 5d276a50421aa9117c982845 --study-file-id 5dd5ae25421aa910a723a337 ingest_expression --taxon-name 'Homo sapiens' --taxon-common-name human --ncbi-taxid 9606 --matrix-file ../tests/data/dense_matrix_19_genes_1000_cells.txt --matrix-file-type dense

Expression matrix validity can be tested by pointing a portal core instance to an ingest docker image with a docker build from this PR (eg. gcr.io/broad-singlecellportal-staging/scp-ingest-jlc_fix_image:e42abd1) and running the test_create_default_testing_study integration test:
bin/run_tests.sh -l -t test/integration/study_creation_test.rb
The above test would also validate metadata and cluster ingest.

references: PEP-0353 and bpo-40943
Copy link
Member

@eweitz eweitz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code looks good!

@jlchang jlchang merged commit dd77866 into development Mar 31, 2023
@jlchang jlchang deleted the jlc_fix_image branch March 31, 2023 17:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants