Workflow and tests implementation #490

taha-abdullah · 2024-03-19T14:11:37Z

Developed a workflow to build, run, and test FastSurfer on every push to the dev repository:

The workflow is running on a local runner on the Dirac machine
The workflow builds a FastSurfer GPU docker image and runs it on a sample MRI dataset stored on the local runner memory
File existence test is run on the output to ensure all the necessary log files have been outputted by the FastSurfer run
Error message test is run on the log files to check for any errors which may have occurred during the run

Running this workflow requires a local runner.

For now, the workflow is run manually using a workflow-dispatch trigger for easy testing purposes

- Added yaml files for test_file_existence and test_error_messages - Added error handling in test_file_existence and test_error_handling installing dependencies: PyYAML small workflow to run tests while avoiding building and running FastSurfer Adding a file existence test to make sure all the files that are supposed to be in the results are actually there. adding a job to test file existence making jobs sequential by adding dependencies changed license directory removed read-only flags from MRI data and License Added environment variables for MRI data and output adding docker run command changed PYTHONPATH to variable renaming the workflow to 'FastSurfer Docker Image Build' Create docker_build.yml Building GPU FastSurfer Image configuring python commenting out PYTHONPATH Building GPU FastSurfer Image and running analysis simple workflow to test if self hosted runner 'dirac' is running correctly

test_error_messages.py Rename docker_build.yml to fastsurfer_build.yml Update tests.yml separating test-file-existence into new job and adding test-error-messages job Update tests.yml

…ead of yaml files

updated test_file_existence update tests.yml fixed issues in test_file_existence removed debug print statements updated test_error_messages

dkuegler

There are also multiple extra files which should be deleted.

folder structure for test should be

/test
/test/quick_test/...files....
/test/quick_test/data/...files...

dkuegler · 2024-03-19T17:59:13Z

test/__pycache__/__init__.cpython-310.pyc

You should not commit __pycache__ files.

dkuegler · 2024-03-19T18:00:39Z

rough_work/file_existence.py

+
+if __name__ == '__main__':
+
+    if len(sys.argv) > 1:


please use argparse for parameters.

dkuegler · 2024-03-19T18:03:54Z

.github/workflows/dirac-docker.yml

+jobs:
+
+  build:
+    runs-on: dirac


Suggested change

runs-on: dirac

runs-on: ci-gpu

dkuegler · 2024-03-19T18:04:11Z

.github/workflows/dirac-test.yml

+    # The runner that 'build' will run on
+    runs-on:
+      # Self hosted runner called 'dirac'
+      - dirac


Suggested change

- dirac

- ci-gpu

dkuegler · 2024-03-19T18:27:02Z

.github/workflows/dirac-docker.yml

I think we want to run all of these tests as singularity containers. I think we do not want to always build the image for that container, but just mount the "checked out" version into the container. But to get the version information in there we need to run the build script.

So:

uses: actions/checkout

prepare job: build.py --device cuda --dry_only (on the checkout -- this creates the BUILD.info file there)

run singularity, but "mount" the "checked out" directory as /fastsurfer-dev as read-only and set the environment variable FASTSURFER_HOME to /fastsurfer-dev run inside the container

bash <<EOF
ln -sf /fastsurfer-dev/checkpoints /fastsurfer/checkpoints
brun_fastsurfer.sh <<EEOF
subject07=/data/...
subject08=/data/...
subject10)=/data/...
EOF

Similarly run the tests

dkuegler · 2024-03-19T18:39:36Z

rough_work/error_messages.py

Please add docstrings to python files

dkuegler · 2024-03-19T18:44:39Z

rough_work/file_existence.py

+        # Get the folder path from the YAML file
+        cls.folder_path = cls.data.get('folder_path')
+        if not cls.folder_path:
+            raise Exception("The 'folder_path' key was not found in the YAML file")


Suggested change

raise Exception("The 'folder_path' key was not found in the YAML file")

raise ValueError("The 'folder_path' key was not found in the YAML file")

dkuegler · 2024-03-19T18:46:46Z

rough_work/file_existence.py

+            with open(yaml_file_path, 'r') as file:
+                cls.data = yaml.safe_load(file)
+        except FileNotFoundError:
+            raise Exception(f"YAML file not found at path: {yaml_file_path}")


Suggested change

raise Exception(f"YAML file not found at path: {yaml_file_path}")

raise FileNotFoundError(f"YAML file not found at path: {yaml_file_path}")

dkuegler · 2024-03-19T18:47:21Z

rough_work/file_existence.py

+        except FileNotFoundError:
+            raise Exception(f"YAML file not found at path: {yaml_file_path}")
+        except yaml.YAMLError as e:
+            raise Exception(f"Error parsing YAML file: {e}")


Suggested change

raise Exception(f"Error parsing YAML file: {e}")

raise RuntimeError(f"Error parsing YAML file: {e}") from e

dkuegler · 2024-03-19T18:52:27Z

rough_work/file_existence.py

+
+        This method is executed once before any test methods in the class.
+        """
+        yaml_file_path = './test/files.yaml'  # './' refers to the current directory


Best use Path(__file__).parents[0] / "files.yaml"

added fastsurfer_singularity.yaml to run FastSurfer in singularity container

dkuegler · 2024-03-25T13:21:37Z

superceeded by #496

taha-abdullah and others added 5 commits March 19, 2024 14:43

Changed assertIn to assertNotIn in

836e764

test_error_messages.py Rename docker_build.yml to fastsurfer_build.yml Update tests.yml separating test-file-existence into new job and adding test-error-messages job Update tests.yml

Taking yaml files using command line arguments

fe300b1

completed implementation of tests through command line arguments inst…

61754c7

…ead of yaml files

fixed issues with test cases

bc93dd6

updated test_file_existence update tests.yml fixed issues in test_file_existence removed debug print statements updated test_error_messages

dkuegler requested changes Mar 19, 2024

View reviewed changes

Resolved issues from code review

8a110d2

added fastsurfer_singularity.yaml to run FastSurfer in singularity container

dkuegler closed this Mar 25, 2024

	raise Exception("The 'folder_path' key was not found in the YAML file")
	raise ValueError("The 'folder_path' key was not found in the YAML file")

	raise Exception(f"YAML file not found at path: {yaml_file_path}")
	raise FileNotFoundError(f"YAML file not found at path: {yaml_file_path}")

	raise Exception(f"Error parsing YAML file: {e}")
	raise RuntimeError(f"Error parsing YAML file: {e}") from e

Workflow and tests implementation #490

Workflow and tests implementation #490

Uh oh!

Conversation

taha-abdullah commented Mar 19, 2024

Uh oh!

dkuegler left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dkuegler commented Mar 25, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants