Investigating additional stage axes. #1094

AdvancedImagingUTSW · 2025-04-30T15:46:43Z

I'm trying to investigate whether or not we will run into any issues with the multi-position table when additional stage axes are added.

After adding a new axis to the configuration file, I do see the axis listed in the multi-position table, but without any entries:

If I do not correctly specify the flip flags, I get this exception:

Traceback

2025-04-30 10:25:51,170 - model - WARNING - base: Stage z1 limits not set in configuration file.
2025-04-30 10:25:51,171 - model - WARNING - base: Stage z2 limits not set in configuration file.
Traceback (most recent call last):
  File "/opt/miniconda3/envs/navigate/bin/navigate", line 8, in <module>
    sys.exit(main())
  File "/Users/Dean/Documents/GitHub/navigate/src/navigate/main.py", line 114, in main
    Controller(
  File "/Users/Dean/Documents/GitHub/navigate/src/navigate/controller/controller.py", line 338, in __init__
    self.populate_experiment_setting(in_initialize=True)
  File "/Users/Dean/Documents/GitHub/navigate/src/navigate/controller/controller.py", line 483, in populate_experiment_setting
    self.channels_tab_controller.populate_experiment_values()
  File "/Users/Dean/Documents/GitHub/navigate/src/navigate/controller/sub_controllers/channels_tab.py", line 261, in populate_experiment_values
    self.update_z_steps()
  File "/Users/Dean/Documents/GitHub/navigate/src/navigate/controller/sub_controllers/channels_tab.py", line 430, in update_z_steps
    if flip_flags["z"]:
KeyError: 'z'
^CFatal Python error: PyEval_RestoreThread: the function must be called with the GIL held, but the GIL is released (the current Python thread state is NULL)
Python runtime state: initialized

Thread 0x000000037168f000 (most recent call first):
  File "/opt/miniconda3/envs/navigate/lib/python3.9/tkinter/__init__.py", line 377 in set
  File "/Users/Dean/Documents/GitHub/navigate/src/navigate/controller/sub_controllers/stages.py", line 373 in set_position_silent
  File "/Users/Dean/Documents/GitHub/navigate/src/navigate/controller/controller.py", line 1380 in update_stage_controller_silent
  File "/Users/Dean/Documents/GitHub/navigate/src/navigate/controller/controller.py", line 1406 in update_event
  File "/opt/miniconda3/envs/navigate/lib/python3.9/threading.py", line 910 in run
  File "/opt/miniconda3/envs/navigate/lib/python3.9/threading.py", line 973 in _bootstrap_inner
  File "/opt/miniconda3/envs/navigate/lib/python3.9/threading.py", line 930 in _bootstrap

Current thread 0x00000001f4c8cc80 (most recent call first):
  File "/opt/miniconda3/envs/navigate/lib/python3.9/threading.py", line 1448 in _shutdown

To prevent this particular error from happening, I added a .get call that defaults to False.

If I run a multi-position acquisition with empty values, I get an exception.

Traceback

zsh: abort      navigate -sh
(navigate) Dean@SW589085 navigate % 
(navigate) Dean@SW589085 navigate % navigate -sh
WARNING: navigate was built to operate on a Windows platform. While much of the software will work for evaluation purposes, some unanticipated behaviors may occur. For example, it is known that the Tkinter-based GUI does not grid symmetrically, nor resize properly on MacOS. Testing on Linux operating systems has not been performed.
2025-04-30 10:26:28,491 - model - WARNING - base: Stage z2 limits not set in configuration file.
Exception in thread camera:
Traceback (most recent call last):
  File "/Users/Dean/Documents/GitHub/navigate/src/navigate/controller/thread_pool.py", line 248, in func
    target(*args, **kwargs)
  File "/Users/Dean/Documents/GitHub/navigate/src/navigate/controller/controller.py", line 1161, in capture_image
    self.acquire_bar_controller.progress_bar(
  File "/Users/Dean/Documents/GitHub/navigate/src/navigate/controller/sub_controllers/acquire_bar.py", line 213, in progress_bar
    images_received / bottom_anticipated_images
ZeroDivisionError: division by zero

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/Dean/Documents/GitHub/navigate/src/navigate/controller/thread_pool.py", line 98, in run
    self._target(*self._args, **self._kwargs)
  File "/Users/Dean/Documents/GitHub/navigate/src/navigate/controller/thread_pool.py", line 250, in func
    logger.exception(
  File "/opt/miniconda3/envs/navigate/lib/python3.9/logging/__init__.py", line 1481, in exception
    self.error(msg, *args, exc_info=exc_info, **kwargs)
  File "/opt/miniconda3/envs/navigate/lib/python3.9/logging/__init__.py", line 1475, in error
    self._log(ERROR, msg, args, **kwargs)
  File "/opt/miniconda3/envs/navigate/lib/python3.9/logging/__init__.py", line 1589, in _log
    self.handle(record)
  File "/opt/miniconda3/envs/navigate/lib/python3.9/logging/__init__.py", line 1599, in handle
    self.callHandlers(record)
  File "/opt/miniconda3/envs/navigate/lib/python3.9/logging/__init__.py", line 1661, in callHandlers
    hdlr.handle(record)
  File "/opt/miniconda3/envs/navigate/lib/python3.9/logging/__init__.py", line 948, in handle
    rv = self.filter(record)
  File "/opt/miniconda3/envs/navigate/lib/python3.9/logging/__init__.py", line 806, in filter
    result = f.filter(record)
  File "/Users/Dean/Documents/GitHub/navigate/src/navigate/log_files/filters.py", line 95, in filter
    if record.getMessage().startswith("Performance"):
  File "/opt/miniconda3/envs/navigate/lib/python3.9/logging/__init__.py", line 367, in getMessage
    msg = msg % self.args
TypeError: not all arguments converted during string formatting

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/miniconda3/envs/navigate/lib/python3.9/threading.py", line 973, in _bootstrap_inner
    self.run()
  File "/Users/Dean/Documents/GitHub/navigate/src/navigate/controller/thread_pool.py", line 100, in run
    logger.exception(
  File "/opt/miniconda3/envs/navigate/lib/python3.9/logging/__init__.py", line 1481, in exception
    self.error(msg, *args, exc_info=exc_info, **kwargs)
  File "/opt/miniconda3/envs/navigate/lib/python3.9/logging/__init__.py", line 1475, in error
    self._log(ERROR, msg, args, **kwargs)
  File "/opt/miniconda3/envs/navigate/lib/python3.9/logging/__init__.py", line 1589, in _log
    self.handle(record)
  File "/opt/miniconda3/envs/navigate/lib/python3.9/logging/__init__.py", line 1599, in handle
    self.callHandlers(record)
  File "/opt/miniconda3/envs/navigate/lib/python3.9/logging/__init__.py", line 1661, in callHandlers
    hdlr.handle(record)
  File "/opt/miniconda3/envs/navigate/lib/python3.9/logging/__init__.py", line 948, in handle
    rv = self.filter(record)
  File "/opt/miniconda3/envs/navigate/lib/python3.9/logging/__init__.py", line 806, in filter
    result = f.filter(record)
  File "/Users/Dean/Documents/GitHub/navigate/src/navigate/log_files/filters.py", line 95, in filter
    if record.getMessage().startswith("Performance"):
  File "/opt/miniconda3/envs/navigate/lib/python3.9/logging/__init__.py", line 367, in getMessage
    msg = msg % self.args
TypeError: not all arguments converted during string formatting

For this one, I wrapped the bottom_anticipated_images in a try/except statement.

And finally, after rebooting the software, and running a multi position acquisition, I get this error in the logs:

Traceback

2025-04-30 10:43:44,056 - model - DEBUG - feature_container: SignalContainer - Traceback (most recent call last):
  File "/Users/Dean/Documents/GitHub/navigate/src/navigate/model/features/feature_container.py", line 558, in run
    result, is_end = self.curr_node.run(*args, wait_response=wait_response)
  File "/Users/Dean/Documents/GitHub/navigate/src/navigate/model/features/feature_container.py", line 243, in run
    self.node_funcs["init"]()
  File "/Users/Dean/Documents/GitHub/navigate/src/navigate/model/features/common_features.py", line 1180, in pre_signal_func
    [self.positions[0][i] for i in self.axes_index],
  File "/Users/Dean/Documents/GitHub/navigate/src/navigate/model/features/common_features.py", line 1180, in <listcomp>
    [self.positions[0][i] for i in self.axes_index],
IndexError: list index out of range

Default to 0.0 instead of np.nan.

AdvancedImagingUTSW · 2025-04-30T15:58:25Z

We were supposed to fill any empty values here:

            data[name] = list(
                pos[i] if i < len(pos) else np.nan for pos in positions[start_index:]
            )

However, the multi-position table was not being populated with np.nan, and instead remained empty. If I instead populate it with 0.0, it works. The multi-position table is populated and the multi-position acquisition goes without error. This does feel a bit risky though as it could send a stage to position zero. It might make more sense to populate it with the stage's current position.

codecov · 2025-04-30T16:20:13Z

Codecov Report

Attention: Patch coverage is 83.87097% with 10 lines in your changes missing coverage. Please review.

Project coverage is 55.03%. Comparing base (b5b2c61) to head (e108a3f).

Files with missing lines	Patch %	Lines
...vigate/controller/sub_controllers/multiposition.py	77.27%	5 Missing ⚠️
...navigate/controller/sub_controllers/acquire_bar.py	50.00%	3 Missing ⚠️
...rc/navigate/controller/configuration_controller.py	96.87%	1 Missing ⚠️
...avigate/controller/sub_controllers/channels_tab.py	50.00%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #1094      +/-   ##
===========================================
+ Coverage    55.01%   55.03%   +0.02%     
===========================================
  Files          184      184              
  Lines        21043    21050       +7     
===========================================
+ Hits         11576    11585       +9     
+ Misses        9467     9465       -2

Flag	Coverage Δ
unittests	`55.03% <83.87%> (+0.02%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

AdvancedImagingUTSW · 2025-05-05T19:08:53Z

The inspiration for this...

2025-04-16 10:30:59,016 - model - DEBUG - feature_container: SignalContainer - Traceback (most recent call last):
  File "C:\Users\Nikon\Documents\GitHub\navigate\src\navigate\model\features\feature_container.py", line 558, in run
    result, is_end = self.curr_node.run(*args, wait_response=wait_response)
  File "C:\Users\Nikon\Documents\GitHub\navigate\src\navigate\model\features\feature_container.py", line 243, in run
    self.node_funcs["init"]()
  File "C:\Users\Nikon\Documents\GitHub\navigate\src\navigate\model\features\common_features.py", line 1039, in pre_signal_func
    self.axes_index = [self.position_headers.index(axis.upper()) for axis in ["x", "y", "z", "theta", "f"]]
  File "C:\Users\Nikon\Documents\GitHub\navigate\src\navigate\model\features\common_features.py", line 1039, in <listcomp>
    self.axes_index = [self.position_headers.index(axis.upper()) for axis in ["x", "y", "z", "theta", "f"]]
ValueError: 'THETA' is not in list

AdvancedImagingUTSW added 3 commits April 30, 2025 10:28

Investigating additional stage axes.

1b7f74d

Default dictionary entry and try/except

59ba563

Update multiposition.py

e108a3f

Default to 0.0 instead of np.nan.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Investigating additional stage axes. #1094

Investigating additional stage axes. #1094

Uh oh!

AdvancedImagingUTSW commented Apr 30, 2025

Uh oh!

AdvancedImagingUTSW commented Apr 30, 2025

Uh oh!

codecov bot commented Apr 30, 2025

Uh oh!

AdvancedImagingUTSW commented May 5, 2025

Uh oh!

Uh oh!

Investigating additional stage axes. #1094

Are you sure you want to change the base?

Investigating additional stage axes. #1094

Uh oh!

Conversation

AdvancedImagingUTSW commented Apr 30, 2025

Uh oh!

AdvancedImagingUTSW commented Apr 30, 2025

Uh oh!

codecov bot commented Apr 30, 2025

Codecov Report

Uh oh!

AdvancedImagingUTSW commented May 5, 2025

Uh oh!

Uh oh!