Errors in rendered widget image with 16-bit screen

[aganders3]

This came up in napari/docs#283 where ultimately it was determined to be caused by my poor choice of default for the headless-gui action screen size.

The issue is that the code here assumes 8-bit 4-channel image, but QPixmap::toImage() may return different formats depending on the system.

magicgui/src/magicgui/backends/_qtpy/widgets.py

Lines 183 to 190 in 33c5e60

	img = self._qwidget.grab().toImage()
	bits = img.constBits()
	h, w, c = img.height(), img.width(), 4
	if qtpy.API_NAME.startswith("PySide"):
	arr = np.array(bits).reshape(h, w, c)
	else:
	bits.setsize(h * w * c)
	arr = np.frombuffer(bits, np.uint8).reshape(h, w, c) # type: ignore

QImage::convertTo looks like a possible solution (I am happy to make a PR for this), but might result in a copy which it looks like this code at least tries to avoid. I don't know how common this might be, so I'm honestly not sure if this is even worth fixing. Either way we have worked around it by fixing headless-gui, so no worries.

---

Depending on the backend, we get a corrupt image:

Or an exception:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
File /opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/site-packages/IPython/core/formatters.py:344, in BaseFormatter.__call__(self, obj)
    342     method = get_real_method(obj, self.print_method)
    343     if method is not None:
--> 344         return method()
    345     return None
    346 else:

File /opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/site-packages/magicgui/widgets/bases/_widget.py:403, in Widget._repr_png_(self)
    397     print(
    398         "(For a nicer magicgui widget representation in "
    399         "Jupyter, please `pip install imageio`)"
    400     )
    401     return None
--> 403 rendered = self.render()
    404 if rendered is not None:
    405     with BytesIO() as file_obj:

File /opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/site-packages/magicgui/widgets/bases/_widget.py:384, in Widget.render(self)
    382 def render(self) -> np.ndarray:
    383     """Return an RGBA (MxNx4) numpy array bitmap of the rendered widget."""
--> 384     return self._widget._mgui_render()

File /opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/site-packages/magicgui/backends/_qtpy/widgets.py:187, in QBaseWidget._mgui_render(self)
    185 h, w, c = img.height(), img.width(), 4
    186 if qtpy.API_NAME.startswith("PySide"):
--> 187     arr = np.array(bits).reshape(h, w, c)
    188 else:
    189     bits.setsize(h * w * c)

ValueError: cannot reshape array of size 204160 into shape (290,352,4)

[tlambert03]

QImage::convertTo looks like a possible solution (I am happy to make a PR for this), but might result in a copy which it looks like this code at least tries to avoid.

thanks for reporting @aganders3 and thanks for the careful analysis. I'm more than happy to incur a possible copy if it results in more robust behavior here. This isn't a super commonly used method (it's more for docs and stuff like this), so performance and memory consumption are less important than "simply working". If you have an idea for a PR that would make this just work, i'm all for it.

[tlambert03]

yep, looks like the corresponding code in napari does that check ... let's do it here too