HardFault with DEBUG build on Metro M4 due to caught AttributeError

[FoamyGuy]

CircuitPython version

Adafruit CircuitPython 8.0.0-beta.4-29-g2c479f4fc-dirty on 2022-11-26; Adafruit Metro M4 Express with samd51j19
Board ID:metro_m4_express
UID:B54635543254375320202039372C0BFF

Code/REPL

>>> from displayio import Group
>>> g = Group()
>>> g = Group(scale=2)  # <- creating Group works fine
 
>>> from adafruit_display_text import label
>>> import terminalio
>>> label.Label(terminalio.FONT)  # <- Hard Fault occurs when we run this

Behavior

On a Metro M4 with DEBUG=1 build I am getting a Hard Fault from creating a Label object in the REPL without doing anything else with it. Haven't initialized or shown anything on a display, only created the Label with it's constructor and that leads to the Hard Fault.

Label class extends LabelBase which itself extends displayio.Group. I tried creating a Group directly with it's constructor and that does execute successfully with no hardfault.

I also attempted to create the LabelBase class and that also results in the HardFault:

>>> from adafruit_display_text import LabelBase
>>> import terminalio
>>> lb = LabelBase(terminalio.FONT) # <- this leads to the same HardFault

Running the same code on the same device with a stock build i.e. without DEBUG seems to run fine and does not lead to the Hard Fault. As best as I can tell somehow the DEBUG flag introduces something that is causing or helping to cause the HardFault but I'm not super confident in my findings thus far.

Description

No response

Additional information

I found this while I was attempting to debug a different Hard Fault so I was setup to get the backtrace from it using a J-Link debugger. This is the resulting backtrace from setting a breakpoint at HardFault_Handler

Breakpoint 1, HardFault_Handler () at supervisor/port.c:718
718         reset_into_safe_mode(HARD_CRASH);
(gdb) bt
#0  HardFault_Handler () at supervisor/port.c:718
#1  <signal handler called>
#2  0x00028186 in mp_execute_bytecode (code_state=0x20001934 <_pystack+288>, inject_exc=<optimized out>) at ../../py/vm.c:1240
#3  0x000206d8 in fun_bc_call (self_in=0x20027e20, n_args=<optimized out>, n_kw=0, args=0x200018c4 <_pystack+176>) at ../../py/objfun.c:297
#4  0x000284e8 in mp_execute_bytecode (code_state=0x200018ac <_pystack+152>, inject_exc=<optimized out>) at ../../py/runtime.c:680
#5  0x000206d8 in fun_bc_call (self_in=0x20005ab0, n_args=<optimized out>, n_kw=2, args=0x20004a00) at ../../py/objfun.c:297
#6  0x00047e1a in closure_call (self_in=0x200286a0, n_args=2, n_kw=2, args=0x20001894 <_pystack+128>) at ../../py/objclosure.c:56
#7  0x0002834e in mp_execute_bytecode (code_state=0x20001854 <_pystack+64>, inject_exc=<optimized out>) at ../../py/runtime.c:858
#8  0x000206d8 in fun_bc_call (self_in=0x20006420, n_args=<optimized out>, n_kw=2, args=0x200049c0) at ../../py/objfun.c:297
#9  0x00047e1a in closure_call (self_in=0x20027a60, n_args=2, n_kw=2, args=0x200049a4) at ../../py/objclosure.c:56
#10 0x0002ba8c in mp_obj_instance_make_new (self=<optimized out>, n_args=1, n_kw=2, args=0x20001830 <_pystack+28>) at ../../py/objtype.c:384
#11 0x000284e8 in mp_execute_bytecode (code_state=0x20001814 <_pystack>, inject_exc=<optimized out>) at ../../py/runtime.c:680
#12 0x000206d8 in fun_bc_call (self_in=0x20029570, n_args=<optimized out>, n_kw=0, args=0x0) at ../../py/objfun.c:297
#13 0x00031e30 in parse_compile_execute (source=0x6b0e5, input_kind=<optimized out>, exec_flags=32, result=0x20000ebc <_exec_result.lto_priv.0>)
    at ../../shared/runtime/pyexec.c:146
--Type <RET> for more, q to quit, c to continue without paging--
#14 0x000247b8 in pyexec_file (result=0x20000ebc <_exec_result.lto_priv.0>, filename=<optimized out>) at ../../shared/runtime/pyexec.c:750
#15 maybe_run_list (filenames=filenames@entry=0x68e24 <supported_filenames>) at ../../main.c:250
#16 0x00039144 in run_code_py (simulate_reset=<synthetic pointer>, safe_mode=<optimized out>) at ../../main.c:428
#17 main () at ../../main.c:1018

Note: In order to make a DEBUG build for the metro M4 I had to disable module(s) to save space to get the build to work. I disabled ULAB with CIRCUITPY_ULAB = 0 inside of mpconfig file.

[FoamyGuy]

I also tested this on a Feather S2 TFT with a DEBUG build and it seems to run successfully there without a HardFault. So this seems to be somehow tied specifically to SAMD port and DEBUG version of the build as far as I can tell.

[FoamyGuy]

Trying to distill the issue further down into the bare minimum needed to reproduce, I also tried removing Label from the mix to find the root cause, but was unsuccessful.

I created this class which is structured similarly to LabelBase

import displayio
class HardfaultCheck(displayio.Group):
    def __init__(self, x=0, y=0):
        super().__init__(x=x, y=y, scale=1)
        self._local_group = displayio.Group()
        self.append(self._local_group)

I imported this and created an instance with constructor in the REPL, but this did not lead to the Hard Fault. So it must be something else about what Label does during initialization that causes this. My theory was that it was caused by the way Group is extended and then created from the subclass. But this seems to rule that out as the problem.

When I get back to this I'll attempt to further narrow down the specific line(s) in Label that cause the HardFault most immediately to see if I can make a reproducer that doesn't rely on the library.

[tannewt]

You could try it without the COMPUTED_GOTO flag. Looks like that's where its happening: https://github.com/adafruit/circuitpython/blob/main/py/vm.c#L1240

[dhalbert]

I've been spending a while on this. It doesn't seem to have anything to do with displayio at all, or subclassing native classes, etc., all of which I was suspicious of. Some code is crashing when doing an assignment after causing and catching an AttributeError. Trying to get to the simplest case. More later.

[jepler]

Did this start after chained exceptions were added?

[dhalbert]

Did this start after chained exceptions were added?

I thought it might be related, so I checked beta.4, and it's broken in beta.4 also. I'll double check that, but chained exceptions were added after beta.4, according to the release notes.

[dhalbert]

This is the minimal crash-inducing program:

crash.py:

def crash():
    try:
        None.wrong()
    except AttributeError:
        pass
    i = None

crash()

Necessary elements:

• The try block must be in a function. It does not crash when at the top level.
• I tried raising ZeroDivisionError instead, and it does not cause the problem.
• The local var assignment must be present. It doesn't really matter what it is. In Label, it was an assignment of 0.

The original issue with Label had to do with testing whether a FontProtocol object had a certain method, by catching the AttributeError, which boiled down to the above.

This problem is present in 8.0.0-alpha.0 but not 7.3.3. I tried doing a bisect on what I thought was a straight line of commits, but failed to find a pivotal commit. (The bisect got down to one commit, but the one before it was also bad.) I'm wondering if it's inside the upstream merge from MicroPython.

[dhalbert]

I did some bisect-like builds by hand to track this down. I believe the critical commit is
58485bc. It switched the default SAMx5x builds from -O2 to -Os, and also removed -fno-inline-functions. When DEBUG=1 is set, additional (partly redundant) -O flags -Og -Os are piled on, but the -fno-inline-functions remains removed.

This doesn't really explain the bug, but maybe something about the nlr stuff is sensitive to this.

[jepler]

That's interesting. I had recently noticed that "-O2" is broken in sam 5x builds: #7279 -- however, that stopped circuitpython from coming up at all