itertools segmentation fault

[pochmann3]

Crash report

What happened?

This code showing the first 20 Fibonacci numbers works fine:

from itertools import *

def Fibonacci():
    parts = [(0, 1)]
    whole = chain.from_iterable(parts)
    output, feedback = tee(whole)
    parts.append(map(sum, pairwise(feedback)))
    return output

print(*islice(Fibonacci(), 20))

Output (Attempt This Online!):

0 1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987 1597 2584 4181

But if I keep a reference, it crashes (after successfully printing the numbers):

from itertools import *

def Fibonacci():
    parts = [(0, 1)]
    whole = chain.from_iterable(parts)
    output, feedback = tee(whole)
    parts.append(map(sum, pairwise(feedback)))
    return output

f = Fibonacci()
print(*islice(f, 20))

The error output (Attempt This Online!):

/ATO/runner: line 4:     2 Segmentation fault      (core dumped) /ATO/yargs %1 /ATO/options /ATO/yargs %2 /ATO/arguments python %1 /ATO/code %2 < /ATO/input

@rhettinger

CPython versions tested on:

3.12

Operating systems tested on:

Linux

Output from running 'python -VV' on the command line:

Python 3.12.0 (main, Oct 7 2023, 10:42:35) [GCC 13.2.1 20230801]

[terryjreedy]

I reproduced online with the customized 3rd party ATO build, but as I expected, this code runs fine for me on Windows with 3.13.0a5 installed from python.org or with a local build of main. Did you see this behavior with any current official build? If not, this seems likely specific to ATO and whatever sandboxing code it added. You can report on ATO's github.

[pochmann3]

@terryjreedy Not with an official build (I don't have a device for that), but I did reproduce it here in a GitHub codespace with main from a few days ago (python -VV says 3.13.0a5+), built as instructed by the devguide.

[pochmann3]

GitHub codespace outputs:

First code:

0 1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987 1597 2584 4181

Second code:

0 1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987 1597 2584 4181
python: ./Include/internal/pycore_typeobject.h:110: void *_PyType_GetModuleState(PyTypeObject *): Assertion `et->ht_module' failed.
Aborted (core dumped)

python -VV:

Python 3.13.0a5+ (heads/main:3f1b6efee9, Mar 13 2024, 05:29:59) [Clang 15.0.7 (Fedora 15.0.7-2.fc37)]

[gaogaotiantian]

I'm able to repro it on WSL+Ubuntu:

0x000055555589f680 in teedataobject_clear (tdo=<optimized out>) at ./Modules/itertoolsmodule.c:841
841         teedataobject_safe_decref(tmp, state->teedataobject_type);
(gdb) bt
#0  0x000055555589f680 in teedataobject_clear (tdo=<optimized out>) at ./Modules/itertoolsmodule.c:841
#1  0x00005555557e0c2e in delete_garbage (state=0x7fffffffd8a0) at Python/gc_free_threading.c:766
#2  gc_collect_internal (interp=<optimized out>, state=state@entry=0x7fffffffd8a0) at Python/gc_free_threading.c:1072
#3  0x00005555557e21e4 in gc_collect_main (generation=2, reason=_Py_GC_REASON_SHUTDOWN, tstate=0x555555b06ab8 <_PyRuntime+259192>) at Python/gc_free_threading.c:1153
#4  _PyGC_CollectNoFail (tstate=tstate@entry=0x555555b06ab8 <_PyRuntime+259192>) at Python/gc_free_threading.c:1475
#5  0x0000555555819599 in finalize_modules (tstate=tstate@entry=0x555555b06ab8 <_PyRuntime+259192>) at Python/pylifecycle.c:1671
#6  0x0000555555819da0 in Py_FinalizeEx () at Python/pylifecycle.c:1973
#7  0x000055555581d80d in Py_FinalizeEx () at Python/pylifecycle.c:1871
#8  0x0000555555851c14 in Py_RunMain () at Modules/main.c:709
#9  pymain_main (args=0x7fffffffda30) at Modules/main.c:737
#10 Py_BytesMain (argc=<optimized out>, argv=<optimized out>) at Modules/main.c:761
#11 0x00007ffff7c68083 in __libc_start_main (main=0x5555555d3850 <main>, argc=2, argv=0x7fffffffdb98, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffffffdb88)
#12 0x00005555555e6a2e in _start () at Python/pylifecycle.c:3024

Seems like a gc issue with interpreter level state? I did not look deep into it as I thought @rhettinger would realize what happened much faster. However, if no one is interested, I'll try to investigate.

[Eclips4]

There's a similar issue related to teedataobject deallocation: #115874.

However, cannot reproduce this on current main (macOS Sonoma). I'll try this later on my WSL+Ubuntu and Windows setups.

[erlend-aasland]

@gaogaotiantian, can you reproduce from builds off of the current main and 3.12 branches? I suspect this was fixed in main by #116204 and in 3.12 by #116955.

Note: there has been no 3.12 nor 3.13 release after those fixes were merged.

[gaogaotiantian]

@erlend-aasland you are right. My main is a bit out of date. I can't repro it once I pull the latest changes.

[erlend-aasland]

Great, thanks for verifying, @gaogaotiantian! Closing as already-fixed.

Thanks for the report, though!

[erlend-aasland]

Or closing as duplicate, if you wish :)