test_os.TimerfdTests is flaky

[pepoluan]

Bug report

Bug description:

When compiling dev-lang/python-3.13.0 on Gentoo, the following tests failed:

======================================================================
FAIL: test_timerfd_interval (test.test_os.TimerfdTests.test_timerfd_interval)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/var/tmp/portage/dev-lang/python-3.13.0/work/Python-3.13.0/Lib/test/test_os.py", line 4238, in test_timerfd_interval
    self.assertEqual(self.read_count_signaled(fd), 1)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 4 != 1

======================================================================
FAIL: test_timerfd_ns_select (test.test_os.TimerfdTests.test_timerfd_ns_select)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/var/tmp/portage/dev-lang/python-3.13.0/work/Python-3.13.0/Lib/test/test_os.py", line 4457, in test_timerfd_ns_select
    self.assertEqual(self.read_count_signaled(fd), 1)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 2 != 1

======================================================================
FAIL: test_timerfd_poll (test.test_os.TimerfdTests.test_timerfd_poll)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/var/tmp/portage/dev-lang/python-3.13.0/work/Python-3.13.0/Lib/test/test_os.py", line 4349, in test_timerfd_poll
    self.check_timerfd_poll(False)
    ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^
  File "/var/tmp/portage/dev-lang/python-3.13.0/work/Python-3.13.0/Lib/test/test_os.py", line 4337, in check_timerfd_poll
    self.assertEqual(self.read_count_signaled(fd), 1)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 2 != 1

======================================================================
FAIL: test_timerfd_select (test.test_os.TimerfdTests.test_timerfd_select)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/var/tmp/portage/dev-lang/python-3.13.0/work/Python-3.13.0/Lib/test/test_os.py", line 4292, in test_timerfd_select
    self.assertEqual(self.read_count_signaled(fd), 1)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 5 != 1

----------------------------------------------------------------------
Ran 366 tests in 32.208s

FAILED (failures=4, skipped=62)
test test_os failed

USE flags used are:

USE="ensurepip gdbm ncurses pgo readline sqlite ssl -bluetooth -build -debug -examples (-jit) -libedit* -test -tk -valgrind -verify-sig"

which resulted in the following ./configure invocation:

./configure --prefix=/usr --build=x86_64-pc-linux-gnu --host=x86_64-pc-linux-gnu --mandir=/usr/share/man --infodir=/usr/share/info --datadir=/usr/share --sysconfdir=/etc --localstatedir=/var/lib --datarootdir=/usr/share --docdir=/usr/share/doc/python-3.13.0 --htmldir=/usr/share/doc/python-3.13.0/html --libdir=/usr/lib64 ac_cv_header_stropts_h=no --enable-shared --without-static-libpython --enable-ipv6 --infodir=${prefix}/share/info --mandir=${prefix}/share/man --with-computed-gotos --with-dbmliborder=gdbm --with-libc= --enable-loadable-sqlite-extensions --without-ensurepip --without-lto --with-system-expat --with-system-libmpdec --with-platlibdir=lib --with-pkg-config=yes --with-wheel-pkg-dir=/usr/lib/python/ensurepip --enable-gil --without-assertions --disable-experimental-jit --enable-optimizations --with-readline=readline --without-valgrind --with-lto

CPython versions tested on:

3.13.0, 3.13.1

Operating systems tested on:

Linux (Gentoo)

[pepoluan]

I think this bug is similar to #124873 , though that one was on Android, while this one is on a (heavily-customized) Gentoo Linux.

[pepoluan]

Update:

Upgrading to 3.13.1 still resulted in a similar error:

======================================================================
FAIL: test_timerfd_TFD_TIMER_ABSTIME (test.test_os.TimerfdTests.test_timerfd_TFD_TIMER_ABSTIME)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/var/tmp/portage/dev-lang/python-3.13.1/work/Python-3.13.1/Lib/test/test_os.py", line 4266, in test_timerfd_TFD_TIMER_ABSTIME
    self.assertAlmostEqual(next_expiration, offset, places=self.CLOCK_RES_PLACES)
    ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 0.99945497 != 1 within 3 places (0.0005450299999999464 difference)

======================================================================
FAIL: test_timerfd_ns_poll (test.test_os.TimerfdTests.test_timerfd_ns_poll)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/var/tmp/portage/dev-lang/python-3.13.1/work/Python-3.13.1/Lib/test/test_os.py", line 4353, in test_timerfd_ns_poll
    self.check_timerfd_poll(True)
    ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/var/tmp/portage/dev-lang/python-3.13.1/work/Python-3.13.1/Lib/test/test_os.py", line 4338, in check_timerfd_poll
    self.assertEqual(self.read_count_signaled(fd), 1)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 4 != 1

----------------------------------------------------------------------
Ran 366 tests in 34.558s

FAILED (failures=2, skipped=62)
test test_os failed

[mhsmith]

Interesting – I couldn't see how the test could fail this way unless the process had been suspended for 1/8 of a second. This seemed unlikely even on an Android emulator, but possible given Android's customized process scheduler. But if it's failing even on Linux, there's probably some other cause.

[rossburton]

We've seen this in the Yocto Project when running the python tests inside a qemu on a heavily loaded system: https://bugzilla.yoctoproject.org/show_bug.cgi?id=15743.

[mhsmith]

The Yocto failure isn't in the same test, it's in a different timerfd test which is much more timing-sensitive. See discussion in #130508.

[mhsmith]

I notice that most of the timerfd tests create an FD and never close it. This shouldn't directly lead to the failure above, because every test uses a separate FD, but it would still be worth cleaning up.

[picnixz]

FTR, it also failed on AMD Debian: #129797 (comment)

[colesbury]

I saw this locally on Ubuntu 22.04 x86-64:

======================================================================
FAIL: test_timerfd_TFD_TIMER_ABSTIME (test.test_os.TimerfdTests.test_timerfd_TFD_TIMER_ABSTIME)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/raid/sgross/cpython/Lib/test/test_os.py", line 4419, in test_timerfd_TFD_TIMER_ABSTIME
    self.assertAlmostEqual(next_expiration, offset, places=self.CLOCK_RES_PLACES)
    ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 0.999380538 != 1 within 3 places (0.0006194619999999595 difference)

----------------------------------------------------------------------

[furkanonder]

@colesbury It also fails on NetBSD, see #131266

[vstinner]

test_time fails randomly on s390x Fedora Stable LTO + PGO 3.x.

Example: https://buildbot.python.org/#/builders/1627/builds/314

FAIL: test_timerfd_ns_poll (test.test_os.TimerfdTests.test_timerfd_ns_poll)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-s390x.lto-pgo/build/Lib/test/test_os.py", line 4512, in test_timerfd_ns_poll
    self.check_timerfd_poll(True)
    ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-s390x.lto-pgo/build/Lib/test/test_os.py", line 4497, in check_timerfd_poll
    self.assertEqual(self.read_count_signaled(fd), 1)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 2 != 1