Segmentation Fault in `zend_class_init_statics` when using `opcache.preload`

[yyny]

Description

We started receiving "Segmentation Fault" errors on startup after upgrading our Docker image to PHP 8.3.3 using the latest release of php:fpm-alpine3.18, with some additional dependencies:

RUN docker-php-ext-configure gd --with-freetype --with-jpeg && \
    docker-php-ext-configure zip && \
    docker-php-ext-configure intl
RUN pecl install redis && \
    docker-php-ext-install -j$(getconf _NPROCESSORS_ONLN) pdo pdo_mysql pcntl zip ffi gd intl
RUN docker-php-ext-enable redis opcache ffi pcntl

The bug also occurs with -d 'opcache.optimization_level=0' -d 'opcache.jit=0000', but not when preloading or opcache is disabled completely.

I tried to create an MCVE, however that will still take a while since at a certain point removing a single file will fix the bug or give different errors (Some of which might be caused by memory corruption). The bug seems to be related to the order in which classes are preloaded.

Here is the backtrace:

(gdb) r
Starting program: /usr/local/bin/php artisan
[Detaching after fork from child process 590]

Program received signal SIGSEGV, Segmentation fault.
0x0000555555cc9861 in zend_class_init_statics (class_type=0x55554de44df0) at /usr/src/php/Zend/zend_object_handlers.c:1573
1573					ZVAL_COPY_OR_DUP(&CE_STATIC_MEMBERS(class_type)[i], p);
(gdb) bt
#0  0x0000555555cc9861 in zend_class_init_statics (class_type=0x55554de44df0) at /usr/src/php/Zend/zend_object_handlers.c:1573
#1  0x0000555555cc95fc in zend_class_init_statics (class_type=0x55554e955fe0) at /usr/src/php/Zend/zend_object_handlers.c:1562
#2  0x0000555555cc9acb in zend_std_get_static_property_with_info (ce=0x55554e955fe0, property_name=0x55554d91d180, type=3, property_info_ptr=0x7fffffffb780) at /usr/src/php/Zend/zend_object_handlers.c:1623
#3  0x0000555555c03283 in zend_fetch_static_property_address_ex (retval=0x7fffffffb840, prop_info=0x7fffffffb800, cache_slot=0, fetch_type=3) at /usr/src/php/Zend/zend_execute.c:3399
#4  0x0000555555c0358e in zend_fetch_static_property_address (retval=0x7fffffffb840, prop_info=0x0, cache_slot=0, fetch_type=3, flags=0) at /usr/src/php/Zend/zend_execute.c:3452
#5  0x0000555555c09ede in zend_fetch_static_prop_helper_SPEC (type=3) at /usr/src/php/Zend/zend_vm_execute.h:849
#6  0x0000555555c0a0c1 in ZEND_FETCH_STATIC_PROP_IS_SPEC_HANDLER () at /usr/src/php/Zend/zend_vm_execute.h:898
#7  0x0000555555c8b0a0 in execute_ex (ex=0x7ffff7016390) at /usr/src/php/Zend/zend_vm_execute.h:57045
#8  0x0000555555baa261 in zend_call_function (fci=0x7fffffffbd80, fci_cache=0x7fffffffbee0) at /usr/src/php/Zend/zend_execute_API.c:957
#9  0x000055555599820b in php_array_walk (context=0x7fffffffbea0, array=0x7ffff5cef108, userdata=0x0, recursive=false) at /usr/src/php/ext/standard/array.c:1489
#10 0x0000555555998971 in zif_array_walk (execute_data=0x7ffff7016320, return_value=0x7fffffffbf40) at /usr/src/php/ext/standard/array.c:1541
#11 0x0000555555c0c0a7 in ZEND_DO_FCALL_BY_NAME_SPEC_RETVAL_UNUSED_HANDLER () at /usr/src/php/Zend/zend_vm_execute.h:1567
#12 0x0000555555c8b7e5 in execute_ex (ex=0x7ffff7016020) at /usr/src/php/Zend/zend_vm_execute.h:57236
#13 0x0000555555c90103 in zend_execute (op_array=0x7ffff707f000, return_value=0x0) at /usr/src/php/Zend/zend_vm_execute.h:61604
#14 0x0000555555bc62f0 in zend_execute_scripts (type=8, retval=0x0, file_count=3) at /usr/src/php/Zend/zend.c:1881
#15 0x0000555555b0b3be in php_execute_script (primary_file=0x7fffffffd730) at /usr/src/php/main/main.c:2507
#16 0x0000555555d536f7 in do_cli (argc=2, argv=0x7ffff7864bf0) at /usr/src/php/sapi/cli/php_cli.c:966
#17 0x0000555555d54538 in main (argc=2, argv=0x7ffff7864bf0) at /usr/src/php/sapi/cli/php_cli.c:1340
(gdb) zbacktrace
[0x7ffff7016a70] Illuminate\Database\Eloquent\Model->bootIfNotBooted() /var/www/html/vendor/laravel/framework/src/Illuminate/Database/Eloquent/Model.php:251 
[0x7ffff7016a10] Illuminate\Database\Eloquent\Model->__construct(array(0)[0x7ffff7016a60]) /var/www/html/vendor/laravel/framework/src/Illuminate/Database/Eloquent/Model.php:235 
[0x7ffff70169a0] App\Models\Account->__construct() /var/www/html/App/Models/Account.php:59 
[0x7ffff7016900] Illuminate\Database\Eloquent\Model->observe(array(1)[0x7ffff7016950]) /var/www/html/vendor/laravel/framework/src/Illuminate/Database/Eloquent/Concerns/HasEvents.php:40 
[0x7ffff7016810] Illuminate\Foundation\Support\Providers\EventServiceProvider->Illuminate\Foundation\Support\Providers\{closure}() /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Support/Providers/EventServiceProvider.php:53 
[0x7ffff7016780] Illuminate\Container\BoundMethod->Illuminate\Container\{closure}() /var/www/html/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php:36 
[0x7ffff7016700] Illuminate\Container\Util->unwrapIfClosure(object[0x7ffff7016750]) /var/www/html/vendor/laravel/framework/src/Illuminate/Container/Util.php:41 
[0x7ffff7016660] Illuminate\Container\BoundMethod->callBoundMethod(object[0x7ffff70166b0], object[0x7ffff70166c0], object[0x7ffff70166d0]) /var/www/html/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php:81 
[0x7ffff70165c0] Illuminate\Container\BoundMethod->call(object[0x7ffff7016610], object[0x7ffff7016620], array(0)[0x7ffff7016630], NULL) /var/www/html/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php:35 
[0x7ffff7016500] Illuminate\Container\Container->call(object[0x7ffff7016550]) /var/www/html/vendor/laravel/framework/src/Illuminate/Container/Container.php:662 
[0x7ffff7016480] Illuminate\Support\ServiceProvider->callBootingCallbacks() /var/www/html/vendor/laravel/framework/src/Illuminate/Support/ServiceProvider.php:103 
[0x7ffff7016410] Illuminate\Foundation\Application->bootProvider(object[0x7ffff7016460]) /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Application.php:1027 
[0x7ffff7016390] Illuminate\Foundation\Application->Illuminate\Foundation\{closure}(object[0x7ffff70163e0], object[0x7ffff70163f0]) /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Application.php:1011 
[0x7ffff7016320] array_walk(reference, object[0x7ffff7016380]) [internal function]
[0x7ffff70162c0] Illuminate\Foundation\Application->boot() /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Application.php:1010 
[0x7ffff7016260] Illuminate\Foundation\Bootstrap\BootProviders->bootstrap(object[0x7ffff70162b0]) /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Bootstrap/BootProviders.php:17 
[0x7ffff70161c0] Illuminate\Foundation\Application->bootstrapWith(array(7)[0x7ffff7016210]) /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Application.php:263 
[0x7ffff7016160] Illuminate\Foundation\Console\Kernel->bootstrap() /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Console/Kernel.php:447 
[0x7ffff70160d0] Illuminate\Foundation\Console\Kernel->handle(object[0x7ffff7016120], object[0x7ffff7016130]) /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Console/Kernel.php:199 
[0x7ffff7016020] (main) /var/www/html/artisan:37 
(gdb) disp/i $pc
1: x/i $pc
=> 0x555555cc9861 <zend_class_init_statics+743>:	mov    0x4(%rax),%eax
(gdb) x/1g $rax
0x7ffff4e69e40:	Cannot access memory at address 0x7ffff4e69e40
(gdb) printzv p
[0x55554de4f850] (refcount=Cannot access memory at address 0x7ffff4e69e40

0x7ffff4e69e40 is a zend-allocated (refcounted) zend_array* zval allocated in the "preload subprocess" (as forked in accel_finish_startup_preload_subprocess):

(gdb) info proc map
	  ...
      0x7fffee000000     0x7ffff4400000  0x6400000        0x0  rw-p   [anon:zend_alloc]
      ...

This zval is last written in /usr/src/php/ext/opcache/zend_persist.c:903:

ce->default_static_members_table = zend_shared_memdup_free(ce->default_static_members_table, sizeof(zval) * ce->default_static_members_count);

The ce->parent->default_static_members_count is greater than the index of the static member that causes the crash, which might be relevant, as it causes the zval to not be "persisted" in the code following the write:

(gdb) print i
$1 = 13
(gdb) print class_type->parent->default_static_members_count
$2 = 26

NOTES

The structure of the class that causes the segfault looks something like this (with each trait/class in a separate file):

<?php
trait TraitA { // = trait Illuminate\Database\Eloquent\Concerns\HasAttributes
    protected static $non_empty_array = ['non-empty'];
}
trait TraitB {
    use TraitA;
}
abstract class ParentClass { // = class Illuminate\Database\Eloquent\Model
	use TraitA;
}
class TheClass extends ParentClass {
	use TraitB;
}

While attempting to make a MCVE, I additionally triggered the following errors/asserts, which might be related:

/usr/src/php/Zend/zend_hash.c(2441) : ht=0x7fbfc4849e40 is being destroyed
Assertion failed: 0 (/usr/src/php/Zend/zend_hash.c: _zend_is_inconsistent: 74)
Aborted

/usr/src/php/Zend/zend_hash.c(2441) : ht=0x7ffff5dd5ae0 is already destroyed
Assertion failed: 0 (/usr/src/php/Zend/zend_hash.c: _zend_is_inconsistent: 74)

Breakpoint 1, 0x00007ffff7f7ff9a in abort () from /lib/ld-musl-x86_64.so.1
(gdb) bt
#0  0x00007ffff7f7ff9a in abort () from /lib/ld-musl-x86_64.so.1
#1  0x00007ffff7f80057 in __assert_fail () from /lib/ld-musl-x86_64.so.1
#2  0x0000555556786d55 in ?? ()
#3  0x0000555555bdc365 in _zend_is_inconsistent (ht=0x7ffff5dd5ae0, file=0x555556786d37 "/usr/src/php/Zend/zend_hash.c", line=2441) at /usr/src/php/Zend/zend_hash.c:74
#4  0x0000555555be42f2 in zend_array_dup (source=0x7ffff5dd5ae0) at /usr/src/php/Zend/zend_hash.c:2441
#5  0x0000555555bc1225 in zval_copy_ctor_func (zvalue=0x7ffff70ddb50) at /usr/src/php/Zend/zend_variables.c:127
#6  0x0000555555cc98b2 in zend_class_init_statics (class_type=0x55554de44650) at /usr/src/php/Zend/zend_object_handlers.c:1573
#7  0x0000555555cc95fc in zend_class_init_statics (class_type=0x55554e90fb10) at /usr/src/php/Zend/zend_object_handlers.c:1562
#8  0x0000555555cc9acb in zend_std_get_static_property_with_info (ce=0x55554e90fb10, property_name=0x55554d919f90, type=3, property_info_ptr=0x7fffffffb780) at /usr/src/php/Zend/zend_object_handlers.c:1623
#9  0x0000555555c03283 in zend_fetch_static_property_address_ex (retval=0x7fffffffb840, prop_info=0x7fffffffb800, cache_slot=0, fetch_type=3) at /usr/src/php/Zend/zend_execute.c:3399
#10 0x0000555555c0358e in zend_fetch_static_property_address (retval=0x7fffffffb840, prop_info=0x0, cache_slot=0, fetch_type=3, flags=0) at /usr/src/php/Zend/zend_execute.c:3452
#11 0x0000555555c09ede in zend_fetch_static_prop_helper_SPEC (type=3) at /usr/src/php/Zend/zend_vm_execute.h:849
#12 0x0000555555c0a0c1 in ZEND_FETCH_STATIC_PROP_IS_SPEC_HANDLER () at /usr/src/php/Zend/zend_vm_execute.h:898
#13 0x0000555555c8b0a0 in execute_ex (ex=0x7ffff7016390) at /usr/src/php/Zend/zend_vm_execute.h:57045
#14 0x0000555555baa261 in zend_call_function (fci=0x7fffffffbd80, fci_cache=0x7fffffffbee0) at /usr/src/php/Zend/zend_execute_API.c:957
#15 0x000055555599820b in php_array_walk (context=0x7fffffffbea0, array=0x7ffff5ba4808, userdata=0x0, recursive=false) at /usr/src/php/ext/standard/array.c:1489
#16 0x0000555555998971 in zif_array_walk (execute_data=0x7ffff7016320, return_value=0x7fffffffbf40) at /usr/src/php/ext/standard/array.c:1541
#17 0x0000555555c0c0a7 in ZEND_DO_FCALL_BY_NAME_SPEC_RETVAL_UNUSED_HANDLER () at /usr/src/php/Zend/zend_vm_execute.h:1567
#18 0x0000555555c8b7e5 in execute_ex (ex=0x7ffff7016020) at /usr/src/php/Zend/zend_vm_execute.h:57236
#19 0x0000555555c90103 in zend_execute (op_array=0x7ffff707f000, return_value=0x0) at /usr/src/php/Zend/zend_vm_execute.h:61604
#20 0x0000555555bc62f0 in zend_execute_scripts (type=8, retval=0x0, file_count=3) at /usr/src/php/Zend/zend.c:1881
#21 0x0000555555b0b3be in php_execute_script (primary_file=0x7fffffffd730) at /usr/src/php/main/main.c:2507
#22 0x0000555555d536f7 in do_cli (argc=2, argv=0x7ffff7864bf0) at /usr/src/php/sapi/cli/php_cli.c:966
#23 0x0000555555d54538 in main (argc=2, argv=0x7ffff7864bf0) at /usr/src/php/sapi/cli/php_cli.c:1340

Assertion failed: p->refcount > 0 (/usr/src/php/Zend/zend_types.h: zend_gc_delref: 1342)

Breakpoint 1, 0x00007ffff7f7ff9a in abort () from /lib/ld-musl-x86_64.so.1
(gdb) bt
#0  0x00007ffff7f7ff9a in abort () from /lib/ld-musl-x86_64.so.1
#1  0x00007ffff7f80057 in __assert_fail () from /lib/ld-musl-x86_64.so.1
#2  0x0000555556781df7 in ?? ()
#3  0x0000555555badc1d in zend_gc_delref (p=0x7ffff5a49e40) at /usr/src/php/Zend/zend_types.h:1342
#4  0x0000555555badfa0 in i_zval_ptr_dtor (zval_ptr=0x7ffff70dd7d0) at /usr/src/php/Zend/zend_variables.h:43
#5  0x0000555555bae7b2 in zend_cleanup_internal_class_data (ce=0x55554dd81250) at /usr/src/php/Zend/zend_opcode.c:189
#6  0x0000555555ba7cc7 in zend_shutdown_executor_values (fast_shutdown=false) at /usr/src/php/Zend/zend_execute_API.c:324
#7  0x0000555555ba832b in shutdown_executor () at /usr/src/php/Zend/zend_execute_API.c:417
#8  0x0000555555bc42f5 in zend_deactivate () at /usr/src/php/Zend/zend.c:1290
#9  0x0000555555b0a418 in php_request_shutdown (dummy=0x0) at /usr/src/php/main/main.c:1889
#10 0x0000555555d53feb in do_cli (argc=2, argv=0x7ffff7864bf0) at /usr/src/php/sapi/cli/php_cli.c:1136
#11 0x0000555555d54538 in main (argc=2, argv=0x7ffff7864bf0) at /usr/src/php/sapi/cli/php_cli.c:134

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff7fb8081 in memcpy () from /lib/ld-musl-x86_64.so.1
(gdb) bt
#0  0x00007ffff7fb8081 in memcpy () from /lib/ld-musl-x86_64.so.1
#1  0x0000555555be46a2 in zend_array_dup (source=0x7ffff5c32600) at /usr/src/php/Zend/zend_hash.c:2472
#2  0x0000555555bc1225 in zval_copy_ctor_func (zvalue=0x7ffff70dded0) at /usr/src/php/Zend/zend_variables.c:127
#3  0x0000555555cc98b2 in zend_class_init_statics (class_type=0x55554de44c90) at /usr/src/php/Zend/zend_object_handlers.c:1573
#4  0x0000555555cc95fc in zend_class_init_statics (class_type=0x55554e987508) at /usr/src/php/Zend/zend_object_handlers.c:1562
#5  0x0000555555cc9acb in zend_std_get_static_property_with_info (ce=0x55554e987508, property_name=0x55554d91c9e8, type=3, property_info_ptr=0x7fffffffb780) at /usr/src/php/Zend/zend_object_handlers.c:1623
#6  0x0000555555c03283 in zend_fetch_static_property_address_ex (retval=0x7fffffffb840, prop_info=0x7fffffffb800, cache_slot=0, fetch_type=3) at /usr/src/php/Zend/zend_execute.c:3399
#7  0x0000555555c0358e in zend_fetch_static_property_address (retval=0x7fffffffb840, prop_info=0x0, cache_slot=0, fetch_type=3, flags=0) at /usr/src/php/Zend/zend_execute.c:3452
#8  0x0000555555c09ede in zend_fetch_static_prop_helper_SPEC (type=3) at /usr/src/php/Zend/zend_vm_execute.h:849
#9  0x0000555555c0a0c1 in ZEND_FETCH_STATIC_PROP_IS_SPEC_HANDLER () at /usr/src/php/Zend/zend_vm_execute.h:898
#10 0x0000555555c8b0a0 in execute_ex (ex=0x7ffff7016390) at /usr/src/php/Zend/zend_vm_execute.h:57045
#11 0x0000555555baa261 in zend_call_function (fci=0x7fffffffbd80, fci_cache=0x7fffffffbee0) at /usr/src/php/Zend/zend_execute_API.c:957
#12 0x000055555599820b in php_array_walk (context=0x7fffffffbea0, array=0x7ffff5b456c8, userdata=0x0, recursive=false) at /usr/src/php/ext/standard/array.c:1489
#13 0x0000555555998971 in zif_array_walk (execute_data=0x7ffff7016320, return_value=0x7fffffffbf40) at /usr/src/php/ext/standard/array.c:1541
#14 0x0000555555c0c0a7 in ZEND_DO_FCALL_BY_NAME_SPEC_RETVAL_UNUSED_HANDLER () at /usr/src/php/Zend/zend_vm_execute.h:1567
#15 0x0000555555c8b7e5 in execute_ex (ex=0x7ffff7016020) at /usr/src/php/Zend/zend_vm_execute.h:57236
#16 0x0000555555c90103 in zend_execute (op_array=0x7ffff707f000, return_value=0x0) at /usr/src/php/Zend/zend_vm_execute.h:61604
#17 0x0000555555bc62f0 in zend_execute_scripts (type=8, retval=0x0, file_count=3) at /usr/src/php/Zend/zend.c:1881
#18 0x0000555555b0b3be in php_execute_script (primary_file=0x7fffffffd730) at /usr/src/php/main/main.c:2507
#19 0x0000555555d536f7 in do_cli (argc=2, argv=0x7ffff7864bf0) at /usr/src/php/sapi/cli/php_cli.c:966
#20 0x0000555555d54538 in main (argc=2, argv=0x7ffff7864bf0) at /usr/src/php/sapi/cli/php_cli.c:1340

   Symfony\Component\ErrorHandler\Error\FatalError 

  Allowed memory size of 268435456 bytes exhausted at /usr/src/php/Zend/zend_hash.c:2470 (tried to allocate 59226964140 bytes)

  at vendor/laravel/framework/src/Illuminate/Database/Eloquent/Model.php:251
    247▕      * @return void
    248▕      */
    249▕     protected function bootIfNotBooted()
    250▕     {
  ➜ 251▕         if (! isset(static::$booted[static::class])) {
    252▕             static::$booted[static::class] = true;
    253▕ 
    254▕             $this->fireModelEvent('booting', false);

PHP Version

PHP 8.3.3

Operating System

Alpine 3.18

[nielsdos]

When debugging and trying to make a reproducing sample for opcache crashes, I always recommend setting opcache.protect_memory=1 in your ini. This has helped me getting crashes more reliably instead of randomly. Once you got it to the point it crashes consistently we can handle it probably, even if it's not yet a minimal example.
Also enabling the address sanitizer during configure will help too.

[nielsdos]

I was able to use the information to create a minimal reproducer, I'll investigate a bit now.

[nielsdos]

Traits seem to be mishandled during persisting, causing their static properties to be skipped in some cases. The consequence is that they keep referring to the old pointer, which is being freed by zend_shared_memdup_put_free because the constant array is not reference counted. To solve this, we should be checking whether the type is indirect I think.
I have a WIP patch that I'll be trying.
EDIT: Bisected to 9a250cc, which makes sense given what I know now. I'll be preparing a PR.

[yyny]

I can confirm that nielsdos:fix-preload-trait-crash fixes the issue in our branch. Good work!