mysql service not coming up for existing data directory mount

[vinaybandakka]

I have data directory with couple of schema and about 25 GB of data. Container starts with volume /var/lib/mysql. When exec to container to see services still mysqld not started by entrypoint script. As you see chown command is still running. Even after 3 hours still chown not finished. It it due to 25GB data mount ? I am using mysql:5.7.20

UID PID PPID C STIME TTY TIME CMD
root 1 0 0 07:29 ? 00:00:00 /bin/bash /usr/local/bin/docker-entrypoint.sh mysqld
root 18 1 0 07:29 ? 00:00:00 chown -R mysql:mysql /var/lib/mysql/
root 19 0 0 07:31 ? 00:00:00 bash

[yosifkit]

That is possible it could take that long on a large database. I don't have a database of that size to test against. Any more information about how/where you are running this that would cause limitations on hard drive access? (docker version, docker info, docker run line or docker-compose.yaml, host information).

Since #161 you can also just run a mysql container as the owner of the files (so that it can't do the chown):

$ docker run -d --user 1000:1000 -e MYSQL_ROOT_PASSWORD=12345 mysql

[vinaybandakka]

Thanks yosifkit

I was using the kubernetes to create the mysql container. It look good now after adding below code

      securityContext:
        runAsUser: 999
        fsGroup: 999

UID for mysql was 999 for me. Thanks again

[yosifkit]

Glad that it is working now!

[user1m]

@vinaybandakka @yosifkit when you started the mysql service what was the password you entered to authenticate successfully? I keep getting su: Authentication failure when running container with --user 999:999 and service mysql start cmd.

Thanks in advance

[yosifkit]

@user1m, service mysql start shouldn't be used; there is no init system running in the container. Just whatever entrypoint cmd is. So using the default mysql image and the following is run:

$ docker run -d --user 1000:1000 -e MYSQL_ROOT_PASSWORD=12345 mysql

Then during the first few seconds a shell script will be running (docker-entrypoint.sh) which will call a few other binaries including running mysqld as a background process that is only available via a socket file. After the script finishes creating the user and running any scripts in /docker-entrypoint-initdb.d/, it then stops mysqld and execs to mysqld listening on the network. In the end, this leaves mysqld running in the container and nothing else.

[user1m]

@yosifkit thanks for the tip. But I'm still running into issues where my container doesn't stay up. Here's the container docker log.

2019-01-22T04:21:04.352259Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
2019-01-22T04:21:11.709133Z 0 [Note] mysqld (mysqld 5.7.21) starting as process 1 ...
2019-01-22T04:21:14.315215Z 0 [Warning] Setting lower_case_table_names=2 because file system for /var/lib/mysql/ is case insensitive
2019-01-22T04:21:14.455799Z 0 [Note] InnoDB: PUNCH HOLE support available
2019-01-22T04:21:14.455896Z 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2019-01-22T04:21:14.455922Z 0 [Note] InnoDB: Uses event mutexes
2019-01-22T04:21:14.455937Z 0 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
2019-01-22T04:21:14.455957Z 0 [Note] InnoDB: Compressed tables use zlib 1.2.3
2019-01-22T04:21:14.455972Z 0 [Note] InnoDB: Using Linux native AIO
2019-01-22T04:21:14.456442Z 0 [Note] InnoDB: Number of pools: 1
2019-01-22T04:21:14.456615Z 0 [Note] InnoDB: Using CPU crc32 instructions
2019-01-22T04:21:14.461848Z 0 [Note] InnoDB: Initializing buffer pool, total size = 128M, instances = 1, chunk size = 128M
2019-01-22T04:21:14.471289Z 0 [Note] InnoDB: Completed initialization of buffer pool
2019-01-22T04:21:14.480363Z 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
2019-01-22T04:21:15.546469Z 0 [ERROR] InnoDB: Cannot read first page of './ibdata1' I/O error
2019-01-22T04:21:15.546569Z 0 [ERROR] InnoDB: Plugin initialization aborted with error I/O error
2019-01-22T04:21:16.155598Z 0 [ERROR] Plugin 'InnoDB' init function returned error.
2019-01-22T04:21:16.155644Z 0 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
2019-01-22T04:21:16.155668Z 0 [ERROR] Failed to initialize builtin plugins.
2019-01-22T04:21:16.155834Z 0 [ERROR] Aborting

2019-01-22T04:21:16.155955Z 0 [Note] Binlog end
2019-01-22T04:21:16.156196Z 0 [Note] Shutting down plugin 'CSV'
2019-01-22T04:21:16.159850Z 0 [Note] mysqld: Shutdown complete

Do you think the ./ibdata1 file is corrupt? I assumed the I/O Error was due to permissions
I'm running the command you have above except mounting a volume via -v $MOUNT_VOLUME_PATH:/var/lib/mysql

[tianon]

Either corrupt or your drive is going bad (which is usually when I start to see IO errors like that).