Latest update caused data corruption

[numericalatina]

Most of my databases use mariadb:latest image

Latest update of this image (17h ago according to DockerHub) caused data corruption in two of my databases, error is:

[ERROR] InnoDB: Tried to read 1048576 bytes at offset 2097152, but was only able to read 0
[ERROR] InnoDB: Operating system error number 5 in a file operation.
[ERROR] InnoDB: Error number 5 means 'Input/output error'
[ERROR] InnoDB: File (unknown): 'read' returned OS error 205. Cannot continue operation

I thos host OS I cannot move ibdata1 either so I suppose this is the corrupted file.
Have I lost my databases?

My other question is: I would like to stick to a fixed version of mariadb, since I cannot afford to have other people's errors to affect my deployment like this. What is the version number just before this? The one that was latest just a few hours ago was working great.

[tianon]

"Input/output error" is usually caused by something more serious like your hardware going bad -- I'd recommend doing a SMART check on your system's drives ASAP.

[numericalatina]

Allow me to add another strange behaviour i experienced today with another of my databases using today's mariadb:latest

The database, user and passwords were not set according to the environment variables, nor the sql script at /docker-entrypoint-initdb.d was executed
Logging into the container I was surprised to find that I could connect as root without root password and that there was an empty database called 'test' weird...
Since I'm suspecting of the latest build, I downgraded to mariadb:10.3.22 where everything worked as it should.
Are you sure this there is nothing wrong with the latest build?

[numericalatina]

Ok so I guess the latter is that I'm hitting #262, the DB initalization didn't happen

Note that I'm using Ansible too in order to deploy a Docker Swarm so it might be related to #298, althought i'm not sure exactly how

[tianon]

Yeah, I'm unable to reproduce a problem -- this is on an NVMe drive, and it took ~20 seconds for the database to fully initialize, but on a spinning disk it'll probably take quite a bit longer.

$ docker pull mariadb:10.4
10.4: Pulling from library/mariadb
Digest: sha256:f72c21e993d4b04cf0972cde00a878f5d1c246887dcf0c0e53de00fa85007e3b
Status: Image is up to date for mariadb:10.4
docker.io/library/mariadb:10.4

$ docker run --rm mariadb:10.4 mysqld --version
mysqld  Ver 10.4.12-MariaDB-1:10.4.12+maria~bionic for debian-linux-gnu on x86_64 (mariadb.org binary distribution)

$ docker run -d --name test -e MYSQL_ROOT_PASSWORD=example mariadb:10.4
00310149f32a99ca56f5dc2d663a4ab693eb3e423a0cebe0d26a7641ade59ccf

$ sleep 20

$ docker logs --tail=2 test
2020-04-24 17:48:50 0 [Note] mysqld: ready for connections.
Version: '10.4.12-MariaDB-1:10.4.12+maria~bionic'  socket: '/var/run/mysqld/mysqld.sock'  port: 3306  mariadb.org binary distribution

$ docker exec -it test mysql -uroot
ERROR 1045 (28000): Access denied for user 'root'@'localhost' (using password: NO)

$ docker exec -it test mysql -uroot -pexample
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 10
Server version: 10.4.12-MariaDB-1:10.4.12+maria~bionic mariadb.org binary distribution

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MariaDB [(none)]> SHOW DATABASES;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| mysql              |
| performance_schema |
+--------------------+
3 rows in set (0.000 sec)

MariaDB [(none)]> 

[numericalatina]

All right I guess we can close the issue.
The original question was about database corruption, which for me must have happened because a docker-compose deploy crashed the databases, which couldn't recover well afterwards. This happened to 2 of more than 10 databases. (I'm trying to check my hard drive anyway, thanks for that suggestion)

Anywayt, I think the biggest issue that should be addressed is #262
IMHO opinion you should be taking in account spinning hard drives.
SSD is a privilege, at least here in latin america assuming everyone to have an SSD is like expecting everyone to have the latest iphone

[tianon]

We simply package MariaDB from upstream -- unfortunately, we cannot influence what they decide to prioritize. 😞