Magento 2.3.1 Cron Deadlocks for cron_schedule #22438

seansan · 2019-04-20T17:49:28Z

Preconditions (*)

Magento 2.3.1
MSI

Steps to reproduce (*)

Vanilla install
Run CRON every 1 minutes 3 jobs main, setup and upgrade

Expected result (*)

No CRON problems, especially no deadlocks for a simple table .... truncating the table only works 1time ... need a structural solution or some kind of retry method

Actual result (*)

See below

This really should not happen

https://magento.stackexchange.com/questions/270856/magento-2-v2-3-1-cron-causing-intermittent-deadlock-for-cron-schedule

https://magento.stackexchange.com/questions/238454/cron-job-error-sqlstate40001-serialization-failure-1213-deadlock-found-when?rq=1

https://magento.stackexchange.com/questions/203107/magento-2-cron-schedule-table-increasing-and-cpu-100

https://magento.stackexchange.com/questions/270813/magento-2-3-1-vanilla-cron-error-serialization-failure-1213-deadlock-found-whe

m2-assistant · 2019-04-20T17:49:30Z

Hi @seansan. Thank you for your report.
To help us process this issue please make sure that you provided the following information:

Summary of the issue
Information on your environment
Steps to reproduce
Expected and actual results

Please make sure that the issue is reproducible on the vanilla Magento instance following Steps to reproduce. To deploy vanilla Magento instance on our environment, please, add a comment to the issue:

@magento-engcom-team give me 2.3-develop instance - upcoming 2.3.x release

For more details, please, review the Magento Contributor Assistant documentation.

@seansan do you confirm that you was able to reproduce the issue on vanilla Magento instance following steps to reproduce?

yes
no

m2-assistant · 2019-04-22T07:38:00Z

Hi @engcom-backlog-nazar. Thank you for working on this issue.
In order to make sure that issue has enough information and ready for development, please read and check the following instruction: 👇

1. Verify that issue has all the required information. (Preconditions, Steps to reproduce, Expected result, Actual result).
Details
If the issue has a valid description, the label Issue: Format is valid will be added to the issue automatically. Please, edit issue description if needed, until label Issue: Format is valid appears.
2. Verify that issue has a meaningful description and provides enough information to reproduce the issue. If the report is valid, add Issue: Clear Description label to the issue by yourself.
3. Add Component: XXXXX label(s) to the ticket, indicating the components it may be related to.
4. Verify that the issue is reproducible on 2.3-develop branch
Details
- Add the comment @magento-engcom-team give me 2.3-develop instance to deploy test instance on Magento infrastructure.
- If the issue is reproducible on 2.3-develop branch, please, add the label Reproduced on 2.3.x.
- If the issue is not reproducible, add your comment that issue is not reproducible and close the issue and stop verification process here!

5. Verify that the issue is reproducible on 2.2-develop branch.
Details
- Add the comment @magento-engcom-team give me 2.2-develop instance to deploy test instance on Magento infrastructure.
- If the issue is reproducible on 2.2-develop branch, please add the label Reproduced on 2.2.x
6. Add label Issue: Confirmed once verification is complete.
7. Make sure that automatic system confirms that report has been added to the backlog.

ghost · 2019-04-22T07:44:10Z

HI @seansan thank you for you report, i'm closing this as duplicate to this -> #8933

seansan · 2019-04-22T08:24:40Z

that is a very very old issue almost 2 years, especially in the code world can something be done to solve this issue you think

…

On Mon, Apr 22, 2019 at 9:45 AM engcom-backlog-nazar < ***@***.***> wrote: Closed #22438 <#22438>. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#22438 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAE7I245SV6UXYMDPCQSJJ3PRVUIHANCNFSM4HHKD3PQ> .

seansan · 2019-04-24T17:36:46Z

Please re=open this issue. the other issue may be about a deadlock but the case is completely different it is about INDEXER and only accurs occasionaly

This is about a very specific CRON schedule

And can influence ALL users that use cron

onlinebizsoft · 2019-06-28T11:05:29Z

I have some sites in 2.2.6 and look like same issue :(

Ctucker9233 · 2019-06-28T17:04:24Z

Same issue here on 2.2.8.

lracicot · 2019-07-12T16:16:54Z

Same problem here with 2.3.1. We import data with an API and I suspect that the cron interferes with my hourly syncs.

driskell · 2019-08-06T21:43:27Z

Also experiencing issue on 2.3.1 with specific deadlocks occurring on the cron_schedule table

manish3m · 2019-08-10T22:59:46Z

Have the same issue. Thought 2.3 will be better than any 1.X version . Made the switch and spend hours making it ready with all prerequisites . The symptoms is if you enable crontab and few crons are run, looks like and overlapp happens and it brings the cpu to 99% . The top on Centos 6 shows PHP and PHPFPM causing this. Eventually found crons are stuck.

I saw a possible solution with https://github.com/magemojo/m2-ce-cron . Has anyone tried this.

manish3m · 2019-08-10T23:02:25Z

m2-assistant · 2019-08-12T07:47:15Z

Hi @engcom-Charlie. Thank you for working on this issue.
In order to make sure that issue has enough information and ready for development, please read and check the following instruction: 👇

1. Verify that issue has all the required information. (Preconditions, Steps to reproduce, Expected result, Actual result).
Details
If the issue has a valid description, the label Issue: Format is valid will be added to the issue automatically. Please, edit issue description if needed, until label Issue: Format is valid appears.
2. Verify that issue has a meaningful description and provides enough information to reproduce the issue. If the report is valid, add Issue: Clear Description label to the issue by yourself.
3. Add Component: XXXXX label(s) to the ticket, indicating the components it may be related to.
4. Verify that the issue is reproducible on 2.3-develop branch
Details
- Add the comment @magento give me 2.3-develop instance to deploy test instance on Magento infrastructure.
- If the issue is reproducible on 2.3-develop branch, please, add the label Reproduced on 2.3.x.
- If the issue is not reproducible, add your comment that issue is not reproducible and close the issue and stop verification process here!
5. Add label Issue: Confirmed once verification is complete.
6. Make sure that automatic system confirms that report has been added to the backlog.

magento-engcom-team · 2019-08-12T08:22:06Z

✅ Confirmed by @engcom-Charlie
Thank you for verifying the issue. Based on the provided information internal tickets MC-19136 were created

Issue Available: @engcom-Charlie, You will be automatically unassigned. Contributors/Maintainers can claim this issue to continue. To reclaim and continue work, reassign the ticket to yourself.

hostep · 2019-08-12T09:22:00Z

@engcom-Charlie: since you were able to reproduce this problem, could you also add the exact steps of how to do this? Since that info doesn't seem to be in this issue yet.

That would be very much appreciated, thanks!

engcom-Charlie · 2019-08-12T10:18:17Z

@hostep I have the same problem as @manish3m #22438 (comment). Maybe I made a mistake when i confirmed this issue.

engcom-Charlie · 2019-08-12T10:23:00Z

@hostep Steps to reproduce :

Vanilla install
Run bin/magento cron:run and bin/magento setup:upgrade
3.Check your CPU load
4.Repeat step 2 and 3
5.Repeat step 2 and 3 again

Actual result:
CPU load will be 96-100%

engcom-Charlie · 2019-08-12T13:43:41Z

@hostep thank you for contributing!
I can't reproduce this issue on Magento 2.3-develop( have no cron errors and deadlocks).
So i close this issue.
@seansan can you please provide more steps to reproduce and maybe more details if you can reproduce it on 2.3-develop branch.

onlinebizsoft · 2019-08-12T14:47:53Z

Quite confusing!!!

hostep · 2019-08-12T18:13:53Z

Yes indeed, very confusing. @engcom-Charlie on what Magento version did you first test it when you were able to reproduce it? This wasn't on the 2.3-develop branch I'm guessing?

engcom-Charlie · 2019-08-13T07:31:31Z

@hostep yes, but i had no deadlocks.Sorry but when i confirmed this issue i reproduced the problem that is described in the #22438 (comment) .
I didn't notice that the problem is completely different.

onlinebizsoft · 2019-10-25T06:30:18Z

@engcom-Charlie maybe they has relation and high CPU can cause deadlock. If so do we have a fix/MR for high CPU?

engcom-Charlie · 2019-10-25T07:38:04Z

@onlinebizsoft
when I tried to reproduce this issue my CPU was 100% loaded

onlinebizsoft · 2019-10-25T09:29:04Z

@engcom-Charlie so there is a bug. Did you fix this CPU issue?

engcom-Charlie · 2019-10-25T13:06:42Z

@onlinebizsoft it is not fixed yet. Please create a new issue for this bug. I can't confirm this issue because it's about deadlocks.

onlinebizsoft · 2019-10-25T14:07:10Z

@manish3m

vincentteyssier · 2019-11-20T05:39:15Z

Same issue on 2.3.3, here is the log :

mmap() failed: [12] Cannot allocate memory

mmap() failed: [12] Cannot allocate memory

mmap() failed: [12] Cannot allocate memory

mmap() failed: [12] Cannot allocate memory

mmap() failed: [12] Cannot allocate memory

mmap() failed: [12] Cannot allocate memory
PHP Fatal error:  Out of memory (allocated 149422080) (tried to allocate 20480 bytes) in /opt/magento/public_html/generated/metadata/crontab.php on line 8
PHP Fatal error:  Out of memory (allocated 155713536) (tried to allocate 20480 bytes) in /opt/magento/public_html/generated/metadata/crontab.php on line 8
PHP Fatal error:  Out of memory (allocated 155713536) (tried to allocate 20480 bytes) in /opt/magento/public_html/generated/metadata/crontab.php on line 8

In Mysql.php line 593:

  SQLSTATE[40001]: Serialization failure: 1213 Deadlock found when trying to
  get lock; try restarting transaction, query was: DELETE FROM `cron_schedule
  ` WHERE (status = 'success') AND (job_code in ('scconnector_verify_website'
  , 'scconnector_retrieve_gtag')) AND (created_at < '2019-11-19 23:39:08')


In Mysql.php line 110:

  SQLSTATE[40001]: Serialization failure: 1213 Deadlock found when trying to
  get lock; try restarting transaction, query was: DELETE FROM `cron_schedule
  ` WHERE (status = 'success') AND (job_code in ('scconnector_verify_website'
  , 'scconnector_retrieve_gtag')) AND (created_at < '2019-11-19 23:39:08')


In Mysql.php line 91:

  SQLSTATE[40001]: Serialization failure: 1213 Deadlock found when trying to
  get lock; try restarting transaction


cron:run [--group GROUP] [--bootstrap BOOTSTRAP]

Vanilla 2.3.3 install. Ubuntu 18.04, PHP 7.3
The error does not happen all the time but approx once every 24 hours on my instance.

Adel-Magebinary · 2020-01-22T22:53:59Z

Still happens 2.3.3

Adel-Magebinary · 2020-01-22T23:07:55Z

------------
TRANSACTIONS
------------
Trx id counter 4287486914
Purge done for trx's n:o < 4287486892 undo n:o < 0 state: running but idle
History list length 8
LIST OF TRANSACTIONS FOR EACH SESSION:
---TRANSACTION 422211093589952, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422211093591088, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422211093585408, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422211093588816, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422211093584272, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422211093583136, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422211093582000, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422211093580864, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422211093571776, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422211093568368, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422211093564960, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422211093561552, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 4287486913, ACTIVE 0 sec
24 lock struct(s), heap size 1136, 50 row lock(s), undo log entries 332
MySQL thread id 183547, OS thread handle 140719452575488, query id 68103188 192.168.130.76 client_db
---TRANSACTION 4287486792, ACTIVE 30 sec starting index read
mysql tables in use 2, locked 2
LOCK WAIT 2 lock struct(s), heap size 1136, 1 row lock(s)
MySQL thread id 183494, OS thread handle 140714939299584, query id 68085316 192.168.130.76 client_db statistics
UPDATE `cron_schedule` AS `current`
 LEFT JOIN `cron_schedule` AS `existing` ON existing.job_code = current.job_code AND (existing.executed_at > UTC_TIMESTAMP() - INTERVAL 1 DAY OR existing.executed_at IS NULL) AND existing.status = 'running'
SET `current`.`status` = 'running'
WHERE (current.schedule_id = '681166') AND (current.status = 'pending') AND (existing.schedule_id IS NULL)
------- TRX HAS BEEN WAITING 30 SEC FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 39229 page no 193 n bits 112 index PRIMARY of table `client_db_m2_live`.`cron_schedule` trx id 4287486792 lock_mode X locks rec but not gap waiting
------------------
---TRANSACTION 4287486494, ACTIVE 77 sec fetching rows
mysql tables in use 2, locked 2
LOCK WAIT 10 lock struct(s), heap size 1136, 3 row lock(s)
MySQL thread id 183321, OS thread handle 140731744249600, query id 68035699 192.168.130.76 client_db Sending data
UPDATE `cron_schedule` AS `current`
 LEFT JOIN `cron_schedule` AS `existing` ON existing.job_code = current.job_code AND (existing.executed_at > UTC_TIMESTAMP() - INTERVAL 1 DAY OR existing.executed_at IS NULL) AND existing.status = 'running'
SET `current`.`status` = 'running'
WHERE (current.schedule_id = '681770') AND (current.status = 'pending') AND (existing.schedule_id IS NULL)
------- TRX HAS BEEN WAITING 77 SEC FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 39229 page no 215 n bits 160 index PRIMARY of table `client_db_m2_live`.`cron_schedule` trx id 4287486494 lock mode S locks rec but not gap waiting
------------------
---TRANSACTION 4287486491, ACTIVE 77 sec starting index read
mysql tables in use 2, locked 2
LOCK WAIT 2 lock struct(s), heap size 1136, 1 row lock(s)
MySQL thread id 183324, OS thread handle 140737152739072, query id 68035697 192.168.130.76 client_db statistics
UPDATE `cron_schedule` AS `current`
 LEFT JOIN `cron_schedule` AS `existing` ON existing.job_code = current.job_code AND (existing.executed_at > UTC_TIMESTAMP() - INTERVAL 1 DAY OR existing.executed_at IS NULL) AND existing.status = 'running'
SET `current`.`status` = 'running'
WHERE (current.schedule_id = '681166') AND (current.status = 'pending') AND (existing.schedule_id IS NULL)
------- TRX HAS BEEN WAITING 77 SEC FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 39229 page no 193 n bits 112 index PRIMARY of table `client_db_m2_live`.`cron_schedule` trx id 4287486491 lock_mode X locks rec but not gap waiting
------------------
---TRANSACTION 4287486487, ACTIVE 77 sec starting index read
mysql tables in use 2, locked 2
LOCK WAIT 2 lock struct(s), heap size 1136, 1 row lock(s)
MySQL thread id 183323, OS thread handle 140718890538752, query id 68035690 192.168.130.76 client_db statistics
UPDATE `cron_schedule` AS `current`
 LEFT JOIN `cron_schedule` AS `existing` ON existing.job_code = current.job_code AND (existing.executed_at > UTC_TIMESTAMP() - INTERVAL 1 DAY OR existing.executed_at IS NULL) AND existing.status = 'running'
SET `current`.`status` = 'running'
WHERE (current.schedule_id = '681166') AND (current.status = 'pending') AND (existing.schedule_id IS NULL)
------- TRX HAS BEEN WAITING 77 SEC FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 39229 page no 193 n bits 112 index PRIMARY of table `client_db_m2_live`.`cron_schedule` trx id 4287486487 lock_mode X locks rec but not gap waiting
------------------
---TRANSACTION 4287486483, ACTIVE 77 sec starting index read
mysql tables in use 2, locked 2
LOCK WAIT 2 lock struct(s), heap size 1136, 1 row lock(s)
MySQL thread id 183322, OS thread handle 140721998718720, query id 68035681 192.168.130.76 client_db statistics
UPDATE `cron_schedule` AS `current`
 LEFT JOIN `cron_schedule` AS `existing` ON existing.job_code = current.job_code AND (existing.executed_at > UTC_TIMESTAMP() - INTERVAL 1 DAY OR existing.executed_at IS NULL) AND existing.status = 'running'
SET `current`.`status` = 'running'
WHERE (current.schedule_id = '681166') AND (current.status = 'pending') AND (existing.schedule_id IS NULL)
------- TRX HAS BEEN WAITING 77 SEC FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 39229 page no 193 n bits 112 index PRIMARY of table `client_db_m2_live`.`cron_schedule` trx id 4287486483 lock_mode X locks rec but not gap waiting
------------------
---TRANSACTION 4287486480, ACTIVE 77 sec starting index read
mysql tables in use 1, locked 1
LOCK WAIT 2 lock struct(s), heap size 1136, 1 row lock(s)
MySQL thread id 183320, OS thread handle 140719873898240, query id 68035670 192.168.130.76 client_db updating
UPDATE `cron_schedule` SET `job_code` = 'sales_grid_order_creditmemo_async_insert', `status` = 'error', `messages` = 'PDOStatement::execute(): MySQL server has gone away', `created_at` = '2020-01-22 14:27:03', `scheduled_at` = '2020-01-22 14:42:00', `executed_at` = NULL, `finished_at` = NULL, `origin` = '2', `user` = 'mage', `ip` = NULL, `error_file` = '/var/www/live/M2/vendor/magento/framework/DB/Statement/Pdo/Mysql.php', `error_line` = '91' WHERE (schedule_id='681172')
------- TRX HAS BEEN WAITING 77 SEC FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 39229 page no 193 n bits 112 index PRIMARY of table `client_db_m2_live`.`cron_schedule` trx id 4287486480 lock_mode X locks rec but not gap waiting
------------------
---TRANSACTION 4287486475, ACTIVE 77 sec starting index read
mysql tables in use 2, locked 2
LOCK WAIT 2 lock struct(s), heap size 1136, 1 row lock(s)
MySQL thread id 183316, OS thread handle 140727230654208, query id 68035661 192.168.130.76 client_db statistics
UPDATE `cron_schedule` AS `current`
 LEFT JOIN `cron_schedule` AS `existing` ON existing.job_code = current.job_code AND (existing.executed_at > UTC_TIMESTAMP() - INTERVAL 1 DAY OR existing.executed_at IS NULL) AND existing.status = 'running'
SET `current`.`status` = 'running'
WHERE (current.schedule_id = '681771') AND (current.status = 'pending') AND (existing.schedule_id IS NULL)
------- TRX HAS BEEN WAITING 77 SEC FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 39229 page no 127 n bits 120 index PRIMARY of table `client_db_m2_live`.`cron_schedule` trx id 4287486475 lock_mode X locks rec but not gap waiting
------------------
---TRANSACTION 4287486474, ACTIVE 77 sec starting index read
mysql tables in use 2, locked 2
LOCK WAIT 2 lock struct(s), heap size 1136, 1 row lock(s)
MySQL thread id 183318, OS thread handle 140730339157760, query id 68035658 192.168.130.76 client_db statistics
UPDATE `cron_schedule` AS `current`
 LEFT JOIN `cron_schedule` AS `existing` ON existing.job_code = current.job_code AND (existing.executed_at > UTC_TIMESTAMP() - INTERVAL 1 DAY OR existing.executed_at IS NULL) AND existing.status = 'running'
SET `current`.`status` = 'running'
WHERE (current.schedule_id = '681165') AND (current.status = 'pending') AND (existing.schedule_id IS NULL)
------- TRX HAS BEEN WAITING 77 SEC FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 39229 page no 193 n bits 112 index PRIMARY of table `client_db_m2_live`.`cron_schedule` trx id 4287486474 lock_mode X locks rec but not gap waiting
------------------
---TRANSACTION 4287486470, ACTIVE 77 sec starting index read
mysql tables in use 1, locked 1
LOCK WAIT 2 lock struct(s), heap size 1136, 1 row lock(s)
MySQL thread id 183317, OS thread handle 140735413794560, query id 68035650 192.168.130.76 client_db updating
UPDATE `cron_schedule` SET `job_code` = 'amasty_cron_activity', `status` = 'error', `messages` = 'PDOStatement::execute(): MySQL server has gone away', `created_at` = '2020-01-22 14:42:03', `scheduled_at` = '2020-01-22 14:52:00', `executed_at` = NULL, `finished_at` = NULL, `origin` = '2', `user` = 'mage', `ip` = NULL, `error_file` = '/var/www/live/M2/vendor/magento/framework/DB/Statement/Pdo/Mysql.php', `error_line` = '91' WHERE (schedule_id='681532')
------- TRX HAS BEEN WAITING 77 SEC FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 39229 page no 198 n bits 152 index PRIMARY of table `client_db_m2_live`.`cron_schedule` trx id 4287486470 lock_mode X locks rec but not gap waiting
------------------
---TRANSACTION 4287486466, ACTIVE 77 sec starting index read
mysql tables in use 2, locked 2
LOCK WAIT 2 lock struct(s), heap size 1136, 1 row lock(s)
MySQL thread id 183315, OS thread handle 140736833447680, query id 68035640 192.168.130.76 client_db statistics
UPDATE `cron_schedule` AS `current`
 LEFT JOIN `cron_schedule` AS `existing` ON existing.job_code = current.job_code AND (existing.executed_at > UTC_TIMESTAMP() - INTERVAL 1 DAY OR existing.executed_at IS NULL) AND existing.status = 'running'
SET `current`.`status` = 'running'
WHERE (current.schedule_id = '681165') AND (current.status = 'pending') AND (existing.schedule_id IS NULL)
------- TRX HAS BEEN WAITING 77 SEC FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 39229 page no 193 n bits 112 index PRIMARY of table `client_db_m2_live`.`cron_schedule` trx id 4287486466 lock_mode X locks rec but not gap waiting
------------------
---TRANSACTION 4287486465, ACTIVE 77 sec fetching rows
mysql tables in use 2, locked 2
LOCK WAIT 10 lock struct(s), heap size 1136, 3 row lock(s)
MySQL thread id 183312, OS thread handle 140714658752256, query id 68035636 192.168.130.76 client_db Sending data
UPDATE `cron_schedule` AS `current`
 LEFT JOIN `cron_schedule` AS `existing` ON existing.job_code = current.job_code AND (existing.executed_at > UTC_TIMESTAMP() - INTERVAL 1 DAY OR existing.executed_at IS NULL) AND existing.status = 'running'
SET `current`.`status` = 'running'
WHERE (current.schedule_id = '681771') AND (current.status = 'pending') AND (existing.schedule_id IS NULL)
------- TRX HAS BEEN WAITING 77 SEC FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 39229 page no 215 n bits 160 index PRIMARY of table `client_db_m2_live`.`cron_schedule` trx id 4287486465 lock mode S locks rec but not gap waiting
------------------
---TRANSACTION 4287486464, ACTIVE 77 sec starting index read
mysql tables in use 2, locked 2
LOCK WAIT 2 lock struct(s), heap size 1136, 1 row lock(s)
MySQL thread id 183314, OS thread handle 140719874369280, query id 68035634 192.168.130.76 client_db statistics
UPDATE `cron_schedule` AS `current`
 LEFT JOIN `cron_schedule` AS `existing` ON existing.job_code = current.job_code AND (existing.executed_at > UTC_TIMESTAMP() - INTERVAL 1 DAY OR existing.executed_at IS NULL) AND existing.status = 'running'
SET `current`.`status` = 'running'
WHERE (current.schedule_id = '681165') AND (current.status = 'pending') AND (existing.schedule_id IS NULL)
------- TRX HAS BEEN WAITING 77 SEC FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 39229 page no 193 n bits 112 index PRIMARY of table `client_db_m2_live`.`cron_schedule` trx id 4287486464 lock_mode X locks rec but not gap waiting
------------------
---TRANSACTION 4287486456, ACTIVE 77 sec fetching rows
mysql tables in use 2, locked 2
LOCK WAIT 13 lock struct(s), heap size 1136, 3 row lock(s)
MySQL thread id 183313, OS thread handle 140737146840832, query id 68035618 192.168.130.76 client_db Sending data
UPDATE `cron_schedule` AS `current`
 LEFT JOIN `cron_schedule` AS `existing` ON existing.job_code = current.job_code AND (existing.executed_at > UTC_TIMESTAMP() - INTERVAL 1 DAY OR existing.executed_at IS NULL) AND existing.status = 'running'
SET `current`.`status` = 'running'
WHERE (current.schedule_id = '681656') AND (current.status = 'pending') AND (existing.schedule_id IS NULL)
------- TRX HAS BEEN WAITING 77 SEC FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 39229 page no 193 n bits 112 index PRIMARY of table `client_db_m2_live`.`cron_schedule` trx id 4287486456 lock mode S locks rec but not gap waiting
------------------
---TRANSACTION 4287486451, ACTIVE 77 sec fetching rows
mysql tables in use 2, locked 2
LOCK WAIT 10 lock struct(s), heap size 1136, 3 row lock(s)
MySQL thread id 183310, OS thread handle 140733166118656, query id 68035603 192.168.130.76 client_db Sending data
UPDATE `cron_schedule` AS `current`
 LEFT JOIN `cron_schedule` AS `existing` ON existing.job_code = current.job_code AND (existing.executed_at > UTC_TIMESTAMP() - INTERVAL 1 DAY OR existing.executed_at IS NULL) AND existing.status = 'running'
SET `current`.`status` = 'running'
WHERE (current.schedule_id = '681298') AND (current.status = 'pending') AND (existing.schedule_id IS NULL)
------- TRX HAS BEEN WAITING 77 SEC FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 39229 page no 215 n bits 160 index PRIMARY of table `client_db_m2_live`.`cron_schedule` trx id 4287486451 lock mode S locks rec but not gap waiting
------------------
---TRANSACTION 4287486450, ACTIVE 77 sec fetching rows
mysql tables in use 2, locked 2
LOCK WAIT 13 lock struct(s), heap size 1136, 3 row lock(s)
MySQL thread id 183309, OS thread handle 140718469277440, query id 68035598 192.168.130.76 client_db Sending data
UPDATE `cron_schedule` AS `current`
 LEFT JOIN `cron_schedule` AS `existing` ON existing.job_code = current.job_code AND (existing.executed_at > UTC_TIMESTAMP() - INTERVAL 1 DAY OR existing.executed_at IS NULL) AND existing.status = 'running'
SET `current`.`status` = 'running'
WHERE (current.schedule_id = '681166') AND (current.status = 'pending') AND (existing.schedule_id IS NULL)
------- TRX HAS BEEN WAITING 77 SEC FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 39229 page no 193 n bits 112 index PRIMARY of table `client_db_m2_live`.`cron_schedule` trx id 4287486450 lock mode S locks rec but not gap waiting
------------------
---TRANSACTION 4287486449, ACTIVE 77 sec fetching rows
mysql tables in use 2, locked 2
LOCK WAIT 13 lock struct(s), heap size 1136, 3 row lock(s)
MySQL thread id 183308, OS thread handle 140721014953728, query id 68035592 192.168.130.76 client_db Sending data
UPDATE `cron_schedule` AS `current`
 LEFT JOIN `cron_schedule` AS `existing` ON existing.job_code = current.job_code AND (existing.executed_at > UTC_TIMESTAMP() - INTERVAL 1 DAY OR existing.executed_at IS NULL) AND existing.status = 'running'
SET `current`.`status` = 'running'
WHERE (current.schedule_id = '681418') AND (current.status = 'pending') AND (existing.schedule_id IS NULL)
------- TRX HAS BEEN WAITING 77 SEC FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 39229 page no 193 n bits 112 index PRIMARY of table `client_db_m2_live`.`cron_schedule` trx id 4287486449 lock mode S locks rec but not gap waiting
------------------
---TRANSACTION 4287486448, ACTIVE 77 sec starting index read
mysql tables in use 1, locked 1
LOCK WAIT 2 lock struct(s), heap size 1136, 1 row lock(s)
MySQL thread id 183307, OS thread handle 140722420045568, query id 68035589 192.168.130.76 client_db updating
UPDATE `cron_schedule` SET `job_code` = 'sales_grid_order_creditmemo_async_insert', `status` = 'error', `messages` = 'PDOStatement::execute(): MySQL server has gone away', `created_at` = '2020-01-22 14:27:03', `scheduled_at` = '2020-01-22 14:33:00', `executed_at` = NULL, `finished_at` = NULL, `origin` = '2', `user` = 'mage', `ip` = NULL, `error_file` = '/var/www/live/M2/vendor/magento/framework/DB/Statement/Pdo/Mysql.php', `error_line` = '91' WHERE (schedule_id='681163')
------- TRX HAS BEEN WAITING 77 SEC FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 39229 page no 193 n bits 112 index PRIMARY of table `client_db_m2_live`.`cron_schedule` trx id 4287486448 lock_mode X locks rec but not gap waiting
------------------
---TRANSACTION 4287486447, ACTIVE 77 sec starting index read
mysql tables in use 2, locked 2
LOCK WAIT 2 lock struct(s), heap size 1136, 1 row lock(s)
MySQL thread id 183306, OS thread handle 140732727547648, query id 68035587 192.168.130.76 client_db statistics
UPDATE `cron_schedule` AS `current`
 LEFT JOIN `cron_schedule` AS `existing` ON existing.job_code = current.job_code AND (existing.executed_at > UTC_TIMESTAMP() - INTERVAL 1 DAY OR existing.executed_at IS NULL) AND existing.status = 'running'
SET `current`.`status` = 'running'
WHERE (current.schedule_id = '681192') AND (current.status = 'pending') AND (existing.schedule_id IS NULL)
------- TRX HAS BEEN WAITING 77 SEC FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 39229 page no 193 n bits 112 index PRIMARY of table `client_db_m2_live`.`cron_schedule` trx id 4287486447 lock_mode X locks rec but not gap waiting
------------------
---TRANSACTION 4287486446, ACTIVE 77 sec starting index read
mysql tables in use 2, locked 2
LOCK WAIT 2 lock struct(s), heap size 1136, 1 row lock(s)
MySQL thread id 183301, OS thread handle 140731181741824, query id 68035582 192.168.130.76 client_db statistics
UPDATE `cron_schedule` AS `current`
 LEFT JOIN `cron_schedule` AS `existing` ON existing.job_code = current.job_code AND (existing.executed_at > UTC_TIMESTAMP() - INTERVAL 1 DAY OR existing.executed_at IS NULL) AND existing.status = 'running'
SET `current`.`status` = 'running'
WHERE (current.schedule_id = '681190') AND (current.status = 'pending') AND (existing.schedule_id IS NULL)
------- TRX HAS BEEN WAITING 77 SEC FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 39229 page no 193 n bits 112 index PRIMARY of table `client_db_m2_live`.`cron_schedule` trx id 4287486446 lock_mode X locks rec but not gap waiting
------------------
---TRANSACTION 4287486439, ACTIVE 77 sec starting index read
mysql tables in use 1, locked 1
LOCK WAIT 2 lock struct(s), heap size 1136, 1 row lock(s)
MySQL thread id 183296, OS thread handle 140732867852032, query id 68035574 192.168.130.76 client_db updating
UPDATE `cron_schedule` SET `job_code` = 'sales_send_order_emails', `status` = 'pending', `messages` = NULL, `created_at` = '2020-01-22 14:27:03', `scheduled_at` = '2020-01-22 14:40:00', `executed_at` = NULL, `finished_at` = NULL, `origin` = '2', `user` = 'mage', `ip` = NULL, `error_file` = NULL, `error_line` = NULL WHERE (schedule_id='681186')
------- TRX HAS BEEN WAITING 77 SEC FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 39229 page no 193 n bits 112 index PRIMARY of table `client_db_m2_live`.`cron_schedule` trx id 4287486439 lock_mode X locks rec but not gap waiting
------------------
---TRANSACTION 4287486434, ACTIVE 77 sec fetching rows
mysql tables in use 2, locked 2
LOCK WAIT 11 lock struct(s), heap size 1136, 5 row lock(s)
MySQL thread id 183293, OS thread handle 140716625614592, query id 68035564 192.168.130.76 client_db Sending data
UPDATE `cron_schedule` AS `current`
 LEFT JOIN `cron_schedule` AS `existing` ON existing.job_code = current.job_code AND (existing.executed_at > UTC_TIMESTAMP() - INTERVAL 1 DAY OR existing.executed_at IS NULL) AND existing.status = 'running'
SET `current`.`status` = 'running'
WHERE (current.schedule_id = '681192') AND (current.status = 'pending') AND (existing.schedule_id IS NULL)
------- TRX HAS BEEN WAITING 77 SEC FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 39229 page no 193 n bits 112 index PRIMARY of table `client_db_m2_live`.`cron_schedule` trx id 4287486434 lock mode S locks rec but not gap waiting
------------------
---TRANSACTION 4287486429, ACTIVE 77 sec fetching rows
mysql tables in use 2, locked 2
LOCK WAIT 12 lock struct(s), heap size 1136, 8 row lock(s)
MySQL thread id 183303, OS thread handle 140727652640512, query id 68035558 192.168.130.76 client_db Sending data
UPDATE `cron_schedule` AS `current`
 LEFT JOIN `cron_schedule` AS `existing` ON existing.job_code = current.job_code AND (existing.executed_at > UTC_TIMESTAMP() - INTERVAL 1 DAY OR existing.executed_at IS NULL) AND existing.status = 'running'
SET `current`.`status` = 'running'
WHERE (current.schedule_id = '681190') AND (current.status = 'pending') AND (existing.schedule_id IS NULL)
------- TRX HAS BEEN WAITING 77 SEC FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 39229 page no 193 n bits 112 index PRIMARY of table `client_db_m2_live`.`cron_schedule` trx id 4287486429 lock mode S locks rec but not gap waiting
------------------
---TRANSACTION 4287486422, ACTIVE 77 sec starting index read
mysql tables in use 2, locked 2
LOCK WAIT 2 lock struct(s), heap size 1136, 1 row lock(s)
MySQL thread id 183305, OS thread handle 140717626222336, query id 68035545 192.168.130.76 client_db statistics
UPDATE `cron_schedule` AS `current`
 LEFT JOIN `cron_schedule` AS `existing` ON existing.job_code = current.job_code AND (existing.executed_at > UTC_TIMESTAMP() - INTERVAL 1 DAY OR existing.executed_at IS NULL) AND existing.status = 'running'
SET `current`.`status` = 'running'
WHERE (current.schedule_id = '681172') AND (current.status = 'pending') AND (existing.schedule_id IS NULL)
------- TRX HAS BEEN WAITING 77 SEC FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 39229 page no 193 n bits 112 index PRIMARY of table `client_db_m2_live`.`cron_schedule` trx id 4287486422 lock_mode X locks rec but not gap waiting
------------------
---TRANSACTION 4287486407, ACTIVE 77 sec starting index read
mysql tables in use 2, locked 2
LOCK WAIT 2 lock struct(s), heap size 1136, 1 row lock(s)
MySQL thread id 183304, OS thread handle 140720295630592, query id 68035524 192.168.130.76 client_db statistics
UPDATE `cron_schedule` AS `current`
 LEFT JOIN `cron_schedule` AS `existing` ON existing.job_code = current.job_code AND (existing.executed_at > UTC_TIMESTAMP() - INTERVAL 1 DAY OR existing.executed_at IS NULL) AND existing.status = 'running'
SET `current`.`status` = 'running'
WHERE (current.schedule_id = '681172') AND (current.status = 'pending') AND (existing.schedule_id IS NULL)
------- TRX HAS BEEN WAITING 77 SEC FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 39229 page no 193 n bits 112 index PRIMARY of table `client_db_m2_live`.`cron_schedule` trx id 4287486407 lock_mode X locks rec but not gap waiting
------------------
---TRANSACTION 4287486400, ACTIVE 77 sec fetching rows
mysql tables in use 2, locked 2
LOCK WAIT 15 lock struct(s), heap size 1136, 7 row lock(s)
MySQL thread id 183302, OS thread handle 140724825417472, query id 68035506 192.168.130.76 client_db Sending data
UPDATE `cron_schedule` AS `current`
 LEFT JOIN `cron_schedule` AS `existing` ON existing.job_code = current.job_code AND (existing.executed_at > UTC_TIMESTAMP() - INTERVAL 1 DAY OR existing.executed_at IS NULL) AND existing.status = 'running'
SET `current`.`status` = 'running'
WHERE (current.schedule_id = '681174') AND (current.status = 'pending') AND (existing.schedule_id IS NULL)
------- TRX HAS BEEN WAITING 77 SEC FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 39229 page no 193 n bits 112 index PRIMARY of table `client_db_m2_live`.`cron_schedule` trx id 4287486400 lock mode S locks rec but not gap waiting
------------------
---TRANSACTION 4287486382, ACTIVE 77 sec starting index read
mysql tables in use 2, locked 2
LOCK WAIT 2 lock struct(s), heap size 1136, 1 row lock(s)
MySQL thread id 183295, OS thread handle 140714517915392, query id 68035454 192.168.130.76 client_db statistics
UPDATE `cron_schedule` AS `current`
 LEFT JOIN `cron_schedule` AS `existing` ON existing.job_code = current.job_code AND (existing.executed_at > UTC_TIMESTAMP() - INTERVAL 1 DAY OR existing.executed_at IS NULL) AND existing.status = 'running'
SET `current`.`status` = 'running'
WHERE (current.schedule_id = '681203') AND (current.status = 'pending') AND (existing.schedule_id IS NULL)
------- TRX HAS BEEN WAITING 77 SEC FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 39229 page no 205 n bits 120 index PRIMARY of table `client_db_m2_live`.`cron_schedule` trx id 4287486382 lock_mode X locks rec but not gap waiting
------------------
---TRANSACTION 4287486373, ACTIVE 77 sec starting index read
mysql tables in use 2, locked 2
LOCK WAIT 2 lock struct(s), heap size 1136, 1 row lock(s)
MySQL thread id 183285, OS thread handle 140737163552512, query id 68035439 192.168.130.76 client_db statistics
UPDATE `cron_schedule` AS `current`
 LEFT JOIN `cron_schedule` AS `existing` ON existing.job_code = current.job_code AND (existing.executed_at > UTC_TIMESTAMP() - INTERVAL 1 DAY OR existing.executed_at IS NULL) AND existing.status = 'running'
SET `current`.`status` = 'running'
WHERE (current.schedule_id = '681458') AND (current.status = 'pending') AND (existing.schedule_id IS NULL)
------- TRX HAS BEEN WAITING 77 SEC FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 39229 page no 221 n bits 120 index PRIMARY of table `client_db_m2_live`.`cron_schedule` trx id 4287486373 lock_mode X locks rec but not gap waiting
------------------
---TRANSACTION 4287486372, ACTIVE 77 sec starting index read
mysql tables in use 2, locked 2
LOCK WAIT 2 lock struct(s), heap size 1136, 1 row lock(s)
MySQL thread id 183294, OS thread handle 140731603474176, query id 68035435 192.168.130.76 client_db statistics
UPDATE `cron_schedule` AS `current`
 LEFT JOIN `cron_schedule` AS `existing` ON existing.job_code = current.job_code AND (existing.executed_at > UTC_TIMESTAMP() - INTERVAL 1 DAY OR existing.executed_at IS NULL) AND existing.status = 'running'
SET `current`.`status` = 'running'
WHERE (current.schedule_id = '681221') AND (current.status = 'pending') AND (existing.schedule_id IS NULL)
------- TRX HAS BEEN WAITING 77 SEC FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 39229 page no 205 n bits 120 index PRIMARY of table `client_db_m2_live`.`cron_schedule` trx id 4287486372 lock_mode X locks rec but not gap waiting
------------------
---TRANSACTION 4287486368, ACTIVE 77 sec fetching rows
mysql tables in use 2, locked 2
LOCK WAIT 14 lock struct(s), heap size 1136, 3 row lock(s)
MySQL thread id 183292, OS thread handle 140735695017728, query id 68035421 192.168.130.76 client_db Sending data
UPDATE `cron_schedule` AS `current`
 LEFT JOIN `cron_schedule` AS `existing` ON existing.job_code = current.job_code AND (existing.executed_at > UTC_TIMESTAMP() - INTERVAL 1 DAY OR existing.executed_at IS NULL) AND existing.status = 'running'
SET `current`.`status` = 'running'
WHERE (current.schedule_id = '681172') AND (current.status = 'pending') AND (existing.schedule_id IS NULL)
------- TRX HAS BEEN WAITING 77 SEC FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 39229 page no 193 n bits 112 index PRIMARY of table `client_db_m2_live`.`cron_schedule` trx id 4287486368 lock mode S locks rec but not gap waiting
------------------
---TRANSACTION 4287486367, ACTIVE 77 sec starting index read
mysql tables in use 2, locked 2
LOCK WAIT 2 lock struct(s), heap size 1136, 1 row lock(s)
MySQL thread id 183291, OS thread handle 140730057668352, query id 68035415 192.168.130.76 client_db statistics
UPDATE `cron_schedule` AS `current`
 LEFT JOIN `cron_schedule` AS `existing` ON existing.job_code = current.job_code AND (existing.executed_at > UTC_TIMESTAMP() - INTERVAL 1 DAY OR existing.executed_at IS NULL) AND existing.status = 'running'
SET `current`.`status` = 'running'
WHERE (current.schedule_id = '681203') AND (current.status = 'pending') AND (existing.schedule_id IS NULL)
------- TRX HAS BEEN WAITING 77 SEC FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 39229 page no 205 n bits 120 index PRIMARY of table `client_db_m2_live`.`cron_schedule` trx id 4287486367 lock_mode X locks rec but not gap waiting
------------------
---TRANSACTION 4287486364, ACTIVE 77 sec fetching rows
mysql tables in use 2, locked 2
LOCK WAIT 13 lock struct(s), heap size 1136, 8 row lock(s)
MySQL thread id 183283, OS thread handle 140732446529280, query id 68035402 192.168.130.76 client_db Sending data
UPDATE `cron_schedule` AS `current`
 LEFT JOIN `cron_schedule` AS `existing` ON existing.job_code = current.job_code AND (existing.executed_at > UTC_TIMESTAMP() - INTERVAL 1 DAY OR existing.executed_at IS NULL) AND existing.status = 'running'
SET `current`.`status` = 'running'
WHERE (current.schedule_id = '681458') AND (current.status = 'pending') AND (existing.schedule_id IS NULL)
------- TRX HAS BEEN WAITING 77 SEC FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 39229 page no 221 n bits 120 index PRIMARY of table `client_db_m2_live`.`cron_schedule` trx id 4287486364 lock mode S locks rec but not gap waiting
------------------
---TRANSACTION 4287486360, ACTIVE 77 sec starting index read
mysql tables in use 2, locked 2
LOCK WAIT 2 lock struct(s), heap size 1136, 1 row lock(s)
MySQL thread id 183280, OS thread handle 140729900586752, query id 68035400 192.168.130.76 client_db statistics
UPDATE `cron_schedule` AS `current`
 LEFT JOIN `cron_schedule` AS `existing` ON existing.job_code = current.job_code AND (existing.executed_at > UTC_TIMESTAMP() - INTERVAL 1 DAY OR existing.executed_at IS NULL) AND existing.status = 'running'
SET `current`.`status` = 'running'
WHERE (current.schedule_id = '681465') AND (current.status = 'pending') AND (existing.schedule_id IS NULL)
------- TRX HAS BEEN WAITING 77 SEC FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 39229 page no 221 n bits 120 index PRIMARY of table `client_db_m2_live`.`cron_schedule` trx id 4287486360 lock_mode X locks rec but not gap waiting
------------------
---TRANSACTION 4287486356, ACTIVE 77 sec fetching rows
mysql tables in use 2, locked 2
LOCK WAIT 10 lock struct(s), heap size 1136, 4 row lock(s)
MySQL thread id 183289, OS thread handle 140736842098432, query id 68035395 192.168.130.76 client_db Sending data
UPDATE `cron_schedule` AS `current`
 LEFT JOIN `cron_schedule` AS `existing` ON existing.job_code = current.job_code AND (existing.executed_at > UTC_TIMESTAMP() - INTERVAL 1 DAY OR existing.executed_at IS NULL) AND existing.status = 'running'
SET `current`.`status` = 'running'
WHERE (current.schedule_id = '681296') AND (current.status = 'pending') AND (existing.schedule_id IS NULL)
------- TRX HAS BEEN WAITING 77 SEC FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 39229 page no 215 n bits 160 index PRIMARY of table `client_db_m2_live`.`cron_schedule` trx id 4287486356 lock mode S locks rec but not gap waiting
------------------
---TRANSACTION 4287486353, ACTIVE 77 sec fetching rows
mysql tables in use 2, locked 2
LOCK WAIT 13 lock struct(s), heap size 1136, 7 row lock(s)
MySQL thread id 183277, OS thread handle 140726230570752, query id 68035384 192.168.130.76 client_db Sending data
UPDATE `cron_schedule` AS `current`
 LEFT JOIN `cron_schedule` AS `existing` ON existing.job_code = current.job_code AND (existing.executed_at > UTC_TIMESTAMP() - INTERVAL 1 DAY OR existing.executed_at IS NULL) AND existing.status = 'running'
SET `current`.`status` = 'running'
WHERE (current.schedule_id = '681465') AND (current.status = 'pending') AND (existing.schedule_id IS NULL)
------- TRX HAS BEEN WAITING 77 SEC FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 39229 page no 221 n bits 120 index PRIMARY of table `client_db_m2_live`.`cron_schedule` trx id 4287486353 lock mode S locks rec but not gap waiting
------------------
---TRANSACTION 4287486347, ACTIVE 77 sec fetching rows
mysql tables in use 2, locked 2
LOCK WAIT 10 lock struct(s), heap size 1136, 3 row lock(s)
MySQL thread id 183287, OS thread handle 140736836724480, query id 68035377 192.168.130.76 client_db Sending data
UPDATE `cron_schedule` AS `current`
 LEFT JOIN `cron_schedule` AS `existing` ON existing.job_code = current.job_code AND (existing.executed_at > UTC_TIMESTAMP() - INTERVAL 1 DAY OR existing.executed_at IS NULL) AND existing.status = 'running'
SET `current`.`status` = 'running'
WHERE (current.schedule_id = '681442') AND (current.status = 'pending') AND (existing.schedule_id IS NULL)
------- TRX HAS BEEN WAITING 77 SEC FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 39229 page no 205 n bits 120 index PRIMARY of table `client_db_m2_live`.`cron_schedule` trx id 4287486347 lock mode S locks rec but not gap waiting
------------------
---TRANSACTION 4287486344, ACTIVE 77 sec starting index read
mysql tables in use 2, locked 2
LOCK WAIT 2 lock struct(s), heap size 1136, 1 row lock(s)
MySQL thread id 183286, OS thread handle 140714798995200, query id 68035363 192.168.130.76 client_db statistics
UPDATE `cron_schedule` AS `current`
 LEFT JOIN `cron_schedule` AS `existing` ON existing.job_code = current.job_code AND (existing.executed_at > UTC_TIMESTAMP() - INTERVAL 1 DAY OR existing.executed_at IS NULL) AND existing.status = 'running'
SET `current`.`status` = 'running'
WHERE (current.schedule_id = '681195') AND (current.status = 'pending') AND (existing.schedule_id IS NULL)
------- TRX HAS BEEN WAITING 77 SEC FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 39229 page no 205 n bits 120 index PRIMARY of table `client_db_m2_live`.`cron_schedule` trx id 4287486344 lock_mode X locks rec but not gap waiting
------------------
---TRANSACTION 4287486341, ACTIVE 77 sec fetching rows
mysql tables in use 2, locked 2
LOCK WAIT 13 lock struct(s), heap size 1136, 7 row lock(s)
MySQL thread id 183284, OS thread handle 140722982348544, query id 68035350 192.168.130.76 client_db Sending data
UPDATE `cron_schedule` AS `current`
 LEFT JOIN `cron_schedule` AS `existing` ON existing.job_code = current.job_code AND (existing.executed_at > UTC_TIMESTAMP() - INTERVAL 1 DAY OR existing.executed_at IS NULL) AND existing.status = 'running'
SET `current`.`status` = 'running'
WHERE (current.schedule_id = '681444') AND (current.status = 'pending') AND (existing.schedule_id IS NULL)
------- TRX HAS BEEN WAITING 77 SEC FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 39229 page no 221 n bits 120 index PRIMARY of table `client_db_m2_live`.`cron_schedule` trx id 4287486341 lock mode S locks rec but not gap waiting
------------------
---TRANSACTION 4287486339, ACTIVE 77 sec fetching rows
mysql tables in use 2, locked 2
LOCK WAIT 13 lock struct(s), heap size 1136, 7 row lock(s)
MySQL thread id 183282, OS thread handle 140736828532480, query id 68035339 192.168.130.76 client_db Sending data
UPDATE `cron_schedule` AS `current`
 LEFT JOIN `cron_schedule` AS `existing` ON existing.job_code = current.job_code AND (existing.executed_at > UTC_TIMESTAMP() - INTERVAL 1 DAY OR existing.executed_at IS NULL) AND existing.status = 'running'
SET `current`.`status` = 'running'
WHERE (current.schedule_id = '681685') AND (current.status = 'pending') AND (existing.schedule_id IS NULL)
------- TRX HAS BEEN WAITING 77 SEC FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 39229 page no 221 n bits 120 index PRIMARY of table `client_db_m2_live`.`cron_schedule` trx id 4287486339 lock mode S locks rec but not gap waiting
------------------
---TRANSACTION 4287486338, ACTIVE 77 sec fetching rows
mysql tables in use 2, locked 2
LOCK WAIT 10 lock struct(s), heap size 1136, 3 row lock(s)
MySQL thread id 183281, OS thread handle 140726949893888, query id 68035333 192.168.130.76 client_db Sending data
UPDATE `cron_schedule` AS `current`
 LEFT JOIN `cron_schedule` AS `existing` ON existing.job_code = current.job_code AND (existing.executed_at > UTC_TIMESTAMP() - INTERVAL 1 DAY OR existing.executed_at IS NULL) AND existing.status = 'running'
SET `current`.`status` = 'running'
WHERE (current.schedule_id = '681532') AND (current.status = 'pending') AND (existing.schedule_id IS NULL)
------- TRX HAS BEEN WAITING 77 SEC FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 39229 page no 215 n bits 160 index PRIMARY of table `client_db_m2_live`.`cron_schedule` trx id 4287486338 lock mode S locks rec but not gap waiting
------------------
---TRANSACTION 4287486336, ACTIVE 77 sec fetching rows
mysql tables in use 2, locked 2
LOCK WAIT 16 lock struct(s), heap size 1136, 7 row lock(s)
MySQL thread id 183279, OS thread handle 140721576732416, query id 68035316 192.168.130.76 client_db Sending data
UPDATE `cron_schedule` AS `current`
 LEFT JOIN `cron_schedule` AS `existing` ON existing.job_code = current.job_code AND (existing.executed_at > UTC_TIMESTAMP() - INTERVAL 1 DAY OR existing.executed_at IS NULL) AND existing.status = 'running'
SET `current`.`status` = 'running'
WHERE (current.schedule_id = '681660') AND (current.status = 'pending') AND (existing.schedule_id IS NULL)
------- TRX HAS BEEN WAITING 77 SEC FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 39229 page no 193 n bits 112 index PRIMARY of table `client_db_m2_live`.`cron_schedule` trx id 4287486336 lock mode S locks rec but not gap waiting
------------------
---TRANSACTION 4287486335, ACTIVE 77 sec fetching rows
mysql tables in use 2, locked 2
LOCK WAIT 16 lock struct(s), heap size 1136, 7 row lock(s)
MySQL thread id 183278, OS thread handle 140722841573120, query id 68035310 192.168.130.76 client_db Sending data
UPDATE `cron_schedule` AS `current`
 LEFT JOIN `cron_schedule` AS `existing` ON existing.job_code = current.job_code AND (existing.executed_at > UTC_TIMESTAMP() - INTERVAL 1 DAY OR existing.executed_at IS NULL) AND existing.status = 'running'
SET `current`.`status` = 'running'
WHERE (current.schedule_id = '681661') AND (current.status = 'pending') AND (existing.schedule_id IS NULL)
------- TRX HAS BEEN WAITING 77 SEC FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 39229 page no 193 n bits 112 index PRIMARY of table `client_db_m2_live`.`cron_schedule` trx id 4287486335 lock mode S locks rec but not gap waiting
------------------
--------
FILE I/O

magento-engcom-team · 2020-12-22T00:35:54Z

Hi @seansan. Thank you for your report.
The issue has been fixed in #28007 by @driskell in 2.4-develop branch
Related commit(s):

The fix will be available with the upcoming 2.4.3 release.

ctadlock · 2025-04-16T17:38:39Z

Still an issue in Magento 2.4.7-p5.

What a joke.

magento-engcom-team added the Issue: Format is valid Gate 1 Passed. Automatic verification of issue format passed label Apr 20, 2019

ghost self-assigned this Apr 22, 2019

ghost closed this as completed Apr 22, 2019

seansan mentioned this issue Apr 22, 2019

1213 Deadlock found when trying to get lock #8933

Closed

driskell reopened this Aug 6, 2019

ghost removed their assignment Aug 6, 2019

engcom-Charlie self-assigned this Aug 12, 2019

ghost unassigned engcom-Charlie Aug 12, 2019

magento-engcom-team added the Issue: Ready for Work Gate 4. Acknowledged. Issue is added to backlog and ready for development label Aug 12, 2019

engcom-Charlie removed the Issue: Confirmed Gate 3 Passed. Manual verification of the issue completed. Issue is confirmed label Aug 12, 2019

engcom-Charlie closed this as completed Aug 12, 2019

engcom-Charlie added Issue: Cannot Reproduce Cannot reproduce the issue on the latest `2.4-develop` branch and removed Reproduced on 2.3.x The issue has been reproduced on latest 2.3 release labels Aug 12, 2019

roni-sooryen mentioned this issue Feb 21, 2020

Cronjobs increasing CPU usage and slow queries #26507

Closed

driskell mentioned this issue Apr 28, 2020

Cron cleanup repeatedly hits deadlocks on large environments where groups can overlap #28007

Merged

4 tasks

magento-engcom-team added the Fixed in 2.4.x The issue has been fixed in 2.4-develop branch label Dec 22, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Magento 2.3.1 Cron Deadlocks for cron_schedule #22438

Magento 2.3.1 Cron Deadlocks for cron_schedule #22438

seansan commented Apr 20, 2019 •

edited

Loading

m2-assistant bot commented Apr 20, 2019 •

edited by kozie

Loading

m2-assistant bot commented Apr 22, 2019

ghost commented Apr 22, 2019

seansan commented Apr 22, 2019 via email

seansan commented Apr 24, 2019

onlinebizsoft commented Jun 28, 2019

Ctucker9233 commented Jun 28, 2019

lracicot commented Jul 12, 2019

driskell commented Aug 6, 2019

manish3m commented Aug 10, 2019

manish3m commented Aug 10, 2019

m2-assistant bot commented Aug 12, 2019 •

edited by engcom-Charlie

Loading

magento-engcom-team commented Aug 12, 2019

hostep commented Aug 12, 2019

engcom-Charlie commented Aug 12, 2019

engcom-Charlie commented Aug 12, 2019

engcom-Charlie commented Aug 12, 2019 •

edited

Loading

onlinebizsoft commented Aug 12, 2019

hostep commented Aug 12, 2019

engcom-Charlie commented Aug 13, 2019 •

edited

Loading

onlinebizsoft commented Oct 25, 2019

engcom-Charlie commented Oct 25, 2019

onlinebizsoft commented Oct 25, 2019

engcom-Charlie commented Oct 25, 2019

onlinebizsoft commented Oct 25, 2019

vincentteyssier commented Nov 20, 2019

Adel-Magebinary commented Jan 22, 2020

Adel-Magebinary commented Jan 22, 2020

magento-engcom-team commented Dec 22, 2020

ctadlock commented Apr 16, 2025

Magento 2.3.1 Cron Deadlocks for cron_schedule #22438

Magento 2.3.1 Cron Deadlocks for cron_schedule #22438

Comments

seansan commented Apr 20, 2019 • edited Loading

Preconditions (*)

Steps to reproduce (*)

Expected result (*)

Actual result (*)

m2-assistant bot commented Apr 20, 2019 • edited by kozie Loading

m2-assistant bot commented Apr 22, 2019

ghost commented Apr 22, 2019

seansan commented Apr 22, 2019 via email

seansan commented Apr 24, 2019

onlinebizsoft commented Jun 28, 2019

Ctucker9233 commented Jun 28, 2019

lracicot commented Jul 12, 2019

driskell commented Aug 6, 2019

manish3m commented Aug 10, 2019

manish3m commented Aug 10, 2019

m2-assistant bot commented Aug 12, 2019 • edited by engcom-Charlie Loading

magento-engcom-team commented Aug 12, 2019

hostep commented Aug 12, 2019

engcom-Charlie commented Aug 12, 2019

engcom-Charlie commented Aug 12, 2019

engcom-Charlie commented Aug 12, 2019 • edited Loading

onlinebizsoft commented Aug 12, 2019

hostep commented Aug 12, 2019

engcom-Charlie commented Aug 13, 2019 • edited Loading

onlinebizsoft commented Oct 25, 2019

engcom-Charlie commented Oct 25, 2019

onlinebizsoft commented Oct 25, 2019

engcom-Charlie commented Oct 25, 2019

onlinebizsoft commented Oct 25, 2019

vincentteyssier commented Nov 20, 2019

Adel-Magebinary commented Jan 22, 2020

Adel-Magebinary commented Jan 22, 2020

magento-engcom-team commented Dec 22, 2020

ctadlock commented Apr 16, 2025

seansan commented Apr 20, 2019 •

edited

Loading

m2-assistant bot commented Apr 20, 2019 •

edited by kozie

Loading

m2-assistant bot commented Aug 12, 2019 •

edited by engcom-Charlie

Loading

engcom-Charlie commented Aug 12, 2019 •

edited

Loading

engcom-Charlie commented Aug 13, 2019 •

edited

Loading