Cron may double schedule jobs in edge case

[scottsb]

Preconditions

Magento 2.1.3 (but I can see problem code has not changed in develop branch)

Steps to reproduce

These are a fairly artificial set of steps to reproduce the issue consistently, but the same situation can happen "in the wild" if the timings line up just right:

1. Identify a cron job that takes a while to run (over a minute).
2. Watch for selected job to run.
3. While selected job is running, and during the same minute on the clock that original job was scheduled, flush cache and run cron again, forcing scheduler to run.

Expected result

When scheduler runs in step 3 above, it schedules a job for the next time increment beyond the cron run that is currently still in process.

Actual result

Instead, scheduler creates a pending job at the exact same time as the cron job that is in progress. Example result in the cron_schedule table:

Root cause analysis

The problem is that when ProcessCronQueueObserver::_generate() assembles the array of existing cron schedules that later will be skipped in saveSchedule(), it includes jobs only in pending status, not running status. At a minimum it should include running jobs as well, and I would probably argue it shouldn't pay attention to the status of the job at all: if a record for the job exists in cron_schedule for the given time, regardless of status, it shouldn't ever be rescheduled for the same time.

[magento-engcom-team]

@scottsb thank you for your report.

Closed as duplicate #10650