Module Catalog: product "Save and Duplicate" causes getting infinite loop

[oleksii-lisovyi]

Preconditions

1. Magento 2.2.x.
2. You should have product(s) with existing URL rewrites.
   2.1. In our case there are products migrated from Magento 1, but it's no necessary.

Steps to reproduce

1. Go to Admin panel -> Catalog -> Inventory -> Products.
2. Open a product with existing URL rewrite.
3. Click on "Save and Duplicate" button.

Expected result

Original product is saved and an error message is displayed for Admin user.

Actual result

1. Admin user observes infinitely loading in browser and decides to close the browser tab.
2. There is infinite PHP process, that causes next issues:

• there are duplicated images creating process, that increase the media storage volume quite fast (see Duplicating product copies product images couple of hundred times #9466, for example);
• there are multiple cache invalidate requests to the caching server (Redis in our case), that increases CPU load by only 1 product saving (see Product Duplication crash #4096, for example).

3. [extra issue causes by this one] making any changes, that requires running catalog reindex process, for example adding/removing product(s) from a category etc., causes getting MySQL Deadlocks, that leads to products are disappearing from the website, because of broken index.

Technical details

The issue is caused by incorrect implementation of \Magento\Catalog\Model\Product\Copier::copy():

// ...
do {
    $urlKey = $duplicate->getUrlKey();
    $urlKey = preg_match('/(.*)-(\d+)$/', $urlKey, $matches)
        ? $matches[1] . '-' . ($matches[2] + 1)
        : $urlKey . '-1';
    $duplicate->setUrlKey($urlKey);
    try {
        $duplicate->save();
        $isDuplicateSaved = true;
    } catch (\Magento\Framework\Exception\AlreadyExistsException $e) {
    } catch (\Magento\UrlRewrite\Model\Exception\UrlAlreadyExistsException $e) {
        // Here is the exception is catched infinitely
    }
} while (!$isDuplicateSaved);
// ...

Here is a callstack of the exception:

PDOException: SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '<some product index created>' for key 'URL_REWRITE_REQUEST_PATH_STORE_ID' in <magento_root>/vendor/magento/zendframework1/library/Zend/Db/Statement/Pdo.php:228 Stack trace: 
#0 <magento_root>/vendor/magento/zendframework1/library/Zend/Db/Statement/Pdo.php(228): PDOStatement->execute(Array) 
#1 <magento_root>/vendor/magento/framework/DB/Statement/Pdo/Mysql.php(93): Zend_Db_Statement_Pdo->_execute(Array) 
#2 <magento_root>/vendor/magento/zendframework1/library/Zend/Db/Statement.php(303): Magento\Framework\DB\Statement\Pdo\Mysql->_execute(Array) 
#3 <magento_root>/vendor/magento/zendframework1/library/Zend/Db/Adapter/Abstract.php(480): Zend_Db_Statement->execute(Array) 
#4 <magento_root>/vendor/magento/zendframework1/library/Zend/Db/Adapter/Pdo/Abstract.php(238): Zend_Db_Adapter_Abstract->query('INSERT INTO `u...', Array) 
#5 <magento_root>/vendor/magento/framework/DB/Adapter/Pdo/Mysql.php(533): Zend_Db_Adapter_Pdo_Abstract->query('INSERT INTO `u...', Array) 
#6 <magento_root>/vendor/magento/framework/DB/Adapter/Pdo/Mysql.php(596): Magento\Framework\DB\Adapter\Pdo\Mysql->_query('INSERT INTO `u...', Array) 
#7 <magento_root>/vendor/magento/framework/DB/Adapter/Pdo/Mysql.php(2021): Magento\Framework\DB\Adapter\Pdo\Mysql->query('INSERT INTO `u...', Array) 
#8 <magento_root>/vendor/magento/framework/DB/Adapter/Pdo/Mysql.php(1978): Magento\Framework\DB\Adapter\Pdo\Mysql->insertArray('url_rewrite', Array, Array) 
#9 <magento_root>/vendor/magento/module-url-rewrite/Model/Storage/DbStorage.php(245): Magento\Framework\DB\Adapter\Pdo\Mysql->insertMultiple('url_rewrite', Array) 
#10 <magento_root>/vendor/magento/module-url-rewrite/Model/Storage/DbStorage.php(204): Magento\UrlRewrite\Model\Storage\DbStorage->insertMultiple(Array) 
#11 <magento_root>/vendor/magento/module-url-rewrite/Model/Storage/AbstractStorage.php(87): Magento\UrlRewrite\Model\Storage\DbStorage->doReplace(Array) 
#12 <magento_root>/vendor/magento/framework/Interception/Interceptor.php(58): Magento\UrlRewrite\Model\Storage\AbstractStorage->replace(Array) 
#13 <magento_root>/vendor/magento/framework/Interception/Interceptor.php(138): Magento\UrlRewrite\Model\Storage\DbStorage\Interceptor->___callParent('replace', Array) 
#14 <magento_root>/vendor/magento/framework/Interception/Interceptor.php(153): Magento\UrlRewrite\Model\Storage\DbStorage\Interceptor->Magento\Framework\Interception\{closure}(Array) 
#15 <magento_root>/generated/code/Magento/UrlRewrite/Model/Storage/DbStorage/Interceptor.php(39): Magento\UrlRewrite\Model\Storage\DbStorage\Interceptor->___callPlugins('replace', Array, Array) 
#16 <magento_root>/vendor/magento/module-catalog-url-rewrite/Observer/ProductProcessUrlRewriteSavingObserver.php(54): Magento\UrlRewrite\Model\Storage\DbStorage\Interceptor->replace(Array) 
#17 <magento_root>/vendor/magento/framework/Event/Invoker/InvokerDefault.php(72): Magento\CatalogUrlRewrite\Observer\ProductProcessUrlRewriteSavingObserver->execute(Object(Magento\Framework\Event\Observer)) 
#18 <magento_root>/vendor/magento/framework/Event/Invoker/InvokerDefault.php(60): Magento\Framework\Event\Invoker\InvokerDefault->_callObserverMethod(Object(Magento\CatalogUrlRewrite\Observer\ProductProcessUrlRewriteSavingObserver), Object(Magento\Framework\Event\Observer)) 
#19 <magento_root>/vendor/magento/framework/Event/Manager.php(66): Magento\Framework\Event\Invoker\InvokerDefault->dispatch(Array, Object(Magento\Framework\Event\Observer)) 
#20 <magento_root>/generated/code/Magento/Framework/Event/Manager/Proxy.php(95): Magento\Framework\Event\Manager->dispatch('catalog_product...', Array) 
#21 <magento_root>/vendor/magento/framework/Model/AbstractModel.php(818): Magento\Framework\Event\Manager\Proxy->dispatch('catalog_product...', Array) 
#22 <magento_root>/vendor/magento/module-catalog/Model/Product.php(929): Magento\Framework\Model\AbstractModel->afterSave() 
#23 <magento_root>/vendor/magento/framework/EntityManager/Observer/AfterEntitySave.php(31): Magento\Catalog\Model\Product->afterSave() 
#24 <magento_root>/vendor/magento/framework/Event/Invoker/InvokerDefault.php(72): Magento\Framework\EntityManager\Observer\AfterEntitySave->execute(Object(Magento\Framework\Event\Observer)) 
#25 <magento_root>/vendor/magento/framework/Event/Invoker/InvokerDefault.php(60): Magento\Framework\Event\Invoker\InvokerDefault->_callObserverMethod(Object(Magento\Framework\EntityManager\Observer\AfterEntitySave), Object(Magento\Framework\Event\Observer)) 
#26 <magento_root>/vendor/magento/framework/Event/Manager.php(66): Magento\Framework\Event\Invoker\InvokerDefault->dispatch(Array, Object(Magento\Framework\Event\Observer)) 
#27 <magento_root>/generated/code/Magento/Framework/Event/Manager/Proxy.php(95): Magento\Framework\Event\Manager->dispatch('magento_catalog...', Array) 
#28 <magento_root>/vendor/magento/framework/EntityManager/EventManager.php(52): Magento\Framework\Event\Manager\Proxy->dispatch('magento_catalog...', Array) 
#29 <magento_root>/vendor/magento/framework/EntityManager/Operation/Create.php(123): Magento\Framework\EntityManager\EventManager->dispatchEntityEvent('Magento\\Catalog...', 'save_after', Array) 
#30 <magento_root>/vendor/magento/framework/EntityManager/EntityManager.php(96): Magento\Framework\EntityManager\Operation\Create->execute(Object(Magento\Catalog\Model\Product\Interceptor), Array) 
#31 <magento_root>/vendor/magento/module-catalog/Model/ResourceModel/Product.php(655): Magento\Framework\EntityManager\EntityManager->save(Object(Magento\Catalog\Model\Product\Interceptor)) 
#32 <magento_root>/vendor/magento/framework/Interception/Interceptor.php(58): Magento\Catalog\Model\ResourceModel\Product->save(Object(Magento\Catalog\Model\Product\Interceptor)) 
#33 <magento_root>/vendor/magento/framework/Interception/Interceptor.php(138): Magento\Catalog\Model\ResourceModel\Product\Interceptor->___callParent('save', Array) 
#34 <magento_root>/vendor/magento/module-catalog-search/Model/Indexer/Fulltext/Plugin/Product.php(51): Magento\Catalog\Model\ResourceModel\Product\Interceptor->Magento\Framework\Interception\{closure}(Object(Magento\Catalog\Model\Product\Interceptor)) 
#35 <magento_root>/vendor/magento/module-catalog-search/Model/Indexer/Fulltext/Plugin/Product.php(24): Magento\CatalogSearch\Model\Indexer\Fulltext\Plugin\Product->addCommitCallback(Object(Magento\Catalog\Model\ResourceModel\Product\Interceptor), Object(Closure), Object(Magento\Catalog\Model\Product\Interceptor)) 
#36 <magento_root>/vendor/magento/framework/Interception/Interceptor.php(135): Magento\CatalogSearch\Model\Indexer\Fulltext\Plugin\Product->aroundSave(Object(Magento\Catalog\Model\ResourceModel\Product\Interceptor), Object(Closure), Object(Magento\Catalog\Model\Product\Interceptor)) 
#37 <magento_root>/vendor/magento/framework/App/Cache/FlushCacheByTags.php(68): Magento\Catalog\Model\ResourceModel\Product\Interceptor->Magento\Framework\Interception\{closure}(Object(Magento\Catalog\Model\Product\Interceptor)) 
#38 <magento_root>/vendor/magento/framework/Interception/Interceptor.php(135): Magento\Framework\App\Cache\FlushCacheByTags->aroundSave(Object(Magento\Catalog\Model\ResourceModel\Product\Interceptor), Object(Closure), Object(Magento\Catalog\Model\Product\Interceptor)) 
#39 <magento_root>/vendor/magento/framework/Interception/Interceptor.php(153): Magento\Catalog\Model\ResourceModel\Product\Interceptor->Magento\Framework\Interception\{closure}(Object(Magento\Catalog\Model\Product\Interceptor)) 
#40 <magento_root>/generated/code/Magento/Catalog/Model/ResourceModel/Product/Interceptor.php(39): Magento\Catalog\Model\ResourceModel\Product\Interceptor->___callPlugins('save', Array, Array) 
#41 <magento_root>/vendor/magento/framework/Model/AbstractModel.php(647): Magento\Catalog\Model\ResourceModel\Product\Interceptor->save(Object(Magento\Catalog\Model\Product\Interceptor)) 
#42 <magento_root>/vendor/magento/module-catalog/Model/Product/Copier.php(87): Magento\Framework\Model\AbstractModel->save() 
#43 <magento_root>/vendor/magento/module-catalog/Controller/Adminhtml/Product/Save.php(144): Magento\Catalog\Model\Product\Copier->copy(Object(Magento\Catalog\Model\Product\Interceptor)) 
#44 <magento_root>/vendor/magento/framework/Interception/Interceptor.php(58): Magento\Catalog\Controller\Adminhtml\Product\Save->execute() 
#45 <magento_root>/vendor/magento/framework/Interception/Interceptor.php(138): Magento\Catalog\Controller\Adminhtml\Product\Save\Interceptor->___callParent('execute', Array) 
#46 <magento_root>/vendor/magento/framework/Interception/Interceptor.php(153): Magento\Catalog\Controller\Adminhtml\Product\Save\Interceptor->Magento\Framework\Interception\{closure}() 
#47 <magento_root>/generated/code/Magento/Catalog/Controller/Adminhtml/Product/Save/Interceptor.php(26): Magento\Catalog\Controller\Adminhtml\Product\Save\Interceptor->___callPlugins('execute', Array, Array) 
#48 <magento_root>/vendor/magento/framework/App/Action/Action.php(107): Magento\Catalog\Controller\Adminhtml\Product\Save\Interceptor->execute() 
#49 <magento_root>/vendor/magento/module-backend/App/AbstractAction.php(229): Magento\Framework\App\Action\Action->dispatch(Object(Magento\Framework\App\Request\Http)) #50 <magento_root>/vendor/magento/framework/Interception/Interceptor.php(58): Magento\Backend\App\AbstractAction->dispatch(Object(Magento\Framework\App\Request\Http)) 
#51 <magento_root>/vendor/magento/framework/Interception/Interceptor.php(138): Magento\Catalog\Controller\Adminhtml\Product\Save\Interceptor->___callParent('dispatch', Array) 
#52 <magento_root>/vendor/magento/module-backend/App/Action/Plugin/Authentication.php(143): Magento\Catalog\Controller\Adminhtml\Product\Save\Interceptor->Magento\Framework\Interception\{closure}(Object(Magento\Framework\App\Request\Http)) 
#53 <magento_root>/vendor/magento/framework/Interception/Interceptor.php(135): Magento\Backend\App\Action\Plugin\Authentication->aroundDispatch(Object(Magento\Catalog\Controller\Adminhtml\Product\Save\Interceptor), Object(Closure), Object(Magento\Framework\App\Request\Http)) 
#54 <magento_root>/vendor/magento/framework/Interception/Interceptor.php(153): Magento\Catalog\Controller\Adminhtml\Product\Save\Interceptor->Magento\Framework\Interception\{closure}(Object(Magento\Framework\App\Request\Http)) 
#55 <magento_root>/generated/code/Magento/Catalog/Controller/Adminhtml/Product/Save/Interceptor.php(39): Magento\Catalog\Controller\Adminhtml\Product\Save\Interceptor->___callPlugins('dispatch', Array, NULL) 
#56 <magento_root>/vendor/magento/framework/App/FrontController.php(55): Magento\Catalog\Controller\Adminhtml\Product\Save\Interceptor->dispatch(Object(Magento\Framework\App\Request\Http)) 
#57 <magento_root>/vendor/magento/framework/Interception/Interceptor.php(58): Magento\Framework\App\FrontController->dispatch(Object(Magento\Framework\App\Request\Http)) 
#58 <magento_root>/vendor/magento/framework/Interception/Interceptor.php(138): Magento\Framework\App\FrontController\Interceptor->___callParent('dispatch', Array) 
#59 <magento_root>/vendor/magento/framework/Interception/Interceptor.php(153): Magento\Framework\App\FrontController\Interceptor->Magento\Framework\Interception\{closure}(Object(Magento\Framework\App\Request\Http)) 
#60 <magento_root>/generated/code/Magento/Framework/App/FrontController/Interceptor.php(26): Magento\Framework\App\FrontController\Interceptor->___callPlugins('dispatch', Array, Array) 
#61 <magento_root>/vendor/magento/framework/App/Http.php(135): Magento\Framework\App\FrontController\Interceptor->dispatch(Object(Magento\Framework\App\Request\Http)) 
#62 <magento_root>/vendor/magento/framework/App/Bootstrap.php(256): Magento\Framework\App\Http->launch() 
#63 <magento_root>/pub/index.php(37): Magento\Framework\App\Bootstrap->run(Object(Magento\Framework\App\Http\Interceptor)) 

Related issues

#9466
#4096
#12412

[magento-engcom-team]

Hi @oleksii-lisovyi. Thank you for your report.
To help us process this issue please make sure that you provided the following information:

• Summary of the issue
• Information on your environment
• Steps to reproduce
• Expected and actual results

Please make sure that the issue is reproducible on the vanilla Magento instance following Steps to reproduce. To deploy vanilla Magento instance on our environment, please, add a comment to the issue:

@magento-engcom-team give me $VERSION instance

where $VERSION is version tags (starting from 2.2.0+) or develop branches (for example: 2.3-develop).
For more details, please, review the Magento Contributor Assistant documentation.

@oleksii-lisovyi do you confirm that you was able to reproduce the issue on vanilla Magento instance following steps to reproduce?

• yes
• no

[magento-engcom-team]

Hi @oleksii-lisovyi. Thank you for your report.
The issue has been fixed in #18566 by @oleksii-lisovyi in 2.2-develop branch
Related commit(s):

• 7169f0442658ec21c6a5c215269d8bafcff71093
• 9b0586e96248c6753688a417cde68a89e532907c

The fix will be available with the upcoming 2.2.8 release.

[magento-engcom-team]

Hi @oleksii-lisovyi. Thank you for your report.
The issue has been fixed in #19170 by @p-bystritsky in 2.3-develop branch
Related commit(s):

• 284daec007cc9cc5bacc736e3d6ce00ebdf751ef

The fix will be available with the upcoming 2.3.1 release.

[Grumag]

Well, this issue happened yesterday to our site that is running 2.3.5-p1. It was a 1.9 migration, to 2.3.3 fresh install, which was later upgraded to 2.3.5-p1.

1. Duplicated a product and got an infinite loop, closed the tab, and did it again and got the same issue
2. After few hours 15gb of new data was created

So not fixed?

[hostep]

@Grumag: according to this comment, it should get fixed finally in Magento 2.4.0, no idea why they didn't backport the fix to 2.3.x ...

Maybe you can try to apply the changes from #25875 to your installation and see if that solves it?