Import Speed Issues - What LAMP stack works?

[hgpit]

Hi all,

Sorry for the odd subject title. Didn't know how to word it!

I have already posted on this forum with speed issues relating to importing products, but I am getting nowhere fast and my deadline for a usable platform is quickly approaching. There is clearly a problem with Magento 2.0.x but it's either some random/unknown variable with our builds or some bad code deep within M2.

We have tried (physical and VMs) CentOS 6.7, Apache 2.2, MariaDB 10/10.1/MySQL 5.6, PHP5.6/7, Magento CE/EE 2/2.0.1 in various combinations and every time our product imports start fast (between 1000~2500 per minute) but then crawls to ~14 per minute (MySQL looks idle while the import is running but HTTPD is constantly at 100% CPU). We can not go into production with this problem.

In other posts you have said that you cannot replicate this issue, but I am now on my 12th build and see the same problem.

Please can you tell me exactly what combination of hardware/software (OS/Apache/MySQL/PHP) you use when testing M2 and importing up to 80,000 products? Have you really tested it with this many products (each with images)?

From all our testing, it seems that M2 is incapable of handling the import of more than ~10000 products before winding down to a crawl.

Feedback very much appreciated.

Best Regards,

Elliot

[daim2k5]

Hi @hgpit because you don't link the other issue can you give us a link? How do you import stuff? Soap or Import/export? and maybe important can you maybe create a playground for the magento 2 team? So they can reproduce the issue?

[ktomk]

HTTPD is constantly at 100% CPU

Are you importing over HTTP or via CLI? Just asking because I can't imagine someone importing 80k products via HTTP at all but the high CPU use on HTTPD suggests that.

[andimov]

According to contributor guide, tickets without response for two weeks should be closed.
Please feel free to reopen if it's needed.