Bug - Magento_Eav - eav_attribute_group (repository) filtering

[LucianMihalache]

There is a bug on the Magento_Eav Module, to be more precise, there is a bug when you want to filter the attribute_group collection by multiple values (IN).

*The bug affects rest api too

Preconditions (*)

1. Any Magento 2.3.1 that does not overwrite Magento_Eav module
2. Some data on the site, I tested with sampledata

Steps to reproduce (*)

1. Try to filter the eav_attribute_group (AttributeGroupRepository) by multiple attribute_set_id's or by multiple attribute_group_code

If you try to filter by attribute_group_code, you won't be able to filter at all.

/**
 * @var \Magento\Eav\Api\AttributeGroupRepositoryInterface
 */
protected $attributeGroupRepository;

/**
 * @var \Magento\Framework\Api\Search\SearchCriteriaBuilderFactory
 */
protected $searchCriteriaBuilderFactory;

/**
 * @var \Magento\Framework\Api\FilterBuilder
 */
protected $filterBuilder;

public function __construct(
    \Magento\Eav\Api\AttributeGroupRepositoryInterface $attributeGroupRepository,
    \Magento\Framework\Api\Search\SearchCriteriaBuilderFactory >$searchCriteriaBuilderFactory,
    \Magento\Framework\Api\FilterBuilder $filterBuilder
) {
    $this->attributeGroupRepository = $attributeGroupRepository;
    $this->searchCriteriaBuilderFactory = $searchCriteriaBuilderFactory;
    $this->filterBuilder = $filterBuilder;
}

public function test()
{
    /** @var \Magento\Framework\Api\Search\SearchCriteriaBuilder $criteriaBuilder */
    $criteriaBuilder = $this->searchCriteriaBuilderFactory->create();
    $setIdFilter = $this->filterBuilder
        ->setField('attribute_set_id')
        ->setValue('15,10,4,14,11,12,13,9') // All the attribute_set_id from that are present on clean install with sampledata
        ->setConditionType('in')
        ->create();

    $groupCodeFilter = $this->filterBuilder
        ->setField('attribute_group_code')
        ->setValue('"advanced-pricing", "autosettings", "bundle-items", "content"')
        ->setConditionType('in')
        ->create();

/* I am using directly imploded strings because this is the format I need when I am doing rest api 
   get, and since is the same repository, it works the same way.*/

    $criteria = $criteriaBuilder->addFilter($setIdFilter)->create();
    //$criteria = $criteriaBuilder->addFilter($groupCodeFilter)->create();

    $groupList = $this->attributeGroupRepository->getList($criteria);
    //Do something with $groupList
    var_dump($groupList->getSearchCriteria()->getFilterGroups()[0]->getFilters());
    var_dump($groupList->getTotalCount());
    foreach ($groupList->getItems() as $group) {
        var_dump('AttributeSetId: ' . $group->getAttributeSetId());
    }
    die('asdfasfasdf');
}

Expected result (*)

1. Returning all the groups assigned to all the attribute_set_id's given.
   In the database, that means, 80 records (sampledata)

Actual result (*)

    ....php:102:
    array (size=1)
   0 => 
     object(Magento\Framework\Api\Filter)[1784]
       protected '_data' => 
         array (size=3)
           'field' => string 'attribute_set_id' (length=16)
           'value' => string '15,10,4,14,11,12,13,9' (length=21)
           'condition_type' => string 'in' (length=2)
     ....php:103:int 10
     ....php:105:string 'AttributeSetId: 15' (length=18)
     ....php:105:string 'AttributeSetId: 15' (length=18)
     ....php:105:string 'AttributeSetId: 15' (length=18)
     ....php:105:string 'AttributeSetId: 15' (length=18)
     ....php:105:string 'AttributeSetId: 15' (length=18)
     ....php:105:string 'AttributeSetId: 15' (length=18)
     ....php:105:string 'AttributeSetId: 15' (length=18)
     ....php:105:string 'AttributeSetId: 15' (length=18)
     ....php:105:string 'AttributeSetId: 15' (length=18)
     ....php:105:string 'AttributeSetId: 15' (length=18)

The query string after applying the filters:

 SELECT `main_table`.* FROM `eav_attribute_group` AS `main_table` WHERE (`attribute_set_id` = '15,10,4,14,11,12,13,9')

The problem, as I said in some other post (#2892 (comment)) is:

After some debugging, I found the problem:
When the AttributeGroupRepository processes the collection, the process goes through:

vendor/magento/framework/Api/SearchCriteria/CollectionProcessor/FilterProcessor.php
    addFilterGroupToCollection(...)

Here, on the line 66-70:

        $customFilter = $this->getCustomFilterForField($filter->getField());
        if ($customFilter) {
            $isApplied = $customFilter->apply($filter, $collection);
        }

There IS a custom field for my search (attribute_set_id), and the code goes in:

 vendor/magento/module-eav/Model/Api/SearchCriteria/CollectionProcessor/FilterProcessor/AttributeGroupAttributeSetIdFilter.php

Here there is:

public function apply(Filter $filter, AbstractDb $collection)
{
    /** @var \Magento\Eav\Model\ResourceModel\Entity\Attribute\Group\Collection $collection */
    $collection->setAttributeSetFilter($filter->getValue());

    return true;
}

Which goes on:

 vendor/magento/module-eav/Model/ResourceModel/Entity/Attribute/Group/Collection.php

public function setAttributeSetFilter($setId)
{
    $this->addFieldToFilter('attribute_set_id', ['eq' => $setId]);
    $this->setOrder('sort_order');
    return $this;
}

Here we can see that the eav_attribute_group collection does not even care about what we want us to do with the filter, it simply forces us on equal.

I don't want to believe that this was intentional, but instead, I believe that this was some old code that was done before the whole process-filter-thing, and it was forgotten.

The same goes for all filters from the eav_attribute_group:

      vendor/magento/module-eav/Model/Api/SearchCriteria/CollectionProcessor/FilterProcessor/AttributeGroupAttributeSetIdFilter.php

vendor/magento/module-eav/Model/Api/SearchCriteria/CollectionProcessor/FilterProcessor/AttributeGroupCodeFilter.php

vendor/magento/module-eav/Model/Api/SearchCriteria/CollectionProcessor/FilterProcessor/AttributeSetEntityTypeCodeFilter.php

All of them, don't bother to see what condition type we want to use on the collection.

For fixes, I think that the best solution is to let the processor do its job. That means:

vendor/magento/module-eav/etc/di.xml

Remove the lines 184-154:

<virtualType name="Magento\Eav\Model\Api\SearchCriteria\CollectionProcessor\AttributeSetFilterProcessor" type="Magento\Framework\Api\SearchCriteria\CollectionProcessor\FilterProcessor">
    <arguments>
        <argument name="customFilters" xsi:type="array">
            <item name="entity_type_code" xsi:type="object">Magento\Eav\Model\Api\SearchCriteria\CollectionProcessor\FilterProcessor\AttributeSetEntityTypeCodeFilter</item>
        </argument>
    </arguments>
</virtualType>

Remove the lines: 169-179

<virtualType name="Magento\Eav\Model\Api\SearchCriteria\CollectionProcessor\AttributeGroupFilterProcessor" type="Magento\Framework\Api\SearchCriteria\CollectionProcessor\FilterProcessor">
    <arguments>
        <argument name="fieldMapping" xsi:type="array">
            <item name="attribute_group_code" xsi:type="string">main_table.attribute_group_code</item>
        </argument>
        <argument name="customFilters" xsi:type="array">
            <item name="attribute_set_id" xsi:type="object">Magento\Eav\Model\Api\SearchCriteria\CollectionProcessor\FilterProcessor\AttributeGroupAttributeSetIdFilter</item>
            <item name="attribute_group_code" xsi:type="object">Magento\Eav\Model\Api\SearchCriteria\CollectionProcessor\FilterProcessor\AttributeGroupCodeFilter</item>
        </argument>
    </arguments>
</virtualType>

And of course, remove the not used (anymore) code...

[m2-assistant]

Hi @LucianMihalache. Thank you for your report.
To help us process this issue please make sure that you provided the following information:

• Summary of the issue
• Information on your environment
• Steps to reproduce
• Expected and actual results

Please make sure that the issue is reproducible on the vanilla Magento instance following Steps to reproduce. To deploy vanilla Magento instance on our environment, please, add a comment to the issue:

@magento give me 2.3-develop instance - upcoming 2.3.x release

For more details, please, review the Magento Contributor Assistant documentation.

@LucianMihalache do you confirm that you were able to reproduce the issue on vanilla Magento instance following steps to reproduce?

• yes
• no

---

• Join Magento Community Engineering Slack and ask your questions in #github channel.

[LucianMihalache]

@magento-engcom-team Yes, I can reproduce it on multiple instances of 2.3.1
Can somebody work on this?
There is a little problem because I need to do api requests, and by doing N requests to get some attribute groups is really... inneficient, nobody wants to wait 10-20 seconds just to do some checks on the other end...

[m2-assistant]

Hi @engcom-Charlie. Thank you for working on this issue.
In order to make sure that issue has enough information and ready for development, please read and check the following instruction: 👇

• 1. Verify that issue has all the required information. (Preconditions, Steps to reproduce, Expected result, Actual result).

  Details

  If the issue has a valid description, the label Issue: Format is valid will be added to the issue automatically. Please, edit issue description if needed, until label Issue: Format is valid appears.

• 2. Verify that issue has a meaningful description and provides enough information to reproduce the issue. If the report is valid, add Issue: Clear Description label to the issue by yourself.

• 3. Add Component: XXXXX label(s) to the ticket, indicating the components it may be related to.

• 4. Verify that the issue is reproducible on 2.3-develop branch

  Details

  - Add the comment @magento give me 2.3-develop instance to deploy test instance on Magento infrastructure.
  - If the issue is reproducible on 2.3-develop branch, please, add the label Reproduced on 2.3.x.
  - If the issue is not reproducible, add your comment that issue is not reproducible and close the issue and stop verification process here!

• 5. Verify that the issue is reproducible on 2.2-develop branch.

  Details

  - Add the comment @magento give me 2.2-develop instance to deploy test instance on Magento infrastructure.
  - If the issue is reproducible on 2.2-develop branch, please add the label Reproduced on 2.2.x

• 6. Add label Issue: Confirmed once verification is complete.

• 7. Make sure that automatic system confirms that report has been added to the backlog.

[engcom-Charlie]

Hello @LucianMihalache ! Thank you for your report !
I`ve just tested this on Vanilla Magento 2.3-develop without Sample Data and got this result

[LucianMihalache]

@engcom-Charlie so, for you, a filter of AttributeGroupRepository by multiple attribute_set_id applied the correct filter, 'IN' ?
Can I get a develop instance?
I wanna try a rest api call for that repository to see if the response is the same.
What is the exact version of your instance? Release 2.3.1, or the develop version?

[LucianMihalache]

@magento give me 2.3-develop instance

[magento-engcom-team]

Hi @LucianMihalache. Thank you for your request. I'm working on Magento 2.3-develop instance for you

[magento-engcom-team]

Hi @LucianMihalache, here is your Magento instance.
Admin access: https://i-23172-2-3-develop.instances.magento-community.engineering/admin
Login: admin Password: 123123q
Instance will be terminated in up to 3 hours.

[LucianMihalache]

@engcom-Charlie , Look, on the develop instance, run this curl:

This one is for you to see how many groups are there, and note their attribute_set_id.
I created 3 more attribute_sets

And the curl is to get all the groups assigned to the attribute sets.
I made it with swagger.
https://i-23172-2-3-develop.instances.magento-community.engineering/swagger
Api key: lbpbbo4he8kp8sit9ln4uwkj7t9scfb1

Firstly, lets get all the groups of all attribute_sets that exists on the site (4,9,10,11)

And the result:

As you can see, there are no groups to any other attribute set.

Than, lets see how many groups are there (including the customer, etc..., we will just ignore them)

To do that, we will just sort the group table.

And the result:

As you can see, there all our attribute sets. But, the search for all of them, does not work.
It will give you only the first value searched in our query: 4,9,10,11 - that means, the value 4.
Because the query run into the database by the repository looks like this:

Select * from eav_attribute_group where attribute_set_id = '4,9,10,11';

And the problem (+answer) why there is '=' instead of 'in', is written above.

curl -X GET "https://i-23172-2-3-develop.instances.magento-community.engineering/rest/all/V1/products/attribute-sets/groups/list?searchCriteria%5BfilterGroups%5D%5B0%5D%5Bfilters%5D%5B0%5D%5Bfield%5D=attribute_set_id&searchCriteria%5BfilterGroups%5D%5B0%5D%5Bfilters%5D%5B0%5D%5Bvalue%5D=4%2C9%2C10%2C11&searchCriteria%5BfilterGroups%5D%5B0%5D%5Bfilters%5D%5B0%5D%5BconditionType%5D=in&searchCriteria%5BsortOrders%5D%5B0%5D%5Bfield%5D=attribute_set_id&searchCriteria%5BsortOrders%5D%5B0%5D%5Bdirection%5D=asc" -H "accept: application/json" -H "Authorization: Bearer lbpbbo4he8kp8sit9ln4uwkj7t9scfb1"