Skip to content

Bug: Feature Flags - Faulty Test Cases #2058

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
ajwad-shaikh opened this issue Mar 28, 2023 · 1 comment · Fixed by #2059
Closed

Bug: Feature Flags - Faulty Test Cases #2058

ajwad-shaikh opened this issue Mar 28, 2023 · 1 comment · Fixed by #2059
Labels
bug Something isn't working

Comments

@ajwad-shaikh
Copy link
Contributor

ajwad-shaikh commented Mar 28, 2023

Expected Behaviour

Feature Flags match type test cases must have the non-expected value as default and the condition when_match must be equal to expected_value

reference - #2052 (review)

Current Behaviour

For the list of test cases below, the expected_value is assigned as the default value which is returned when the condition does not match. The test only passes because both the expected and non-expected values are the same - True meaning that the test cannot fail ever.

  • test_flags_not_equal_match
  • test_flags_less_than_match
  • test_flags_less_than_or_equal_match_1
  • test_flags_less_than_or_equal_match_2
  • test_flags_greater_than_match
  • test_flags_greater_than_or_equal_match_1
  • test_flags_greater_than_or_equal_match_2

Code snippet

# One of the test cases from above list

def test_flags_not_equal_match(mocker, config):
    expected_value = True
    mocked_app_config_schema = {
        "my_feature": {
            "default": expected_value,
            "rules": {
                "tenant id not equals 345345435": {
                    "when_match": True,
                    "conditions": [
                        {
                            "action": RuleAction.NOT_EQUALS.value,
                            "key": "tenant_id",
                            "value": "345345435",
                        }
                    ],
                }
            },
        }
    }
    feature_flags = init_feature_flags(mocker, mocked_app_config_schema, config)
    toggle = feature_flags.evaluate(name="my_feature", context={"tenant_id": "", "username": "a"}, default=False)
    assert toggle == expected_value

Possible Solution

For match cases - expected_value, must be assigned to the when_match of condition rather than default and default must return False

def test_flags_not_equal_match(mocker, config):
    expected_value = True
    mocked_app_config_schema = {
        "my_feature": {
            "default": False,
            "rules": {
                "tenant id not equals 345345435": {
                    "when_match": expected_value,
                    "conditions": [
                        {
                            "action": RuleAction.NOT_EQUALS.value,
                            "key": "tenant_id",
                            "value": "345345435",
                        }
                    ],
                }
            },
        }
    }
    feature_flags = init_feature_flags(mocker, mocked_app_config_schema, config)
    toggle = feature_flags.evaluate(name="my_feature", context={"tenant_id": "", "username": "a"}, default=False)
    assert toggle == expected_value

Steps to Reproduce

N/A

AWS Lambda Powertools for Python version

latest

AWS Lambda function runtime

3.6

Packaging format used

PyPi

Debugging logs

No response

@ajwad-shaikh ajwad-shaikh added bug Something isn't working triage Pending triage from maintainers labels Mar 28, 2023
@heitorlessa heitorlessa linked a pull request Mar 28, 2023 that will close this issue
7 tasks
@github-actions
Copy link
Contributor

⚠️COMMENT VISIBILITY WARNING⚠️

This issue is now closed. Please be mindful that future comments are hard for our team to see.

If you need more assistance, please either tag a team member or open a new issue that references this one.

If you wish to keep having a conversation with other community members under this issue feel free to do so.

@heitorlessa heitorlessa removed the triage Pending triage from maintainers label Mar 29, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants