[FSSDK-11546] holdout + decision service impl. #386
Conversation
junaed-optimizely
requested review from
mikechu-optimizely,
muzahidul-opti and
Copilot
and removed request for
mikechu-optimizely
There was a problem hiding this comment.
Pull Request Overview
This PR adds holdout support to the Optimizely .NET SDK's decision service, enabling the SDK to evaluate holdouts (experiments that exclude users from treatment) alongside existing experiments and rollouts.
- Refactored the inheritance hierarchy to use
ExperimentCoreas the base class for both experiments and holdouts - Implemented holdout evaluation logic in the decision service with proper priority ordering (holdouts → experiments → rollouts)
- Added comprehensive test coverage for holdout functionality including bucketing, decision reasons, and user context operations
Reviewed Changes
Copilot reviewed 18 out of 18 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| ExperimentCore.cs | Added base class with common experiment/holdout properties and moved shared logic from Experiment class |
| Holdout.cs | Modified to override LayerId property to always return empty string as holdouts don't belong to layers |
| Experiment.cs | Removed duplicate properties now inherited from ExperimentCore |
| DecisionService.cs | Added GetDecisionForFlag method implementing holdout evaluation with proper priority ordering |
| Bucketer.cs | Updated to accept ExperimentCore instead of Experiment for holdout compatibility |
| FeatureDecision.cs | Added holdout decision source constant and updated to use ExperimentCore |
| Various other files | Updated method signatures to use ExperimentCore for holdout compatibility |
| Test files | Added comprehensive holdout test coverage including bucketing, decision service, and user context tests |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
| public bool IsExperimentRunning => | ||
| !string.IsNullOrEmpty(Status) && Status == STATUS_RUNNING; | ||
|
|
||
| public bool IsActivated => IsExperimentRunning; |
There was a problem hiding this comment.
The IsActivated property delegates to IsExperimentRunning, but this creates an indirect dependency on a misleadingly named property. Consider implementing this directly as !string.IsNullOrEmpty(Status) && Status == STATUS_RUNNING for clarity.
| public bool IsActivated => IsExperimentRunning; | |
| public bool IsActivated => !string.IsNullOrEmpty(Status) && Status == STATUS_RUNNING; |
Copilot uses AI. Check for mistakes.
There was a problem hiding this comment.
Would it be ok if we use existing isExperimentRunning for holdouts as well ? I think we dont need this newly introduced prop then in C# ? @muzahidul-opti ?
There was a problem hiding this comment.
Yah, we can do that since only running ho will be in the datafile.
There was a problem hiding this comment.
We can keep it generic, isRunning what do you think?
There was a problem hiding this comment.
Yeah. Since Experiment, ExperimentCore and Holdout all are internal, we can change this to isRunning and it should not be breaking! Unless I am getting this completely wrong!
CC: @mikechu-optimizely!
| { | ||
| _VariationKeyMap[holdout.Key][variation.Key] = variation; | ||
| _VariationIdMap[holdout.Key][variation.Id] = variation; | ||
| _VariationKeyMapByExperimentId[holdout.Id][variation.Key] = variation; |
There was a problem hiding this comment.
Looks good to me. Since the ho should have only one variation, we may add instance method to HO like this:
func getVariation(id: String) -> Variation? {
return variations.filter { $0.id == id }.first
}
There was a problem hiding this comment.
C# Uses Variation key map for bucketing and this fits well with that pattern. I am not sure about the instance method and its implication. Could you clarify please ?
Summary
Added holdout support in decision service.
Test plan
Tests: Comprehensive tests addition
Issues
FSSDK-11546