Adding Merge config capabilities

[abhishekkumams]

Why make this change?

• Closes Should we do union of configuration properties specified in dab-config.json and dab-config.<Environment>.json? #714 , Issues with multiple config files #722
  • To Allow dotnet like experience during local development where Development config gets merged with the original one.

What is this change?

• Added a new Class to enable merging of two different configs.
• This merge feature is only applicable with dab start.
• If the user provide a config file it will still be honored to ensure backward compatibility.
• When the user doesn't provide a config file with dab start. It will check if DAB_ENVIRONMENT is set. if yes, then it will try to merge the dab-config.json with the dab-config.{DAB_ENVIRONMENT}.json and use the generated merged file dab-config.{DAB_ENVIRONMENT}.merged.json to startup the engine.
• If only dab-config.{DAB_ENVIRONMENT}.json is available, it will be used. if not it will check for dab-config.json. if that is also not available, an error will be thrown.

How the merge is performed?

• if the type is same, then object from second config is given preference. merging of json object is done recurrsively.
• if the second object is null, property from first config is given preference.
• in case of JsonArray, the elements from second config completely overrides that of the original config, i.e. the merged config will have elements in the array from only second config.

How was this tested?

• Unit Tests

Sample Request(s)

• set the DAB_ENVIRONMENT variable in your environment settings. for ex: Development.
• you can keep your local changes in a file called dab-config.Development.json. It will have the connection string and other information that is for local development.
• And you can have a file dab-config.json which can act as base file.
• when we do dab start the merge will be performed, and the generated merged file dab-config.Development.merged.json will be used to fire up the engine.

Example Scenarios:

Case1:

Environment value: $env:DAB_ENVIROMENT=""

command: dab start

Files in Directory:

• dab-config.json
• dab-config.DEVELOPMENT.json

Config Used for startup: dab-config.json

Reasoning: Environment value not setup

Case2:

Environment value: $env:DAB_ENVIROMENT="PRODUCTION"

command: dab start

Files in Directory:

• dab-config.json
• dab-config.DEVELOPMENT.json

Config Used for startup: dab-config.json

Reasoning: dab-config.PRODUCTION.json is not present in the directory

Case3:

Environment value: $env:DAB_ENVIROMENT="DEVELOPMENT"

command: dab start

Files in Directory:

• dab-config.json
• dab-config.DEVELOPMENT.json

Config Used for startup: dab-config.json + dab-config.DEVELOPMENT.json (Merged)

Reasoning: both dab-config.json and dab-config.DEVELOPMENT.json is present in the directory

Case4:

Environment value: $env:DAB_ENVIROMENT="DEVELOPMENT"

command: dab start -c dab-config.json

Files in Directory:

• dab-config.json
• dab-config.DEVELOPMENT.json

Config Used for startup: dab-config.json

Reasoning: If config is provided by the user it will always get precedence irrespective of the ENVIRONMENT value.

[aaronpowell]

Could I make a request that we be really cautious on this work as I'm getting close to completing #1402 which will see a major overhaul of the config type system in place, which may impact the way that merging needs to be undertaken.

[ayush3797]

Please update the comment here: it is no longer only used for testing. Probably explicitly write what it is initialized with.

data-api-builder/src/Config/RuntimeConfigPath.cs

Line 191 in e55f458

// Used for testing

[Aniruddh25]

Looking for few answers

[seantleonard]

last few nits and question based on latest changes

[seantleonard]

Thank you for adding this capability!

[Aniruddh25]

LGTM, thanks for clarifying my questions!