[Enhancement]: Support AKV

[JerryNixon]

For developers who want their secrets in Azure Key Vault, support a syntax in the config that allows it.

Added properties

{
  "azure-key-vault" : {
    "endpoint": "url", (string, required when @akv() is present)
    "retry-policy": {
        "mode": "fixed | exponential", (enum, default: exponential)
        "max-count": 3, (integer, default: 3)
        "delay-seconds": 1, (integer, default: 1)
        "max-delay-seconds": 100 (integer, default: 60),
        "network-timeout-seconds": 100 (integer, default: 60),
    }
  }
}

Based on: https://github.com/Azure/azure-sdk-for-net/blob/dc70df6e1bb2595cf351269fdbba0ddd84515de5/sdk/core/Azure.Core/src/RetryOptions.cs#L67C42-L67C46

Setting	Meaning	Default	Min	Max
endpoint	Key Vault URL	(not required)
retry-policy / mode	Retry strategy	exponential	-	-
retry-policy / max-count	Max retry attempts	3 (for both strategies)	0	int.MaxValue
retry-policy / delay-seconds	Initial delay between retries	1 (for both strategies)	1	int.MaxValue
retry-policy / max-delay-seconds	Max delay cap	60 (only for exponential)	1	int.MaxValue
retry-policy / network-timeout-seconds	Max network timeout per try	60	1	int.MaxValue

JSON Schema

Update our JSON schema. Add properties and constraints and defaults.

1. azure-key-vault is not required.
2. azure-key-vault.endpoint is required when azure-key-vault is present.

CLI updates

Add command line support.

1. Add dab configure --azure-key-vault.endpoint
2. Add dab configure --azure-key-vault.retry-policy.mode
3. Add dab configure --azure-key-vault.retry-policy.max-count
4. Add dab configure --azure-key-vault.retry-policy.delay-seconds
5. Add dab configure --azure-key-vault.retry-policy.max-delay-seconds
6. Add dab configure --azure-key-vault.retry-policy.network-timeout-seconds

Added syntax

Support new @akv() method to replace property values in our DAB configuration.

{
  "data-source": {
    "connection-string": "@akv('my-connection-string')"
  }
}

This works. Because env runs first.

{
  "data-source": {
    "connection-string": "@akv('my-connection-string')"
  },
  "azure-key-vault": {
    "endpoint": "@env('my-akv-endpoint')"
  }
}

Details

1. @env() executes before @akv(). So, azure-key-vault.endpoint can be set by @env().
2. For @akv('key-value'), "key-value" is the secret name.
3. Like @env() DAB supports @akv() for any config property.
4. Using @akv() requires either system-assigned or user-assigned managed identity.
5. The @akv() function works in both mode development and production.
6. The syntax @env('@akv('value')') is not supported and would look for an env named @akv('value').
7. However, the syntax @akv('@env('value')') is supported.
8. The DAB config does not monitor, listen to or poll AKV for secret changes. Requires restart.

Key rules (based on AKV docs)

Rule	Constraint
Allowed chars	Alphanumeric and - (hyphen)
Disallowed	No spaces, no _, /, @, ., etc.
Start/end	Must start and end with an alphanumeric character
Length	1 to 127 characters
Case	Case-insensitive (MySecret = mysecret)
Uniqueness	Must be unique within the Key Vault

Multiple configs

DAB supports multiple configuration files, where the top-level config is the master and others inherit runtime settings from it. Child configs define their own connection strings in the data-source property and may also have unique azure-key-vault.endpoint values. If a child omits the endpoint, it inherits the setting from the master config, which is required only if the child omits it.

1. Each config file can define a unique azure-key-vault.endpoint value.
2. If a child omits azure-key-vault.endpoint, it inherits from the master.
3. A child may use @akv() even if the master does not define an endpoint. However, in this case, the child configuration must define an endpoint value.

Consideration

1. Include an OTEL activity wrapping the replacement.
2. (would be nice) Include an OTEL activity wrapping the replacement for @env().

Errors

1. When @akv() appears without azure-key-vault.endpoint, we will log (as error) and fail to start.
2. When @akv() connection fails (for any reason), we will log the reason (as error) and fail to start.
3. When @akv() secret not found, we will log "secret not found: {property-name}" (as error) and fail to start.
4. When @akv() fails during hot reload, we will log "@akv error: {message}", then return to "last known good".

Flow

sequenceDiagram
  actor Engine as Engine
  participant ConfigInMem as ConfigInMem
  participant Environment as Environment
  participant AKV as AKV
  participant Config as ConfigFile

  Engine ->> Engine: Pre-Start
  Engine ->> Config: Load Config
  Config -->> Engine: Config Data
  Engine ->> ConfigInMem: Create In-Memory Config

  Note over Engine: Perform Config Replacements

  activate Engine
  ConfigInMem -->> Engine: Parse @env Values
  Engine ->> Environment: Get
  Environment -->> Engine: Values
  Engine ->> ConfigInMem: Replace @env Values
  deactivate Engine

  Note over Engine: Use Config With ENV Replacements

  activate Engine
  ConfigInMem -->> Engine: Parse @akv Values
  Engine ->> AKV: Request 
  AKV -->> Engine: Secrets
  Engine ->> ConfigInMem: Replace @akv Values
  deactivate Engine

  Note over Engine: Use Config With AKV Replacements

  Engine ->> Engine: Start

Loading