Skip to content

🐞 Bug: Azure Functions Deployment Slot Fails to Load Experimental Extension Bundle (MCP) #27

@jillbourque

Description

@jillbourque

Summary

After deploying an Azure Function App that utilizes the Experimental MCP Extension Bundle (Microsoft.Azure.Functions.ExtensionBundle.Experimental) to a deployment slot, the Functions Host consistently fails to detect, download, or register the experimental bundle. This leaves the application in a broken state, preventing MCP tools (like the hello function) from loading. The issue requires manual intervention (SSH cache clearing and restart) after every deployment to the affected slot.

Environment & Configuration

  • App Service Plan: EP1 (Elastic Premium)
  • Operating System: Linux
  • Functions Host Runtime Version: 4.1043.200.1
  • Function App Hosting: App Service Plan with Deployment Slots enabled.
  • Extension Bundle ID: Microsoft.Azure.Functions.ExtensionBundle.Experimental
  • Minimal Reproduction Code: Using the official Azure MCP Sample: https://github.com/Azure-Samples/remote-mcp-functions-typescript

Symptoms (Failure State After Deployment to staging slot)

  1. Missing Functions in Portal: The MCP function (hello: mcpToolTrigger) is missing from the Azure Portal's Functions list for the staging slot.
  2. Missing System Key: The required system key for the MCP endpoint is not generated:
    az functionapp keys list --name <app-name> --slot staging --query "systemKeys.mcp_extension" -o tsv
    # Returns empty output
  3. Endpoint Failure: Attempts to call the MCP endpoint fail:
    # curl command attempts to hit the runtime/webhooks/mcp endpoint
    # Returns: 404 Not Found or 401 Unauthorized

Root Cause Analysis

The host runtime appears to be aggressively caching the standard extension bundle (Microsoft.Azure.Functions.ExtensionBundle) and fails to detect the need to download the experimental one (.Experimental) during a "cold" start or initialization, which frequently occurs following slot-specific operations (initial deployment, post-swap sync).

SSH Evidence from Failed Slot:

Checking the function app's cache directory via Kudu SSH shows only the standard bundle is present:

cd /FuncExtensionBundles/
ls -la

 Output:
# drwxr-xr-x 5 root root ... Microsoft.Azure.Functions.ExtensionBundle
#
# Missing: Microsoft.Azure.Functions.ExtensionBundle.Experimental

Current Workaround (Manual Fix)

We must execute the following steps manually after every deployment to the staging slot. This confirms that clearing the cache forces the runtime to download the required bundle on the subsequent restart.

  1. Access Kudu Console/SSH for the deployment slot.

  2. Clear the extension bundle cache:

cd /FuncExtensionBundles/
rm -rf *
  1. Restart the Function App slot using a full STOP and START.

  2. The MCP tools then register successfully.

Desired Permanent Solution

The manual step breaks our automated CI/CD pipeline and adds significant time to the deployment workflow. We request a fix for the underlying caching bug:

  1. Primary Request (Host Fix): The Functions Host runtime should reliably check the host.json for the required bundle ID and ensure it is downloaded and loaded during startup, even when the cache directory is empty.

  2. Alternative (Configuration Switch): If a full runtime fix is not immediately feasible, provide a documented and supported Application Setting (e.g., WEBSITE_FUNCTIONS_FORCE_BUNDLE_REFRESH=1) that can be set on the deployment slot to force the host to ignore cache and re-download bundles on startup.

Additional Thoughts (General Slot Deployment Instability)

We have observed a broader pattern where, even for non-experimental HTTP functions, a RESTART is consistently necessary on staging slot deployments when a brand new function is added. This suggests that the host runtime on a deployment slot is not reliably detecting file system or configuration changes upon deployment, making the experimental bundle cache failure a critical symptom of a more general host configuration refresh issue on slots.

Metadata

Metadata

Labels

bugSomething isn't working

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions