Azure Container-Based Hosted Agents: Setup, Usage & Best Practices

Azure's container-based hosted agents let you deploy custom agent logic in Docker containers managed by Azure AI Foundry. This skill enables you to build agents that run your own code with custom dependencies, tools, and behavior while leveraging Azure's managed infrastructure for scaling, monitoring, and integration.

What This Skill Does

This framework uses ImageBasedHostedAgentDefinition from the Azure AI Projects SDK to deploy containerized agents. Unlike the standard agent framework that runs on Microsoft's predefined runtime, hosted agents execute your own Docker images with full control over the environment. You specify the container image, resource allocation (CPU and memory), tools, and environment variables, and Azure handles the hosting, scaling, and lifecycle management.

The workflow begins with building a Docker image containing your agent's code and dependencies, pushing it to Azure Container Registry, then creating an agent version that references your image. Azure pulls the image, starts containers with your specified resources, and makes the agent available through the AI Projects API. Your container communicates using the RESPONSES protocol, enabling standard agent interactions while running your custom logic.

This approach bridges the gap between fully managed agent services and custom infrastructure. You get Azure's operational benefits—managed hosting, monitoring, integration with other Azure services—while maintaining complete control over your agent's implementation. It's particularly powerful when you need specific libraries, custom models, proprietary logic, or integration with systems that require specialized SDKs.

Getting Started

Install the Azure AI Projects SDK with hosted agent support:

pip install azure-ai-projects>=2.0.0b3 azure-identity

Set your Azure AI project endpoint:

export AZURE_AI_PROJECT_ENDPOINT=https://<resource>.services.ai.azure.com/api/projects/<project>

Prerequisites before creating agents:

Container Image: Build and push your agent image to Azure Container Registry

ACR Permissions: Grant your project's managed identity the AcrPull role on your registry

Capability Host: Configure an account-level capability host with enablePublicHostingEnvironment=true

SDK Version: Verify you have azure-ai-projects>=2.0.0b3

Authentication uses Azure's standard credential chain. In production, DefaultAzureCredential automatically handles managed identities, service principals, and other auth methods.

Key Features

Custom Container Runtime: Run any code that fits in a Docker image. Use specialized Python libraries, non-Python runtimes, custom models, or proprietary algorithms not available in standard agent frameworks.

Resource Allocation Control: Specify CPU (0.5-4 cores) and memory (1-8 GiB) per agent. Start small for simple tasks, scale up for compute-intensive workloads like data processing or model inference.

Protocol Versioning: Agents declare which protocols they support via ProtocolVersionRecord. Currently supports AgentProtocol.RESPONSES v1 for standard request-response patterns.

Tool Integration: Equip agents with code interpreters, file search, and MCP tools. Tools are configured during agent creation and accessible to your container at runtime.

Environment Configuration: Pass environment variables to containers for API keys, model names, feature flags, and runtime configuration. Never hardcode secrets—use environment variables or Azure Key Vault.

Version Management: Create multiple versions of the same agent. List, update, and delete versions independently. Useful for canary deployments, A/B testing, and rollbacks.

Usage Examples

Basic Hosted Agent: Create an agent with a custom image and standard tools:

from azure.ai.projects import AIProjectClient
from azure.ai.projects.models import (
    ImageBasedHostedAgentDefinition,
    ProtocolVersionRecord,
    AgentProtocol
)
from azure.identity import DefaultAzureCredential

client = AIProjectClient(
    endpoint=os.environ["AZURE_AI_PROJECT_ENDPOINT"],
    credential=DefaultAzureCredential()
)

agent = client.agents.create_version(
    agent_name="data-processor",
    definition=ImageBasedHostedAgentDefinition(
        container_protocol_versions=[
            ProtocolVersionRecord(protocol=AgentProtocol.RESPONSES, version="v1")
        ],
        image="myregistry.azurecr.io/processor:v1.0",
        cpu="2",
        memory="4Gi",
        tools=[{"type": "code_interpreter"}]
    )
)

With Environment Variables: Configure runtime behavior through environment:

agent = client.agents.create_version(
    agent_name="ml-inference",
    definition=ImageBasedHostedAgentDefinition(
        container_protocol_versions=[
            ProtocolVersionRecord(protocol=AgentProtocol.RESPONSES, version="v1")
        ],
        image="myregistry.azurecr.io/ml-agent:latest",
        cpu="4",
        memory="8Gi",
        environment_variables={
            "MODEL_NAME": "gpt-4o-mini",
            "MAX_RETRIES": "3",
            "LOG_LEVEL": "INFO"
        }
    )
)

Multiple Tools: Combine code interpreter, file search, and MCP tools:

agent = client.agents.create_version(
    agent_name="research-assistant",
    definition=ImageBasedHostedAgentDefinition(
        container_protocol_versions=[
            ProtocolVersionRecord(protocol=AgentProtocol.RESPONSES, version="v1")
        ],
        image="myregistry.azurecr.io/research:latest",
        tools=[
            {"type": "code_interpreter"},
            {"type": "file_search"},
            {
                "type": "mcp",
                "server_label": "research-tools",
                "server_url": "https://mcp.example.com"
            }
        ]
    )
)

List and Delete Versions: Manage agent lifecycle:

# List all versions
versions = client.agents.list_versions(agent_name="data-processor")
for v in versions:
    print(f"Version {v.version}: {v.state}")

# Delete old version
client.agents.delete_version(agent_name="data-processor", version="v1.0")

Best Practices

Tag Images Explicitly: Never use latest in production. Tag images with semantic versions (v1.2.3) or commit SHAs for reproducibility and easy rollbacks.

Start Small on Resources: Begin with minimal CPU and memory allocation. Monitor actual usage and scale up only when needed. Overprovisioning wastes budget; underprovisioning causes performance issues.

Use Environment Variables for All Config: Hardcoding values makes updates require image rebuilds. Environment variables enable runtime configuration changes without redeployment.

Grant Minimal Permissions: Give your project's managed identity only the AcrPull role on specific registries. Don't grant broader permissions that could expose other resources.

Test Protocol Versions: Ensure your container correctly implements the RESPONSES protocol v1. Mismatches cause runtime failures that are harder to debug than build-time errors.

Clean Up Unused Versions: Delete agent versions you no longer need. Each version consumes resources and appears in listings, cluttering your environment.

Implement Health Checks: Build health check endpoints into your container. Azure can use these to verify agent readiness and restart unhealthy instances.

When to Use / When NOT to Use

Use hosted agents when:

• You need custom dependencies not available in standard runtimes


• Your agent requires proprietary code or licensed software


• You're integrating with systems that need specific SDKs


• You want to run non-Python code (Go, Rust, Java, etc.)


• You need full control over the execution environment


• You're implementing custom ML models or data processing pipelines


• Standard agent frameworks don't support your use case

Avoid hosted agents when:

• The standard agent framework meets your needs (simpler, faster to deploy)


• You're building a simple chatbot without custom logic


• You lack container expertise or CI/CD pipelines


• Your workload fits in serverless functions (use Azure Functions instead)


• You need sub-second cold starts (containers take longer to initialize)


• You want to avoid Docker image maintenance overhead

Related Skills

• agent-framework-azure-ai-py: Standard Azure agent framework without containers
• azure-ai-projects-py: Azure AI project management and orchestration
• azd-deployment: Azure Developer CLI for deployment automation

Source

Maintained by Microsoft. View on GitHub