Building Multi-Agent Conversations with Microsoft AutoGen

By · · AI Engineering

Single-agent systems hit a ceiling fast. When you need an AI to write code, review it, and then fix bugs based on the review, a single prompt-response loop struggles to manage the different perspectives required. Multi-agent systems solve this by letting specialized agents collaborate, each with its own role and instructions. Microsoft's AutoGen framework makes this pattern accessible with a clean Python API. We will build a multi-agent conversation system using AutoGen 0.2.

Why multi-agent?

The core idea is simple: different tasks benefit from different personas. A code-writing agent should be creative and thorough. A code-reviewing agent should be critical and detail-oriented. By separating these roles into distinct agents, you get better results than asking a single agent to wear all hats. AutoGen provides the infrastructure to manage the conversation flow between these agents automatically.

Installation

AutoGen 0.2 is available on PyPI. Install it along with the OpenAI integration:

pip install pyautogen

Set your OpenAI API key as an environment variable:

export OPENAI_API_KEY="sk-your-api-key-here"

Configuring the LLM

AutoGen uses a configuration list to define which models are available. This allows you to set up fallbacks and filter by model name:

import autogen

config_list = [
    {
        "model": "gpt-4-turbo-preview",
        "api_key": os.environ["OPENAI_API_KEY"],
    }
]

llm_config = {
    "config_list": config_list,
    "temperature": 0,
    "seed": 42,
}

The seed parameter helps with reproducibility during development, though results may still vary slightly between runs.

A two-agent conversation

The simplest multi-agent setup involves an AssistantAgent and a UserProxyAgent. The assistant is powered by the LLM, while the user proxy can execute code and relay results back.

import autogen

assistant = autogen.AssistantAgent(
    name="Coder",
    llm_config=llm_config,
    system_message="""You are a senior Python developer. When asked to solve
    a problem, write clean, well-documented Python code. Always include
    error handling and type hints.""",
)

user_proxy = autogen.UserProxyAgent(
    name="User",
    human_input_mode="NEVER",
    max_consecutive_auto_reply=5,
    is_termination_msg=lambda x: x.get("content", "").rstrip().endswith("TERMINATE"),
    code_execution_config={
        "work_dir": "coding_output",
        "use_docker": False,
    },
)

user_proxy.initiate_chat(
    assistant,
    message="Write a Python function that finds the longest palindromic substring in a given string. Include unit tests.",
)

When you run this, the user proxy sends the task to the assistant. The assistant writes code, the user proxy executes it, and if there are errors, the assistant sees the output and fixes the code. This back-and-forth continues until the task is complete or the maximum number of replies is reached.

Adding a code reviewer agent

Two agents are good, but three agents are better for code quality. Let us add a dedicated reviewer:

reviewer = autogen.AssistantAgent(
    name="Reviewer",
    llm_config=llm_config,
    system_message="""You are a senior code reviewer. Your job is to review
    Python code for:
    - Correctness and edge cases
    - Performance and time complexity
    - Code style and readability
    - Proper error handling

    Be specific in your feedback. If the code is good, say 'APPROVED'.
    If changes are needed, explain exactly what should change.""",
)

Group chat for multi-agent collaboration

AutoGen's GroupChat lets multiple agents converse in a managed setting. The GroupChatManager decides which agent speaks next:

groupchat = autogen.GroupChat(
    agents=[user_proxy, assistant, reviewer],
    messages=[],
    max_round=12,
)

manager = autogen.GroupChatManager(
    groupchat=groupchat,
    llm_config=llm_config,
)

user_proxy.initiate_chat(
    manager,
    message="""Create a Python class for a thread-safe LRU cache with the
    following requirements:
    1. Support get and put operations with O(1) time complexity
    2. Be thread-safe using threading locks
    3. Include comprehensive unit tests
    4. The reviewer must approve the final code""",
)

The conversation unfolds naturally. The Coder writes the implementation, the Reviewer critiques it, the Coder revises based on feedback, and the cycle continues until the Reviewer approves or the round limit is hit.

Customizing agent selection

By default, the GroupChatManager uses the LLM to decide which agent speaks next. You can customize this with a speaker selection function:

def custom_speaker_selection(last_speaker, groupchat):
    messages = groupchat.messages

    if last_speaker is user_proxy:
        return assistant  # User tasks go to the coder first
    elif last_speaker is assistant:
        return reviewer   # Code goes to review
    elif last_speaker is reviewer:
        last_message = messages[-1]["content"]
        if "APPROVED" in last_message:
            return None   # End the conversation
        return assistant   # Send back for revision

    return "auto"

groupchat = autogen.GroupChat(
    agents=[user_proxy, assistant, reviewer],
    messages=[],
    max_round=12,
    speaker_selection_method=custom_speaker_selection,
)

This gives you deterministic control over the conversation flow while still using the LLM for the actual work.

A practical example: data analysis pipeline

Let us build something more realistic. Here is a multi-agent setup for automated data analysis:

analyst = autogen.AssistantAgent(
    name="DataAnalyst",
    llm_config=llm_config,
    system_message="""You are a data analyst. When given a dataset or
    analysis task:
    1. Write Python code using pandas and matplotlib
    2. Include descriptive statistics
    3. Create visualizations
    4. Summarize findings in plain language""",
)

statistician = autogen.AssistantAgent(
    name="Statistician",
    llm_config=llm_config,
    system_message="""You are a statistician. Review data analysis code and
    results for:
    - Statistical validity
    - Appropriate use of methods
    - Potential biases or confounders
    - Whether conclusions are supported by the data
    Suggest additional analyses when appropriate.""",
)

executor = autogen.UserProxyAgent(
    name="Executor",
    human_input_mode="NEVER",
    max_consecutive_auto_reply=8,
    is_termination_msg=lambda x: x.get("content", "").rstrip().endswith("TERMINATE"),
    code_execution_config={
        "work_dir": "analysis_output",
        "use_docker": False,
    },
)

groupchat = autogen.GroupChat(
    agents=[executor, analyst, statistician],
    messages=[],
    max_round=15,
)

manager = autogen.GroupChatManager(
    groupchat=groupchat,
    llm_config=llm_config,
)

executor.initiate_chat(
    manager,
    message="""Analyze the Iris dataset from sklearn.datasets. Perform
    exploratory data analysis, test whether sepal length differs
    significantly across species, and build a simple classifier.
    Present all findings with visualizations.""",
)

The DataAnalyst writes the analysis code, the Executor runs it, and the Statistician reviews the methodology and results. This creates a workflow that mirrors how a real data science team operates.

Error handling and guardrails

In production, you need to handle failures gracefully. AutoGen provides several mechanisms:

user_proxy = autogen.UserProxyAgent(
    name="User",
    human_input_mode="TERMINATE",  # Ask for human input before terminating
    max_consecutive_auto_reply=3,   # Limit retries
    code_execution_config={
        "work_dir": "output",
        "use_docker": True,         # Sandbox code execution
        "timeout": 60,              # Kill long-running code
    },
)

Setting use_docker to True is strongly recommended for any scenario where agents execute arbitrary code. It sandboxes execution so a misbehaving agent cannot harm your system.

Takeaways

After building several multi-agent systems with AutoGen, a few patterns have emerged:

AutoGen makes multi-agent orchestration accessible without requiring you to build the conversation management infrastructure yourself. The framework handles message routing, code execution, and agent coordination while you focus on defining the roles and tasks. It is a powerful tool for building AI systems that go beyond what a single prompt can achieve.