Back to all posts

AI agent orchestration (with OpenAI Agents SDK)

Learn to build an effective multiagent system with AI agent orchestration.
by
Talha Irfan
Talha Irfan

Managing multiple AI agents for complex tasks can quickly become difficult without clear coordination. If you don't have an effective system, agents can end up performing duplicate work, wasting resources, or failing tasks altogether. The solution is to use an orchestrator - such as the OpenAI Agents SDK - to coordinate agents effectively.

In this article, you'll learn how agent orchestration works, see practical examples, and build a basic orchestration setup yourself.

What is AI agent orchestration?

AI agent orchestration is the process of coordinating multiple specialized AI agents through an orchestrator to achieve specific goals.

As demand grows for agent-based systems, multiagent architectures - with specialized agents handling different tasks - are becoming increasingly common. Several orchestration approaches exist, including:

  • Role-based orchestration, where each agent has a clearly defined role.
  • Task-based orchestration, where complex tasks are broken down into subtasks and assigned hierarchically to specific agents.
  • Other advanced methods, such as reinforcement learning (RL) or human-in-the-loop systems.

Despite differences in these methods, orchestration architectures typically share common core components. To clarify how this works, we'll consider a multiagent travel planning system as an example.

Example of orchestrating a multiagent system

To understand AI agent orchestration, we need to have a grasp of AI agents and their architectures. A multiagent system consists of several specialized agents working together, which is why they need an orchestrator to optimize their productivity. For instance, in our travel agent scenario, these specialized agents include:

  • Flight agent: Search and reserve flights.
  • Hotel agent: These agents can be role-based, with some dedicated agents for different tasks.
📌
Not every component of a multiagent system is an agent. Some can be simple tools, like an API call or a data retrieval function.

The orchestration engine

The orchestration engine is the manager of the whole system. It can also be a structured, rule-based system or an LLM model (for flexible and dynamic systems).

  • Rule-based orchestrators

Workflow-management engines mostly use rule-based orchestration. A good example is Apache Airflow, which uses DAGs. These systems are good for inflexible/rigid systems.

  • LLM-based orchestrators

LLM-based systems offer much better flexibility with natural language understanding and reasoning capabilities. In some LLM-based systems, the reasoning model also acts as the orchestrator (like ChatGPT), while in other architectures (like CrewAI or LangGraph-based systems), orchestration is handled separately. I think it deserves some explanation here.

  • Reasoning model as an orchestrator: Recent versions of ChatGPT are a good example of a large language model (like GPT-4) acting not only as the reasoning agent but also as an orchestrator.
  • External orchestrators: Systems like AutoGen, CrewAI, or LangGraph work in a more modular way and use a dedicated orchestrator to manage other agents.
💡
Related: Learn how to use CrewAI with Apify to build AI agents, and how to publish and monetize them on the Apify platform

In our case, we can use either of the LLM-based orchestrators; it depends on the implementation: for a simple chatGPT-based solution, it can be the reasoning model itself or a dedicated one for AutoGen or LangGraph-based implementation.

📌
Graph-based orchestrators, like LangGraph, use a hybrid approach that combines a Directed Acyclic Graph (DAG) for structured task execution and LLMs for dynamic decision-making when needed.

Communication protocols

For multiagent systems to collaborate, agents need to exchange data efficiently. The communication protocols depend on the architecture and other factors. Examples include:

  • Message queues: Systems like Kafka and RabbitMQ facilitate asynchronous communication via queues.
  • Direct function calls: Frameworks such as LangChain use straightforward function-calling mechanisms for simpler agent interactions.

Decision-making mechanisms

Agent orchestrators decide which agents to activate and when, using strategies such as rule-based logic or dynamic LLM-driven reasoning. Let’s illustrate this with an example:

Find flights from Toronto to New York on April 25, 2025 and accommodation in New York.

The orchestration workflow would proceed as follows:

  • The reasoning agent identifies the required tasks: checking availability and finding flights.
  • The orchestrator assigns these tasks to appropriate sub-agents:
    • The Booking agent checks for available hotels in New York on particular dates.
    • The Flight agent retrieves flight details (New York → Boston) for those dates.
  • Finally, an output agent decides the best format (such as a PDF summary or voice message) to deliver the results clearly back to the user.

How to implement AI agent orchestration with OpenAI Agents SDK

Now, let's move from theory into practice by building a simple orchestration demo. This architecture will remain pretty scalable to the real problem you would like to port to.

1. Set up your environment

In this implementation, we'll use the OpenAI Agents SDK, a straightforward and flexible option suitable for multiagent systems.

pip install openai openai-agents

Here, we're using an Apify Actor, so we'll need to use its library, too.

pip install apify_client
  • Verification

We can confirm the installation by running this Python snippet:


import openai
import openai-agents
from apify_client import ApifyClient
  • API keys

The installation is done. We can now set up the keys too. Again, it will take just a minute.

import os
os.environ["OPENAI_API_KEY"] = "xxxxxxx"
os.environ["APIFY_API_TOKEN"] = "xxxxxxx"

2. Define your agents

After setting up libraries and APIs, the next step is defining each agent clearly according to its specific role.

1. Flights Search Agent

Searching for flights is a core part of any travel planner app. Since flight prices change dynamically, we can’t cache/fetch them from a database and need to retrieve them fresh from either the airline directly, some APIs (like Amadeus), or third-party flight search engines like SkyScanner or Google Flights. I would prefer SkyScanner here.

To retrieve flight data from SkyScanner, we can use their official API or scrape the information directly. For simplicity, this demo uses the scraping method with SkyScanner Scraper. It requires the following parameters (IATA codes for airports):

  • Departure airport, date
  • Arrival airport, date

To make things easier, we'll use the functional calling facility; the function will get data from the SkyScanner scraper, and our agent will call it.

import asyncio
import os

from agents import Agent, Runner, function_tool
from agents.extensions.handoff_prompt import RECOMMENDED_PROMPT_PREFIX
from apify_client import ApifyClient
from openai import AsyncOpenAI

openai_client = AsyncOpenAI(api_key=os.getenv("OPENAI_API_KEY"))
apify_client = ApifyClient(token=os.getenv("APIFY_TOKEN"))

@function_tool
def skyscanner_scraper_tool(from_location: str, to_location: str, depart_date: str, return_date: str) -> list | None:
    """
    Use skyscanner to search for flights from a given location to a destination on a given date. Always use the IATA code for the location.
    Example:
    {
      "from": "YYZ",
      "to": "JFK",
      "departDate": "2025-04-25",
      "returnDate": "2025-04-27"
    }
    """
    run_input = {
        "from": from_location,
        "to": to_location,
        "departDate": depart_date,
        "returnDate": return_date,
    }
    run = apify_client.actor("harvest/skyscanner-scraper").call(run_input=run_input)
    return [
        {"Price": item.get("price")["raw"], "Departure": item.get("legs")[0].get("departure"), "Arrival": item.get("legs")[0].get("arrival")}
        for item in apify_client.dataset(run["defaultDatasetId"]).iterate_items()
    ]

We have defined the skyscanner_scraper_tool here to fetch respective flights from the Skyscanner scraper. Now function tooling can use it (tools=[skyscanner_scraper_tool]).

flight_agent = Agent(
    name="Flight agent",
    handoff_description="Specialist agent for Flight search fetch the data from the flight_skyscanner_",
    instructions=f"""{RECOMMENDED_PROMPT_PREFIX} You are a flight agent. You are given a user query and flight details.
        Use the context to extract flight information (origin, destination, departure, price)
        by calling skyscanner_scraper_tool with the provided context.
        Return the results in a natural language format, such as 'Flight from LHE to BKK: $753.4,
        departs June 1, 2025, 23:40, arrives June 2, 2025, 06:10'.""",
    tools=[skyscanner_scraper_tool],
)

Let’s save all the code into apify_agent.py and run it with the query: Find flights from New York to Boston on April 25, 2025.

result = asyncio.run(Runner.run(flight_agent,"Find flights from Toronto to New York on 25th April 2025"))
print(result.final_output)
### Flights from New York to Boston on 25th April 2025:

1. **Flight**: $196.97
   - **Departure**: 06:00
   - **Arrival**: 07:28

2. **Flight**: $481.97
   - **Departure**: 13:10
   - **Arrival**: 14:36

3. **Flight**: $201.97
   - **Departure**: 06:00
   - **Arrival**: 07:28
   
.....

2. Booking Agent

We'll also have a booking agent to search for hotels in a given location on a particular date. 

@function_tool
def booking_scraper_tool(search: str, check_in: str, check_out: str) -> list | None:
    """
    Use booking.com to search for hotels in a given location on a given date.
    Example:
    {
      "search": "New York",
      "checkIn": "2025-04-25",
      "checkOut": "2025-04-26"
      "maxItems": 5,
    }
    """

    run_input = {
        "search": search,
        "checkIn": check_in,
        "checkOut": check_out,
        "maxItems": 5,
    }

    run = apify_client.actor("voyager/booking-scraper").call(run_input=run_input)
    return [
        {
            "Name": item.get("name"),
            "Description": item.get("description"),
            "Address": item.get("address"),
            "Stars": item.get("stars"),
            "Rating": item.get("rating"),
            "Reviews": item.get("reviews"),
            "Price": item.get("price"),
            "CheckIn": item.get("checkIn"),
            "CheckOut": item.get("checkOut"),
        }
        for item in apify_client.dataset(run["defaultDatasetId"]).iterate_items()
    ]

3. Orchestrate the agents

The orchestrator itself will be the LLM agent (GPT4). It will also be defined similarly to the agents we defined before. As you can see, an orchestrator can be defined simply by a prompt, followed by the tools/agents it's supposed to orchestrate.

orchestrator_agent = Agent(
    name="orchestrator_agent",
    instructions=(
        "You are a travel orchestrator agent. Based on the user's request, call the appropriate tools: "
        "- For flight requests, use the Flight agent."
        "- For booking requests, use the Booking agent."
        "- For combined requests, use the both agents, Flight and Booking."
        "Never generate data yourself—always use the provided tools. Pass results to the SynthesizerAgent."
    ),
    tools=[
        flight_agent.as_tool(
            tool_name="flight_agent",
            tool_description="Flight agent to fetch the data from the flight_skyscanner_",
        ),
        booking_agent.as_tool(
            tool_name="booking_agent",
            tool_description="Booking agent to fetch the data from the booking_tool",
        ),
    ],
)

4. Test and refine

Now, we're ready to test this AI agent orchestration system with a couple of examples.

    result = asyncio.run(Runner.run(orchestrator_agent, "I want to find flights from New York to Boston on 25th April 2025 and also find a hotel in Boston for 2 nights from 25th April 2025"))
    print(result.final_output)

The orchestrator will guide both agents to retrieve the flights as well as booking data.

### Flights from New York to Boston on 25th April 2025:

1. **Flight**: $196.97
   - **Departure**: 06:00
   - **Arrival**: 07:28

2. **Flight**: $481.97
   - **Departure**: 13:10
   - **Arrival**: 14:36

3. **Flight**: $201.97
   - **Departure**: 06:00
   - **Arrival**: 07:28
   
.....

### Hotels in Boston from 25th to 27th April 2025:

1. **Hyatt Centric Faneuil Hall Boston**
   - **Price**: $766.24
   - **Rating**: 8.4 (2437 reviews)
   - **Stars**: 4
   - **Address**: 54-68 Devonshire St, Financial District, MA 02109

2. **Hotel AKA Boston Common**
   - **Price**: $931.60
   - **Rating**: 7.9 (397 reviews)
   - **Stars**: 4
   - **Address**: 90 Tremont Street, MA 02108

3. **Clarendon Square**
   - **Price**: $1066.56
   - **Rating**: 8.1 (25 reviews)
   - **Address**: 198 West Brookline Street, South End, MA 02118
   
.....

We can play around with it some more by trying new prompts and dates and also adding some new agents in the pipeline (the orchestrator will be directed to handle that, too). I would invite you to try that too.

Recap

Here's a quick reminder of the whole implementation process:

Step Description
1. Set up your environment Install Python, OpenAI SDK, Apify client, and set up OpenAI API key.
2. Define your agents Identify roles, e.g., Information Searcher for web queries, Flights Searcher for flights, Weather agent for weather updates, etc.
3. Orchestrate the agents Use OpenAI SDK (or LangChain, LangGraph, etc.) to define the workflow, ensuring Agent 1 and Agent 2 work in sync as per the user’s query.
4. Test and refine Run with sample queries, observe outputs, and iterate to optimize performance.

The benefits of AI agent orchestration

  • Efficiency: By coordinating multiple agents, we can automate complex workflows. A good example is to compare the efficiency of the earlier versions of chatGPT (” Sorry, I am updated only as of x date and can’t update you there”) vs. now, where it uses multiple agents, including the web search, cross-checking the information or some other custom-built agent (like Scholar GPT’s engine).
  • Cost optimization: Most APIs, including these OpenAI APIs, etc., are paid, and it's important to use them efficiently. Orchestrators are pretty good at cost optimization by avoiding redundant API calls or other computational resources.
  • Scalability: We can easily add, remove, or update agents without disrupting the whole system. We have seen how these systems (like Amazon’s Fulfillment AI or social media AI) are capable of handling tens of thousands of requests per second.
💡
With plug-and-play protocols like Model Context Protocol (MCP), it is becoming even easier to make scalable multiagent systems.

AI agent orchestration challenges

AI agent orchestration is quite appealing (and should be), but it comes with its share of challenges too. Some of these challenges are:

  • Decision-making complexity: While orchestration is pretty good at dynamic decision-making, deciding which agent to call in complex scenarios is still a challenge. For example, a user is booking a trip to Turkïye in the next week, but their embassy takes at least 4 weeks to issue a visa, so should we still issue a ticket?
  • Scalability: Even advanced models like GPT-4 can orchestrate workflows with an average of only 6.1 actions and normal workflows are much more complex.
  • Communication: It reminds me of “The Mythical Man-Month” from our university days. Is having a team of agents guaranteed to improve productivity? Likely but not guaranteed. Communication deadlocks and handling priority queues, etc, are all challenges associated with agent orchestration.
  • Error handling: Error handling is necessary for single agents too, but even if we ensure it for every agent, still some unforeseen issue(s) in a single agent can stall the whole pipeline. I found myself in a similar situation once where API itself got down, and agents had no idea where to fall back.

The encouraging thing is the growing focus of the AI research community on both agent-based systems and the challenges of AI agent orchestration (check out recently published studies here and here).

Conclusion

Multiagent systems are growing in demand, and agent orchestration is an important component in designing them. Here, we've used the new OpenAI Agent SDK to demonstrate how we can simply use tool calling (optional in many cases) and prompting to create AI agent orchestration systems.

Frequently asked questions

Why use AI agent orchestration?

AI agent orchestration allows us to use the power of multiple agents in parallel. Even some basic AI systems require multiple agents, and it's mainly down to orchestrators to efficiently use their resources. Compared to static methods, orchestrators are dynamic and scalable.

What is the difference between AI agent orchestration and traditional AI systems?

The difference between AI agent orchestration and traditional AI systems is multifold. Traditional AI systems are quite rigid and single-agent (or monolithic), while AI agent orchestration is dynamic and uses the power of multiple agents in parallel.

What is an agent orchestrator?

The agent orchestrator is a system that handles all the agents in a multiagent system. An orchestrator may or may not be an agent itself.

More about AI agents

Talha Irfan
Talha Irfan
I love reading Russian Classics, History, Cricket, Nature, Philosophy, and Science (especially Physics)— a lifelong learner. My goal is to learn and facilitate the learners.
On this page

Build the scraper you want

No credit card required

Start building

Related articles