2. Basics of AI agents¶

Questions

What do we mean by an AI agent, and how is it more than just calling an LLM?
Why can’t a plain language model tell you the time or remember what you just said?
When an agent has several tools, how does it decide which one to use?
What happens when tools are “chained” together?

Objectives

Understand what an AI agent is and the role of the harness and tools.
Build up an agent step by step: adding a tool, conversation history, and a second tool.
See why tools and message history are needed by running the example scripts.

Defining agents¶

The term agent is quite overloaded these days. It is often used to refer to any application using LLMs in some capacity, even one as simple as our “hello world” program from the previous section. In this course, we understand an AI agent to mean a computer program (a harness in AI jargon) that leverages an LLM to interact with the program’s environment in order to accomplish some task. This is enabled by outfitting the harness with capabilities, which may be implemented as, e.g., Python functions.

        flowchart TB
    In([In])
    Out([Out])
    LLM[LLM]

    In --> LLM
    LLM --> Out

    subgraph Capabilities["Capabilities"]
      direction LR
      Planning[Planning]
      Tools[Tools, MCP, Skills..]
      Memory[Memory]
    end

    LLM <-.->|TODO lists| Planning
    LLM <-.->|Action/Feedback| Tools
    LLM <-.->|Read/Write| Memory

    style In fill:#ffdac1
    style Out fill:#ffdac1
    style LLM fill:#e2f0cb
    style Tools fill:#f0c9e8
    style Memory fill:#ffe6cc

At the time of writing these materials, there are many frameworks for building AI agents, each providing roughly the same features. This course uses Pydantic AI. Other popular options include LangChain and OpenAI Agents SDK. The code block below shows how one would use Pydantic AI to build an agent on top of the OpenAI client used in the previous section.

client = AsyncOpenAI(
    api_key=os.getenv("OPENAI_API_KEY", default="EMPTY"),
    base_url=os.getenv("MODEL_BASE_URL", "https://aitta-api.csc.fi/openai/v1"),
)

model = OpenAIChatModel(
    os.getenv("MODEL_NAME", "google/gemma-4-31b-it"),
    provider=OpenAIProvider(openai_client=client),
)
agent = Agent(model)

Here Pydantic AI’s Agent wraps the same Aitta-hosted client we built in the previous episode, turning it into an agent that we can later extend with tools and conversation history.

We run the agent in a loop as you can see below. It is constantly waiting for new prompts or processing prompts you just sent.

async def main():
    history = []
    while True:
        try:
            prompt = input("You: ")
        except (EOFError, KeyboardInterrupt):
            break
        if prompt.strip().lower() in {"quit", "exit", ""}:
            break

        async with agent.run_stream(prompt, message_history=history) as result:
            print("Agent: ", end="")
            async for text in result.stream_text(delta=True):
                print(text, end="", flush=True)

Discussion

So where is the compute happening? Which part of this program runs on your own machine, and which part runs elsewhere?

Exercise 2.1: Ask the agent for the time¶

Exercise

We can try to use the agent with the following command.

uv run 02a_time.py

Try asking the agent about the time and observe the result:

What is the time currently in Helsinki?

Discussion

What do you see? Is this surprising to you?

The agent most likely made up a time or refused to answer. An LLM only generates text and has no way to look up the current time on its own. To give it that ability, we provide tools.

Tools¶

The previous section introduced the concept of a language model as a text generation algorithm. The role of an LLM in an AI agent is to guide the harness program’s execution, so that it can leverage the provided tools in a useful manner. This is accomplished by extracting pre-determined patterns from text generated by the LLM. An inference server running the model, such as vLLM or SGLang, parses tool calls from the LLM’s output and returns a structured response to the harness.

<bos><|turn>system
You are a helpful assistant.<turn|>
<|turn>user
What's the temperature in London?<turn|>
<|turn>model
<|tool_call>call:get_current_weather{location:<|"|>London<|"|>}<tool_call|><|tool_response>

[
  {
    "role": "system",
    "content": "You are a helpful assistant."
  },
  {
    "role": "user",
    "content": "What's the temperature in London?"
  },
  {
    "role": "assistant",
    "tool_calls": [
      {
        "function": {
          "name": "get_current_weather",
          "arguments": {
            "location": "London"
          }
        }
      }
    ]
  }
]

When a user inputs a message to a harness with access to tools, a list of available tools is passed along with the message to the inference server running the model. If the model ends its generated output with a tool call, the tool call is included as structured data in the returned response. The harness adds the response to the conversation history, executes the tool call and sends a new request to the inference server with the tool call result appended. The model then either answers or calls another tool.

        sequenceDiagram
    participant Harness
    participant Inference as Inference server

    Harness->>Inference: Request with tool list
    Note over Inference: Generate output
    Note over Inference: Parse tool call
    Inference-->>Harness: Response with tool call
    Note over Harness: Execute tool call
    Harness->>Inference: Request with tool result
    Note over Inference: Generate answer

Note

Just like the parsed response example shown at the start of the current section, the request sent by the harness is structured data rather than raw text. The raw-text prompt used for generation is rendered from a chat template by filling in the request data. For an example, we suggest looking at the Gemma 4 chat template, which was used to render the example text.

Below you can see the tool get_current_datetime that returns the current time given a timezone. While the LLM has no access to the current time, your computer knows the time very well. This tool simply returns the time of your computer to the agent. Note that the tool’s docstring tells the agent when to use it.

@agent.tool_plain
def get_current_datetime(timezone: str | None = None) -> str:
    """Get the current date and time.

    Args:
        timezone: IANA timezone name, e.g. 'Europe/Stockholm'. Omit for the user's local time.
    """
    logfire.info("tool called with {timezone=}", timezone=timezone)
    tz = ZoneInfo(timezone) if timezone else None
    now = datetime.datetime.now(tz).astimezone(tz)
    result = now.strftime("%Y-%m-%d %H:%M:%S %Z (UTC%z)")
    logfire.info("tool result: {result}", result=result)
    return result

Discussion

What other tools could you imagine?

Exercise 2.2: Enable a tool¶

Exercise

We can try the agent with the get_current_datetime tool enabled by passing --enable_tools. The agent will call this tool if it seems appropriate for replying to your prompt.

uv run 02a_time.py --enable_tools

Try asking the agent again about the time and observe the result (note that you can see potential tool calls in the log):

What is the time currently in Helsinki?

This works now! However, what about asking multiple things that build on top of each other?

The instructor lives in Helsinki.

Followed by the second prompt:

What is the time currently in the city that the instructor lives in?

You will notice that this is not possible for the LLM to answer. We will now explore why.

Discussion

Why do you think the agent could not answer the second question?

History¶

LLMs are stateless, which means that a given input does not affect how consecutive inputs are processed. A practical consequence of this is that when you input a message to a model, it does not remember previous messages. In order to hold a conversation with a model, every new input must include all previous inputs and responses. While chat history can be naïvely implemented by manually appending messages to a list, pre-made harnesses typically take care of this on behalf of the programmer.

async def main():
    history = []
    while True:
        try:
            prompt = input("You: ")
        except (EOFError, KeyboardInterrupt):
            break
        if prompt.strip().lower() in {"quit", "exit", ""}:
            break

        async with agent.run_stream(prompt, message_history=history) as result:
            print("Agent: ", end="")
            async for text in result.stream_text(delta=True):
                print(text, end="", flush=True)
            print()
        history = result.all_messages() if args.enable_history else []

Exercise 2.3: Enable conversation history¶

Exercise

We can try the agent by additionally enabling the history with --enable_history.

uv run 02a_time.py --enable_tools --enable_history

This example will now work:

The instructor lives in Helsinki.

Followed by the second prompt:

What is the time currently in the city that the instructor lives in?

Now the LLM should reply with the current time in Helsinki.

Multiple tools and tool chaining¶

An agent is not limited to a single tool. We can register several tools on the same agent, and the LLM decides which one to call for a given prompt. Tools can also be chained: the result of one tool becomes the input to another.

The 02b_flight_search.py example adds a search_flights tool alongside get_current_datetime. The flight search only understands concrete dates, so its docstring instructs the agent to resolve relative dates like “tomorrow” by calling get_current_datetime first and then passing the resolved date to search_flights.

See the search_flights tool below. It is quite complicated, but in the end it just allows access to a flight database. The results from that database can be filtered. If you are interested, you can find the full mock flight database under data/flights.csv.

@agent.tool_plain
def search_flights(
    origin: str | None = None,
    destination: str | None = None,
    date: str | None = None,
) -> str:
    """Search the weekly flight schedule. Flights recur on fixed weekdays.

    Args:
        origin: departure city, e.g. 'Helsinki'. Omit to match any.
        destination: arrival city, e.g. 'Stockholm'. Omit to match any.
        date: travel date 'YYYY-MM-DD'; returns flights on that weekday. Resolve
            relative dates like 'tomorrow' with get_current_datetime first.
    """
    logfire.info("tool called with {origin=} {destination=} {date=}", origin=origin, destination=destination, date=date)
    flights = pd.read_csv(Path(__file__).parent / "data" / "flights.csv")
    if origin:
        flights = flights[flights["origin"].str.lower() == origin.lower()]
    if destination:
        flights = flights[flights["destination"].str.lower() == destination.lower()]
    if date:
        try:
            weekday = WEEKDAYS[datetime.date.fromisoformat(date).weekday()]
        except ValueError:
            return f"Invalid date {date!r}, expected 'YYYY-MM-DD'."
        flights = flights[flights["weekdays"].str.contains(weekday)]

    if flights.empty:
        result = "No matching flights found."
    else:
        result = flights.head(20).to_string(index=False)
    logfire.info("tool result: {result}", result=result)
    return result

Discussion

Before running it, which tools do you expect the agent to call, and in what order?

Exercise 2.4: Chain two tools¶

Exercise

Run the flight search agent, which has both tools enabled:

uv run 02b_flight_search.py

Ask a question that depends on today’s date, and watch the log to see the agent call get_current_datetime first and then search_flights:

Are there any flights from Paris to Rome tomorrow?

The agent should have returned a list of flights.

Summary¶

An AI agent is a program (a harness) that uses an LLM to decide how to interact with its environment through tools. In this episode we used Pydantic AI to wrap the model from the previous episode into an agent.

Because an LLM only generates text, tools are necessary to do more useful things: the harness adds the available tools to the prompt, detects when the model’s output is a tool call, runs the tool, and returns the result to the model. A tool’s docstring tells the agent when and how to use it.

LLMs are stateless, so to hold a conversation the harness must send the full message history with every request. With tools and history in place, the agent can chain several tool calls together and build on earlier turns, for example resolving a relative date before searching for flights.