---
title: Evaluate Google ADK Agents
description: Evaluate Google ADK Python agents with Promptfoo tracing, sessions, tools, callbacks, plugins, artifacts, and workflow-agent checks.
sidebar_position: 27
---

# Evaluate Google ADK Agents

Use Google ADK's Python SDK with Promptfoo by wrapping your app as a Python provider. That keeps the ADK runtime in process, so Promptfoo can inspect the same sessions, artifacts, and native OpenTelemetry spans that the agent produced.

:::note
This guide targets ADK 2.x and requires Python 3.10 or later. The bundled example uses in-memory sessions and artifacts. Before upgrading a production app, review [ADK's migration guidance](https://adk.dev/2.0/#adk-python-1x-compatibility): persisted event schemas gain `node_info` and `output`, custom execution overrides change, and the runtime must own event emission and retries.
:::

## Quick Start

```bash
npx promptfoo@latest init --example integration-google-adk
cd integration-google-adk

python3 -m venv .venv
source .venv/bin/activate
python -m pip install -r requirements.txt

export GOOGLE_API_KEY=your_google_api_key_here
npx promptfoo@latest eval -c promptfooconfig.yaml --no-cache
npx promptfoo@latest eval -c promptfooconfig.workflow.yaml --no-cache
npx promptfoo@latest view
```

The example defaults to `gemini-2.5-flash`. If you want to use another ADK-supported model, set `ADK_MODEL`. Provider-style model strings such as `openai/gpt-5.4-mini` require the optional LiteLLM adapter:

```bash
python -m pip install 'litellm>=1.101,<2'
export OPENAI_API_KEY=your_openai_api_key_here
export ADK_MODEL=openai/gpt-5.4-mini
```

If Promptfoo runs outside the activated virtual environment, set the interpreter explicitly:

```bash
PROMPTFOO_PYTHON=.venv/bin/python npx promptfoo@latest eval -c promptfooconfig.yaml --no-cache
```

## What The Example Covers

`promptfooconfig.yaml` evaluates one conversational task across three user turns against a single `Agent`:

```python title="agent.py"
root_agent = Agent(
    name="weather_agent",
    model=model,
    instruction="...travel weather assistant. Call get_weather... Call save_trip_note...",
    tools=[get_weather, save_trip_note],
    before_agent_callback=before_agent_callback,
)
app = App(name=APP_NAME, root_agent=root_agent, plugins=[audit_plugin])
```

The runtime wires up an `InMemorySessionService`, an `InMemoryArtifactService`, an app-wide `BasePlugin` (`AuditPlugin`), and the native ADK OpenTelemetry exporter. `promptfooconfig.workflow.yaml` evaluates a `SequentialAgent` flow with two child agents and asserts that the weather lookup runs before the briefing.

## Why Use A Python Provider

An in-process Python provider lets one Promptfoo row drive a multi-turn task, read session state, load artifacts, observe plugin and callback side effects, and assert against ADK's native trajectory spans — all without going over the wire. The HTTP shape around `adk api_server` cannot expose any of those without a parallel inspection channel.

Pick the HTTP provider when the deployed HTTP contract itself is what you want to validate (auth, request shape, status codes). Pick the Python provider for everything else.

## How Native ADK Tracing Fits Promptfoo

ADK 2.x emits OpenTelemetry spans such as:

- `invocation`
- `invoke_agent weather_agent`
- `call_llm`
- `execute_tool get_weather`

The example provider preserves Promptfoo's W3C `traceparent`, starts a small provider span beneath it, and exports ADK's child spans to Promptfoo's built-in OTLP receiver. ADK already records `gen_ai.tool.name` and tool-call arguments, so Promptfoo can normalize them into trajectory steps without a custom span translator.

```yaml
assert:
  - type: trajectory:tool-used
    value:
      - get_weather
      - save_trip_note

  - type: trajectory:tool-args-match
    value:
      name: get_weather
      args:
        city: London
      mode: partial

  - type: trajectory:tool-sequence
    value:
      steps:
        - get_weather
        - save_trip_note
```

Use `trace-span-count` and `trace-error-spans` alongside trajectory assertions when you also want to prove that the ADK runtime emitted the expected framework spans and stayed error-free.

After the eval, inspect the row in the Trace Timeline. The bundled conversational run should show:

- one Promptfoo provider span beneath the injected parent trace
- `invoke_agent weather_agent` once per user turn
- `call_llm` spans around the model hops
- `execute_tool get_weather`
- `execute_tool save_trip_note`
- `gen_ai.tool.name` on tool spans
- serialized ADK tool arguments in `gcp.vertex.agent.tool_call_args`

## Sessions, State, Plugins, And Artifacts

State is mutated through `ToolContext`. Artifacts go through the same context but a different method:

```python title="agent.py"
async def save_trip_note(city: str, summary: str, tool_context: ToolContext):
    artifact = types.Part.from_bytes(
        data=f"# Trip note for {city}\n\n{summary}\n".encode("utf-8"),
        mime_type="text/markdown",
    )
    await tool_context.save_artifact(filename=f"{city}-trip-note.md", artifact=artifact)
    tool_context.state["last_saved_artifact"] = f"{city}-trip-note.md"
```

The provider drains both back out and returns them as one JSON payload. Deterministic `contains` and `is-json` assertions can then check internal effects without a model grader:

```json
{
  "artifact_names": ["london-trip-note.md"],
  "plugin_events": ["before_run:...", "after_run:..."],
  "session_state": {
    "callback_invocations": 3,
    "last_city": "London",
    "last_saved_artifact": "london-trip-note.md"
  }
}
```

One eval row now covers the assistant's reply _and_ the agent's internal bookkeeping.

## Workflow Agents

ADK 2.9.2 deprecates `SequentialAgent` in favor of `Workflow`, but still supports
the sequential-agent example below. For new graph-based apps, use
[ADK workflows](https://adk.dev/workflows/) and match trace assertions to that runtime.

When the order of work is fixed, encode it as a workflow agent instead of relying on the LLM to route. The bundled example chains a lookup agent into a briefing agent:

```python title="agent.py"
weather_lookup_agent = Agent(
    name="weather_lookup_agent",
    model=model,
    tools=[get_weather],
    output_key="weather_snapshot",  # writes the final response to session state
)
briefing_agent = Agent(name="briefing_agent", model=model, instruction="Use weather_snapshot...")
workflow = SequentialAgent(
    name="trip_planning_workflow",
    sub_agents=[weather_lookup_agent, briefing_agent],
)
```

The provider pattern is unchanged — `_run_workflow_provider` still calls `runner.run_async`. Swap `SequentialAgent` for `ParallelAgent`, `LoopAgent`, a custom `BaseAgent`, or any tree that uses `sub_agents` / `AgentTool`. Add a `trace-span-count` for each named workflow agent, and keep `trajectory:tool-used` focused on the tool calls that must happen.

## Structured Outputs, Memory, And Advanced Tools

The same provider shape covers the other ADK features. Map each ADK feature to the assertion type that proves it works:

| ADK feature                          | Assertion to add                                                           |
| ------------------------------------ | -------------------------------------------------------------------------- |
| `output_schema` or `output_key`      | `is-json` with a JSON schema in `value`, plus `contains` on returned state |
| `MemoryService`                      | paired test cases that share a session id vs use distinct ones             |
| MCP, OpenAPI, or authenticated tools | `trajectory:tool-used`, `trajectory:tool-args-match`, `is-refusal`         |
| code executors                       | `trace-span-count` on `execute_tool *` plus safety `contains` / `regex`    |
| multi-agent trees                    | one `trace-span-count` per `invoke_agent <name>` you require               |
| long-running / resumable apps        | `contains` on `session_state` snapshots before and after resume            |

ADK ships its own `adk eval` stack — use it for ADK-native eval sets and ADK-specific metrics. Promptfoo is the better fit when one harness has to compare ADK against other frameworks, run red teams against the same surface, or assert on OpenTelemetry traces alongside the final output.

## Production Notes

- Keep the provider span small. ADK emits the framework spans; the wrapper only has to preserve Promptfoo's parent trace and flush before the worker exits.
- The bundled example uses in-memory services so runs are deterministic. Swap in your real `SessionService`, `ArtifactService`, or `MemoryService` when persistence is part of the behavior under test.
- Reach for state and artifact assertions first; reserve model-graded assertions for outcomes that actually require semantics (tone, factuality, refusal quality).
- The optional LiteLLM adapter adds provider dependencies. Install it only when you need provider-prefixed model strings (`openai/...`, `anthropic/...`), and expect upstream warnings unrelated to your eval.

## Source References

- [ADK technical overview](https://adk.dev/get-started/about/)
- [ADK Python quickstart](https://adk.dev/get-started/python/)
- [ADK sessions](https://adk.dev/sessions/)
- [ADK callbacks](https://adk.dev/callbacks/)
- [ADK artifacts](https://adk.dev/artifacts/)
- [ADK agents](https://adk.dev/agents/)