INTEGRATION GUIDE

Connecting your agentic system to Emet

This guide details the path from an Emet account to a verified agent in production. It covers what Emet checks and what you get back, connecting your data, what you build, the data you send, edge cases, going live, and handling errors.

SECTION 1

At a glance

What Emet does for your agent

Your agent answers questions with figures, comparisons and trends drawn from its tools.

Emet examines each answer, recomputes its figures and calculations in deterministic code, and checks every claim against the evidence your agent actually received and, with your data connected, against the source itself. No model ever judges its own output. Every response your agent makes receives an Emet Accuracy Report: an assessment of its accuracy and completeness. Inaccurate figures or claims are logged, and Emet provides a clarification or correction based on the available evidence.

This way you find out which answers hold up, which ones do not, and exactly which claim, tool result, and model were responsible.

What you get back

Every checked answer produces an Emet Accuracy Report in the Emet Observability Dashboard, where everyone on your team can log in and review it at any time. Each report shows every claim in the answer with its verdict and reason, the evidence behind it, and the calculation behind every derived figure.

Each claim receives one of four verdicts, with its reason shown alongside it.

VERIFIED
The claim matches the evidence.
CLARIFIED
The claim is right but missing context a reader needs, such as its scope or time window. Emet adds it.
CORRECTED
The claim is wrong, and the evidence establishes the right answer. Emet auto-corrects it and shows the correction against the original wording.
FLAGGED
The claim cannot be supported or corrected from the evidence. Emet flags it rather than guess (or hallucinate an answer).

The Emet Observability Dashboard

An Accuracy Report covers one answer. The Emet Observability Dashboard covers all of them, across your company and by team, built from the same traces:

  • Overview

    Accuracy rate, hallucination rate and coverage (the share of claims Emet could check) over time. Accuracy is broken out by application, by model and by data source, with the reason behind every miss: incorrect value, misattributed figure, calculation error or ungrounded claim.

  • Reports

    Every Accuracy Report, filtered by status, application and team.

  • Usage & cost

    Agent runs, model calls, tool calls, tokens, estimated cost and run duration, by model and by tool.

  • AI Analytics

    Where your tokens go and where spend can be cut, ranked by monthly saving. Emet only recommends a change that keeps the hallucination rate flat.

The dashboard fills in from the moment you integrate. Already sending traces to another tracing backend? Keep it, and add Emet as a second export.

Example: right data, wrong reading

The question was “How did CTV perform against display last week?” The agent's SQL was correct and returned four rows:

ChannelWeek ofConversionsCPA
CTV31 August1,248$50.00
CTV7 September1,340$43.96
Display31 August1,030$40.00
Display7 September1,075$41.67

The Accuracy Report showed:

Claim
“CTV conversions rose 7% week over week to 1,340”
Verdict
VERIFIED
Why
The result shows 1,248 rising to 1,340, an increase of 7.4%.
Claim
“the most of any channel”
Verdict
CLARIFIED
Why
True of the two channels the query covered. Emet adds the scope: “the most of the two channels queried.”
Claim
“Its CPA fell to $43.96, now below display”
Verdict
CORRECTED
Why
Display's CPA was $41.67 the same week, so CTV's is still above it. Emet corrects it to “Its CPA fell to $43.96, still above display's $41.67.”
Claim
“The lift was driven by the new creative”
Verdict
FLAGGED
Why
Nothing in the result describes creative, so the cause can be neither supported nor corrected from this evidence.

What is AI observability?

AI agents are non-deterministic: the same question can take a different route each time, so you can't test every behavior before launch. Traditional monitoring tells you an agent is running, what it cost and how many tokens it used. AI observability goes further and shows you what your agent actually did in production: what it was asked, which tools it called, what came back, and what it finally said. On its own, neither tells you whether the answer was right.

This works through instrumentation, which means adding a small amount of logging to your agent, typically through an SDK or an integration with your existing framework. After that, every run is captured automatically as structured data, organized in three levels:

  • Conversation

    The full multi-turn exchange with a user.

  • Trace

    A single turn, meaning one question, the agent's work, and its response.

  • Span

    One step within a trace, such as a sub-agent or a tool call and its result.

Because Emet sees the tool results your agent received at every step, and can query your connected data sources directly, it checks each claim in the final answer against both the evidence your agent saw and the source itself. Emet gives you both: what your agent did, and whether what it said is true.

What you build

Send each turn to Emet as one JSON document over HTTPS, using the open-source Emet SDK for Python with Pydantic AI, or our reference exporter for TypeScript (section 5). The document follows the OpenTelemetry GenAI conventions, so an agent that already records them maps onto it directly. Direct OpenTelemetry ingestion (OTLP) is available on request.

What you do not have to touch

Nothing in your request path changes
  • No proxy required.

    Your model calls go straight to your provider, exactly as today.

  • One exporter, at most.

    The Emet SDK for Python with Pydantic AI, or one exporter for your stack, sends plain HTTPS and JSON, under your control.

  • No inbound access to your agent.

    Emet never calls into your application. Traces go one way, from your exporter to Emet. The only thing Emet connects to is the data source you choose to connect: read-only, scoped by you, revocable at once (section 3).

  • No added latency.

    Exports run in the background, after your user already has the answer.

  • No production risk.

    An export that fails is logged and dropped. It never reaches the request that served your user.

How it fits

Your agent works exactly as it does today. Emet only enters the picture after your user already has the answer, and reads your connected data only to check it.

Sequence: the user asks a question; the agent calls its tools with arguments and gets results; the agent answers the user. Only then does the agent export the turn to Emet and receive an acceptance. Emet runs read-only queries against your data, gets the source results, checks each claim, and produces the Accuracy Report.

The path to a first report

Six steps in order: 1. Set up, 2. Connect, 3. Instrument, 4. Export, 5. Check, 6. Roll out.

Step
1. Set up
What happens
Create a team, register the application, mint a key
Who
Whoever owns the Emet account
Section
Step
2. Connect
What happens
Connect your data warehouse in the Emet dashboard, read-only
Who
Whoever administers your warehouse
Section
Step
3. Instrument
What happens
Find where each turn finishes and what it holds: messages, tool arguments and results
Who
The engineer who owns the agent
Section
Step
4. Export
What happens
Hook in the exporter after each turn
Who
The same engineer
Section
4 and 5
Step
5. Check
What happens
Send real turns and read the first reports
Who
The same engineer
Section
Step
6. Roll out
What happens
Move from a trial application to full traffic
Who
Your team
Section

Steps 3 and 4 happen in your codebase.

SECTION 2

Before you start

Two short lists: the answers your team needs before writing code, and the three things to create in the Emet dashboard.

Questions to answer first

Every one of these is about your own system, and each one maps to a field Emet needs.

Question
Which framework runs your agent, and does it already emit OpenTelemetry traces?
Why it matters
Most frameworks already hold the whole turn in a message history or in OpenTelemetry GenAI spans, which makes the exporter short.
Becomes
The Emet SDK for Python with Pydantic AI, or the exporter to start from
Question
Where does a turn finish in your code?
Why it matters
That is where the exporter hooks in, after the answer has gone out.
Becomes
The place you call the exporter
Question
Which identifier tells you whose work a turn is? Usually a workspace, account or client id your agent already holds.
Why it matters
Emet attributes every turn to a team by it. It cannot be added to traces already sent.
Becomes
tenant_ref
Question
Which identifier groups the turns of one conversation?
Why it matters
Emet shows a turn next to the ones around it.
Becomes
conversation_id
Question
Can you read each tool call's arguments as well as its result?
Why it matters
A claim can only be traced back to the query that produced it if the arguments were kept. This cannot be added later either.
Becomes
The arguments on each tool span

If your agent already records OpenTelemetry GenAI spans with content capture on, most of this data is already flowing, and section 4 shows how each attribute maps to the envelope. On Python with Pydantic AI, the Emet SDK reads those spans for you.

Set up the account

Create the team from Team in the sidebar. Register the application and mint its key under Applications.

  1. 01
    Create a team

    Open Team. One for each part of your company whose work should be reported separately. Give each one the external ID your agent will send as tenant_ref.

  2. 02
    Register the application

    Open Applications and add the agent system that will send traces. A name and a short description are enough.

  3. 03
    Mint a key

    For the application, leaving the team blank, so each trace is attributed by its own tenant_ref. The key is shown once; store it the way you store any other production credential.

One application per environment

Register a separate application for each environment that will send traces, such as “Nova Analyst (staging)” and “Nova Analyst (production)”, each with its own key. Trial traffic and real traffic stay apart on every report, rate limits are counted per application so a load test never throttles production, and revoking a staging key never touches production. Keys can carry an expiry date, rotate in pairs without downtime, and can be revoked at once.

What to hand your engineer

  • A key for the trial application. Emet's API is at https://app.emet.so.
  • The external ID of at least one team.
  • The answers to the questions above.
SECTION 3

Connecting your data

Your traces tell Emet what your agent saw. A data connection lets Emet check it against the source. With your warehouse connected, Emet runs its own read-only queries to confirm that the results your agent received match your data, that the query asked the right question of it, and that each claim in the answer holds up against the source, not only against the trace.

Emet connects to Databricks and other major data warehouses. You connect yours in the Emet dashboard, which walks you through it.

What the connection is

  • Read-only.

    Emet runs read queries only. It never writes, alters or deletes.

  • Scoped by you.

    Emet sees only the data you grant it access to.

  • Revocable at once.

    Remove Emet's access in your warehouse, and it stops immediately.

  • Visible in your own logs.

    Every Emet query runs through the connection you create, so it appears in your warehouse's query history alongside everything else.

How Emet learns your data

During onboarding, Emet maps your definitions: which tables hold which metrics, your attribution windows, fiscal calendar, currencies and time zones. That way its queries ask the same question your agent did, by your rules.

Before you connect

Compute
Emet's queries run on the warehouse compute you connect, so you control where they run and what they cost.
Credentials
Stored encrypted and used only for verification queries. You can rotate or revoke them at any time.
What Emet reads
Only the data behind the claims in your agent's answers. Emet re-runs the numbers; it does not browse or copy your tables.
Query results
Kept with the report they support, as the evidence behind each verdict.
SECTION 4

Capturing a turn

Emet takes each turn as one JSON document, the envelope, posted to a single endpoint. It carries the evidence: every model call and tool call in the turn, with what went in and what came out. Its fields follow the OpenTelemetry GenAI semantic conventions, so an agent already instrumented with them maps onto it field by field.

Sending JSON

Your exporter sends Emet one envelope per turn. It names whose work the turn was and carries the turn as a small tree of spans. Every model call and every tool call is its own span, and all of them hang off the turn: a turn with five tool calls across three model calls has nine spans. Every field and limit below is what the endpoint enforces today.

The request

Method and path
POST /api/v1/traces/
Authentication
Authorization: Bearer, followed by your application key
Body
The envelope, as JSON
Posting an envelope
curl -X POST "https://app.emet.so/api/v1/traces/" \ -H "Authorization: Bearer $EMET_APPLICATION_KEY" \ -H "Content-Type: application/json" \ --data @envelope.json

The envelope

Field
tenant_ref
Required
Yes
Meaning
The team's external ID. A value that matches no team is accepted and marked unattributed. A request without it is rejected.
Field
conversation_id
Required
Yes
Meaning
Groups the turns of one conversation.
Field
trace_id
Required
Yes
Meaning
Identifies this turn. Resending the same trace_id adds to the same trace.
Field
ingestion_mode
Required
Yes
Meaning
The route the turn took. "sdk" when an exporter in your app posts the envelope: the Emet SDK, a reference exporter, or your own. "telemetry" when the envelope is built from OpenTelemetry spans outside your app.
Field
spans
Required
Yes
Meaning
The spans of the turn, 1 to 1,000 per request.
Field
end
Required
No
Meaning
true on the last request of a turn, so Emet starts checking at once. A request that carries the turn span has the same effect.
Field
semconv_version
Required
No
Meaning
Which vocabulary your payload keys follow, stored with the trace so it stays readable as that vocabulary changes. The Emet SDK and the reference exporter send "emet-genai-1", the vocabulary on this page. An exporter that passes OpenTelemetry attributes through unchanged sends the GenAI conventions version its instrumentation emits.
Field
session_ref
Required
No
Meaning
A non-secret id that groups the turns of one agent session. Never a login token or cookie.
Field
enduser_ref
Required
No
Meaning
Who triggered the turn, like OpenTelemetry's enduser.id. Emet matches it to a member of your organization by email; other values are stored but not shown. Set it from your server's signed-in identity, never from a value your users can edit, and send it on the first request of the turn: the first value Emet receives is the one it keeps.
Field
resource
Required
No
Meaning
Describes the sending service: its name, version and deployment environment.

Rules the endpoint enforces

  • conversation_id, trace_id and span_id accept letters, digits and the characters _ . : - only, and an id made only of dots is rejected. They may be up to 255, 128 and 64 characters long.
  • Every timestamp carries a UTC offset, and no span ends before it starts.
  • A span_id appears once per request, and only one span per trace is the final output.
  • A request carries at most 1,000 spans and 32 MB. attributes and resource hold small metadata: 16 KiB each.
  • Text must not contain a NUL character, and numbers must be finite.

Span fields

Field
span_id
Required
Yes
Meaning
Unique within the trace, such as model.0 or tool.call_1.
Field
parent_span_id
Required
No
Meaning
Empty for the turn, the turn's span_id for everything else.
Field
name
Required
Yes
Meaning
A readable name, such as "chat" or "execute_tool get_spend".
Field
kind
Required
No
Meaning
turn, model, tool, retrieval or memory for most agents.
Field
started_at, ended_at
Required
Yes
Meaning
ISO 8601 timestamps with a UTC offset.
Field
status
Required
No
Meaning
ok or error. Set error on a tool that failed.
Field
is_final_output
Required
No
Meaning
true on the one model span that produced the answer.
Field
model_name, provider, tokens, finish_reason
Required
No
Meaning
For model spans. Optional, but they power accuracy by model, Usage & cost and AI Analytics in the Emet Observability Dashboard.
Field
tool_name, tool_call_id
Required
No
Meaning
For tool spans. tool_call_id links the span to the model's call.
Field
error_message
Required
No
Meaning
What went wrong, on a span with status error.
Field
attributes
Required
No
Meaning
Small key-value metadata of your own, such as which retry attempt a span belongs to.
Field
payload
Required
No
Meaning
The content Emet verifies. See below.

The payload

The payload is the evidence, and it is the part of the envelope that matters most. Its keys depend on the span kind:

Span kind
tool
Payload keys
arguments, result
Holds
What the tool was called with and what it returned. Each is a JSON object or a string.
Span kind
model
Payload keys
input_messages, output_messages
Holds
The conversation the model saw, in order, and what it produced: text, tool calls, or both.
Span kind
model
Payload keys
system_instructions, tool_definitions
Holds
Optional. A hash of your system prompt (never its text), and the tools the model was offered.
Span kind
retrieval
Payload keys
query, documents
Holds
What was searched for and what came back.
Span kind
memory
Payload keys
query, records
Holds
What was read from or written to memory.

Each message is a role (user, assistant or tool) and a list of typed parts: text for anything the user or the model wrote, tool_call for the model asking for a tool (id, name, arguments), and tool_call_response for the tool's answer as the model saw it (id, response). Some frameworks, PydanticAI among them, send tool answers in a user message made only of tool_call_response parts; Emet skips those when it picks the turn's question.

A worked example

One question, one tool call, one answer: “What did we spend on paid search last week?”

The turn span carries timing only. Under it hang three spans: model.0, which calls a tool; tool.call_1, which carries the arguments and the result; and model.1, which writes the answer and is marked as the final output.

Span
turn
Kind
turn
What it carries
The start and end of the whole turn. No payload.
Span
model.0
Kind
model
What it carries
Input: the user's question. Output: a call to get_spend for paid search, 7 to 13 September.
Span
tool.call_1
Kind
tool
What it carries
Arguments: channel paid search, 7 to 13 September. Result: spend 48,210.55 USD.
Span
model.1
Kind
model
What it carries
Input: the question, the tool call and its result. Output: “Paid search spend last week was $48,210.55.” Marked as the final output.

The span below is the one that carries the evidence. The arguments say which question was asked of the data; the result is what came back.

The tool span from this example
{ "span_id": "tool.call_1", "parent_span_id": "turn", "name": "execute_tool get_spend", "kind": "tool", "status": "ok", "tool_name": "get_spend", "tool_call_id": "call_1", "started_at": "2026-09-14T09:12:05.020Z", "ended_at": "2026-09-14T09:12:06.410Z", "payload": { "arguments": { "channel": "paid_search", "start": "2026-09-07", "end": "2026-09-13" }, "result": { "spend": 48210.55, "currency": "USD" } } }

If your agent emits OpenTelemetry

On Python with Pydantic AI, the Emet SDK builds the envelope from your spans in your own process, and you can skip this part. Otherwise, build it from your spans: each GenAI attribute below maps to a field in it. Direct OpenTelemetry ingestion (OTLP) is available on request; your Emet contact gives you the endpoint and setup.

gen_ai.input.messages, gen_ai.output.messages
input_messages, output_messages on a model span
gen_ai.system_instructions, gen_ai.tool.definitions
system_instructions, tool_definitions on a model span
gen_ai.tool.call.arguments, gen_ai.tool.call.result
arguments, result on a tool span
gen_ai.retrieval.query.text, gen_ai.retrieval.documents
query, documents on a retrieval span
gen_ai.memory.query.text, gen_ai.memory.records
query, records on a memory span
gen_ai.conversation.id
conversation_id
enduser.id
enduser_ref, set to the user's email. Emet matches members by email, so if enduser.id holds an internal id, look up the email or leave the field out.

What your spans need

  • Content capture on.

    Without it, messages, tool arguments and results are never recorded, and Emet has no evidence to check.

  • The turn's team.

    Each turn carries the team's external ID from section 2, so its report lands with the right team.

  • The turn's conversation.

    Set gen_ai.conversation.id, so Emet can show a turn next to the ones around it.

  • Tool arguments as well as results.

    A claim can only be traced back to the query that produced it when the arguments are there.

  • Your system prompt stays private.

    Standard instrumentation records it as text. To keep it private, hash or remove gen_ai.system_instructions before export, as the Emet SDK and the reference exporter do.

SECTION 5

Building the exporter

The exporter builds the envelope from section 4. It is one function in your codebase: it turns a finished agent turn into the envelope and posts it.

Python with the Emet SDK

On Python with Pydantic AI, you do not build an exporter. The open-source Emet SDK, emet-sdk on PyPI (FUSEis/emet-py), reads the OpenTelemetry spans Pydantic AI already emits and sends each turn for you, applying the design rules below. It runs on Python 3.10 to 3.14, and Pydantic AI on Python 3.11 or later.

With a coding agent

emet-cli sets up the SDK with your coding agent, such as Claude Code or Cursor. You type the key into your own terminal, never into the agent's chat. login saves it to the app's .env, which the setup tells the agent not to read. To back that up, deny your agent read access to .env, for example with "Read(./.env)" under permissions.deny in Claude Code's .claude/settings.json. That rule covers Claude Code's file tools and shell commands such as cat, but not a script that opens the file itself; only Claude Code's sandbox blocks every process.

  1. 01
    Save your key.

    In your own terminal, in the app's folder, run uvx emet-cli@0.1.3 login and paste a key for your trial application. uvx comes with uv.

  2. 02
    Hand the setup to your agent.

    Tell your coding agent: “Follow https://github.com/FUSEis/emet-py/blob/v0.1.3/cli/AGENT_SETUP.md to set up Emet.” It installs the SDK, wires it into your code, shows you the diff, and asks you to send one question through the app.

  3. 03
    Check the turn.

    Once the app has answered that one question, run uvx emet-cli@0.1.3 check --last. It reads the most recent turn's result and says whether it reached Emet whole, and the fix if it did not.

By hand

Install the SDK
uv add emet-sdk # or: pip install emet-sdk
Wrapping a turn (FastAPI and Pydantic AI)
import emet_sdk from pydantic_ai import Agent # Once, at startup, after your own tracing setup emet_sdk.configure( service_name="support-agent", on_delivery=emet_sdk.write_delivery_file(".emet/last-delivery.json"), ) Agent.instrument_all() @app.post("/chat") async def chat(body: ChatIn, user=Depends(current_user)): # From your server's own auth and storage, never the request body thread = await threads.get_for_user(user, body.thread_id) # All three values are strings with emet_sdk.turn(tenant_ref=user.team_external_id, conversation_id=str(thread.id), user=user.email): result = await agent.run(body.message) return result.output

Set EMET_APPLICATION_KEY in the app's environment. Without it, the app logs “Emet is off” once and runs as before. write_delivery_file saves each turn's result for emet-cli check --last, and writes only when its .emet folder exists, which emet-cli login creates. Setting up by hand, create it with mkdir .emet and add it to .gitignore. The SDK's README covers streaming responses, the delivery result and every option.

What the SDK handles for you

  • Only spans started inside turn() are sent. Your other agents, HTTP calls and database queries never leave the app.
  • It adds itself to your existing OpenTelemetry setup, so Logfire or any other exporter keeps receiving the same spans.
  • Each turn is sent on a background thread after the answer, split across requests within Emet's limits (a single span over 32 MB is refused), with one retry on a timeout, a 5xx or a short 429. Nothing that goes wrong while tracing or sending raises into your app, and a turn without a team or conversation is skipped rather than failing the request.
  • Known secret formats, such as provider API keys, tokens, private keys and passwords in URLs, are redacted from every string. Remove any other sensitive values from tool results yourself.
  • The system prompt is sent as a SHA-256 hash, and files and images become their type, size and digest.

The rest of this section is for TypeScript and any other stack, where you build the exporter yourself.

Three steps, any stack

  1. 01
    Collect during the turn.

    For each model call keep the messages in and out, the model name, token counts and timing. For each tool call keep the name, the arguments, the result and timing. Most agent frameworks already hold all of this in the run's result once the turn completes.

  2. 02
    Build the envelope when the turn completes.

    One model span per model call and one tool span per tool call, all under one turn span. Mark the model span that produced the answer as the final output.

  3. 03
    Send it in the background.

    After the answer has gone out, post the envelope with a short timeout.

Design rules

Export after the answer is sent, never before.
Your users see no added latency.
Never let the export throw into the request.
A failed export is a missed report, not a failed answer. Log it and move on.
Use a short timeout, around 8 seconds, and retry only 429 and 5xx, briefly.
A slow network never holds a worker open, and a passing hiccup does not cost a report.
Take tenant_ref, the conversation and its history from your own authorization and storage, never from the request body.
Whoever controls tenant_ref decides which team's reports a turn lands in, and history taken from the browser could plant evidence the model never saw.
Keep tool arguments, not only results.
A claim can only be traced back to its query when the arguments are there.
Send a hash of the system prompt, not the text.
It answers “did these turns run under the same prompt?” without sending the prompt itself.
Send the signed-in user's email as enduser_ref, from your server's session, and nowhere else. Send no credentials or connection strings.
Emet attributes usage and alerts to the member of your organization with that email. For anyone without an Emet login the value is stored but matched to no one, so leave it blank if you would rather not share it. Strip secrets from tool arguments, results and error messages.
Replace files and images with a short descriptor: media type, size and digest.
Emet verifies figures, not binary content, and the request stays small.

The reference exporter

For the Vercel AI SDK, we maintain a reference exporter that applies every rule above. Your team can adapt it rather than write one from scratch; we share the source files with you.

Framework
Vercel AI SDK (TypeScript)
Tested with
ai 6.0.116, strict type checking
Reads the turn from
Each step of result.steps. The fields it reads mean the same in AI SDK 6 and 7.
Where it runs
In Next.js, inside after(), which runs once the response has been sent.
Framework
Any other framework
Tested with
—
Reads the turn from
The run's own record of the turn: every framework keeps one.
Where it runs
Any background task or queue that runs after the reply.

In a Next.js route, the only change is to note when each step finishes, then hand the finished run to the exporter inside after(), which runs once the response is out.

Hooking into a Next.js route (Vercel AI SDK)
const result = await generateText({ model, system, tools, messages, onStepFinish: () => { stepEndedAt.push(new Date()); }, }); after(() => sendToEmet(buildEnvelope({ tenantRef: workspace.id, // from your auth, never the request body conversationId: thread.id, enduserRef: session.user.email, // the signed-in user, from your session traceId: crypto.randomUUID(), startedAt, endedAt: new Date(), system, messages, steps: result.steps, // every model call, tool call and result stepEndedAt, })));

Test before you ship

Point the exporter at your trial application and run a handful of real questions, including one where a tool fails and one that takes several tool rounds. Each post should come back 202 with no capture gaps.

What the reference exporter handles

A turn with several rounds of tool calls
Rebuilds, for every model call, exactly the conversation the model saw at that point.
Earlier turns in the same conversation
Includes them in the history the model saw, and emits spans for the current turn only.
Parallel tool calls
One tool span per call, each linked to the model's call by its id.
A tool that fails
Marks the span as an error, keeps the arguments, and sends no result. Emet does not count it as a capture gap.
A tool that returns nothing
Sends an explicit empty result, so the call is not mistaken for missing evidence.
Lists, numbers, decimals, dates or ids
Converts them to plain JSON that Emet accepts.
Files and images returned by a tool
Replaces them with media type, size and SHA-256 digest.
The system prompt
Sends a SHA-256 hash that stays the same across every call and turn under the same prompt.
Identifiers from your own systems
Replaces characters Emet does not accept and keeps them within the length limits.
A 429 or a 5xx from Emet
Tries up to three times, waiting at least as long as Emet asks. Gives up on a wait longer than ten seconds so it never holds your worker.
SECTION 6

Edge cases

Real agents retry, stream, call tools in parallel and hand work to sub-agents. None of it needs special support from Emet, but each case has one right way to show up in the envelope.

A tool call that fails
Send the span anyway, with status error and what went wrong in error_message. Keep the arguments and leave the result out. A span with no result and no error status is reported as a capture gap, because Emet cannot tell a failure from a result that was never recorded.
Parallel tool calls
One tool span per call, each with its own tool_call_id, all under the turn.
Retries and fallback models
One model span per attempt, each with its own model name, timing and status. Only the attempt that produced the answer is the final output. A tool call made by a superseded attempt still happened, so send it.
Streaming
Stream to your user as usual and export once the stream has finished, from the finished steps or message history rather than the text chunks: the chunks carry prose, the steps carry the evidence.
Long multi-step loops
One span per call, all under the same turn. A request carries up to 1,000 spans.
Sub-agents
Add a span of kind agent for the sub-agent and point its model and tool spans at it. If the sub-agent's work is its own conversation with its own answer, export it as its own turn.
Long conversations
The history grows with every turn, and that is useful: an earlier tool result can be the evidence for a later claim. The only ceiling is 32 MB per request.
Very large tool results
Send them whole; they are evidence. If one turn truly exceeds 32 MB, split its spans across several requests with the same trace_id.
One turn in several requests
Post batches with the same trace_id; order does not matter and duplicates are stored once. Emet starts checking as soon as a request carries the turn span or sets end, so send those only with the last request. Otherwise it checks 30 seconds after the last span arrives, and anything later is stored but not checked again.
A turn that belongs to no team
It is accepted and marked unattributed, and filed under your organization. Worth an alert on your side: it usually means a new client whose team does not exist yet in Emet.

A failed tool call carries its status, the message and the arguments. No result, and no capture gap.

A failed tool call (fragment: name and timing omitted)
{ "span_id": "tool.call_2", "kind": "tool", "status": "error", "tool_name": "get_budget", "error_message": "budget API timed out", "payload": { "arguments": { "channel": "paid_search" } } }
SECTION 7

Going live

How to know the integration is right before it carries real traffic, and what to watch once it does.

Check the first turns

Send a handful of real questions through the trial application, including one that makes a tool fail, and check each one. The first four checks read the endpoint's response. If OpenTelemetry ingestion is enabled for your account, confirm your exporter logs each export as successful, then use the last three.

With the Emet SDK, send test questions one at a time and run uvx emet-cli@0.1.3 check --last after each, since it reads the most recent turn's result. It reads the SDK's delivery result and covers the first four checks at once: a turn is captured when every request was accepted, the turn found its team, nothing arrived late and no gaps were reported. Otherwise it says what went wrong and the fix.

Check
The post is accepted
Where
Your exporter's log
Healthy
202
Check
Tool arguments and results arrived
Where
The gaps in the response
Healthy
All three lists empty
Check
The turn found its team
Where
unattributed in the response
Healthy
false
Check
The turn was complete when checked
Where
late in the response
Healthy
false
Check
The report exists
Where
Reports in the dashboard
Healthy
One row per turn, named by the question
Check
Claims name their evidence
Where
The open report
Healthy
Each claim names the tool its evidence came from
Check
Teams are right
Where
The report's team line
Healthy
Changes with who asked, never the same team for everyone
A healthy response
{ "trace": "5f0c2b8e-1d4a-4c7e-9a31-2f6b8d0e4c19", "conversation": "a3e91f47-6b2d-4f08-8c5e-7d1a9b3c6e20", "accepted": 4, "late": false, "unattributed": false, "gaps": { "missing_tool_arguments": [], "missing_tool_results": [], "conflicting_final_outputs": [] } }

Roll out in stages

  1. 01
    Trial application.

    Real questions, trial key, until every check above passes.

  2. 02
    Production application, part of the traffic.

    A new application and key for production. Export a share of turns first if you want to watch volume on your side.

  3. 03
    All traffic.

    Export every turn. Reports accumulate per turn and stay available in your dashboard.

What to monitor

Responses other than 202
A mapping bug (400), a key problem (403), or a burst over the rate limit (429).
Non-empty gaps
A tool whose arguments or result stopped being captured, often after a framework upgrade, or a turn that marked more than one model call as its final answer.
unattributed: true
A client whose team does not exist yet in Emet, or an identifier that changed.
late: true
Spans sent after the turn was already being checked.

The Emet SDK logs under the emet_sdk logger on the first turn and whenever the outcome changes: a warning when a turn is not captured, and an info line when it is, which Python hides by default. It also hands every turn's delivery result to the on_delivery callback you pass to configure(), for your own monitoring.

The rate limit is 10,000 requests an hour per application. If your production traffic will run above that, talk to us before you switch on step 3.

Keys over time

Keys rotate in pairs, so a rotation never needs a coordinated deploy: mint the new key, deploy it at your own pace, then revoke the old one. Revoking is immediate, which also makes it the fastest way to stop an export.

After a framework upgrade

Agent frameworks change how they record a turn between major versions. After upgrading yours, send a few turns through the trial application and repeat the checks above before the upgrade reaches production.

SECTION 8

Handling errors

Every response Emet can send, what it means, and what your exporter should do about it. Branch on the status and the error body's code, never on the message text.

Status
202
Meaning
Accepted. The spans are stored and the turn will be checked.
What to do
Read gaps, unattributed and late, and log anything unexpected.
Status
400
Meaning
The envelope did not validate. The body names every field at fault.
What to do
Log the body and fix the mapping. Retrying unchanged fails the same way.
Status
403, code not_authenticated
Meaning
No Authorization: Bearer header. Any other scheme counts as none.
What to do
Send the key as Authorization: Bearer <key>.
Status
403, code authentication_failed
Meaning
A Bearer header whose key is unknown, expired, revoked or malformed (missing, or with extra parts), or whose Application or organization is deactivated.
What to do
Check the header carries exactly one key, then the key itself in the dashboard. Every one of these gets the same response on purpose.
Status
403, code permission_denied
Meaning
The key is pinned to an environment, and the batch's resource names a different deployment.environment.name or none at all.
What to do
Send the key's environment as deployment.environment.name in resource, or use that environment's key.
Status
413
Meaning
The request is over 32 MB.
What to do
Split the turn across several requests with the same trace_id.
Status
429
Meaning
Over 10,000 requests an hour for this application.
What to do
Wait for the Retry-After header, then retry.
Status
5xx
Meaning
Emet could not take the request.
What to do
Retry with backoff. Duplicate spans are dropped, so a retry never double-counts.

With the Emet SDK

The SDK handles these for you. It retries once on a timeout, a 5xx, or a 429 whose Retry-After is 5 seconds or less, splits each turn across requests within the limits, and stops a turn at the first request Emet refuses. A single span over 32 MB cannot be split; Emet refuses it with 413, and the delivery result reports span_too_large. Each turn's outcome is in its delivery result, and uvx emet-cli@0.1.3 check --last explains it with the fix.

For a key pinned to an environment, the SDK sends deployment.environment.name from your OpenTelemetry resource. Set it with OTEL_RESOURCE_ATTRIBUTES=deployment.environment.name=production, or use that environment's key.

What an error body contains

Field
error
Present on
Every error
Holds
A readable message, such as “Invalid input.”
Field
code
Present on
Every error
Holds
A stable code, such as invalid, throttled or request_too_large.
Field
status_code
Present on
Every error
Holds
The HTTP status.
Field
details
Present on
400, 403, 429
Holds
For a 400, the field errors, with each span's position in the spans list.
Field
limit_bytes
Present on
413
Holds
The request size limit.

Below, the first span was sent with a timestamp that has no timezone. details points at span 0 and names the field.

A real 400
{ "error": "Invalid input.", "code": "invalid", "status_code": 400, "details": { "spans": { "0": { "started_at": ["Timestamps must carry a timezone offset."] } } } }

Typical field errors

tenant_ref
This field is required.
spans, position 0, started_at
Timestamps must carry a timezone offset.
spans, position 2, payload result
Expected an object or a string.
trace_id
Identifiers may only contain letters, digits, and the characters _ . : - and may not be dots alone
spans
Only one span per trace may be the final output

Troubleshooting

Symptom
400 on a tool result or arguments
Likely cause
A tool returned a list or a bare number
Fix
Serialize it to a JSON string, or wrap it in an object.
Symptom
400 on a timestamp
Likely cause
A timestamp without a UTC offset
Fix
Send ISO 8601 with Z or an explicit offset.
Symptom
400 on an identifier
Likely cause
An upstream id with spaces, slashes or other characters
Fix
Replace characters outside the allowed set before sending.
Symptom
202, but gaps lists tool spans
Likely cause
Tool arguments or results were not captured
Fix
Read them from the framework's step or message history, not from the streamed text.
Symptom
202, but no report to open
Likely cause
The turn had nothing to check, or only metadata arrived
Fix
Make sure the payloads carry messages and tool results.
Symptom
Every report lands on the same team
Likely cause
tenant_ref is hardcoded in the exporter
Fix
Send the identifier of whoever the turn was for.
Symptom
unattributed: true on every turn
Likely cause
The team's external ID does not match what the agent sends
Fix
Compare the team's external ID in the dashboard with the value in the envelope.
Symptom
late: true
Likely cause
Spans arrived after the turn was already being checked
Fix
Send the turn span, or end, only with the last request of a turn.
Symptom
Claims without a verdict, data not captured
Likely cause
The tool result behind the claim is missing
Fix
Check that every tool span carries its result.

Ready to trust your AI data analysis?