Telemetry as infrastructure for agentic engineering

The design of telemetry determines which questions can be answered from a system’s recorded activity, including questions nobody anticipated when the instrumentation was written. Successful operations, usage patterns, resource consumption and outcomes can all provide useful evidence.

An engineering agent can take on the repeated work of querying records, following a pattern into the source and checking an explanation against further data. The operator can redirect that investigation as they learn, without manually carrying out every query and code lookup.

The operator sets the purpose: which outcomes matter, what constraints apply and whether a finding warrants a change. Telemetry becomes reusable input to that engineering process, alongside source code, tests and documentation. Its quality affects how readily an agent can pursue a new question, not just how well an existing report answers a familiar one.

Learning from normal operation

Consider a hypothetical reporting service. Its requests complete correctly and within the expected response times. An operator wants to add another output format without increasing the compute budget and asks an agent how that budget is currently used.

The agent groups successful requests by input revision and processing stage, keeping releases and configurations separate. It finds requests for a table, a chart and an export using the same data snapshot. Each runs the preparation stage before producing its output. Most of their recorded processor time is spent in preparation.

That suggests a question: could those requests share the prepared data? The agent follows the stage records to the deployed source and checks what preparation depends on. The operator explains that reuse would be useful across formats, but each result must reflect the requested data revision and the caller’s access rights.

The next query estimates how often requests meet those conditions. A few repeated examples would not justify an architectural change. Frequency, potential savings and the cost of retaining prepared data help the operator decide whether sharing it is worth testing.

Recording useful observations

The useful observations extend beyond execution timings. Which capabilities are used together? Which outputs are generated but never consumed? Where do people repeatedly correct an automated result?

A request completing, an output being delivered and a user accepting that output are different events. A duration needs a defined boundary and unit. A usage count needs a population and time window. Without those definitions, an agent can produce a precise calculation that answers the wrong question.

The same applies to the engineering process itself. Build results, review decisions and human interventions can help reveal which kinds of changes repeatedly need rework. Passing tests and having a change accepted are distinct outcomes here too.

Recording these distinctions does not require storing every payload. References to inputs and results can preserve relationships while their contents remain subject to the appropriate access and retention rules.

Connecting observations to the system

In the reporting example, a total processor-time metric would show consumption but not which stages or requests offered an opportunity to reduce it. The useful analysis depends on links between a request, its input revision, processing stages and result. Work that crosses queues or service boundaries needs to retain those relationships; retries need separate identities so they are not mistaken for new demand.

Execution records also need a route to the code and effective configuration used at the time. The current checkout or today’s settings may describe different behaviour. A reference is only useful if it still resolves to the historical version.

These links let an agent move from a pattern in the data to the part of the system that could explain it, then back to the records to test that explanation. The data can remain in different stores. Recorded relationships avoid having to reconstruct connections from similar timestamps or message text. Where a connection is inferred, that distinction should remain visible.

Exploring with an agent

Queries can perform the grouping and arithmetic before results enter the model’s context. A useful result states what was measured, over which population and interval, and retains the query and references to supporting records. The agent can then reason over a focused set of evidence rather than spend its context searching an entire log stream. Existing query tools, filters and pagination may be sufficient.

The limits belong with the results: sampling, query limits, missing instrumentation and expired records can all affect a conclusion. The operator also brings knowledge that telemetry may not contain. Low usage could mean a capability is unhelpful, hard to discover, or reserved for an important seasonal task. The observation starts that discussion; it does not settle it.

Evaluating a change

Returning to the hypothetical reporting service, suppose the operator chooses to test shared preparation. The intended improvement is lower compute use for eligible requests, while preserving correct results, freshness, access boundaries and response times. A higher reuse rate alone would not establish that the change helped.

Compare the original and revised behaviour on equivalent workloads, retaining input, release and configuration references. Check whether preparation work actually decreased, what storage or coordination costs were added, and what happened to requests that could not share results. Tests check the conditions under which reuse is allowed; telemetry shows how the implementation behaves under the observed workload.

If the workload changes, a lower total cost is not by itself evidence of a better implementation. The agent can break the comparison down by request type and input size; the operator decides whether the measured trade-offs meet the original goal.

Ask an agent

Ask your agent to read this article and evaluate your system. Choose a question that matters to you: how the system is used, where work is spent, or what changed after a release. See where the agent can follow the evidence and where it has to guess. Those gaps are a useful starting point for improving what you can learn from the system.

If you’d like a second opinion on the findings, or help making the changes, contact us at Software Foundry.