How Metra stores operational evidence
Metra stores and queries time series inside your Agent. Explore and dashboards read that retained evidence; the browser is not the metrics database. You remain responsible for the Agent’s cycles, storage, controller security, and recovery.
The path from canister to chart
Section titled “The path from canister to chart”- Collect: Monitor uses authorized platform reads; producers submit application measurements.
- Validate: the Agent checks Registry/forwarder authority before decoding; Metra checks descriptors, groups, timestamp order, sequences, and capacity.
- Store: Monitor writes frames; protocol V3 producers write bounded strips. Identity, timestamps, attributes, and values survive acceptance.
- Service: one host scheduler runs bounded sealing, folding into tiles, retention, and quarantine work.
- Query: Native retrieves original observations; PromQL computes supported expressions over projected values.
- Use: views/reports present evidence; Custra evaluates rules through its controlled path.
Platform monitoring needs no separate Metra canister. For producer SDKs and standalone interfaces, see custom metric integrations.
Embedded engine settings
Section titled “Embedded engine settings”Fresh Wasm installs leave Metrics, Logs and Traces Not enabled. The creation review selects Metrics by default, then initializes it through the same Admin management API used in Agent › Storage. Enabling Metrics does not start Monitor. Custra and manual Cycles operations remain core Agent features.
An active Admin configures a finite Metrics arena ceiling in 8 MiB steps (default 4 GiB), optional positive age and occupied-byte retention limits (default unlimited age and no occupied-byte budget), and refusal or oldest-data expiry at the occupied budget (default refusal). The ceiling cannot be removed or lowered below physical allocation, occupied plus reserved bytes, or the retention budget. The combined engine plan also applies.
Every change binds the exact Agent revision and operation identity, validates before allocation, applies synchronously, and records an audit outcome. Lost replies reconcile the original request; a retry never substitutes a new revision. Raising a ceiling does not reserve subnet capacity or promise successful growth.
Turning Metrics off first requires explicitly stopping Monitor and collected logs, disabling or redirecting active local rules and scheduled reports, and redirecting known embedded RUM consumers. Retirement stops intake and revokes lanes while preserving reads, retention and native maintenance. Unlimited retention may leave it Retiring indefinitely. Parking requires native evidence; re-enable retains the same native state and allocated storage. No purge or reset is available. Monitoring-dependent automatic top-ups stop evaluating when their monitoring stops; historical metrics remain readable during retirement, and manual Cycles operations remain available.
An Admin can still disable and clear the Prometheus reader credential while Metrics is retiring, disabled or never enabled. This does not initialize the engine. Issuing a new reader credential requires enabled Metrics.
Fresh-Agent producers return notReady until index initialization succeeds; failed initialization retains a recovery wake without publishing data. See engine initialization and capacity.
Who supplies the evidence?
Section titled “Who supplies the evidence?”- Monitor: cycles, consumption, memory, configuration, and query workload. Platform selectors require its protected origin; similarly named custom instruments cannot substitute.
- Registered producers: V3 writes bind to active Registry resource/revision with protected Agent/resource identity. SDK descriptors may use application scope.
- OpenTelemetry: send OTLP/HTTP to
metra-forwarder, never directly to the Agent. An Admin grants metrics-only V3 writes to a dedicated principal outside the owner, stored members, and unarchived Registry. Protected identity/origin gives no management role.
Write lifecycle
Section titled “Write lifecycle”Streams have nonreused IDs, generations, and advancing sequences. Registration establishes descriptor handles/groups; append_strips validates the whole bounded batch before publishing. Rejected batches remain unpublished.
Registry replacement/reactivation can advance generation. Old handles grant no continued authority; resolve pending work explicitly before using the new registration.
Storage lifecycle
Section titled “Storage lifecycle”Queries cover accepted open frames, strips, and checked tiles. Folding reclaims reusable extents. Corruption is quarantined as explicit failure; superseded revision-bound indexes fail closed. Metra owns sealing, folding, retention, summaries, and quarantine, scheduled under a bounded instruction meter.
Ordinary reads have a ninety-day window, independent of retention. Collection starts, failures, and cleanup may reduce coverage. Logical retention can hide observations before physical reclamation and does not immediately shrink stable memory.
Admission and internal registration reads use the Agent’s real absolute instruction counter and runtime heap measurements. Maintenance integrity faults are recorded as failed service outcomes in Agent audit; a later successful service call does not erase that historical evidence. A refused plan can reserve physical backing or record bounded allocation diagnostics without publishing observations, registrations or sequence advancement. Allocated stable-memory pages remain part of the storage bill.
Check metraStatus usage, capacity, and refusal reasons. Engine deadlines schedule further physical work. Closed groups reject writes as group(unknownGroup) until registered again; closing preserves retained observations.
Read lifecycle and trust
Section titled “Read lifecycle and trust”The Agent authorizes Candid reads before accepting scope. Negative/regex matchers cannot override protected attributes. Browser state stays scoped to Source, network, identity, execution mode, and Agent.
Queries bound instructions, heap, series, points, time, and output; Native batches share one instruction allowance. Custra instant queries require complete observations. Exhaustion never produces a falsely complete answer. Eligible ranges may split into bounded requests; inconsistent/incomplete assembly stays visibly incomplete.
HTTP Prometheus responses are uncertified. Metrics grant no settings or spending authority.
Capacity
Section titled “Capacity”Agent status shows engine usage, capacity, and query limits. Each write separately checks admission and reserves growth against measured instruction headroom. icLens applies per-producer quotas without a global series ceiling or active-series budget. Refusals never authorize silently dropping observations. See capacity fields and refusal codes.
Prefer bounded attributes such as operation/outcome. User IDs, request IDs, and free-form URLs can create a series per event and exhaust capacity. Keep private arguments and identifiers out of metrics.
Explicit histograms and cumulative counters can carry aggregates for future real user monitoring. This engine update does not add browser collection, public browser intake, sessions, logs or traces. Aggregate counts must remain population-weighted; never average percentiles. A useful application overview may need separately published application totals alongside bounded detailed dimensions. Metra’s standalone RUM qualification is not a capacity or cost guarantee for the Agent’s shared runtime.
Use Agent Storage to inspect maintenance and pressure, and Metra troubleshooting when queries or producers are blocked.
Write refusals
Section titled “Write refusals”A producer write that the engine refuses returns one of these codes. Backpressure is explicit: nothing is dropped silently.
| Engine refusal | Meaning and next step |
|---|---|
notReady |
The first timer wake has not completed random index keying. Wait for its retained recovery wake; do not replace the key with a fixed seed. |
maintenanceDue |
Growable indexes need bounded splits. Let scheduled engine service progress, then reconcile or retry the prepared request. |
indexCapacity |
This batch or collision chain cannot fit its bounded index. Reduce or reshape the request; retrying maintenance alone cannot repair it. |
registrationReadBudgetExceeded |
Registration’s internal epoch lookup reached its instruction or heap-growth budget. No registration was published. Reduce pending strip work or resource pressure before retrying. This is not corruption. |
storageExhausted |
A host ceiling or platform growth refusal prevented allocation. Inspect the capacity report, cycles, configured ceiling and retention before reviewing a change. There is no default 512 MiB engine ceiling. |
activeSeriesBudget |
The embedding host’s optional budget was exceeded. icLens does not configure such a budget. |
group(unknownGroup) after closure |
Idle retention or owner/stream closure retired the write binding. Register the group again; retained old data can remain readable. |