> ## Documentation Index
> Fetch the complete documentation index at: https://docs.formation.texturehq.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Error tracking

> Configure failure reporting and understand what is captured or scrubbed.

Formation can report failures to Sentry, its currently implemented error-tracking
adapter. Reporting is optional and off by default. It complements [run
inspection](https://docs.formation.texturehq.com/runtime/observability.md) and [OpenTelemetry](https://docs.formation.texturehq.com/telemetry.md). Once reports
reach Sentry, [Sentry alert routing](https://docs.formation.texturehq.com/operations/sentry-alert-routing.md) covers
which issues page a human, where that lands in Slack, and the beta triage SLAs.

## Configure in the dashboard

Open **Settings → Error tracking**, enter a DSN, select a sample rate, enable
**Send reports**, and save. The setting applies to the running process. The DSN
is encrypted with `MESH_AGENT_MASTER_KEY`; reads return only presence and a
redacted hint. Leaving the input blank keeps the saved value.

* **Send a test report** sends a synthetic event using the candidate or effective
  configuration. It does not save settings; it does send data to the provider.
* **Clear stored DSN** removes the saved credential and disables reporting.
* **Enable tracing** optionally enables the provider's own APM tracing. It is
  separate from error reports and off by default. OTLP remains independently
  configured through environment variables.

A save may return `status: "saved_restart_required"` if persistence succeeded but
live replacement failed. The previous reporter remains active until the saved
configuration can be applied; follow the dashboard's status rather than assuming
that a saved setting is already effective.

## Configure with environment variables

| Variable | Meaning |
| - | - |
| `MESH_ERROR_TRACKER` | `sentry` selects the environment-managed adapter. When unset, database settings may apply. |
| `MESH_ERROR_TRACKER_DSN` | Required endpoint/credential for the selected adapter. |
| `MESH_ERROR_TRACKER_SAMPLE_RATE` | Fraction of error reports, in `(0,1]`; default `1`. |
| `MESH_ERROR_TRACKER_TRACING` | Enable the adapter's APM tracing; default off. |
| `MESH_ERROR_TRACKER_TRACES_SAMPLE_RATE` | Fraction of traces when tracing is on, in `(0,1]`; default `1`. |

A nonempty `MESH_ERROR_TRACKER` claims the entire section: provider, DSN, reporting
and tracing options. Database settings are not mixed with it. The dashboard
shows the section as environment-managed and refuses writes with
`409 settings_env_managed`. With no environment selection or stored enabled
configuration, reporting is off.

Failure to initialize reporting is advisory: Formation logs the problem and continues
starting. It does not make a broken database or another required dependency
optional. Reports flush within a bounded shutdown window.

### Why a managed deployment sets these in the environment

The two configuration paths are not equivalent in posture, and for a hosted
deployment the difference is the whole point:

* **Dashboard (database) settings** turn reporting on *after* someone opens the
  settings page. Until then — including across a fresh deploy or a restart before
  anyone logs in — the earliest boot failures, the schema step, and the first
  requests report nowhere.
* **Environment variables** are read as the error tracker is installed during
  boot, before the database pool is opened and before migrations run, so
  reporting is live for the schema step and the first requests. The failures an
  operator most wants paged about are these early failures, which is exactly the
  window the dashboard path — which cannot report anything until the stored
  configuration is loaded after migrations, and only if someone saved it — cannot
  cover.

A managed install should therefore treat error tracking as infrastructure and set
it through the environment, so it is on from boot and cannot be left off by a
forgotten dashboard toggle. A self-host operator who prefers to configure it from
the dashboard loses nothing: with no environment selection, the database path is
unchanged.

### Managed deployment on Fly.io

On Fly.io the error-tracking variables are set as app secrets, which Fly injects
into the machine environment at boot. Set the secret **names** below; never commit
a real DSN to `fly.toml` or source control — a DSN is a credential and belongs in
`fly secrets`, alongside `DATABASE_URL` and `MESH_AGENT_MASTER_KEY`.

| Secret name | Value in a managed deploy |
| - | - |
| `MESH_ERROR_TRACKER` | `sentry` — selects the adapter and claims the whole section. |
| `MESH_ERROR_TRACKER_DSN` | The project's Sentry DSN. Credential; never in `fly.toml`. |
| `MESH_ERROR_TRACKER_SAMPLE_RATE` | Optional. Fraction of reports in `(0,1]`; omit for `1`. |
| `MESH_ERROR_TRACKER_TRACING` | Optional. `true` to enable Sentry APM tracing; omit for off. |
| `MESH_ERROR_TRACKER_TRACES_SAMPLE_RATE` | Optional. Fraction of traces in `(0,1]` when tracing is on. |
| `MESH_RELEASE` | Optional. The deployed image's revision, for regression grouping; a clean-checkout build otherwise derives it from the embedded VCS stamp. |

Set them with `fly secrets set` (one invocation restages the machines once, rather
than a restart per variable):

```sh theme={null}
fly secrets set \
  MESH_ERROR_TRACKER=sentry \
  MESH_ERROR_TRACKER_DSN="$SENTRY_DSN" \
  MESH_RELEASE="mesh@$MESH_IMAGE_REVISION" \
  --app <your-mesh-app>
```

Set `MESH_RELEASE` to the immutable revision of the **image being deployed**
(`$MESH_IMAGE_REVISION` above — the commit or tag the deploy pipeline built the
image from), not the revision of whatever checkout happens to run `fly secrets`.
The two can differ, and the value must track the image so `errtrack.Setup` groups
issues by the build that actually produced them; a clean-checkout build without it
falls back to the embedded VCS stamp, which is already the deployed revision. Keep
it in step with the image on each deploy. Supply the DSN from your own secret
store or shell (`$SENTRY_DSN` above); do not paste the literal value into a script
that lands in version control. `fly secrets`
values are not readable back from the platform, which is the property that keeps
the DSN out of `fly.toml` and logs.

Once these are set, the dashboard's **Settings → Error tracking** section renders
as environment-managed and read-only, and a `PATCH` to the settings API returns
`409 settings_env_managed`. That is intended: the deployment owns the
configuration, so the dashboard reports the DSN hint and source without offering
an edit that precedence would ignore. To hand control back to the dashboard on a
given install, unset the secrets (`fly secrets unset MESH_ERROR_TRACKER
MESH_ERROR_TRACKER_DSN --app <your-mesh-app>`); the section becomes editable again
and any previously stored configuration applies.

## What is reported

Capture points include HTTP panics, turn and delivery failures, inbound
persistence failures, selected boot failures, unexpected admin API failures and
authenticated dashboard error reports. Routine successful turns, expected setup
conflicts and ordinary observed messages are not incidents.

Delivery failures are distinguished from execution failures so a connector token
problem can be investigated separately from a model-provider problem. Run
records remain the detailed source for the corresponding work.

The dashboard forwards errors through Formation's authenticated, bounded and
rate-limited `POST /api/client-errors` endpoint. It does not ship a Sentry browser
SDK. This path does not provide source maps, automatic browser breadcrumbs or an
offline error queue, and errors before an authenticated session may not reach it.

## Data boundary

Reports carry bounded diagnostics, component/route labels, identifiers and stacks.
Conversation bodies, model prompts and responses, credentials and DSNs are not
intended report fields. The reporting layer applies known-secret and
credential-pattern redaction plus length limits before invoking the adapter.
These controls do not make arbitrary application text safe to log: maintainers
must avoid putting private content into errors and tags.

Detailed content remains behind the database and operator access boundary. For
retention and inspection, see [Observability](https://docs.formation.texturehq.com/runtime/observability.md). For adapter
and redaction implementation, see
[`internal/errtrack`](https://github.com/TextureHQ/formation/tree/main/internal/errtrack).


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.