How To Emit Traces
Overview
Fullsend emits OpenTelemetry traces for every agent run. By default, traces are written to the run-telemetry.jsonl file that gets uploaded into an artifact in the workflow. Fullsend is able to send traces to a remote OpenTelemetry-compatible endpoint.
Follow this guide to configure a GitHub repository or organization to send traces to a backend like MLflow, Jaeger, Grafana Tempo, etc.
Before you begin
- An OTLP/HTTP-compatible endpoint and its URL (e.g.
https://mlflow.example.com:4318/v1/traces). - Authentication credentials for the endpoint (bearer token, basic auth, or API key) if required.
- Network reachability from where runs execute (your machine or CI runners) to the backend endpoint.
- The
ghCLI installed and authenticated, or access to the GitHub UI for setting variables and secrets.
Send traces from a repository
Determine the endpoint URL where your backend receives traces.
For most systems this is
<host>:4318/v1/traces.Set the endpoint as a GitHub Actions variable on the repository.
bashgh variable set OTEL_EXPORTER_OTLP_ENDPOINT --body "https://example.com:4318" --repo <owner/repo>OTEL_EXPORTER_OTLP_ENDPOINTgets/v1/tracesappended automatically by the SDK. If your backend uses a different path, useOTEL_EXPORTER_OTLP_TRACES_ENDPOINTand specify the full URL instead.Build the authentication header for your endpoint.
The header format depends on your backend. Common patterns include:
- Bearer token:
Authorization=Bearer%20<token> - Basic auth:
Authorization=Basic%20<base64 of user:password>
%20is the encoding for space, don't use spaces directly.- Bearer token:
Set the authentication header as a GitHub Actions secret on the repository.
bashgh secret set OTEL_EXPORTER_OTLP_TRACES_HEADERS --body "<authentication-header>" --repo <owner/repo>(Optional) Add backend-specific routing headers to the same secret.
Some backends require additional headers. For MLflow, append the experiment ID:
bashgh secret set OTEL_EXPORTER_OTLP_TRACES_HEADERS \ --body "<authentication-header>,x-mlflow-experiment-id=<id>" \ --repo <owner/repo>(Optional) Set resource attributes to tag traces with metadata.
bashgh variable set OTEL_RESOURCE_ATTRIBUTES \ --body "deployment.env=prod,category=a" \ --repo <owner/repo>Resource attributes may become filterable trace tags depending on your backend.
Send traces to an endpoint from a GitHub organization
Set the endpoint as an organization-level variable.
To make it visible to all repositories:
bashgh variable set OTEL_EXPORTER_OTLP_ENDPOINT \ --body "https://example.com:4318" \ --org <org> --visibility allTo restrict visibility to specific repositories:
bashgh variable set OTEL_EXPORTER_OTLP_ENDPOINT \ --body "https://example.com:4318" \ --org <org> --repos repo1,repo2,repo3Set the authentication header as an organization-level secret with the same visibility.
bashgh secret set OTEL_EXPORTER_OTLP_TRACES_HEADERS \ --body "<authentication-header>" \ --org <org> --visibility allTo restrict visibility to specific repositories:
bashgh secret set OTEL_EXPORTER_OTLP_TRACES_HEADERS \ --body "<authentication-header>" \ --org <org> --repos repo1,repo2,repo3
Disable trace export
Remove the endpoint variable and header secret from the repository or organization to stop sending traces to the endpoint. The local run-telemetry.jsonl file continues to be produced.
gh variable delete OTEL_EXPORTER_OTLP_ENDPOINT --repo <owner/repo>
gh secret delete OTEL_EXPORTER_OTLP_TRACES_HEADERS --repo <owner/repo>Disable the local trace file
Set OTEL_SDK_DISABLED to suppress all telemetry output, including the local file and any configured endpoint export.
gh variable set OTEL_SDK_DISABLED --body "true" --repo <owner/repo>To disable at the organization level and then re-enable individual repositories:
gh variable set OTEL_SDK_DISABLED --body "true" --org <org> --visibility all
gh variable set OTEL_SDK_DISABLED --body "false" --repo <owner/repo>See also
- Tracing with MLflow: experiment routing, Basic auth encoding, org-level organization, and cost column caveats
- Tracing Reference: span schemas, attribute tables, and environment variable reference
- Tracing Development Guide: implementation details for contributors
