Incidentary Docs

Go

Instrument your Go services with the Incidentary SDK.

Go Quickstart

This guide gets you from zero to your first captured trace in a Go service.

Prerequisites

  • Go 1.21+
  • An Incidentary workspace and API key (from the dashboard under Settings → API Keys)

Install

go get github.com/incidentary/sdk-go

Initialize the client

import (
    "os"
    incidentary "github.com/incidentary/sdk-go/incidentary"
)

cfg := incidentary.DefaultConfig(os.Getenv("INCIDENTARY_API_KEY"), "my-service")
client := incidentary.New(cfg)
defer client.Teardown() // clean up integrations on shutdown

Inbound middleware

net/http

mux := http.NewServeMux()
mux.HandleFunc("/orders/", handleOrders)
http.ListenAndServe(":8080", incidentary.Middleware(client, mux))

chi

r := chi.NewRouter()
r.Use(incidentary.ChiMiddleware(client))

gin

handler := incidentary.GinMiddleware(client, router)
http.ListenAndServe(":8080", handler)

echo

e.Use(echo.WrapMiddleware(incidentary.EchoMiddlewareFunc(client)))

The middleware captures every inbound HTTP request automatically with no blocking overhead.

Outbound HTTP — one-line setup

Wrap your http.Client transport once, and all outbound HTTP calls automatically get trace headers injected:

httpClient := &http.Client{
    Transport: incidentary.WrapTransport(client, http.DefaultTransport),
}

// Every request through this client now propagates trace context
resp, err := httpClient.Do(req)

Alternatively, use InstrumentedDo for individual calls with retry metadata:

resp, err := incidentary.InstrumentedDo(client, httpClient, ctx, req, nil)

Queue instrumentation

The SDK provides helpers for propagating trace context through message queues:

// Producer: inject trace context into message headers
headers := incidentary.InjectQueueContext(ctx, incidentary.QueueHeaders{
    "content-type": "application/json",
})
// Publish message with these headers

// Consumer: wrap your handler to auto-extract context
handler := incidentary.WrapQueueConsumer(client, func(ctx context.Context, headers incidentary.QueueHeaders, body []byte) error {
    // ctx now has trace context from the producer
    // all downstream calls (HTTP, DB, etc.) inherit this context
    return processMessage(ctx, body)
})

Database instrumentation

Wrap your database/sql connector to auto-record query events:

import "database/sql"

// Wrap the connector
connector := incidentary.WrapConnector(client, baseConnector)
db := sql.OpenDB(connector)

// All queries through this db now emit db_query events
rows, err := db.QueryContext(ctx, "SELECT * FROM orders WHERE id = $1", orderID)

Database events appear as INTERNAL context events in the trace — SQL text is truncated to 500 characters, and query parameters are never recorded.

gRPC support

// Outbound: inject trace context into gRPC metadata
md := incidentary.InjectGRPCContext(ctx, incidentary.GRPCMetadataCarrier{})
// Set md as gRPC metadata on your call

// Inbound: wrap handler to extract context
handler := incidentary.WrapGRPCHandler(client, incomingMetadata, func(ctx context.Context) error {
    return handleRPC(ctx)
})

Recording custom events

Queue, job, and webhook events can also be recorded manually:

client.RecordQueuePublish(incidentary.RecordEventOptions{
    EventAttrs: map[string]interface{}{"topic": "orders.created"},
})
client.RecordQueueConsume(incidentary.RecordEventOptions{})
client.RecordJobStart(incidentary.RecordEventOptions{ParentCeID: parentCeID})
client.RecordJobEnd(incidentary.RecordEventOptions{})

Environment variables

VariableRequiredDefaultDescription
INCIDENTARY_API_KEYYesWorkspace API key (sk_...) from the dashboard
INCIDENTARY_SERVICE_NAMENoAlternative to passing service name in code
INCIDENTARY_ENVIRONMENTNoproductionEnvironment label
INCIDENTARY_API_URLNohttps://api.incidentary.comOverride for self-hosted or local dev

Verify capture

After deploying with the SDK installed:

  1. Make a request to your service
  2. Open the Incidentary dashboard → Traces
  3. A trace should appear within a few seconds

Debugging

state := client.GetPreArmDebugState()
// state contains trigger counters, lifecycle metrics, and retry quality distribution

Troubleshooting

401 from ingest: Verify your API key is a workspace key (sk_...), not a user token.

No traces in the UI: Ensure the middleware wraps your handler, not the other way around. The middleware must be outermost.

426 version rejection: Upgrade to sdk-go 0.2.0 or newer.

Services not linked in the trace: You are not using InstrumentedDo for outbound calls. See the outbound section above.

On this page