Add OpenTelemetry traces and metrics

Approach

Configure OpenTelemetry with ASP.NET Core and HttpClient instrumentation plus the runtime and Kestrel meters, export over OTLP to a collector, and add your own ActivitySource and Meter for business operations.

Solution

using System.Diagnostics;
using System.Diagnostics.Metrics;
using OpenTelemetry.Metrics;
using OpenTelemetry.Resources;
using OpenTelemetry.Trace;

var builder = WebApplication.CreateBuilder(args);
builder.Services.AddOpenTelemetry()
    .ConfigureResource(r => r.AddService("orders-api", serviceVersion: "1.4.0"))
    .WithTracing(t => t
        .AddAspNetCoreInstrumentation()
        .AddHttpClientInstrumentation()
        .AddSource(Checkout.Source.Name)
        .AddOtlpExporter())                              // OTEL_EXPORTER_OTLP_ENDPOINT
    .WithMetrics(m => m
        .AddAspNetCoreInstrumentation()
        .AddMeter("Microsoft.AspNetCore.Server.Kestrel", Checkout.Meter.Name)
        .AddOtlpExporter());
var app = builder.Build();
app.MapPost("/api/checkout", () => Checkout.Run());
app.Run();

public static class Checkout
{
    public static readonly ActivitySource Source = new("Shop.Checkout");
    public static readonly Meter Meter = new("Shop.Checkout");
    static readonly Counter<long> Orders = Meter.CreateCounter<long>("shop.orders.placed");

    public static IResult Run()
    {
        using var activity = Source.StartActivity("checkout");
        activity?.SetTag("shop.payment_method", "card");
        Orders.Add(1, new KeyValuePair<string, object?>("shop.channel", "web"));
        return Results.Ok();
    }
}

Pitfalls

• High-cardinality tags (user IDs, order IDs) on metrics, exploding storage costs; keep them on traces.
• Exporting directly to vendors from every service instead of through a collector.
• Sampling everything at 100% in high-traffic production; use tail or ratio sampling.

Variations

• .NET Aspire's service defaults configure the same OpenTelemetry setup for you.
• Correlate logs by adding the OpenTelemetry logging provider (builder.Logging.AddOpenTelemetry).

Further reading: Observability with OpenTelemetry