Latency percentiles on Fly.io
Fly.io latency percentiles come from a histogram: its proxy records every HTTP response time in a Prometheus histogram, fly_edge_http_response_time_seconds, but you only get a percentile by querying it with PromQL, and Fly.io has no built-in alerting. For what each percentile means, see P50 vs P95 vs P99 latency explained.
Get P50, P95 and P99 with PromQL
Apps with an HTTP handler publish the histogram with app, region, host and status labels. In the managed Grafana, or any Prometheus client pointed at your organization’s endpoint:
histogram_quantile(0.95, sum by (le) (rate(fly_edge_http_response_time_seconds_bucket{app="my-app"}[5m])))Swap 0.95 for 0.5 or 0.99, and add region to the by clause to compare regions. The 5xx error rate comes from the same histogram’s _count series, filtered on status.
Fly.io’s Prometheus keeps about 15 days of data.
Alerting on Fly.io
Fly.io leaves alerting to you: create Grafana alert rules on the Fly.io Prometheus data source with a contact point such as email or Slack, or run your own Prometheus that federates from Fly.io and use Alertmanager.
Which percentile to alert on
Alert on P95 over a 5 to 10 minute window, next to the 5xx error rate. A histogram percentile is an estimate from its buckets, which is fine for alerting but can flatten very slow outliers, so look at P99 when you investigate.