Help Center/Integrations

Grafana OnCall

For admins

Send Grafana OnCall alert groups into Oncall, so they become incidents and page whoever is on call. You need to be an admin. It has two parts: a connection in Oncall, and an outgoing webhook in Grafana.

Add a connection in Oncall

  1. Open Settings → Integrations → Grafana OnCall and click Add connection.
  2. Give it a name. Under Page this team, choose the team to page. The default is your company’s default team.
  3. Oncall shows a Webhook URL and a Secret. The secret is shown once only.
Copy the secret when it is shown. If you lose it, rotate it and update Grafana.

Set up the webhook in Grafana

  1. In Grafana OnCall, go to Outgoing webhooks and create a new one.
  2. For the URL, use the webhook URL from Oncall.
  3. For Headers, enter {"X-Webhook-Token": "<secret>"} with your secret. If your Grafana version has no Headers field, add ?token=<secret> to the end of the URL instead.
  4. For Data, paste the template below. Oncall reads these fields, so keep them as they are.
  5. Set the trigger to Alert Group Created and save.
  6. Create a second outgoing webhook with the same URL, Headers and Data, and set its trigger to Status Change. This one carries acknowledgements and resolutions.
Data
{
  "alert_group_id": "{{ alert_group_id }}",
  "event_type": "{{ event.type }}",
  "title": "{{ alert_payload.labels.alertname }}",
  "severity": "{{ alert_payload.labels.severity }}"
}

From then on, Grafana alert groups arrive in Oncall as incidents. They are created, acknowledged and resolved in step with Grafana.

Manage a connection

Each connection has these controls. Send test alert, Rotate secret and Delete are icon buttons: hover over one to see its name.

  • Send test alert opens a real SEV4 test incident through this connection and pages whoever is on call, so you can check routing and paging. It doesn’t check your tool’s settings.
  • Rotate secret to get a new secret. Update Grafana with it.
  • Enable or Disable to turn the connection on or off.
  • Delete asks you to confirm: press and hold Hold to delete.

Related