Datadog Integration
Prerequisites
| Requirement | Details |
|---|---|
| Datadog account | Admin role to create API/App keys and configure webhooks |
| Datadog plan | Pro or Enterprise (Webhooks integration required) |
| StackFlow role | super_admin |
Step 1: Generate API and App Keys
- In Datadog, go to Organization Settings → API Keys → New Key
- Name it StackFlow and copy the API Key
- Go to Organization Settings → Application Keys → New Key
- Name it StackFlow App Key and copy the Application Key
Key Scopes: Datadog Application Keys inherit the permissions of the user who creates them. Create a dedicated service account with read-only access for monitoring and incident query operations. The webhook delivery uses the API key only.
Step 2: Configure Monitor Webhook
- In Datadog, go to Integrations → Webhooks → New
- Set the URL to:
https://your-instance.stackflow-tech.com/prod/api/webhooks/inbound/datadog - Enable Use custom payload and paste the StackFlow payload template:
{
"alert_title": "$ALERT_TITLE",
"alert_type": "$ALERT_TYPE",
"alert_transition": "$ALERT_TRANSITION",
"priority": "$PRIORITY",
"hostname": "$HOSTNAME",
"tags": "$TAGS",
"url": "$LINK",
"monitor_id": "$ALERT_ID",
"timestamp": "$TIMESTAMP"
}
- In the Custom Headers section, add:
X-StackFlow-Secret: YOUR_WEBHOOK_SECRET - Click Save
Step 3: Tag Mapping
Configure how Datadog tags map to StackFlow fields. Tags follow the format key:value:
| Datadog Tag | StackFlow Field | Example |
|---|---|---|
env:production | Environment filter | Only alert on production monitors |
team:platform | assignment_group | Routes to Platform Engineering |
service:api | category | Sets incident category to api |
severity:critical | priority | Maps to P1 |
ci:CI0000789 | ci_id | Links CI in CMDB |
Step 4: Connect in StackFlow
- Navigate to Admin → Integration Hub → Datadog → Configure
- Enter the Datadog API Key and Application Key
- Enter your Datadog site (e.g.,
datadoghq.comordatadoghq.eu) - Configure tag-to-field mapping
- Set priority mapping: ALERT→P1, WARNING→P2, NO DATA→P3, INFO→P4
- Click Save & Test
Monitor to Incident Auto-Creation
When a Datadog monitor fires, StackFlow evaluates configured rules and automatically creates an incident. Deduplication ensures only one incident is created per monitor until it resolves:
# View auto-created incidents from Datadog
curl -H "Authorization: Bearer $TOKEN" "https://your-instance.stackflow-tech.com/prod/api/incidents?source=datadog&state=new"
Alert Deduplication: StackFlow uses the Datadog
monitor_id as a dedup key. If a monitor fires again before the existing incident is resolved, a work note is added to the existing incident rather than creating a duplicate.
Alert Routing Rules
{
"routing_rules": [
{
"condition": "tags.env == 'production' AND alert_type == 'metric alert'",
"action": "create_incident",
"assignment_group": "Platform Engineering",
"priority_override": null
},
{
"condition": "tags.service == 'payments'",
"action": "create_incident",
"assignment_group": "Payments SRE",
"priority_override": "P1",
"notify_slack_channel": "#incidents-payments"
},
{
"condition": "tags.env != 'production'",
"action": "ignore"
}
]
}
Field Reference
| Field | Required | Description |
|---|---|---|
| API Key | Yes | Datadog API key for inbound event authentication |
| App Key | Yes | Datadog Application key for query operations |
| Datadog Site | Yes | datadoghq.com, datadoghq.eu, us3.datadoghq.com |
| Webhook Secret | Yes | Secret for inbound webhook signature verification |
| Priority Mapping | No | Map Datadog alert levels to StackFlow priorities |
| Tag Mapping | No | Map Datadog tags to StackFlow incident fields |
| Auto-Resolve | No | Resolve SF incident when Datadog monitor recovers (default: true) |
Testing the Integration
- In Datadog, trigger a test webhook: Integrations → Webhooks → [your webhook] → Test
- Verify the StackFlow incident was created in ITSM → Incidents
- Resolve the Datadog monitor and verify the StackFlow incident moves to
resolved
# Manually send a test Datadog-format payload
curl -X POST https://your-instance.stackflow-tech.com/prod/api/webhooks/inbound/datadog -H "Content-Type: application/json" -H "X-StackFlow-Secret: YOUR_SECRET" -d '{"alert_title":"Test Monitor","alert_type":"metric alert","alert_transition":"Triggered","priority":"P2","hostname":"test-host","tags":"env:production,team:platform"}'
Troubleshooting
| Issue | Cause | Fix |
|---|---|---|
| Incidents not created | Routing rule filtering out alerts | Check routing rules in Integration Hub; test with action: create_incident rule without conditions |
| Wrong priority | Priority mapping misconfigured | Review priority mapping; verify Datadog $PRIORITY variable in payload |
| Duplicate incidents | Dedup not matching | Verify monitor_id field is present in the webhook payload template |
| API key 403 errors | App key permissions | Ensure App Key user has incidents_read and monitors_read permissions |