Skip to main content

Azure Monitor Source Node

The azuremonitorsource node queries Azure Monitor Log Analytics workspaces using KQL and emits each result row as a pipeline record.

The source is compatible with Azure Sentinel (Microsoft Sentinel) workspaces and authenticates with Azure Service Principal credentials (Entra ID client credentials). Each row returned by the KQL query becomes one record in the pipeline.

Key Features​

  • KQL queries: run any Kusto Query Language expression against a Log Analytics workspace
  • Azure Sentinel compatibility: works with Microsoft Sentinel workspaces without additional configuration
  • Service Principal authentication: authenticates with Azure Service Principal credentials (client ID / client secret) supplied as a username/password credential
  • Configurable batch size: limit the number of rows returned per query execution via batchSize

Configuration​

FieldTypeRequiredDefaultDescription
workspaceIdStringYes—Log Analytics workspace ID to query
tenantIdStringYes—Azure Active Directory tenant ID
kqlQueryStringYes—Kusto Query Language (KQL) query to run against the workspace
credentialIdStringYes—ID of Azure Service Principal credentials (username/password) in jobContext.otherProperties
batchSizeintNo1000Maximum rows returned per query execution (must be at least 1)

KQL Query Tips​

KQL queries can filter by time range, severity, or any field available in the target table. A typical security-focused query:

SecurityEvent
| where TimeGenerated > ago(1h)
| where EventLevelName == "Error"
| project TimeGenerated, Computer, Activity

Key tips:

  • Always scope queries with a TimeGenerated filter to avoid scanning the entire table
  • Use project to return only the fields needed downstream — this reduces record payload size
  • The batchSize field caps the number of rows returned; add | top N by TimeGenerated desc inside the query to control ordering before the limit is applied

DAG Example​

jobContext:
  otherProperties:
    azure-cred:
      username: my-client-id
      password: my-client-secret
  metricTags: {}
  dlqConfig:

dag:
  - id: "source"
    commandName: "azuremonitorsource"
    config:
      workspaceId: "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
      tenantId: "yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy"
      kqlQuery: "SecurityEvent | where TimeGenerated > ago(1h) | where EventLevelName == 'Error' | project TimeGenerated, Computer, Activity"
      credentialId: "azure-cred"
      batchSize: 1000
    outputs:
      - "sink"

  - id: "sink"
    commandName: "stdout"
    config:
      encodingType: "JSON_OBJECT"