Azure Data Factory Monitoring | Cloudmon

Azure Data Factory

Microsoft Azure

Azure Data Factory

Discover the data factories in your connected Azure account, see each one with its version, public access setting, and integration runtimes, and track the health and performance measures Azure Monitor provides.

Overview

Azure Data Factory is a cloud data integration service for building pipelines that move and transform data.

Cloudmon automatically discovers the data factories in your connected Azure account. Navigate to Cloud → Azure → [Select Account] → Azure Data Factory to view the list.

To connect an Azure account, see How to Set Up Azure Monitoring.

Turning On Monitoring for Azure Data Factory

The Services tab of the Azure account shows a card for each Azure service. Select All to see every supported service. A service that is not yet monitored for the account shows an Enable button instead of a resource count.

To choose the services discovered for an account, edit the account and select them under Services to be Discovered, as described in Actions in Azure Monitoring.

Azure Data Factory List

The list displays all discovered data factories with the following details:

ColumnDescription
NameThe name of the resource as configured in Azure.
Resource GroupThe resource group the resource belongs to.
LocationThe Azure region where the resource is deployed.
VersionThe Data Factory version.
Public AccessWhether public network access to the data factory is enabled.
Integration RuntimesThe integration runtimes in the data factory.
StatusThe current status of the resource.
ActionsOptions for managing monitoring of the resource.

Metrics

Azure Monitor provides the following metrics for data factories:

MetricDescription
Succeeded, Failed, and Cancelled pipeline runsPipeline runs by outcome.
Succeeded, Failed, and Cancelled activity runsActivity runs by outcome.
Succeeded, Failed, and Cancelled trigger runsTrigger runs by outcome.
Integration runtime CPU utilizationCPU use of the integration runtime nodes.
Integration runtime available memoryFree memory on the integration runtime nodes.
Integration runtime queue length, Integration runtime queue durationHow many tasks are waiting for the integration runtime, and how long they wait.
Total entities count, Maximum allowed entities countThe number of pipelines, datasets, and other entities, compared with the limit.

Use Cases

  • Alert on Failed pipeline runs so broken data loads are noticed the same day.
  • Watch Integration runtime queue length and queue duration to see when the runtime needs more nodes.
  • Compare Total entities count with the maximum to avoid hitting the factory limit.

Troubleshooting

IssueWhat to check
The list shows "There are no Azure Data Factory Available"The account has no CDN profiles, or discovery has not run since they were created. Navigate to Cloud → Azure → [Account], open the menu (☰), and click Rediscover.
The service card shows EnableThe service is not monitored for this account. Edit the account and select Azure Data Factory under Services to be Discovered.
Resources are missing after discoveryConfirm that the application registered for Cloudmon has the Reader role on the subscriptions that contain the resources. See How to Set Up Azure Monitoring.