Configure Dependency Collection Using Insights SDK for .NET in Azure
Learn how to lower your data ingestion costs by customizing Application Insights to toggle dependency collection capabilities.
Oct 20, 2020 • 5 Minute Read
Introduction
Application Insights SDK for .NET offers powerful dependency collection capabilities. However, sometimes it can log too much information. Since Application Insights and Azure Monitor charge per GB of data ingestion, if your application's dependency collection is not configured correctly, your monthly Azure bill may be much higher than what it should be.
In this guide, you will learn how to customize Application Insights SDK for .NET to toggle different dependency collection capabilities and write a custom TelemetryProcessor to filter out telemetry, which can reduce the amount of telemetry your application emits and lower your data ingestion costs.
How Dependency Collection Works in .NET Applications
.NET offers two powerful features for collecting data about your application. These include:
- Diagnostic Source, a feature that allows libraries to publish messages such as instrumentation that other packages can listen to and even modify. Application Insights can automatically track dependencies to supported targets since these libraries implement Diagnostic Source publishers. Using this hook, it can read messages, enrich them with extra data, and log them.
- Event Source, a similar feature that can publish low-level events that other tools and libraries can use. These tools are not able to modify data in the context of your application like Diagnostic Source. Some examples of event source data that can be collected include SQL requests, garbage collection (GC) statistics, and memory allocations.
Azure Application Insights SDKs for .NET uses these two features to track and log dependency information for your application from supported targets.
Customize Dependency Collection
If you are using ASP.NET Core with the Microsoft.ApplicationInsights.AspnetCore package, there are a few global options you can configure to customize what information is logged.
services.ConfigureTelemetryModule<DependencyTrackingTelemetryModule>((module, options) =>
{
// disable all dependency collection
module.DisableDiagnosticSourceInstrumentation = false;
// disable command text
module.EnableSqlCommandTextInstrumentation = false;
});
You can use the ConfigureTelementryModule to globally disable SQL Text capture and all dependency collection. The SQL text can sum to a large amount of data, especially if your application is very chatty with a database.
Note: Version 2.14.0 of the SDK has disabled command text collection by default.
Create a Telemetry Processor to Conditionally Filter Telemetry
Rather than customize dependency collection globally, you can use telemetry processors to filter telemetry conditionally.
This feature allows you to inspect all telemetry inside your application and decide whether it should be logged or not.
If your application polls another service because it implements health checks or needs to check to maintain a singleton lock or active connection to a queue, then you will be publishing a lot of meaningless dependency logs to Application Insights.
/// filters out dependencies like polling queues that are not attached to any larger operation.
public class AzureDependencyFilterTelemetryProcessor : ITelemetryProcessor
{
private readonly ITelemetryProcessor _inner;
public AzureDependencyFilterTelemetryProcessor(ITelemetryProcessor inner)
{
_inner = inner;
}
public void Process(ITelemetry item)
{
if (item is Microsoft.ApplicationInsights.DataContracts.DependencyTelemetry dependency
&& dependency.Success == true
&& dependency.Context.Operation.Name == null
&& (dependency.Type == "Azure Service Bus"
|| dependency.Type == "Azure table"
|| dependency.Type == "Azure blob"
|| dependency.Type == "Azure queue"))
{
return;
}
_inner.Process(item);
}
}
The code snippet above demonstrates how you can filter out these low-value dependency events. If your application is polling a queue to check for new messages, this task will not be part of an operation. You can use operation.Context.Operation.Name == null to check whether the dependency was called in the context of processing an incoming HTTP request or background message or not. If the dependency is part of an operation, then you may want to keep those dependency traces.
Finally, you need to register your the TelemetryProcessor with the IoC container.
services.AddApplicationInsightsTelemetryProcessor<AzureDependencyFilterTelemetryProcessor>();
By filtering these dependency traces, you can reduce the number of events collected by hundreds of thousands of events if your application subscribes to many different queues.
Conclusion
Application Insights provides a rich set of capabilities for monitoring, logging, and diagnostics. If you are curious about some of the low-level details, you can learn more by reading How Azure’s Application Insights correlates telemetry.
