Log retrieval from Azure Log Analytics using Python

In this digitized world, software industry is one of the fast-growing. Sometimes more than products going into individual users, software companies provide services for software companies. This is kind of a B2B architecture. Companies like Google, Amazon and Microsoft provide services to smaller scale companies. A similar service provided by azure is azure log analytics.

Azure is a cloud based solution provided by Microsoft. This involves many sub-services. Today we are looking into the Log Analytics service. Some business applications will be dealing with larger scale servers and comparative to simple applications, these server logs contain much high value. Since these are large scale, we need separate services to manage them. AWS Cloudwatch and Azure Log Analytics are similar services for log management. My previous blog contains details on how to retrieve logs from Cloudwatch. In this blog will focus on Azure Log Analytics.

Before going into retrieval, we need to understand a few things that are different from AWS Cloudwatch. In log analytics we are required to use Kusto QL to query. IMO this is very similar to SQL. So anyone who has a basic understanding of how SQL works will be able to get the hang of Kusto QL easily.

There’s three ways that you can retrieve logs out of Log Analytics. First is pretty simple. Once you log into the portal and navigate to the log analytics section, you can find a section to query for logs. When you query and get the response, you can download the response in a suitable format you want. This is the easy first method.

The second method is to retrieve via the Log Analytics python SDK. Click here to go to the SDK documentation. You can execute

pip install azure-loganalytics

and the SDK will be installed locally. After installing you should run

az login

and it will direct you from the terminal to your default web browser and you will be prompted to log into azure portal. So here you should log into the portal which has access to the log analytics workspace you need to access. Once you login, the SDK will authorize into that account and then you can do all required communication through that SDK. This communication happens through the rest API provided by log analytics. But we will not need to manually access the API. The SDK will manage that task. Will look into a small example.

First you will have to import the SDK.

from azure.loganalytics import LogAnalyticsDataClient
from azure.common.credentials import get_azure_cli_credentials
from azure.loganalytics.models import QueryBody

Next we cam add the the query and use the SDK to retrieve the logs.

creds, _ = get_azure_cli_credentials(resource="https://api.loganalytics.io")
log_client = LogAnalyticsDataClient(creds)
work_space_id = 'abc'
result = log_client.query(work_space_id, QueryBody(**{'query': 'query'}))

Here, you will have to replace the workspace id and the query with actual values. After replacing you can go ahead and execute this program. You can print ‘result’ to see the response.

Now lets look into the third method. Here we will be using an AD Application to retrieve the logs. You will have to create an active directory application from azure end and once you do, you will receive some credentials. A tenant id, a client id and a client secret. We can use these credentials to generate a bearer token. Using the barer token, we can authorize to log analytics and query for logs. This communication is done by us directly through the rest API.

The documentation has provided with a step by step process from creating the AD app to retrieving the logs through the REST API here.

That’s all about it. If you require to explore more about this service, visit the original documentation.

Kudos