Import Office 365 Message Trace logs into Azure Sentinel — Part 1

During my experiments with Azure Sentinel, I noticed that the Office 365 connector does not support Message Trace. A workaround is described in this article, and allows to analyze email trafic with fields like sender, receiver, date and subject from Azure Sentinel.

This first part deals with punctual import i.e. the import of logs only once. A second part will deal with periodic log import.

What is message trace ?

Message Trace is a Office 365 feature that allows administrators, or any allowed user, to get emails data (receiver, sender, etc…). Reports are built with these data, and are available for download via an ODATA API, located at the URL below.

Message Trace report — XML-formatted https://reports.office365.com/ecp/reportingwebservice/reporting.svc/MessageTrace.

If you clic on the link above, you are prompted to authenticate. You can use your global admin account. By default, the data returned is XML-formatted. This can be changed by adding ?$format=json at the end of our API URL:

Message Trace report — Json-formatted https://reports.office365.com/ecp/reportingwebservice/reporting.svc/MessageTrace?$format=json

More details regarding the returned data can be found here.

Integrate Message Trace logs with Azure Sentinel

When it comes to import custom logs into a Log Analytics workspace, there is a simple method detailed here. However, since our data are not a classic log format but either XML or JSON, we cannot use this method.

Instead we can create a simple Azure Logic App.

1. Download a JSON file from the Message Trace API. I named this file messagetrace.json.
2. Then, store this file within a Cloud solution (OneDrive, Google Drive, DropBox, etc…), and make sure it is supported by Logic Apps.
3. Create a Logic App that fetch this file and sends its content to our Log Analytics workspace.

Let’s get into this third step: the Logic App creation.

The first block of our Logic App is a Schedule trigger, that will fire our Logic App each 3 minutes. We actually don’t care about the frequency, because we will manually fire our Logic App once, then disable it.

Block 1 — Recurrence as a trigger

The second block of our Logic App is an action that read our messagetrace.json file content. Here I used OneDrive for Business to store my file. If you used any of the other supported Cloud solutions (Google Drive, Dropbox, etc…), the Get file content option (or similar) should be available.

Block 2 — Read file content

The third block of our Logic App is a Data operation connector. This connectors has two parameters: content and schema.

The content field needs input data, in our case, that is an expression:

• body(‘Get_file_content’) is the content of our file, formatted by default to octet-stream. If we don’t convert it to JSON, we won’t be able to retrieve data, and we will get an error at runtime.
• The json() function allows us to convert octet-stream to actual JSON

The JSON schema must be provided for our data to be properly parsed. The schema can be found here, juste copy/paste it.

Block 3 — Parse JSON data

The fourth and last block is a Control connector that allows us to create a foreach loop over all Message Trace records. For each element, we want to send it to our Log Analytics workspace.

Block 4 — Send each element to our Log Analytics workspace.

Finally, run the Logic App manually, then disable it. Depending on how large is your data, you may need to wait some time.

You can now retrieve all the different records from the Azure Sentinel log interface, as shown below. Note that the TimeGenerated attribute is actually the date of the import. Instead, analysts should focus on the Received_t helps to sort emails by date.

Message Trace logs from the Azure Sentinel interface

We figured out to integrate Message Trace logs within Azure Sentinel by using Logic Apps. This can be usefull for punctual needs such as forensics. In the next part, we will see how to periodically integrate Message Trace logs with Azure Sentinel.