How Office 365 Collects and Reports Audit Data

Office 365 Security and Compliance Center

Office 365 Unified Auditing

No one likes being audited. It is intrusive to be asked to account for your actions, but auditing is an important part of IT operations, if only to answer the perpetual question of who did what when.

Office 365 is no different to any other system. Auditing is required to understand what actions happen to create, modify, or remove messages, files, and other items from the various workloads. Originally, audit events were handled by the individually workloads. Now they are gathered from all corners and stored in the Office 365 audit log from where the audit data can be searched through the Office 365 Security and Compliance Center.

 

 

Knowing how auditing happens and how to interrogate the audit data is an important skill for an administrator, so let’s run through the basics of how auditing works inside Office 365.

Enabling Auditing

Before Office 365 gathers audit events for a tenant, the Office 365 audit log must be enabled. To do this, go to the Search & Investigation section of Security and Compliance Center and select Audit Log Search. If a link saying Start recording user and admin activity is present, click it to start the collection of audit events. If not, events are already being collected and you can start to interrogate the audit log. It cannot be emphasized enough that if you don’t enable auditing for a tenant, you won’t be able to see audit events. Also, audit collection only proceeds from the point that you enable auditing. There’s no retrospective availability for events.

It’s reasonable to assume that each workload has its own methods to generate audit data. To ensure commonalty, the events are transformed as they are ingested into the Office 365 audit log using a common schema to ensure that each event contains a defined set of fields in a set format. Workloads also support product-specific schemas to allow events that don’t occur in other workloads to be captured. For example, SharePoint Online supports product-specific schemas for file operations and sharing. This page explains what the properties of audit log entries mean.

Although Exchange Online administrative auditing is enabled by default, you have to enable mailbox auditing for mailboxes because this option is not set when mailboxes are created. A simple PowerShell command is sufficient to look for mailboxes that do not have auditing enabled and then enable auditing for those mailboxes.

[PS] C:\> Get-Mailbox -Filter {AuditEnabled -eq $False} | Set-Mailbox -AuditEnabled $True

Workloads That Generate Audit Events

Table 1 lists the Office 365 workloads that generate audit events and the time lag that exists between an event occurring and when you can expect it to appear in the event log. Microsoft has improved the lag for Exchange Online events (in particular) as these used to take up to 24 hours to appear. The full set of audit events is documented online.

Workload Events Lag
SharePoint Online User (created or modified files) and administrative (create new site, etc.) 30 minutes
OneDrive for Business User and administrative (as for SharePoint Online) 30 minutes
Exchange Online mailbox User, delegate, and administrative access 30 minutes
Exchange Online administrative activities Running administrative cmdlets (in EAC or elsewhere) 30 minutes
Azure Active Directory User Login events 30 minutes
Azure Active Directory Administrative events (added user or group, applied license, etc.) 24 hours
Yammer (Coming, not yet available in any of my tenants) 24 hours
Power BI User (viewed dashboard, downloaded report, etc.) 24 hours
Sway User (created, edited, or deleted Sway, etc.) 24 hours
Security and Compliance Center eDiscovery actions (create case, apply holds, changed search actions, etc.) 24 hours

Table 1: Sources of Office 365 audit events

Some Office 365 applications don’t supply audit data. For example, there’s no trace of events coming in for Microsoft Planner or Microsoft Teams. Although Teams is relatively new and still won’t be generally available until early 2017, it’s surprising that Planner is not yet audit-enabled. The same is true for PowerApps and Flow.

Even more surprising, Data Loss Prevention (DLP) events generated by Exchange transport rules are not captured in the Office 365 audit log. Events generated by Unified DLP rules (which apply to Exchange, SharePoint, and OneDrive for Business) are described in the audit schema but no events have shown up. Given that the schema for DLP events is new, Microsoft might be preparing the way for their capture and storage.

Office 365 keeps 90 days’ worth of audit events for a tenant. If you use the Advanced Security Management (ASM) option, 180 days are retained. ASM is part of the Office 365 E5 plan or is available as an add-on option. All users in a tenant must be licensed for ASM before 180 days of audit data are retained.

Data held in the audit log are immutable. Administrators can read from the audit log. They can download copies of data from the audit log. But they cannot amend or remove data from the audit log.

Searching Audit Events

The Audit Log Search option in the Security and Compliance Center is the basic way to access audit events. The process is straightforward:

  • Select the audit events that you want to look for from the drop-down Activities You are not limited to selecting events from a single workload and can combine events from all of the available Office 365 audit sources in a single search.
  • Define the date range to use (remembering that audit events are only available for 90 days).
  • Select the user that performed the action you are looking for or leave the Users field blank to retrieve data for all users.
  • Add supplementary information for the search such as the name of a file or site URL. If you want to search for a word in a document title, provide the full word rather than a partial substring. For example, if the document is called “Reporting and Auditing”, a search will find it if “Auditing” is provided, but won’t for “Audit”.
  • Click Search.

A list of matching events is displayed (Figure 1). To speed access, up to 150 items are retrieved and displayed. If more items are found, they are displayed if the user scrolls to the bottom of the list. Up to 1,500 items can be viewed in this manner.

Search Office 365 Audit Log
Figure 1: Searching Office 365 audit data (image credit: Tony Redmond)

Activity Alerts

If you’re trying to keep an eye on specific events, it can rapidly become boring to set up and execute the same search on an ongoing basis. To automate the process, you can create an Activity Alert based on a search. Only accounts that hold the Organization Configuration role for the Security and Compliance Center can create alerts.

When you’re happy that a search locates the desired audit data, click Add an Alert. The activities used for the search are copied to pre-populate the new alert. At least one recipient must be present for an alert. The name of the person who creates an alert is automatically added as a recipient and you might want to replace the name or add others to the list.

One thing to remember is that Alerts don’t support filtering based on a file or folder name, so you can’t use them to monitor changes made to a specific item. For instance, if your search is for file check-ins for a document called “Budget” and you use the search to create an alert, the alert will monitor for all file check-ins and not just for that document. When everything’s complete, click Save (Figure 2).

Office 365 Activity Alert
Figure 2: Defining an activity alert (image credit: Tony Redmond)

Alerts monitor the inbound stream of audit events from Office 365 workloads to check for events that match. When a matching event is detected, the alert fires and the notification is sent to the defined recipients (Figure 3).

Office 365 Alert message
Figure 3: An activity alert for a monitored condition (image credit: Tony Redmond)

Because a delay occurs before events are ingested into the Office 365 audit log, an alert won’t fire immediately a monitored action happens. However, it’s still easier to have automatic monitoring and notification in place 24 hours a day than to search the audit log periodically.

You can see the set of alerts defined for a tenant through the Manage Alerts option under Alerts in the Security and Compliance Center.

Alternative Investigation Options

Searching the Office 365 audit log to hone in on particular events is fine for small to medium tenants. However, even smallish tenants are quite capable of generating several thousand audit events daily. This means that it can be quite a task to use the Office 365 audit report to find the proverbial needle in a haystack of events.

If you want to pay more, you can buy additional software to help. The Office 365 audit log is available to third party to build their own version of an audit investigation tool. Microsoft’s Advanced Security Management (ASM) option is expensive but powerful, especially in its ability to pick up suspicious events through correlation and its knowledge of attacks that occur against other tenants. If you use the Office 365 E5 plan, there’s no reason to go anywhere else than ASM because you’re paying for it in the plan. ASM is also available as an add-on, but at $36/user/year it’s pretty expensive, especially as the number of users in a tenant grows.

Third party developers can use the Office 365 Management Activity API to access the Office 365 audit log and create products based on that data. At $6/user/year, Cogmotive Discover & Audit (Figure 4) is a cheaper option than ASM. The interface is customizable with filters and pivot tables to help administrators isolate audit events as they investigate incidents or look for anomalous situations. Cogmotive’s solution doesn’t include some of the high-end functionality found in ASM, but its lower price makes it an interesting option to investigate if you think you need more than the standard Office 365 audit report delivers.

Cogmotive Audit Reports
Figure 4: Cogmotive Discover and Audit (image credit: Tony Redmond)

Apart from taking a different approach to reporting, third-party products usually allow tenants to store audit data for longer periods. This is a big advantage for large enterprises who often need to hold audit information for years to meet regulatory requirements.

PowerShell searches

It’s sometimes faster to use the Search-UnifiedAuditLog cmdlet to interrogate the audit log, especially when a tenant generates a large number of audit events daily. Here’s an example that returns all file modified events for documents with “Auditing” in their title for a specific period:

[PS] C:\> Search-UnifiedAuditLog –StartDate '01-Nov-2016 09:00' –EndDate '01-Dec-2016 17:00'
–ObjectIds 'Auditing' –Operations FileModified -SessionCommand ReturnNextPreviewPage |
Format-Table UserIds, CreationDate, AuditData

 

UserIds                                CreationDate              AuditData

-------                                ------------              ---------

[email protected]  13/11/2016 14:35:44 {"CreationTime":"2016-07-13T14:35:44"

The AuditData field contains the actual audit entry. This field is formatted in Json value-pair notation. To make the output easier to read, use the Formatted parameter:

[PS] C:\> Search-UnifiedAuditLog –StartDate '01-Nov-2016 09:00' –EndDate '01-Dec-2016 17:00'
–ObjectIds 'Auditing' –Operations FileModified -Formatted -SessionCommand ReturnNextPreviewPage

Disabling Auditing

If you need to disable auditing for an Office 365 tenant, you can do so by setting the ingestion enabled property in the audit log configuration to $False as follows:

[PS] C:\> Set-AdminAuditLogConfig -UnifiedAuditLogIngestionEnabled $false

It can take up to 60 minutes before all of the workloads respect the new setting and stop sending events to the audit log.

It’s Good to Know

Auditing is never the favorite activity of an IT administrator. However, when auditing is needed, it’s best when comprehensive coverage is provided, and that’s what is available inside Office 365. Some gaps do exist, mostly in the newer applications, but those gaps are being steadily filled and the auditing infrastructure has matured nicely over the last two years. Let’s hope that progress is maintained.

Follow Tony on Twitter @12Knocksinna.

Want to know more about how to manage Office 365? Find what you need to know in “Office 365 for IT Pros,” the most comprehensive eBook covering all aspects of Office 365. Available in PDF and EPUB formats (suitable for iBooks) or for Amazon Kindle.