Skip to content
Sponsor
3.0 preview2.3.0 stable

These docs describe the preview. Pick the version you run and every page marks what it doesn't have yet. Get the preview →
Maester 2.x docs (archived) →

Run Maester in Azure DevOps

View as MarkdownOpen in ChatGPTOpen in Claudellms.txt for the whole site

This guide will walk you through setting up Maester in Azure DevOps and automate the running of tests using Azure DevOps Pipelines.

Why Azure DevOps?

Section titled “Why Azure DevOps?”

Azure DevOps is a great way to automate the daily running of Maester tests to monitor your tenant. You can use Azure DevOps to run Maester tests on a schedule, such as daily, and view the results in the Azure DevOps interface.

Azure DevOps comes with a free tier that includes 1,800 minutes of Maester test runs per month (unlimited hours if you use a self-hosted agent).

Azure DevOps has native integration with Microsoft Entra including single sign on, user and group management as well as support for conditional access policies.

Set up the Maester repository in Azure DevOps

Section titled “Set up the Maester repository in Azure DevOps”

Pre-requisites

Section titled “Pre-requisites”

Create the repository for your pipeline

Section titled “Create the repository for your pipeline”

The Maester tests ship inside the Maester PowerShell module, so the repository doesn’t hold any tests. Azure Pipelines needs a repository for the pipeline definition, and that is all it has to hold.

  • Select Repos from the left-hand menu
  • In the Initialize main branch with a README or gitignore section, select Initialize to create the default repository for your project.
  • Later, you can commit a maester-config.json to change settings (see Configuration) and a custom folder for your own tests (see Writing custom tests). The pipelines below pick them up when they are there.

Set up the Azure Pipeline

Section titled “Set up the Azure Pipeline”

There are many ways to authenticate with Microsoft Entra in Azure DevOps. We recommend using workload identity federation as it is more secure, requires less maintenance and is the easiest to set up.

If you’re unable to use more advanced options like certificates stored in Azure Key Vault, which need an Azure subscription, there’s also guidance available for using client secrets.

  • Workload identity federation (recommended) uses OpenID Connect (OIDC) to authenticate with Microsoft Entra protected resources without using secrets.
  • Client secret uses a secret to authenticate with Microsoft Entra protected resources.

Pre-requisites

Section titled “Pre-requisites”

Create an empty Azure Resource Group

Section titled “Create an empty Azure Resource Group”

This empty resource group is required to set up workload identity federation authentication. No Azure resources will be created in this resource group and there are no costs associated with it.

  • Browse to the Azure portal and open the Resource groups blade.
  • Select + Create.
  • Enter a name for the resource group (e.g. Maester-Resource-Group)
  • Select any region
  • Select Review + create > Create

Create a new workload identity federation service connection

Section titled “Create a new workload identity federation service connection”
  • In the Azure DevOps project, go to Project settings > Service connections.
  • Select New service connection, and then select Azure Resource Manager.
  • Select Workload identity federation (automatic).
  • Specify the following parameters:
    • Subscription: Select an existing Azure subscription.
    • Resource Group: Select the resource group created in the previous step. (e.g. Maester Resource Group) Leaving this field empty will grant Contribute access to all resources in the subscription.
    • Service connection name: A name for this connection (e.g. Maester Service Connection)
  • Select Save to create the connection.

Grant permissions to Microsoft Graph

Section titled “Grant permissions to Microsoft Graph”
  • Select the service connection you created in the previous step (e.g. Maester Service Connection)
    • Service connections are listed under Project settings > Service connections.
  • Select Manage Service Principal to open the Service Principal in the Entra portal.
  • Select API permissions > Add a permission
  • Select Microsoft Graph > Application permissions
  • Search for each of the permissions and check the box next to each permission:
    • AccessReview.Read.All
    • AuditLog.Read.All
    • DeviceManagementApps.Read.All
    • DeviceManagementConfiguration.Read.All
    • DeviceManagementManagedDevices.Read.All
    • DeviceManagementRBAC.Read.All
    • DeviceManagementServiceConfig.Read.All
    • Directory.Read.All
    • DirectoryRecommendations.Read.All
    • EntitlementManagement.Read.All
    • IdentityRiskEvent.Read.All
    • NetworkAccess.Read.All
    • OnPremDirectorySynchronization.Read.All
    • OrgSettings-AppsAndServices.Read.All
    • OrgSettings-Forms.Read.All
    • Policy.Read.All
    • Policy.Read.ConditionalAccess
    • Reports.Read.All
    • ReportSettings.Read.All
    • RoleEligibilitySchedule.Read.Directory
    • RoleManagement.Read.All
    • RoleManagementAlert.Read.Directory
    • SecurityIdentitiesSensors.Read.All
    • SecurityIdentitiesHealth.Read.All
    • SharePointTenantSettings.Read.All
    • ThreatHunting.Read.All
    • UserAuthenticationMethod.Read.All

    To run preview tests, also grant these application permissions:

    • AgentIdentity.Read.All
    • AgentIdentityBlueprint.Read.All
    • AgentIdentityBlueprintPrincipal.Read.All
    • Application.Read.All
    • IdentityRiskyUser.Read.All
  • Optionally, search for each of the permissions if you want to allow privileged permissions:
    • ReportSettings.ReadWrite.All
      • Required to disable report obfuscation
  • Select Add permissions
  • Select Grant admin consent for [your organization]
  • Select Yes to confirm

Create Azure Pipeline

Section titled “Create Azure Pipeline”
  • Open your Azure DevOps project
  • Select Pipelines > New pipeline
  • Select Azure Repos Git as the location of your code
  • Select the repository you created earlier
  • Select Starter pipeline
  • Replace the content of the azure-pipelines.yml file with the code below
  • Verify the azureSubscription value is set to the service connection you created in the previous step (e.g. Maester Service Connection)
  • Select Validate and save > Save
  • Select Run to run the pipeline
  • Select Job to view the test results
# Maester Daily Tests


trigger:
  - main


schedules:
  - cron: "0 0 * * *"
    displayName: Daily midnight build
    branches:
      include:
        - main


pool:
  vmImage: ubuntu-latest


steps:
  - task: AzurePowerShell@5
    displayName: "Run Maester"
    inputs:
      azureSubscription: "Maester Service Connection"
      pwsh: true
      azurePowerShellVersion: LatestVersion
      ScriptType: InlineScript
      Inline: |
        # Connect to Microsoft Graph
        $accessToken = (Get-AzAccessToken -ResourceTypeName MSGraph).Token
        Connect-MgGraph $accessToken


        # Install Maester
        Install-Module Maester -Force


        # Configure test results (Maester writes the NUnit XML file, Pester is not required)
        $PesterConfiguration = @{
          TestResult = @{
            Enabled      = $true
            OutputPath   = '$(System.DefaultWorkingDirectory)/test-results/test-results.xml'
            OutputFormat = 'NUnitXml'
          }
        }


        # Run Maester tests: the built-in tests run from the module, -Path adds any custom tests and maester-config.json in the repository
        Invoke-Maester -Path '$(System.DefaultWorkingDirectory)' -PesterConfiguration $PesterConfiguration -OutputFolder '$(System.DefaultWorkingDirectory)/test-results'
  - publish: $(System.DefaultWorkingDirectory)/test-results
    displayName: Publish Maester Html Report
    artifact: TestResults
  - task: PublishTestResults@2
    displayName: Publish Maester Test Results
    inputs:
      testResultsFormat: "NUnit"
      testResultsFiles: "**/test-results.xml"
      failTaskOnFailedTests: true

Viewing test results

Section titled “Viewing test results”
  • Select Pipelines > Runs to view the status of the pipeline
  • Select on a run to view the test results

Summary view

Section titled “Summary view”

The summary view shows the status of the pipeline run, the duration, and the number of tests that passed, failed, and were skipped.

Screenshot of Azure DevOps Pipeline Run Summary Page

Maester report

Section titled “Maester report”

The Maester report can be downloaded and viewed by selecting the Published artifact.

Screenshot of the downloaded Maester report

Tests view

Section titled “Tests view”

The Tests tab shows a detailed view of each test, including the test name, duration, and status.

Screenshot of Azure DevOps Pipeline Tests Page

Logs view

Section titled “Logs view”

In the Summary tab select on any of the errors to view the raw logs from Maester.

Screenshot of Azure DevOps Pipeline Logs Page

Keeping your Maester tests up to date

Section titled “Keeping your Maester tests up to date”

The Maester team adds new tests over time. The built-in tests ship inside the Maester PowerShell module, and the pipelines above run Install-Module Maester -Force, so every run uses the latest release and its tests. There is nothing to update in your repository, which holds only the pipeline and, if you added them, your configuration and custom tests.

To control when you move to a new release, pin the version instead, for example Install-Module Maester -RequiredVersion 3.0.0 -Force, and raise it when you are ready.

Maester is free and open source under the MIT licence.SponsorContributorsDiscordMaester Cloud