Skip to main content

Set up Maester in GitHub

This guide will walk you through setting up Maester in GitHub and automate the running of tests using GitHub Actions.

Why GitHub?​

GitHub is the quickest and easiest way to get started with automating Maester. The free tier includes 2,000 minutes per month for private repositories which is more than enough to run your Maester tests daily.

Set up your Maester tests repository in GitHub​

Pre-requisites​

  • If you are new to GitHub, create an account at github.com

Use the latest, or keep control of your versions?​

You may want to always use the latest, or you may want to control the tests running each time. The two options below provide a choice for each.

1. Create a blank new repository to always use the latest available public Maester Tests​

  • Open https://github.com/new
  • Fill in the following fields:
    • Repository name: E.g. maester-tests
    • Add a README file: Select this option to initialize your repository
    • Private: Select this option to keep your tests private
  • Select Create repository

2. Create a new repository and import the Maester Tests repository, to keep updated yourself​

  • Open https://github.com/new/import
  • Fill in the following fields:
    • Your old repository’s clone URL: https://github.com/maester365/maester-tests
    • Repository name: E.g. maester-tests
    • Private: Select this option to keep your tests private
  • Select Begin Import

Set up the GitHub Actions workflow​

There are many ways to authenticate with Microsoft Entra from GitHub Actions. We recommend using workload identity federation as it is more secure, requires less maintenance and is the easiest to set up.

If you’re unable to use more advanced options like certificates stored in Azure Key Vault, which need an Azure subscription, there’s also guidance available for using client secrets.

  • Workload identity federation (recommended) uses OpenID Connect (OIDC) to authenticate with Microsoft Entra protected resources without using secrets.
  • Client secret uses a secret to authenticate with Microsoft Entra protected resources.

This guide is based on Use GitHub Actions to connect to Azure and uses the maester GitHub action.

Pre-requisites​

Create an Entra Application​

  • Open Entra admin center > Identity > Applications > App registrations
  • Select New registration
  • Enter a name for the application (e.g. Maester DevOps Account)
  • Select Register

Grant permissions to Microsoft Graph​

  • Open the application you created in the previous step
  • Select API permissions > Add a permission
  • Select Microsoft Graph > Application permissions
  • Search for each of the permissions and check the box next to each permission:
    • Directory.Read.All
    • DirectoryRecommendations.Read.All
    • IdentityRiskEvent.Read.All
    • Policy.Read.All
    • Policy.Read.ConditionalAccess
    • PrivilegedAccess.Read.AzureAD
    • Reports.Read.All
    • RoleEligibilitySchedule.Read.Directory
    • RoleManagement.Read.All
    • SharePointTenantSettings.Read.All
    • UserAuthenticationMethod.Read.All
  • Optionally, search for each of the permissions if you want to allow privileged permissions:
    • RoleEligibilitySchedule.ReadWrite.Directory
      • Required for eligible role assignments (Reference)
  • Select Add permissions
  • Select Grant admin consent for [your organization]
  • Select Yes to confirm

(Optional) Grant permissions to Exchange Online​

The Exchange Online Role Based Access Control (RBAC) implementation utilizes service specific roles that apply to an application and the below configuration allows the authorization chain to the App Registration you created in the previous steps.

The Exchange Online permissions are necessary to support tests that validate Exchange Online configurations, such as the CISA tests.

  • Open the application you created in the previous step
  • Select API permissions > Add a permission
  • Select APIs that my organization uses > search for Office 365 Exchange Online > Application permissions
  • Search for Exchange.ManageAsApp
  • Select Add permissions
  • Select Grant admin consent for [your organization]
  • Select Yes to confirm
  • Connect to the Exchange Online Management tools and use the following to set the appropriate permissions:
New-ServicePrincipal -AppId <Application ID> -ObjectId <Object ID> -DisplayName <Name>
New-ManagementRoleAssignment -Role "View-Only Configuration" -App <DisplayName from previous command>

(Optional) Grant permissions to Azure​

The Azure Role Based Access Control (RBAC) implementation utilizes Uniform Resource Names (URN) with a "/" separator for heirarchical scoping. There exists resources within the root (e.g., "/") scope that Microsoft retains strict control over by limiting supported interactions. As a Global Administrator you can elevate access to become authorized for these limited interactions.

The Azure RBAC permissions are necessary to support tests that validate Azure configurations, such as the CISA tests.

The following PowerShell script will enable you, with a Global Administrator role assignment, to:

  • Identify the Service Principal Object ID that will be authorized as a Reader and the Subscription ID to authorize for
  • Install the necessary Az module and prompt for connection
  • Elevate your account access to the root scope
  • Create a role assignment for Reader access over the Subscription and objects within
  • Create a role assignment for Reader access over the Entra ID (i.e., aadiam provider)
  • Identify the role assignment authorizing your account access to the root scope
  • Delete the root scope role assignment for your account
$servicePrincipal = "<Object ID of the Entra App>"
$subscription = "<Subscription ID>"
Install-Module Az.Accounts -Force
Install-Module Az.Resources -Force
Connect-AzAccount
#Elevate to root scope access
$elevateAccess = Invoke-AzRestMethod -Path "/providers/Microsoft.Authorization/elevateAccess?api-version=2015-07-01" -Method POST
New-AzRoleAssignment -ObjectId $servicePrincipal -Scope "/subscriptions/$subscription" -RoleDefinitionName "Reader" -ObjectType "ServicePrincipal"
New-AzRoleAssignment -ObjectId $servicePrincipal -Scope "/providers/Microsoft.aadiam" -RoleDefinitionName "Reader" -ObjectType "ServicePrincipal"
#Remove root scope access
$assignment = Get-AzRoleAssignment -RoleDefinitionId 18d7d88d-d35e-4fb5-a5c3-7773c20a72d9|?{$_.Scope -eq "/" -and $_.SignInName -eq (Get-AzContext).Account.Id}
$deleteAssignment = Invoke-AzRestMethod -Path "$($assignment.RoleAssignmentId)?api-version=2018-07-01" -Method DELETE

Add federated credentials​

  • Select Certificates & secrets
  • Select Federated credentials, select Add credential
  • For Federated credential scenario, select GitHub Actions deploying Azure resources
  • Fill in the following fields
    • Organization: Your GitHub organization name or GitHub username. E.g. jasonf
    • Repository: Your GitHub repository name (from the previous step). E.g. maester-tests
    • Entity type: Branch
    • GitHub branch name: main
    • Credential details > Name: E.g. maester-devops
  • Select Add

Create GitHub secrets​

  • Open your maester-tests GitHub repository and go to Settings
  • Select Security > Secrets and variables > Actions
  • Add the secrets listed below by selecting New repository secret
  • To look up these values you will need to use the Entra portal, open the application you created earlier and copy the following values from the Overview page:
    • Name: AZURE_TENANT_ID, Value: The Directory (tenant) ID of the Entra tenant
    • Name: AZURE_CLIENT_ID, Value: The Application (client) ID of the Entra application you created
  • Save each secret by selecting Add secret.

Enable GitHub Actions​

  • Open your maester-tests GitHub repository and go to Settings
  • Select Actions > General > Actions permissions
  • Select Allow all actions
  • Select Save

Create GitHub Action worklow for Maester​

  • Open your maester-tests GitHub repository and go to Actions
  • Select Skip this and set up a workflow yourself
  • Copy and paste the code below into the editor
  • Select Commit changes... to save the workflow
  • Select Actions > maester-daily-tests to view the status of the pipeline
name: Maester Daily Tests

on:
  push:
    branches: ["main"]
  # Run once a day at midnight
  schedule:
    - cron: "0 0 * * *"
  # Allows to run this workflow manually from the Actions tab
  workflow_dispatch:

permissions:
      id-token: write
      contents: read
      checks: write

jobs:
  run-maester-tests:
    name: Run Maester Tests
    runs-on: ubuntu-latest
    steps:
    - name: Run Maester action
      uses: maester365/maester@main
      with:
        client_id: ${{ secrets.AZURE_CLIENT_ID }}
        tenant_id: ${{ secrets.AZURE_TENANT_ID }}
        include_public_tests: true # Optional: Set to false if you are keeping to a certain version of tests or have your own tests
        step_summary: true         # Optional: Set to false if you don't want a summary added to your GitHub Action run
        artifact_upload: true      # Optional: Set to false if you don't want summaries uploaded to GitHub Artifacts
        # Other inputs are available and can be reviewed in the action.yml in the Maester repository

Manually running the Maester tests​

To manually run the Maester tests workflow

  • Open your maester-tests GitHub repository and go to Actions
  • Select Maester Daily Tests from the left pane
  • Select Run workflow drop-down from the right pane
  • Select Run workflow button to start the workflow
  • Select the running workflow to view the status

Viewing test results​

  • Open your maester-tests GitHub repository and go to Actions
  • Select a workflow run to view the results e.g. Maester Daily Tests

Summary view​

The summary view shows the status of the workflow run, the duration, and the number of tests that passed, failed, and were skipped.

Screenshot of GitHub worklflow run summary Page

Maester report​

The detailed Maester report can be downloaded by selecting the maester-test-results... file from the Artifacts section and opening the test-results.html page.

Screenshot of the downloaded Maester report

Summary Maester report​

A detailed summary of the Maester report can be viewed by scrolling down Summary page.

Screenshot of Maester report in the GitHub Summary Page

Logs view​

Select the Run Maester Tests job from Jobs in the left pane to view the raw logs.

Screenshot of GitHub workflow logs page

Keeping your Maester tests up to date​

The Maester team will add new tests over time. To get the latest updates, use the commands below to update your GitHub repository with the latest tests.

  • Clone your fork of the maester-tests repository to your local computer. See Cloning a repository.
  • Update the Maester PowerShell module to the latest version and load it.
  • Change to the maester-tests\tests directory.
  • Run Update-MaesterTests.
cd maester-tests\tests

Update-Module Maester -Force
Import-Module Maester
Update-MaesterTests