Azure DevOps tests for Maester
· 7 min read
We are excited to announce that Azure DevOps tests are now available in Maester!
Description of Azure DevOps tests
Maester now includes an optional suite of Azure DevOps security and resource limit tests. These are shipped alongside the core commands but are only executed when you have an
active connection to an Azure DevOps organization (using the community ADOPS
PowerShell module). Each test
is defined by a pair of files under powershell/public/maester/azuredevops:
a markdown document with rationale and remediation guidance, and a PowerShell
script containing the implementation.
The tests are inspired by Learn - Azure DevOps Security Best Practices and include a subset of the available configurations and settings.
To run the entire set, you can simply connect and run Invoke-Maester:
Install-Module ADOPS
Connect-ADOPS -Organization <name>
Invoke-Maester
Certain cmdlets will use unsupported REST API endpoints in Azure DevOps and may result in error(s) when Azure DevOps endpoints are changed without notice, hence the -Force flag running for certain tests.
Permissions
- At least a basic license in Azure DevOps
- Certain tests require either organization-level permissions such as "Project Collection Administrator" or tenant-level permissions such as "Azure DevOps Administrator".
Available tests
| Test ID | Severity | Description | Link |
|---|---|---|---|
| AZDO.1000 | High | Third-party application access via OAuth should be disabled. | Learn more |
| AZDO.1001 | High | Connecting to Azure DevOps using SSH should be disabled. | Learn more |
| AZDO.1002 | High | Auditing should be enabled. | Learn more |
| AZDO.1003 | High | External guest access to Azure DevOps should be a controlled process. | Learn more |
| AZDO.1004 | High | Externally sourced package versions should be manually approved for internal use to prevent malicious packages. | Learn more |
| AZDO.1005 | High | Conditional Access Policies should be configured for Microsoft Entra ID-backed organizations. | Learn more |
| AZDO.1006 | High | External users access should be a controlled process. | Learn more |
| AZDO.1007 | High | Team and project administrators should not be allowed to invite new users. | Learn more |
| AZDO.1008 | High | Request access to Azure DevOps by email notifications to administrators should be disabled. | Learn more |
| AZDO.1009 | Info | Providing or collecting customer feedback to the product team for Azure DevOps should be enabled. | Learn more |
| AZDO.1010 | High | Audit logs should be retained according to your organization's needs and protected from purging. | Learn more |
| AZDO.1011 | Info | Azure DevOps supports up to 1,000 projects within an organization. | Learn more |
| AZDO.1012 | Info | Azure DevOps supports up to 150,000 tag definitions per organization or collection. | Learn more |
| AZDO.1013 | High | Azure DevOps organization owner should not be assigned to a regular user. | Learn more |
| AZDO.1014 | High | Status badges in Azure DevOps should be disabled for anonymous access. | Learn more |
| AZDO.1015 | High | User-defined variables should not be able to override system variables or variables not defined by the pipeline author. | Learn more |
| AZDO.1016 | High | YAML & build pipelines should have restricted access to only those repositories in the same project as the pipeline. | Learn more |
| AZDO.1017 | High | Release pipelines should have restricted access to only those repositories in the same project as the pipeline. | Learn more |
| AZDO.1018 | High | Access to repositories in YAML pipelines should apply checks and approval before granting access. | Learn more |
| AZDO.1019 | High | Users should not be able to select stages to skip from the Queue Pipeline panel. | Learn more |
| AZDO.1020 | High | Creating classic build pipelines should be disabled. | Learn more |
| AZDO.1021 | High | Creating classic release pipelines should be disabled. | Learn more |
| AZDO.1022 | High | Azure DevOps pipelines should validate contributions from forked GitHub repositories before running. | Learn more |
| AZDO.1023 | High | Disable the ability to install and run tasks from the Marketplace to maintain control. | Learn more |
| AZDO.1024 | High | Disable Node 6 tasks to avoid deprecated runtime environments. | Learn more |
| AZDO.1025 | High | Enable shell task validation to prevent code injection. | Learn more |
| AZDO.1026 | Medium | GitHub Advanced Security for Azure DevOps should be automatically enabled for new projects. | Learn more |
| AZDO.1027 | Medium | Gravatar images should not be exposed for users outside your enterprise. | Learn more |
| AZDO.1028 | High | Creation of Team Foundation Version Control (TFVC) repositories should be disabled. | Learn more |
| AZDO.1029 | Medium | Azure Artifacts storage limit should not be met. | Learn more |
| AZDO.1030 | Critical | Project Collection Administrator membership should be restricted to the minimum number of accounts required and regularly reviewed. | Learn more |
| AZDO.1031 | High | Validation of SSH key expiration date should be enabled. | Learn more |
| AZDO.1032 | High | Restriction of global Personal Access Token creation should be enabled. | Learn more |
| AZDO.1033 | High | Automatic revocation of leaked Personal Access Tokens should be enabled. | Learn more |
| AZDO.1034 | High | Restrict creation of new Azure DevOps organizations. | Learn more |
| AZDO.1035 | High | Restriction of Personal Access Token lifespan should be enabled. | Learn more |
| AZDO.1036 | High | Restriction of full-scoped Personal Access Token creation should be enabled. | Learn more |
Quick Stats
- 🚀 Automated security tests for Azure DevOps using Maester
- 🔢 37 tests in total
- 🔴 1 Critical | 🟠 30 High | 🟡 3 Medium | 🔵 3 Info
Get Started
Follow the step-by-step guide to set up Maester in Azure DevOps with required resources:
- Documentation: Azure DevOps tests