Add a Microsoft Azure connector
With the Microsoft Azure connector, your Harness pipelines can pull Azure artifacts, provision Azure infrastructure, and deploy your applications to Azure.
The Microsoft Azure connector is for ACR, AKS, ARM, Blueprint, Web Apps, and virtual machines for traditional (SSH/WinRM) deployments.
Use the Azure Repos connector to connect to Azure SCM repos
If you're using Harness Cloud Cost Management (CCM), you can Set Up Cloud Cost Management for Azure.
Auth Provider API and TokenRequest API options
Harness provides the option of using the Auth Provider API or TokenRequest API for authentication.
Summary of Auth Provider and TokenRequest API changes
To select which API to use:
- Auth Provider API: this is the current default. You do not have to change the default settings of Harness connectors or the Harness Delegates you use.
- TokenRequest API: you must install the provider-specific plugin on the Harness Delegate(s) to use the TokenRequest API introduced in Kubernetes 1.22.
Install the kubelogin client-go credential (exec) plugin on the delegate
When using the Harness Azure connector with Kubernetes version >= 1.22, you can use the kubelogin client-go credential (exec) plugin to authenticate to AKS cluster.
The Harness Azure connector has 4 authentication types. For each type, you must install the following dependencies in the Harness Delegates you use or Harness will follow the old Auth Provider API format.
- Secret (
SERVICE_PRINCIPAL_SECRET
): Kubelogin binary. - Certificate (
SERVICE_PRINCIPAL_CERT
): Kubelogin binary and azurecli (azurecli is required as kubelogin does not support certificate in PEM format). - System Assigned Managed Identity (
MANAGED_IDENTITY_SYSTEM_ASSIGNED
): Kubelogin binary. - User Assigned Managed Identity (
MANAGED_IDENTITY_USER_ASSIGNED
): Kubelogin binary.
The Secret and Certificate options are available when you select the Specify credentials here option in the Azure connector.
The System Assigned Managed Identity and User Assigned Managed Identity options are available when you select the Use the credentials of a specific Harness Delegate option in the Azure connector.
You can install the kubelogin plugin on the delegate by creating a delegate with an immutable image and updating the following commands in INIT_SCRIPT
:
RHEL 7 OS
Ubuntu
For more information, go to kubelogin from Azure and Delegate installation overview.
Roles, permission, and cluster requirements
This section assumes you're familiar with Azure RBAC. For details, go to the Azure documentation: Assign Azure roles using the Azure portal.
This graphic from Azure is a useful reminder of how Azure manages RBAC:
For security reasons, Harness uses an application object and service principal rather than a user identity. The process is described in the Microsoft Entra documentation: Create a Microsoft Entra application and service principal that can access resources.
Azure Container Repository (ACR) role requirements
The Harness Azure connectors that you'll use to connect Harness to ACR must have the Reader role, at minimum. You can also use a custom role that includes the permissions of the Reader role.
- Reader
- Custom role
The Reader role must be assigned at the Subscription or Resource Group level that is used by the Application (Client) Id that you'll use in the Azure connector's settings. The application must have permission to list all container registries.
Make sure you:
- Don't put the Reader role in a different IAM section of Azure.
- Don't provide only the AcrPull role, instead of Reader. It might appear that the AcrPull role gives access to a specific registry, but Harness needs to list all registries.
The following permissions (actions) are necessary for any Service Principal and/or Managed Identity user, regardless of whether you are using Kubernetes RBAC or Azure RBAC:
Microsoft.ContainerRegistry/registries/read
Microsoft.ContainerRegistry/registries/builds/read
Microsoft.ContainerRegistry/registries/metadata/read
Microsoft.ContainerRegistry/registries/pull/read
Microsoft.ContainerService/managedClusters/read
Microsoft.ContainerService/managedClusters/listClusterUserCredential/action
Microsoft.Resource/subscriptions/resourceGroup/read
For Helm deployments, the version of Helm must be >= 3.2.0. The Harness HELM_VERSION_3_8_0
feature flag must be activated.
You can't use Pod Assigned Managed Identity and System Assigned Managed Identity for the same cluster.
The following JSON sample creates a custom role with the required permissions. To use this sample, replace xxxx
with the role name, subscription Id, and resource group Id.
{
"id": "/subscriptions/xxxx/providers/Microsoft.Authorization/roleDefinitions/xxxx",
"properties": {
"roleName": "xxxx",
"description": "",
"assignableScopes": [
"/subscriptions/xxxx/resourceGroups/xxxx"
],
"permissions": [
{
"actions": [],
"notActions": [],
"dataActions": [
"Microsoft.ContainerService/managedClusters/configmaps/read",
"Microsoft.ContainerService/managedClusters/configmaps/write",
"Microsoft.ContainerService/managedClusters/configmaps/delete",
"Microsoft.ContainerService/managedClusters/secrets/read",
"Microsoft.ContainerService/managedClusters/secrets/write",
"Microsoft.ContainerService/managedClusters/secrets/delete",
"Microsoft.ContainerService/managedClusters/apps/deployments/read",
"Microsoft.ContainerService/managedClusters/apps/deployments/write",
"Microsoft.ContainerService/managedClusters/apps/deployments/delete",
"Microsoft.ContainerService/managedClusters/events/read",
"Microsoft.ContainerService/managedClusters/events/write",
"Microsoft.ContainerService/managedClusters/events/delete",
"Microsoft.ContainerService/managedClusters/namespaces/read",
"Microsoft.ContainerService/managedClusters/nodes/read",
"Microsoft.ContainerService/managedClusters/pods/read",
"Microsoft.ContainerService/managedClusters/pods/write",
"Microsoft.ContainerService/managedClusters/pods/delete",
"Microsoft.ContainerService/managedClusters/services/read",
"Microsoft.ContainerService/managedClusters/services/write",
"Microsoft.ContainerService/managedClusters/services/delete",
"Microsoft.ContainerService/managedClusters/apps/statefulsets/read",
"Microsoft.ContainerService/managedClusters/apps/statefulsets/write",
"Microsoft.ContainerService/managedClusters/apps/statefulsets/delete",
"Microsoft.ContainerService/managedClusters/apps/replicasets/read",
"Microsoft.ContainerService/managedClusters/apps/replicasets/write",
"Microsoft.ContainerService/managedClusters/apps/replicasets/delete"
],
"notDataActions": []
}
]
}
}
Harness supports 500 images from an ACR repo. If you don't see some of your images, then you might have exceeded this limit. This is the result of an Azure API limitation.
If you connect to an ACR repo via the platform-agnostic Docker Connector, the limit is 100.
Azure Web App role requirements
Harness Azure connectors that you'll use to connect to Azure Web Apps with Service Principal or Managed Identity credentials, must have the Contributor role, at minimum. You can also use a custom role that includes the permissions of the Contributor role.
- Contributor permissions
- Custom role
The follow are the Azure RBAC permissions used for System Assigned Managed Identity permissions to perform Azure Web App deployments for container and non-container artifacts:
[
"microsoft.web/sites/slots/deployments/read",
"Microsoft.Web/sites/Read",
"Microsoft.Web/sites/config/Read",
"Microsoft.Web/sites/slots/config/Read",
"microsoft.web/sites/slots/config/appsettings/read",
"Microsoft.Web/sites/slots/*/Read",
"Microsoft.Web/sites/slots/config/list/Action",
"Microsoft.Web/sites/slots/stop/Action",
"Microsoft.Web/sites/slots/start/Action",
"Microsoft.Web/sites/slots/config/Write",
"Microsoft.Web/sites/slots/Write",
"microsoft.web/sites/slots/containerlogs/action",
"Microsoft.Web/sites/config/Write",
"Microsoft.Web/sites/slots/slotsswap/Action",
"Microsoft.Web/sites/config/list/Action",
"Microsoft.Web/sites/start/Action",
"Microsoft.Web/sites/stop/Action",
"Microsoft.Web/sites/Write",
"microsoft.web/sites/containerlogs/action",
"Microsoft.Web/sites/publish/Action",
"Microsoft.Web/sites/slots/publish/Action"
]
The following permissions (actions) are necessary for any Service Principal and/or Managed Identity user, regardless of whether you are using Kubernetes RBAC or Azure RBAC:
Microsoft.ContainerRegistry/registries/read
Microsoft.ContainerRegistry/registries/builds/read
Microsoft.ContainerRegistry/registries/metadata/read
Microsoft.ContainerRegistry/registries/pull/read
Microsoft.ContainerService/managedClusters/read
Microsoft.ContainerService/managedClusters/listClusterUserCredential/action
Microsoft.Resource/subscriptions/resourceGroup/read
For Helm deployments, the version of Helm must be >= 3.2.0. The Harness HELM_VERSION_3_8_0
feature flag must be activated.
You can't use Pod Assigned Managed Identity and System Assigned Managed Identity for the same cluster.
The following JSON sample creates a custom role with the required permissions. To use this sample, replace xxxx
with the role name, subscription Id, and resource group Id.
{
"id": "/subscriptions/xxxx/providers/Microsoft.Authorization/roleDefinitions/xxxx",
"properties": {
"roleName": "xxxx",
"description": "",
"assignableScopes": [
"/subscriptions/xxxx/resourceGroups/xxxx"
],
"permissions": [
{
"actions": [],
"notActions": [],
"dataActions": [
"Microsoft.ContainerService/managedClusters/configmaps/read",
"Microsoft.ContainerService/managedClusters/configmaps/write",
"Microsoft.ContainerService/managedClusters/configmaps/delete",
"Microsoft.ContainerService/managedClusters/secrets/read",
"Microsoft.ContainerService/managedClusters/secrets/write",
"Microsoft.ContainerService/managedClusters/secrets/delete",
"Microsoft.ContainerService/managedClusters/apps/deployments/read",
"Microsoft.ContainerService/managedClusters/apps/deployments/write",
"Microsoft.ContainerService/managedClusters/apps/deployments/delete",
"Microsoft.ContainerService/managedClusters/events/read",
"Microsoft.ContainerService/managedClusters/events/write",
"Microsoft.ContainerService/managedClusters/events/delete",
"Microsoft.ContainerService/managedClusters/namespaces/read",
"Microsoft.ContainerService/managedClusters/nodes/read",
"Microsoft.ContainerService/managedClusters/pods/read",
"Microsoft.ContainerService/managedClusters/pods/write",
"Microsoft.ContainerService/managedClusters/pods/delete",
"Microsoft.ContainerService/managedClusters/services/read",
"Microsoft.ContainerService/managedClusters/services/write",
"Microsoft.ContainerService/managedClusters/services/delete",
"Microsoft.ContainerService/managedClusters/apps/statefulsets/read",
"Microsoft.ContainerService/managedClusters/apps/statefulsets/write",
"Microsoft.ContainerService/managedClusters/apps/statefulsets/delete",
"Microsoft.ContainerService/managedClusters/apps/replicasets/read",
"Microsoft.ContainerService/managedClusters/apps/replicasets/write",
"Microsoft.ContainerService/managedClusters/apps/replicasets/delete"
],
"notDataActions": []
}
]
}
}
Connect Harness to Azure Kubernetes Services (AKS)
There are three options for connecting Harness to an AKS cluster:
- Use the platform-agnostic Kubernetes cluster connector with a Kubernetes or Helm delegate.
- You'll need to install a Kubernetes delegate in the target AKS cluster, and then use the delegate's credentials for the Kubernetes cluster connector's authentication method.
- You won't need to provide Microsoft Azure Service Principal or Managed Identity credentials.
- Use a Microsoft Azure Cloud Provider connector, as described in this topic, with a Kubernetes delegate.
- You'll need to install a Kubernetes delegate in the target AKS cluster, and then use the delegate's credentials for the Azure connector's authentication method.
- You'll need to provide the Microsoft Azure Environment.
- If you use a User Assigned Managed Identity, you'll need to provide the Application (client) Id.
- If you use a System Assigned Managed Identity, you won't need to provide any Ids.
- It is possible to create a connector with a non-existent delegate. This behavior is intended. This design allows customers to replace a delegate with a new one of the same name or tag.
- Use a Microsoft Azure Cloud Connector with Service Principal or Managed Identity credentials, as described in this topic.
- You must assign the Owner role or an equivalent custom role, as explained in AKS role requirements.
AKS cluster setup requirements
- AKS managed AAD, enabled or disabled.
- Kubernetes RBAC, enabled.
- Azure RBAC, enabled or disabled.
For more information, go to the Deployments (CD) section of the Kubernetes cluster connector settings reference.
AKS role requirements
If you use the Microsoft Azure connector to connect to AKS with Service Principal or Managed Identity credentials, you must assign the Owner role or a custom role that includes the permissions of the Owner role.
- Custom role
- Kubernetes RBAC example
- Azure RBAC example
The following permissions (actions) are necessary for any Service Principal and/or Managed Identity user, regardless of whether you are using Kubernetes RBAC or Azure RBAC:
Microsoft.ContainerRegistry/registries/read
Microsoft.ContainerRegistry/registries/builds/read
Microsoft.ContainerRegistry/registries/metadata/read
Microsoft.ContainerRegistry/registries/pull/read
Microsoft.ContainerService/managedClusters/read
Microsoft.ContainerService/managedClusters/listClusterUserCredential/action
Microsoft.Resource/subscriptions/resourceGroup/read
For Helm deployments, the version of Helm must be >= 3.2.0. The Harness HELM_VERSION_3_8_0
feature flag must be activated.
You can't use Pod Assigned Managed Identity and System Assigned Managed Identity for the same cluster.
The following JSON sample creates a custom role with the required permissions. To use this sample, replace xxxx
with the role name, subscription Id, and resource group Id.
{
"id": "/subscriptions/xxxx/providers/Microsoft.Authorization/roleDefinitions/xxxx",
"properties": {
"roleName": "xxxx",
"description": "",
"assignableScopes": [
"/subscriptions/xxxx/resourceGroups/xxxx"
],
"permissions": [
{
"actions": [],
"notActions": [],
"dataActions": [
"Microsoft.ContainerService/managedClusters/configmaps/read",
"Microsoft.ContainerService/managedClusters/configmaps/write",
"Microsoft.ContainerService/managedClusters/configmaps/delete",
"Microsoft.ContainerService/managedClusters/secrets/read",
"Microsoft.ContainerService/managedClusters/secrets/write",
"Microsoft.ContainerService/managedClusters/secrets/delete",
"Microsoft.ContainerService/managedClusters/apps/deployments/read",
"Microsoft.ContainerService/managedClusters/apps/deployments/write",
"Microsoft.ContainerService/managedClusters/apps/deployments/delete",
"Microsoft.ContainerService/managedClusters/events/read",
"Microsoft.ContainerService/managedClusters/events/write",
"Microsoft.ContainerService/managedClusters/events/delete",
"Microsoft.ContainerService/managedClusters/namespaces/read",
"Microsoft.ContainerService/managedClusters/nodes/read",
"Microsoft.ContainerService/managedClusters/pods/read",
"Microsoft.ContainerService/managedClusters/pods/write",
"Microsoft.ContainerService/managedClusters/pods/delete",
"Microsoft.ContainerService/managedClusters/services/read",
"Microsoft.ContainerService/managedClusters/services/write",
"Microsoft.ContainerService/managedClusters/services/delete",
"Microsoft.ContainerService/managedClusters/apps/statefulsets/read",
"Microsoft.ContainerService/managedClusters/apps/statefulsets/write",
"Microsoft.ContainerService/managedClusters/apps/statefulsets/delete",
"Microsoft.ContainerService/managedClusters/apps/replicasets/read",
"Microsoft.ContainerService/managedClusters/apps/replicasets/write",
"Microsoft.ContainerService/managedClusters/apps/replicasets/delete"
],
"notDataActions": []
}
]
}
}
Here's an example of Kubernetes RBAC permissions used for System Assigned Managed Identity.
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: cdp-qa-deployer-role
namespace: cdp-qa-app
rules:
- apiGroups: ["", "apps"]
resources: ["pods", "configmaps", "deployments", "secrets", "events", "services", "replicasets", "deployments/scale", "namespaces", "resourcequotas", "limitranges"]
verbs: ["get", "watch", "list", "create", "update", "patch", "delete"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: cdp-qa-deployer-role-binding
namespace: cdp-qa-app
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: cdp-qa-deployer-role
subjects:
- kind: Group
namespace: cdp-qa-app
name: <AD group id to which the SP and MSI users are assigned>
Here's an example of Azure RBAC permissions used for System Assigned Managed Identity. To use this sample, replace xxxx
with the subscription Id and resource group Id.
{
"id": "/subscriptions/xxxx/providers/Microsoft.Authorization/roleDefinitions/xxxx",
"properties": {
"roleName": "HarnessSysMSIRole",
"description": "",
"assignableScopes": [
"/subscriptions/xxxx/resourceGroups/xxxx"
],
"permissions": [
{
"actions": [],
"notActions": [],
"dataActions": [
"Microsoft.ContainerService/managedClusters/configmaps/read",
"Microsoft.ContainerService/managedClusters/configmaps/write",
"Microsoft.ContainerService/managedClusters/configmaps/delete",
"Microsoft.ContainerService/managedClusters/secrets/read",
"Microsoft.ContainerService/managedClusters/secrets/write",
"Microsoft.ContainerService/managedClusters/secrets/delete",
"Microsoft.ContainerService/managedClusters/apps/deployments/read",
"Microsoft.ContainerService/managedClusters/apps/deployments/write",
"Microsoft.ContainerService/managedClusters/apps/deployments/delete",
"Microsoft.ContainerService/managedClusters/events/read",
"Microsoft.ContainerService/managedClusters/events/write",
"Microsoft.ContainerService/managedClusters/events/delete",
"Microsoft.ContainerService/managedClusters/namespaces/read",
"Microsoft.ContainerService/managedClusters/nodes/read",
"Microsoft.ContainerService/managedClusters/pods/read",
"Microsoft.ContainerService/managedClusters/pods/write",
"Microsoft.ContainerService/managedClusters/pods/delete",
"Microsoft.ContainerService/managedClusters/services/read",
"Microsoft.ContainerService/managedClusters/services/write",
"Microsoft.ContainerService/managedClusters/services/delete",
"Microsoft.ContainerService/managedClusters/apps/statefulsets/read",
"Microsoft.ContainerService/managedClusters/apps/statefulsets/write",
"Microsoft.ContainerService/managedClusters/apps/statefulsets/delete",
"Microsoft.ContainerService/managedClusters/apps/replicasets/read",
"Microsoft.ContainerService/managedClusters/apps/replicasets/write",
"Microsoft.ContainerService/managedClusters/apps/replicasets/delete"
],
"notDataActions": []
}
]
}
}
Azure Resource Management (ARM)
The roles required depend on the scope type of your ARM template:
- Resource group: requires the
Contributor
role. - Subscription: requires the
Contributor
role. - Management group: requires the
Contributor
role. - Tenant: requires the
Contributor
orOwner
role. For example, creating a Tenant requires theContributor
role, but theOwner
role is required to create role assignments. - Key Vault access: to enable access to Key Vaults from the ARM templates you use in Harness, make sure you select the Azure Resource Manager for template deployment option in the Key Vault Access Policy.
The Azure roles provided in the connector must allow Harness to provision the Azure resources in your ARM templates. For example, to create a policy assignment, the Resource Policy Contributor
role is required.
Azure Blueprints
In Azure, the permissions required to create and delete Blueprints are listed in Permissions in Azure Blueprints from Azure.
The Azure roles required on the service principal used by Harness depend on the scope type of your Blueprint definition.
Management Scope
- System-assigned managed identity:
- Contributor role at the management group scope where the Blueprint definitions will be created and published.
- Owner role at subscription scope where the assignment will be done.
- System-assigned user identity:
- Contribute role at the management group scope where Blueprint definitions will be created and published.
Subscription Scope
- System-assigned managed identity:
- Owner role at the subscription scope.
- System-assigned user identity:
- Contribute role to create and publish the Blueprint definition.
Harness does not manage the right and lifecycle of a user-managed identity. You are responsible for managing the right and lifecycle of a user-managed identity that is in charge of assignment.
- Contribute role to create and publish the Blueprint definition.
Add an Azure connector
You can add Azure connectors at the account, org, or project level at any time, or you can add them while setting up pipelines. For example, to add a connector at the project level you would select Project Setup, select Connectors, and then select New Connector.
- From the Connectors library, select Azure under Cloud Providers.
- Input a Name. Harness automatically creates an Id (Entity Identifier) for the connector based on the name. You can edit the Id before saving the connector. Once the connector is saved, the Id is immutable.
- Optionally, you can add a description and tags.
- Select Continue to configure the connector's credentials.
Configure credentials
There are two primary ways for the Harness connector to authenticate with Azure:
- Select Specify credentials here to use an Application (client) and Tenant (directory) Id.
- Select Use the credentials of a specific Harness Delegate to allow the connector to inherit its credentials from the Harness Delegate that is running in your Azure subscription or AKS cluster.
Specify credentials
Inherit credentials from the delegate
Select connectivity mode
-
Select how you want Harness to connect to Azure:
- Connect through Harness Platform: Use a direct, secure communication between Harness and Azure. This connectivity mode is required for Harness Cloud build infrastructure.
- Connect through a Harness Delegate: Harness communicates with Azure through a Harness Delegate in your Azure subscription or AKS cluster. You must choose this option if you chose to inherit delegate credentials.
-
If connecting through a Harness Delegate, select either:
- Use any available Delegate: Harness selects an available delegate at runtime.
- Only use Delegates with all of the following tags: Use tags to match one or more suitable delegates. You can also install a new delegate at this time.
-
Select Save and Continue to run the connection test, and then, if the test succeeds, select Finish. The connection test confirms that your authentication and delegate selections are valid.
If the connection test fails, make sure that your delegate is running and that your credentials are valid. For example, check that the secret has not expired in your App registration.
Using ${HARNESS_KUBE_CONFIG_PATH} with Azure
The Harness ${HARNESS_KUBE_CONFIG_PATH}
expression resolves to the path to a Harness-generated kubeconfig file containing the credentials you provided to Harness.
The credentials can be used by kubectl commands by exporting its value to the KUBECONFIG
environment variable.
For example, you could use this shell script in a Harness Run step:
export KUBECONFIG=${HARNESS_KUBE_CONFIG_PATH} kubectl get pods -n <namespace>
Steps can be executed on any delegate or you can select specific delegates using the Delegate Selector setting.
For Azure deployments, note the following:
- If the Azure connector used in the stage's Infrastructure uses Azure Managed Identity for authentication, then the Shell Script step must use a Delegate Selector for a delegate running in AKS.
- If the Azure connector used in the stage's Infrastructure uses Azure Service Principal for authentication, then the Shell Script step can use any delegate.