Skip to main content

Kubernetes deployments

This topic shows you how to deploy a publicly-available NGINX Docker image and manifest into your Kubernetes cluster using a rolling update strategy in Harness.

In this topic, you'll learn how to:

  • Set up a Kubernetes pipeline.
  • Run the new Kubernetes pipeline and deploy a Docker image to your target cluster.

The following video explains Kubernetes deployments using Harness:

Set up a Kubernetes cluster for deployment​

You'll need a target Kubernetes cluster for the Harness Delegate and deployment. Ensure your cluster meets the following minimum requirements:

  • Number of nodes: 2.
  • vCPUs, Memory, Disk Size: 4vCPUs, 16GB memory, 100GB disk. In GKE, the e2-standard-4 machine type is enough for this quickstart.
  • Networking: outbound HTTPS for the Harness connection to app.harness.io, github.com, and hub.docker.com. Allow TCP port 22 for SSH.
  • A Kubernetes service account with permission to create entities in the target namespace is required. The set of permissions should include list, get, create, and delete permissions. In general, the cluster-admin permission or namespace admin permission is enough.
    For more information, go to User-Facing Roles from Kubernetes.

Create the deploy stage for the Kubernetes pipeline​

Pipelines are collections of stages. For this topic, we'll create a new pipeline and add a single stage.

note

Create a project for your new CD pipeline: if you don't already have a Harness project, create a project for your new CD pipeline. Ensure that you add the Continuous Delivery module to the project. Go to create organizations and projects for more information.

  1. In your Harness project, select Deployments, and then select Create a Pipeline.

  2. Enter the name CD Quickstart and select Start.

    Your pipeline appears.

  3. Select Add Stage and select Deploy.

  4. Enter the name Deploy Service, make sure Service is selected, and then select Set Up Stage.

    The new stage settings appear.

  5. In the Service tab, select New Service.

    note

    Let's take a moment and review Harness services and service definitions (which are explained below). Harness services represent your microservices/apps logically. You can add the same service to as many stages as you need. Service definitions represent your artifacts, manifests, and variables physically. They are the actual files and variable values.

    By separating services and service definitions, you can propagate the same Service across stages while changing the artifacts, manifests, and variables with each stage.

  6. Give the service the name nginx and select Save.

Once you have created a service, it is persistent and can be used throughout the stages of this or any other pipeline in the project.

Add the Kubertnetes manifest for deployment​

Next, we can add a Kubernetes manifest for NGINX. We'll use the publicly-available manifest available from Kubernetes.

  1. In Service Definition, in Deployment Type, select Kubernetes.

  2. In Manifests, select Add Manifest.

    note

    What about Artifacts? In this quickstart we are using a publicly-available NGINX Docker image from DockerHub, and the location of the image is hardcoded in the public manifest from Kubernetes. The Artifacts section is only used when the public artifact is not hardcoded in the manifest or the repo is private. In those cases, you add the image in Artifacts with a Connector for the repo and then reference the image in your values.yaml (image: <+artifact.image>). For more information, go to Add Container Images as Artifacts for Kubernetes Deployments.

  3. Select K8s Manifest, and select Continue.

  4. In Select K8sManifest Store, select GitHub, and then select New GitHub Connector.

  5. The Git Connector settings appear. Enter the following settings.

    • Name: enter a name for the connector.

    • URL Type: select Repository.

    • Connection Type: select HTTP.

    • Git Repository URL: enter https://github.com/kubernetes/website.

    • Username and Token: Enter the username and a Github Personal Access Token (PAT) for your GitHub account. You'll have to create a Harness secret for the password.

      1. In Personal Access Token, click Create or Select a Secret.
      2. Select New Secret Text.
      3. In Secret Name, enter a name for the secret like github-pat.
      4. In Secret Value, paste in a GitHub Personal access token. When you're logged into GitHub, these are typically listed at https://github.com/settings/tokens. For steps on setting up a GitHub PAT, go to creating a personal access token from GitHub. Ensure you PAT has the repo scope selected:

  6. Select Continue.

  7. In Connect to the provider, select Connect through a Harness Delegate, and then select Continue. We don't use the Connect through Harness Platform option here simply because you'll need a Delegate later for the connection to your target Kubernetes cluster. Typically, the Connect through Harness Platform option is a very quick way to make connections without having to use Delegates.

    Expand the section below to learn more about installing delegates.

Use the delegate installation wizard
  1. In your Harness project, select Project Setup.
  2. Select Delegates.
  3. Select Install a Delegate.
  4. Follow the delegate installation wizard.

Use this delegate installation wizard video to guide you through the process.

Install a delegate using the terminal

The Harness Delegate is a lightweight worker process that is installed on your infrastructure and communicates only via outbound HTTP/HTTPS to the Harness Platform. This enables the Harness Platform to leverage the delegate to execute the CI/CD and other tasks on your behalf, without any of your secrets leaving your network.

You can install the Harness Delegate on either Docker or Kubernetes.

note

You might need additional permissions to execute commands in delegate scripts and create Harness users.

Install the default Harness Delegate​

Create a new delegate token​

You can install delegates from the Account, Project, or Org scope. In this example, we'll create a new token in the Account scope.

To create a new delegate token, do the following:

  1. In Harness, select Account Settings, then select Account Resources. The Account Resources page opens.

  2. Select Delegates. The Delegates list page opens.

  3. Select the Tokens tab, then select +New Token. The New Token dialog opens.

  4. Enter a token name, for example firstdeltoken.

  5. Select Apply. Harness generates a new token for you.

  6. Select Copy to copy and store the token in a temporary file.

    You will provide this token as an input parameter in the next installation step. The delegate will use this token to authenticate with the Harness Platform.

Get your Harness account ID​

Along with the delegate token, you will also need to provide your Harness accountId as an input parameter during delegate installation. This accountId is present in every Harness URL. For example, in the following URL:

https://app.harness.io/ng/#/account/6_vVHzo9Qeu9fXvj-AcQCb/settings/overview

6_vVHzo9Qeu9fXvj-AcQCb is the accountId.

note

When you install a delegate via the Harness UI, several dependencies in this topic are prefilled for your convenience. This topic explains where to find the required information for CLI-based installation.

For more information, go to View account info and subscribe to downtime alerts.

Prerequisite​

Ensure that you have access to a Kubernetes cluster. For the purposes of this tutorial, we will use minikube.

info

Harness supports Kubernetes versions 1.25.16, 1.26.10, and 1.27.8 for delegate installation.

Install minikube

  • On Windows

    choco install minikube
    info

    For Chocolatey installation instructions, go to Installing Chocolatey in the Chocolatey documentation.

    For additional options to install minikube on Windows, go to minikube start in the minikube documentation.

  • On macOS:

    brew install minikube
    info

    For Homebrew installation instructions, go to Installation in the Homebrew documentation.

Now start minikube with the following config.

minikube start --memory 4g --cpus 4

Validate that you have kubectl access to your cluster.

kubectl get pods -A

Now that you have access to a Kubernetes cluster, you can install the delegate using any of the options below.

Install the Helm chart​

As a prerequisite, you must have Helm v3 installed on the machine from which you connect to your Kubernetes cluster.

You can now install the delegate using the delegate Helm chart. First, add the harness-delegate Helm chart repo to your local Helm registry.

helm repo add harness-delegate https://app.harness.io/storage/harness-download/delegate-helm-chart/
helm repo update
helm search repo harness-delegate

We will use the harness-delegate/harness-delegate-ng chart in this tutorial.

NAME                                	CHART VERSION	APP VERSION	DESCRIPTION
harness-delegate/harness-delegate-ng	1.0.8        	1.16.0     	A Helm chart for deploying harness-delegate

Now we are ready to install the delegate. The following example installs/upgrades firstk8sdel delegate (which is a Kubernetes workload) in the harness-delegate-ng namespace using the harness-delegate/harness-delegate-ng Helm chart.

You can install delegates from the Account, Project, or Org scope. In this example, we'll install a delegate in the Account scope.

To install a delegate, do the following:

  1. In Harness, select Account Settings, then select Account Resources. The Account Resources page opens.

  2. Select Delegates. The Delegates list page opens.

  3. Select New Delegate. The New Delegate dialog opens.

  4. Under Select where you want to install your Delegate, select Kubernetes.

  5. Under Install your Delegate, select Helm Chart.

  6. Copy the helm upgrade command.

    The command uses the default values.yaml file located in the delegate Helm chart GitHub repo. To make persistent changes to one or more values, you can download and update the values.yaml file according to your requirements. Once you have updated the file, you can use it by running the upgrade command below.

       helm upgrade -i firstk8sdel --namespace harness-delegate-ng --create-namespace \
      harness-delegate/harness-delegate-ng \
      -f values.yaml \
      --set delegateName=firstk8sdel \
      --set accountId=PUT_YOUR_HARNESS_ACCOUNTID_HERE \
      --set delegateToken=PUT_YOUR_DELEGATE_TOKEN_HERE \
      --set managerEndpoint=PUT_YOUR_MANAGER_HOST_AND_PORT_HERE \
      --set delegateDockerImage=harness/delegate:yy.mm.verno \
      --set replicas=1 --set upgrader.enabled=true
note

To install a Helm delegate for Harness Self-Managed Enterprise Edition in an air-gapped environment, you must pass your certificate when you add the Helm repo.

helm repo add harness-delegate --ca-file <.PEM_FILE_PATH> <HELM_CHART_URL_FROM_UI>

For more information on requirements for air-gapped environments, go to Install in an air-gapped environment.

  1. Run the command.

Deploy using a custom role​

During delegate installation, you have the option to deploy using a custom role. To use a custom role, you must edit the delegate YAML file.

Harness supports the following custom roles:

  • cluster-admin
  • cluster-viewer
  • namespace-admin
  • custom cluster roles

To deploy using a custom cluster role, do the following:

  1. Open the delegate YAML file in your text editor.

  2. Add the custom cluster role to the roleRef field in the delegate YAML.

    ---
    apiVersion: rbac.authorization.k8s.io/v1beta1
    kind: ClusterRoleBinding
    metadata:
      name: harness-delegate-cluster-admin
    subjects:
      - kind: ServiceAccount
        name: default
        namespace: harness-delegate-ng
    roleRef:
      kind: ClusterRole
      name: cluster-admin
      apiGroup: rbac.authorization.k8s.io
    ---

    In this example, the cluster-admin role is defined.

  3. Save the delegate YAML file.

Verify delegate connectivity​

Select Continue. After the health checks pass, your delegate is available for you to use. Select Done and verify your new delegate is listed.

Helm chart & Terraform Helm provider​

Delegate Available

Kubernetes manifest​

Delegate Available

Docker​

Delegate Available

You can now route communication to external systems in Harness connectors and pipelines by selecting this delegate via a delegate selector.

Delegate selectors do not override service infrastructure connectors. Delegate selectors only determine the delegate that executes the operations of your pipeline.

Troubleshooting​

The delegate installer provides troubleshooting information for each installation process. If the delegate cannot be verified, select Troubleshoot for steps you can use to resolve the problem. This section includes the same information.

Harness asks for feedback after the troubleshooting steps. You are asked, Did the delegate come up?

If the steps did not resolve the problem, select No, and use the form to describe the issue. You'll also find links to Harness Support and to Delegate docs.

Use the following steps to troubleshoot your installation of the delegate using Helm.

  1. Verify that Helm is correctly installed:

    Check for Helm:

    helm

    And then check for the installed version of Helm:

    helm version

    If you receive the message Error: rendered manifests contain a resource that already exists..., delete the existing namespace, and retry the Helm upgrade command to deploy the delegate.

    For further instructions on troubleshooting your Helm installation, go to Helm troubleshooting guide.

  2. Check the status of the delegate on your cluster:

    kubectl describe pods -n <NAMESPACE>

  3. If the pod did not start, check the delegate logs:

    kubectl logs -f <DELEGATE_NAME> -n <NAMESPACE>

    If the state of the delegate pod is CrashLoopBackOff, check your allocation of compute resources (CPU and memory) to the cluster. A state of CrashLoopBackOff indicates insufficient Kubernetes cluster resources.

  4. If the delegate pod is not healthy, use the kubectl describe command to get more information:

    kubectl describe <POD_NAME> -n <NAMESPACE>

To learn more, watch the Delegate overview video.

  1. Back in Set Up Delegates, you can select the new Delegate. In the list of Delegates, you can see your new Delegate and its tags.
  2. Select the Connect using Delegates with the following Tags option.
  3. Enter the tag of the new Delegate and select Save and Continue. When you are done, the Connector is tested.
  4. Select Continue.
  5. In Manifest Details, enter the following settings, test the connection, and click Submit. We are going to provide connection and path information for a manifest located at https://raw.githubusercontent.com/kubernetes/website/main/content/en/examples/application/nginx-app.yaml.
    • Manifest Identifier: enter nginx.
    • Git Fetch Type**:** select **Latest from Branch**.
    • Branch: enter main.
    • File/Folder path:content/en/examples/application/nginx-app.yaml. This is the path from the repo root.

The manifest is now listed.

  1. Select Next at the bottom of the Service tab.

Now that the artifact and manifest are defined, you can define the target cluster for your deployment.

Define the target cluster for the Kubernetes deployment​

The target cluster is your own Kubernetes cluster, hosted in your cloud environment. This is where we will deploy the Docker image using the manifest you selected.

Harness connects to all of the common cloud platforms and provides a platform-agnostic Kubernetes cluster connection that can connect to Kubernetes anywhere.

  1. In Infrastructure Details, in Specify your environment, select New Environment. Just like with a service, you can create a new environment or selecting an existing one. We'll create a new one.

  2. In New Environment, enter a name, select Pre-Production, and select Save. The new environment appears.

  3. In Infrastructure Definition, select Kubernetes.

    note

    Let's take a moment and review Harness environments and infrastructure definitions. Harness environments represent your deployment targets logically (QA, Prod, etc). You can add the same Environment to as many stages as you need. Infrastructure definitions represent your target infrastructure physically. They are the actual clusters, hosts, etc.

    By separating environments and infrastructure definitions, you can use the same environment in multiple stages while changing the target infrastructure settings with each stage.

  4. An Infrastructure Definition is where you specify the target for your deployment. In this case, your Kubernetes cluster and namespace.

  5. In Cluster Details, in Connector, select Select a connector.

  6. Select New Connector.

  7. The Kubernetes cluster connector appears.

The Kubernetes cluster connector is covered in detail here, but let's quickly walk through it.

Let's look at the steps:

  1. In Kubernetes Cluster Connector, in Name, enter Kubernetes Quickstart, and select Continue.

  2. In Details, select Use the credentials of a specific Harness Delegate. We will select the Delegate next.

  3. Select Continue.

  4. Select the Kubernetes Delegate you added earlier using its Tags, and then select Save and Continue. Harness verifies the Connector.

  5. Select Finish.

  6. Select the new Connector and then select Apply Selected.

  7. Back in Cluster Details, in Namespace, enter the target namespace for the deployment. For example, default. You can use any namespace in your target cluster.

  8. In Advanced, in Release name, enter quickstart. For a release name, you can use a built-in or custom expression or any text you like. Harness requires a release name for tracking. It is applied to the pods as a label. The release name must be unique across the cluster.

  9. When you are done, the Cluster Details will look something like this:

    The target infrastructure is complete. Now we can add our stage steps.

  10. Select Next to move onto Execution.

Add a Kubernetes rollout deployment step to the stage​

Now you can select the deployment strategy for this stage of the pipeline.

  1. In Execution Strategies, select Rolling, and then select Use Strategy.

  2. The Rollout Deployment step is added.

    This is a standard Kubernetes rolling update. By default, Harness uses a 25% max unavailable, 25% max surge strategy.

That's it. Now the pipeline stage is complete and you can deploy.

Deploy the Kubernetes pipeline and review​

  1. Select Save > Save Pipeline and then Run. Now you can select the specific artifact to deploy.

  2. In Primary Artifact, select stable. This is the same as using docker pull nginx:stable.

  3. Select Run Pipeline. Harness will verify the Pipeline and then run it. You can see the status of the deployment, and pause or abort it.

  4. Toggle Console View to watch the deployment with more detailed logging. Select the Rollout Deployment step and expand Wait for Steady State.

    You can see deployment "my-nginx" successfully rolled out.

Congratulations! The deployment was successful.

In your project's Deployments, you can see the deployment listed:

If you run into any errors, it is typically because the cluster does meet the requirements from Before You Begin section or the cluster's network setting does not allow the Delegate to connect to Docker Hub.In this tutorial, you learned how to:

  • Install and launch a Harness Kubernetes Delegate in your target cluster.
  • Connect Harness to your Kubernetes cluster and an artifact server.
  • Add your manifests to Harness.
  • Create an infrastructure definition that targets your cluster and namespace.
  • Add a Kubernetes rolling update.
  • Deploy your Kubernetes pipeline to your target cluster.

Next, try using Harness Continuous Integration to build a codebase, upload it to a repo, and run unit and integrations tests.

Clean up the deployment​

For steps on deleting the Delgate, go to delete a delegate.

Next steps​

See CD tutorials for other deployment features.