Purpose and scope of this Document

This document provides a comprehensive guide on the customized self-hosted agent image usage for namespaces and teams within the KaaS (Kafka as a Service) framework. It serves as a reference for users, project teams, and stakeholders to understand the setup, build, and usage of customized agent images to enhance CI/CD pipelines.

Customized Agent Images: User Guide for Namespace Teams/Projects

Background

The purpose of this functionality is to enable the pre-packaging of custom software dependencies into the Docker image of the self-hosted pipeline agent. Pre-packaging software by building a custom image, rather than installing directly in every single pipeline run, achieves the following benefits: - Increased customization options for the CI/CD execution environment - Faster pipeline execution - More secure management of software dependencies, including security scanning - Reduced privileges and lower attack surface for the actual pipeline execution instance

For installation scripts that require privileged root access or package manager usage, custom images are mandatory since privileged root access is disabled during pipeline executions.

Prerequisites / Justified Use Cases

The following prerequisites are required: - Custom CI/CD dependencies relevant to the team's KaaS CI/CD implementation are known. - This could include dependencies for integration tests or project-specific CI/CD tooling. - Basic knowledge of Azure YAML Pipelines and Docker container builds is required. - Teams must be willing to maintain their own image via self-service. This includes patching and updating the image regularly, which is automated through a pipeline but may require manual intervention. - Teams should request onboarding to custom image functionality for their namespace from the platform team via a support ticket.

Additionally, one or both of the following prerequisites would normally justify use cases: - Installation actions significantly slow down pipeline execution speed if done as part of the pipeline execution. - Some needed installation scripts/actions require or are complicated without privileged root access or package manager usage, as this privilege is disabled for heightened security posture during pipeline executions.

1.1 How to Set Up

• Receive onboarding from the Platform Team, including access to the KaaS workspace.
• Ensure you have write access to the agent-images-ns-<namespace> Git repository.
• Ensure you have run access to the pipeline agent-images-prod-ns-<namespace>.

1.2 How to Build and Patch Your Image

1. Add Docker Contents
   • Commit Dockerfile changes or additional scripts to the agent-images-ns-<namespace> Git repository.
2. Run Build Pipeline
   • Execute the build and patch pipeline, named agent-images-prod-ns-<namespace>. It will automatically build and push updated images based on the configuration in your Dockerfile repository.

1.3 Using the Custom Image in Non-Terraform Pipeline Stages

Note: As of November 2025, KaaS platform pipelines (access-mgmt, stream-gov, etc.) use persistent warm pool baseline agents instead of on-demand agent spinning. The instructions below apply primarily to namespace-specific pipelines that require custom images and still use the on-demand agent provisioning model.

1. Set the imageOwner Variable in Your Deployment Pipeline
   • In your YAML definition file for your deployment pipeline (within your project's repo), add a variable imageOwner: <namespace> to utilize your custom image.

     variables:
       - name: imageOwner
         value: example
     

     Ensure other standard variables, repositories, and parameters for using the pipeline agent are defined in your pipeline definition. A complete example is provided below.

     variables:
       - template: configuration/prod/global-variables.yml@pipeline-agent
       - name: imageOwner
         value: example
     resources:
       repositories:  
       - repository: pipeline-agent
         name: "KaaS (Kafka as a Service)/pipeline-agent"
         type: git
         ref: main
     parameters:
       - name: envVars
         type: object
         default:
           TF_VAR_region: $(aws-region)
           AWS_DEFAULT_REGION: $(aws-region)
           SYSTEM_ACCESSTOKEN: $(System.AccessToken)
     

2. Pass the imageOwner variable as a parameter to the spin-up agent template.

   - template: ../../modules-global/templates/spin-up-demand-agent.yml
     parameters:
       envVars: ${{ parameters.envVars }}
       agentNamePrefix: "example-development"
       imageOwner: $(imageOwner)
   - job: ExampleJob
     timeoutInMinutes: 1500
     pool:
       name: $(agentPoolPrefix)-$(confluentOrganization)
       demands:
       - Agent.Name -equals example-development-$(Build.BuildId)
     dependsOn: SpinUpAgent
   

3. Optionally, set and pass agentDeploymentName if your namespace has multiple images: - If multiple images are defined in the namespace, add an additional parameter agentDeploymentName in the first job (spin up agent) to select the specific image name.

   - template: ../../modules-global/templates/spin-up-demand-agent.yml
     parameters:
       envVars: ${{ parameters.envVars }}
       agentNamePrefix: "example-development"
       imageOwner: $(imageOwner)
       agentDeploymentName: "secondary"
   

1.4 Using the Custom Image in Standard Terraform Pipeline Stages

1. Set the imageOwner Variable in Your Deployment Pipeline
   • In your YAML definition file for your deployment pipeline (within your project’s repo), add a variable imageOwner: <namespace> to utilize your custom image.

     variables:
       - name: imageOwner
         value: example
     

2. Pass This Variable as a Parameter to the Plan and Apply Pipeline Stages: - The parameter is also named imageOwner.

   stages:
   - stage: plan_all_environments
     jobs:
     - template: './confluent-resources/pipeline-stages/plan-stage.yml@kaas-pipeline-templates'
       parameters:
         envVars: ${{ parameters.envVars }}
         environments: ${{ parameters.environments }}
         imageOwner: $(imageOwner)
   - stage: dev_apply_stage
     dependsOn: plan_all_environments
     jobs:
     - template: './confluent-resources/pipeline-stages/apply-stage.yml@kaas-pipeline-templates'
       parameters:
         envVars: ${{ parameters.envVars }}
         environment: 'dev'
         terraform_path: '$(terraformPath)/dev/config'
         imageOwner: $(imageOwner)
   

3. Optionally, set and pass agentDeploymentName if your namespace has multiple images: - If multiple images are defined in the namespace, include an additional parameter agentDeploymentName in the first job (spin up agent).

       parameters:
         envVars: ${{ parameters.envVars }}
         environments: ${{ parameters.environments }}
         imageOwner: $(imageOwner)
         agentDeploymentName: "secondary"
   

1.5 How to Retire/Decommission

1. Coordinate with Platform Team
   • They will remove the YAML entry and delete resources that are no longer needed.
2. Archive Repository
   • If the namespace image is no longer in use, you may discontinue updates to agent-images-ns-<namespace>.
3. Remove imageOwner and Other Added Variables/Parameters
   • Reverse the steps outlined in this guide. Typically, just remove any occurrences of the imageOwner variable/parameter.