Operation Guide: API Key Rotation¶
API Key Rotation¶
Note on AWS Authentication: As of July 2025, KaaS platform uses AWS Service Connections with OIDC, which provide automatically rotating temporary credentials. AWS credentials no longer require manual rotation.
For other API keys (Confluent Cloud, etc.), it is recommended that rotation is performed with a frequency of ideally at least 3 months, or at maximum, 6 months.
Note there may be more specific compliance requirements elsewhere which overrule this recommended frequency, which in that case, will have to be followed. This article should be updated to reflect such a guideline.
Platform API keys - Rotation Guides¶
Since the platform API keys are the 'seed' API keys that provide the access to provision subsequent API keys, they have to be provisioned and therefore rotated manually.
Rotation of platform API keys is handled manually by a designated member of the platform team.
Here the steps will be described for how to rotate the API keys.
Confluent Cloud API key¶
- Open Confluent Cloud with an admin account
- Click on the sidebar (triplebar) icon in the top right corner
- Open Cloud API keys in the menu
- Click + Add key
- Select Granular access
- Select Existing account tf_runner
- Click Next
- Set following in Description tf_runner
- Download and continue.
- Save API key info from text file in PIM
- When applicable (when new API key has been taken into use), go back to Cloud API keys and delete the old API that is owned by service account tf_runner, such that only the new one you created is still there.
AWS Authentication¶
✅ CURRENT APPROACH: AWS Service Connections with OIDC
As of July 2025, KaaS platform uses AWS Service Connections with OIDC authentication which provides temporary credentials that automatically expire. No manual rotation is required for AWS authentication.
Platform pipelines (replacing CloudDevOpsUser):
- Preprod: kaas-preprod-platform-admin
- Prod: kaas-prod-platform-admin
Team pipelines:
- Pattern: kaas-{confluentOrganization}-ns-{namespace}-deploy
- Examples: kaas-preprod-ns-teamname-deploy, kaas-prod-ns-teamname-deploy
- Automatic Rotation: Temporary credentials are generated fresh for each pipeline run
- Security: Trust relationships ensure credentials cannot be misused outside pipeline context
⚠️ LEGACY IAM User Approach (DEPRECATED):
Go to IAM → Users → select relevant user → security credentials, under access key → Create Access Key (maximum of 2).
This access key and secret must be shared with the KaaS project, who will need to update the variable group in their ADO library and paste it there, so they can be used by their pipelines.
Once the new key has been shared with the project and saved in variable groups, delete the old access key, such that only the new one you created remains.
For AWS Service Connection setup details, see: AWS OIDC Service Connection Guide
Team API keys - Rotation guides¶
Here the steps will be described for how to rotate the API keys.
Confluent Resource API keys¶
Each service account will have 2 or more API keys for each environment, for 1) Kafka clusters and 2) Schema registry. These are stored in a team AWS secret for each environment, named confluent_cloud_
These can be rotated by simply deleting the given API key in Confluent Cloud UI, and then running the access-management-pipeline. The access-management-pipeline will then detect that this API key is missing and provision a new one and update the secret with the new API key.
In the future, this may be fully automated by the access-management-pipeline as this is technically feasible.
Kafka clusters¶
This process applies to every environment where rotation should happen
Prerequisites: - Confluent Cloud ResourceKeyAdmin/admin access - Azure DevOps relevant KaaS project access level to run and approve Teams Provisioning Pipeline (TPP) deployments
The naming standard for the service account owner of the API keys will be
Part 1: Confluent old API key deletion
1. Open Confluent Cloud with an ResourceKeyAdmin/admin account
2. Go to "Environments" and select the environment of choice
3. Select the Kafka cluster of choice and go to API keys in the left sidebar
4. Identify the API key of choice owned by a service account with the naming standard
Part 2: Provisioning of new API key by TPP 1. Open TPP in Azure DevOps Pipelines: Link 2. Click "Run Pipeline" 3. Run and approve deployments for each applicable environment that part 1 was executed for
Schema registry¶
This process applies to every environment where rotation should happen
Prerequisites: - Confluent Cloud ResourceKeyAdmin/admin access - Azure DevOps relevant KaaS project access level to run and approve Teams Provisioning Pipeline (TPP) deployments
The naming standard for the service account owner of the API keys will be
Part 1: Confluent old API key deletion
1. Open Confluent Cloud with an ResourceKeyAdmin/admin account
2. Go to "Environments" and select the environment of choice
3. Identify the section "Stream Governance API" in the bottom right corner, which should have the section "Credentials" at the bottom
4. Click "View & Manage" under Credentials, after "
Part 2: Provisioning of new API key by TPP 1. Open TPP in Azure DevOps Pipelines: Link 2. Click "Run Pipeline" 3. Run and approve deployments for each applicable environment that part 1 was executed for
AWS Authentication¶
✅ CURRENT: Teams use AWS Service Connections with OIDC - no manual rotation required.
- Team Service Connections: Pattern
kaas-{confluentOrganization}-ns-{namespace}-deploy - Automatic Rotation: Fresh temporary credentials for each pipeline run
- No Manual Steps: OIDC handles authentication automatically
⚠️ LEGACY (DEPRECATED):
Go to IAM → Users → select relevant user → security credentials, under access key → Create Access Key (maximum of 2).
This access key and secret must be shared with the projects, who will need to create a variable group in their ADO library and paste them there, so they can be used by their pipelines.
Once the new key has been shared with the project and saved in variable groups, delete the old access key, such that only the new one you created remains.
Client (service account) API keys - Provisioning & Rotation guide¶
Client applications within teams will need API keys for the Confluent Cloud service accounts, that are specific to the client.
One API key for Kafka clusters need to be created (and continually rotated) for each client service account, for each environment.
Currently, these are provisioned manually by a platform operator. In a future release, a feature will be created to automatically handle this in Teams Pipeline, requiring a secret storage whose access is managed via Azure AD, so team members can get access to the API keys.
Provisioning of Kafka API keys for Client Service Accounts¶
- Open Confluent Cloud with an ResourceKeyAdmin/admin account
- Go to "Environments" and select the environment of choice
- Select the Kafka cluster of choice and go to API keys in the left sidebar
- Select "Add key" in the right upper corner within the API key list, and select the scope "Create an API key associated with an existing service account"
- Identify the client service account and click next
- Click next in ACLs or add specific scopes
- Store the API key and secret somewhere safe
- Enter a description for the API key. The recommended standard for this is: "
client API key for "
Rotation of client API keys¶
The process of rotation here works by repeating the provisioning steps (part 1) in this section, and creating a new key (note the key ID or append something like "key 2" to the description, to distinguish from the old key).
After that, the old key can be deleted by following exactly the same steps as "Part 1" described in "Team API Keys - Rotation Guides", except the owner of the API key you delete will be the name of the client service account, and the description will be the one created in Part 1 here.
Part 2 of "Team API keys - Rotation Guides" is not relevant here, as the keys aren't managed by Teams Provisioning Pipeline, so running it won't work.