| Title | Kafka as a Service Solution Design - Data Product Pipeline Template |
|---|---|
| Version | 5.0 |
| Approval Pull Request (PR) | PR180699 |
Solution Design - Data Product Pipeline Template¶
Purpose & Scope¶
This Solution Design covers the Data Product Pipeline Template (formerly known as Application Pipeline template), used by Teams to deploy and release on the KaaS Platform. See the Solution Designs section for an overview of the entire KaaS Solution.
Abbreviations and definitions¶
See Abbreviations and definitions
Solution Design Requirements¶
| ID | Name |
|---|---|
| 1 | Application Pipeline Template |
Application Pipeline Template¶
The Data Product pipeline template will allow projects/teams to clone and insert the pipeline Template into their respective ADO projects from the KaaS ADO project.
- After having configured the input files, Teams will be able to deploy Topics and Schemas on KaaS from Development through Production, without the involvement of central platform administrators and operators.
Initial setup and prerequisites by the central team is described in the Platform Handbook: Team Onboarding
Guidance on the usage of the template can be found in the Project Handbook and with the source code.
The Data Product Pipeline template consists of two repos; - kaas-pipeline-templates which contains the core terraform code and logic, and is the repo that in most cases will be updated in case of new features. This repo is part of KaaS ADO Project. - kaas-assets-template which contains the pipeline stages and references all the logic in kaas-pipeline-templates when executed. This repo is part of KaaS Workspace ADO Project.
Detailed design and technical documentation is stored in the kaas-pipeline-templates and an Operation Guide is stored in kaas-assets-template
Traceability between Requirements¶
Traceability between Requirements (URS) and (SDR)¶
| ID | Work Item Type | URS (Status Dynamically Updated) | GIT SDR |
|---|---|---|---|
| 1933853 | URS | #1933853 | Solution Design Requirement: Application Pipeline Template |
Change Log¶
| Version no. | CHG no. | Approved Pull Request | Date | Change Description |
|---|---|---|---|---|
| 1.0 | CHG0100614 | PR102950 | Date of acknowledgement | New document |
| 2.0 | CHG0100614 | PR111467 | See PR | Updated broken links |
| 3.0 | CHG0122187 | PR131425 | See PR | Updated the latest approved URS ID 1933853 |
| 4.0 | CHG0136055 | PR145250 | See PR | Updated name from 'Application Pipeline Template' to 'Data Product Pipeline Template', source code repo links, as part of ADO resource Refactor CHG |
| 5.0 | CHG0159095 | PR180699 | See PR | Updated description and references to GIT repos, as part of KaaS Workspace introduction and 'managed template distribution'. |