| Title | Kafka as a Service Solution Design - Stream Governance Pipeline |
|---|---|
| Version | 2.0 |
| Approval Pull Request (PR) | PR145250 |
Solution Design - Stream Governance Pipeline¶
Purpose & Scope¶
This Solution Design covers the Stream Governance Pipeline, which is used by the KaaS Platform Team to create Business Metadata and Tags. The metadata and tag objects can then be applied by Teams using the Data Product Pipeline Template. See the Solution Designs section for an overview of the entire KaaS Solution.
Abbreviations and definitions¶
See Abbreviations and definitions
Solution Design Requirements¶
| ID | Name |
|---|---|
| 1 | Stream Governance Pipeline |
| 2 | Applying Business Metadata and Tags |
Stream Governance Pipeline¶
The Stream Governance pipeline will allow the configuration and deployment of business metadata and tag 'objects' from a pipeline. Business metadata and tags are used to describe and categorize topics and schemas, to increase context understanding and discovery. Since the configuration is stored in a GIT /configuration folder, there is full traceability of the configuration, and it can be easily redeployed in case of a disaster. The pipeline will enable;
- deploy business metadata
- deploy tags
Through pipeline approval gates between environments.
Detailed solution design, operation guides and source code is stored here stream-gov-modules GIT
Applying Business Metadata and Tags¶
The Business metadata and tags 'objects' created by the Stream Governance Pipeline needs to be applied to specific topics and schemas to add value. This is done by the Data Product Pipeline Template, which allows Teams/projects to add the configuration in a file structure, similar to how topics and schemas are configured.
Traceability between Requirements¶
Traceability between Requirements (URS) and (SDR)¶
| ID | Work Item Type | URS (Status Dynamically Updated) | GIT SDR |
|---|---|---|---|
| 1932759 | URS | #1932759 | Solution Design Requirement: Stream Governance Pipeline |
Change Log¶
| Version no. | CHG no. | Approved Pull Request | Date | Change Description |
|---|---|---|---|---|
| 1.0 | CHG0122187 | PR125748 | See PR | New document |
| 2.0 | CHG0136055 | PR145250 | See PR | Updated source code repo links, as part of ADO resource Refactor CHG |