Skip to content

Azure DevOps Testing: Instructions & Participation List

Purpose & Scope

The purpose of this document is to describe and demonstrate how KaaS works with Azure DevOps for testing. This is to ensure a consistent and reusable approach to testing across releases. It also includes a list of participants, who have received and completed training - either through self-service or a peer workshop. This document is version controlled with GIT in Azure DevOps.

Prerequisites

  • In order to access Azure DevOps test module "Test Plans", you need to request the 'AzureDevOps : Basic plus Test' in novoAccess. Note: there is an increased license cost associated with the access, that NN ADO system management covers at the time of writing this.
  • You need to be a member of ADO project, "KaaS (Kafka as a Service)", requested through novoAccess (Contributor or Administrator)
  • Access to KaaS Artefacts, Reviews and Approvals which contains a matrix of approver roles

Fundamentals

Microsoft have provided guides and documentation on how to work within Test Plans, that all participants should read and understand; - How to Create Test plans - How to Create Test Cases - How to Run manual Test Cases

KaaS Setup

  • Test Plans - containers of test cases & protocol description - can be created by anyone, but can only be final approved by the members of the central NN ADO 'Quality Approver Group'. You also need to be a member of this group to final approve URS. Request "AzureDevOps : Central QA Approver" in novoAccess.
    • A template has been created for KaaS. Please copy the content to a new Test plan. The template can be found here: KaaS_TEMPLATE_Test_Plan for Release 1.0 (ID: 1719014)
      Protocol description should follow existing test plan structure to get a consistent experience
    • Naming should follow existing format, and include a SNOW CHG ID.
  • Test Cases are the actual steps and expected results, and follows the same approval flow as Test Plans
    • Within the test case, the 'Summary' tab should be used for writing the objective and any prerequisites
    • The level of details in steps depends on the test case executor, and can be justified in the test plan and/or summary
    • A test case should always be linked to an URS for easy traceability through the "tested by"
  • Test Runs is the execution and record of results for the test cases.
    • Test runs should be run/executed by someone other than the implementor
    • Test runs should be reviewed and acknowledged by a relevant SME (see KaaS Artefacts, Reviews and Approvals)
    • If the test case execution does not provide the expected result a test defect/bug needs to be raised (see Test Defect Management)