Skip to content

KaaS Defect Management

This process describes how test defects must be handled by the KaaS team.

Handling defects in ADO are based on the ADO qualified setup for GxP IT Systems and aligned with the process flow defined in Validation Deviations Q0300381.

The name Test Defects is used in this process. Please note that in ADO Test Defects are named 'Bugs'. In Validation Deviations Q0300381 Test Defects are named 'Validation Deviations'.

Roles & responsibilities, including Approval

Roles & responsibilities, including Approval of Test Defects are described in the IT Implementation Plan and Report - Kafka as a Service

An excerpt, and further detailing, is provided here:

  • The tester is responsible for creating Test Defects in ADO as work item 'Bug'.
  • SME Test Manager is responsible for triaging defects, for facilitating Test Defects Management meetings, progress and closure of Test Defects.
  • Test Defect Management team is responsible for assessing impact, agree corrective actions and accept closure of Test Defects.
  • SME Test Manager is responsible for acknowledging Test Defects for Unit & Integration test.
  • BO delegate is responsible for acknowledging Test Defects for IV & OV test.

Test Defect Management team consist of: - KaaS BO delegate - KaaS Test Manager - KaaS Implementation Responsible - KaaS SMEs

Process for Test Defects

Create Test Defects

Tester must create an ADO Work Item 'Bug' and provide sufficient information, when:

  1. Acceptance Criteria is not met
  2. Pre-requisites for the verification is not met
  3. Test procedure is not followed
  4. Test script errors that have impact on the result of the test

Typing errors that have not impact on the result of the test does not require a bug.

  • ADO Work Item should have the following information when being created:
    • Title - Provide a meaningful title that can be understood by non-SMEs
    • Repro Steps - Describe and use screenshots
    • Description - Additional Description
    • Root cause - Add root cause

Triage Test Defects

The Test Manager must triage ADO Work Items before presenting the Test Defect at the Test Defect Management meeting: - Triage should ensure that ADO Work Items have the following information: - Clear Title - Repro steps (include release number and environment) - Description - Analysis - Root cause - Type of bug - Suggestion for Priority - Suggestion for Consequence - Bug should be linked to right Testcase or DS - Meta data as defined in the last section in this WIKI page

Agree on Impact & resolution

The purpose of this step is to ensure that relevant stakeholders are informed and participate in assessing the impact of the Defect and agree on the resolution and applicable actions.

The team will also be agreeing to change status from resolved to closed, once corrective actions and re-test been successfully performed.

The Test Manager facilitates Test Defect Management meetings with the Test Defect Management Team to support this. These meetings may be recurring meetings or ad-hoc based on the need.

Outcome of the meeting is an assessment of the following:

  • Assess impact to code, M-unit and integration test
  • Assess impact to RS
  • Assess impact to Risk assessment
  • Assess impact to SD
  • Assess impact to Traceability Matrix
  • Assess impact on testing - can existing test cases be re-used or are new test cases required

The outcome of the assessment is described in the section 'Consequence'

Once assessment is complete, corrective actions and required re-testing can be agreed upon.

The outcome of the agreement is described in the section 'Correction and re-test - proposal'

Correct and Retest

Perform corrective actions and perform re-test.

If code has to be updated based on defect, code review must be performed after code update and before re-test is performed.

Describe actions performed and insert link to build (where Code review report, re-test result and other information can be found) in section Correct and Retest. Described where release number re-tested and on which environment.

State can be set to 'Resolved'.

Close Test Defect

Upon successful correction and re-test, the defect can be closed at the next Test Defect Management meeting, by changing state from 'Resolved' to 'Closed'.

Metadata

Ensure metadata is entered:

Analysis

- Criticality

Choose 'Minor' - based on SOP Validation Deviations - Q0300381

- Root cause

Mandatory drop down picklist value to be picked

- Type of bug

Mandatory drop down picklist value to be picked- this dropdown list is based on the testcase point of view. Nature of bug in relation to testcase.

Close details

- Close reason

Mandatory drop down picklist value to be picked

Planning

- Story points

Type '0'

- Priority

Unless decided at the Bug management meeting, priority is set to 1 Choose '1'

- Severity

Unless decided at the Bug management meeting, Severity is set to 3 - High Choose '3 - High'