Prev Next

Gerard O'ReganConcise Guide to Software EngineeringUndergraduate Topics in Computer Science10.1007/978-3-319-57750-0_9

9. Software Quality Assurance

Gerard O’Regan¹

(1)

SQC Consulting, Cork, Ireland

Gerard O’Regan

Abstract

This chapter discusses software quality assurance and the importance of process quality. It is a premise in the quality field that good processes and conformance to them are essential for the delivery of high-quality product, and this chapter discusses audits and describes how they are carried out.

Keywords

AuditorIndependence of auditorSQA teamAudit planningAudit meetingAudit reportingAudit actionsTracking actionsAudit escalationTraining

9.1 Introduction

The purpose of software quality assurance is to provide visibility to management on the processes being followed and the work products being produced in the organization. It is a systematic enquiry into the way that things are done in the organization, and involves conducting audits of projects, suppliers and departments. It provides:

Visibility into the extent of compliance to the defined processes and standards.
Visibility into the processes and standards in use in the organization.
Visibility into the effectiveness of the defined processes.
Visibility into the fitness for use of the work products produced.

Software quality assurance involves planning and conducting audits; reporting the results to the affected groups; tracking the assigned audit actions to completion; and conducting follow-up audits, as appropriate. It is generally conducted by the SQA group,¹ and this group is independent of the groups being audited. The activities involved include (Table 9.1):

Table 9.1

Auditing activities

Activity	Description
Audit planning	– Select projects/areas to be audited during period – Agree audit dates with affected groups – Agree scope of audit and advise attendees what needs to be brought to the meeting – Book room and send invitation to the attendees – Prepare/update the audit schedule
Audit meeting	– Ask attendees as to their specific role (in the project), the activities performed and determine the extent to which the process is followed – Employ an audit checklist as an aid – Review agreed documentation – Determine if processes are followed and effective
Audit reporting	– Revise notes from the audit meeting and review any appropriate additional documentation – Prepare audit report and record audit actions (consider getting feedback on report prior to publication) – Agree closure dates of the audit actions – Circulate approved report to attendees/management
Track actions	– Track audit actions to closure – Record the audit action status – Escalation (where appropriate) to resolve open actions
Audit closure	– Once all actions are resolved, the audit is closed

All involved in the audit process need to receive appropriate training. This includes the participants in the audit who receive appropriate orientation on the purpose of audits and their role in it. The auditor needs to be trained in interview techniques, including asking open and closed questions, as well as possessing effective documentation skills in report writing, in order to record the results of the audit. The auditor needs to be able to deal with any conflicts that might arise during an audit.²

The flow of activities in a typical audit process is sketched in Fig. 9.1, and they are described in more detail in the following sections.

Fig. 9.1

Sample audit process

9.2 Audit Planning

Organizations vary in size and complexity and so the planning required for audits will vary. In a large organization, the quality manager or auditor is responsible for planning and scheduling the audits. In a small organization, the quality assurance activities may be performed by a part-time auditor who plans and schedules the audits.

A representative sample of projects/areas in the organization will be audited, and the number and types of audits conducted will depend on the current maturity of the organization. Mature organizations with a strong process culture will require fewer audits, whereas immature organizations may need a larger number of audits to ensure that the process is ingrained in the way that work is done.

It is essential that the auditor is independent of the area being audited. That is, the auditor should not be reporting to the manager whose area is being audited, as otherwise important findings in the audit could be omitted from the report. The independence of the auditor helps to ensure that the findings are fair and objective, as the auditor may state the facts as they are without fear of negative consequences.

The auditor needs to be familiar with the process and in a position to judge the extent to which the standards have been followed. The audit report needs to be accurate, as incorrect statements made will damage the credibility of the auditor. The planning and scheduling activities will include:

Project/area to be audited.
Planned date of audit.
Scope of audit.
Checklist to be used.
Documentation required.
Auditor.
Attendees.

The auditor may receive orientation on the project/area to be audited prior to the meeting and may review relevant documentation in advance. A checklist may be employed by the auditor as an aid to structure the interview.

The role requires good verbal and documentation skills, as well as the ability to deal with any conflicts that may arise during the audit. The auditor needs to be fair and objective, and audit criteria will be employed to establish the facts in a non-judgmental manner.

Software quality assurance requires that an independent group (e.g. the SQA group) be set up. This may be a part-time group of one person in a small organization or a team of auditors in a large organization. The auditors must be appropriately trained to carry out their roles. The individuals being audited need to receive orientation on the purpose of audits and their role in the audit.

9.3 Audit Meeting

An audit consists of interviews and document reviews and involves a structured interview of the various team members. The goal is to give the auditor an understanding of the work done, the processes employed and the extent to which they are followed and effective. A checklist tailored to the particular type of audit being conducted is often employed. This will assist in determining relevant facts to judge whether the process is followed and effective. Table 9.2 gives a small selection of questions that may be part of an audit checklist.

Table 9.2

Sample auditing checklist

Item to check
Project management
Has the project planning process been consistently followed?
Is the project plan complete and approved?
Are the risk log, issue log and lessons learned log set up?
Is the Microsoft Schedule (or equivalent) available and up to date?
Are the weekly status reports available and do they follow the template?
Configuration management
Are the appropriate people involved in defining, assessing the impact and approving the change request?
Are the affected deliverables (with the CR) identified and updated?
Are all documents and source code in the repository?
Are checking in/checking out procedures followed?
Supplier management
Is the statement of work complete?
Have the PM skills of the supplier been considered in the evaluation?
Does the formal agreement include strict change control?
Requirements, design and testing
Are the user requirements complete and approved?
Are the system requirements complete and approved?
Is the design complete and approved?
Are the requirements traceable to the design and test deliverables?
Are the unit test scripts available with the results recorded?
Are the system test cases available with results recorded?
Are UAT test cases available with results recorded?
Deployment and support
Are the user manuals complete and available?
Are all open problems documented?

The audit is an enquiry into the particular role of each attendee, the activities performed, the output produced, the standards followed and so on. The auditor needs to be familiar with the process and in a position to judge the extent to which it has been followed.

The auditor opens the meeting with an explanation of the purpose and scope of the audit and usually starts with one or more open questions to get the participants to describe their particular role. Each attendee is asked to describe their specific role, the activities performed, the deliverables produced and the standards followed. Closed questions are employed to obtain specific information when required.

The auditor will take notes during the meeting, and these are reviewed and revised after the audit. There may be a need to review additional documentation after the meeting or to schedule follow-up meetings.

9.4 Audit Reporting

Once the audit meeting and follow-up activities have been completed, the auditor will need to prepare an audit report to communicate the findings from the audit. A draft audit report is prepared and circulated to the attendees, and the auditor reviews any comments received and makes final changes to address any valid feedback.³ The approved audit report is then circulated to the attendees and management.

The audit report will include audit actions that need to be addressed by groups and individuals, and the auditor will track these actions to completion. In rare cases, the auditor may need to escalate the audit actions to management to ensure resolution.

The audit report generally includes three parts, namely the overview, the detailed findings and an action plan. This is described in Table 9.3.

Table 9.3

Sample audit report

Area	Description
Overview of audit	This gives an overview of the audit including the area audited, the date of the audit, its scope, the auditor and attendees, and the number of audit actions raised
Audit findings	These will vary depending on the type of audit, but it may include findings from project management, requirements, design, coding, configuration management, testing and peer reviews, customer support, etc.
Action plan	This will include an action plan to address the findings

9.5 Follow-Up Activity

Once the auditor has circulated the audit report to the affected groups, the focus then moves to closure of the assigned audit actions. The auditor will follow up with the affected individuals to monitor closure of the actions by the agreed date, and where appropriate, a time extension may be granted. The auditor will update the status of an audit action to closed once it has been completed correctly. In rare cases, the auditor may need to escalate the audit action to management for resolution. This may happen when an assigned action has not been dealt with despite one or more time extensions. Once all audit actions have been closed, the audit is closed.

9.6 Audit Escalation

In rare cases, the auditor may encounter resistance from one or more individuals in completing the agreed audit actions. The auditor will remind the individual(s) of the audit process and their responsibilities in the process. In rare cases, where the individual(s) fail to address their assigned action(s) in a reasonable time frame, the auditor will escalate the non-compliance to management. The escalation may involve:

Escalation of actions to middle management.
Escalation to senior management.

Escalation is generally a rare occurrence, especially if good software engineering practices are embedded in the organization.

9.7 Review of Audit Activities

The results of the audit activities will be reviewed with management on a periodic basis. Audits provide important information to management on the processes being used in the organization; the extent to which they are followed; and the extent to which they are effective.

An independent audit (usually a third party or separate internal audit function) of SQA activities may be conducted to ensure that the SQA function is effective. Any non-compliance issues are identified and assigned to the auditor and quality manager for resolution.

9.8 Other Audits

The audit process that we discussed has been focused on process audits conducted during a project. Other audits that may be conducted include supplier audits, where the auditor visits the supplier to determine the extent to which they are following the agreed processes and standards for the outsourced work.

The SQA team is often the point of contact to facilitate customer audits, where an audit team from the customer visits the organization to determine the extent to which they are following processes and standards.

9.9 Review Questions

1.

What is the purpose of an audit?
2.

What planning is done prior to the audit?
3.

Explain why the auditor needs to be independent?
4.

Describe the activities in the audit process.
5.

What happens at an audit meeting?
6.

What happens after an audit meeting?
7.

How will the auditor deal with a situation where the audit actions are still open after the due date?

9.10 Summary

The purpose of software quality assurance is to provide visibility to management on the processes being followed and the work products being produced in the organization. It is a systematic enquiry into the way that things are done in the organization, and it involves conducting audits of projects, suppliers and departments.

It provides visibility into the processes and standards in use, their effectiveness and the extent of compliance to them. It involves planning and conducting audits; reporting the results to the affected groups; tracking the assigned audit actions to completion; and conducting follow-up audits, as appropriate. It is generally conducted by the SQA group, and this group is independent of the groups being audited.

The audit planning is concerned with selecting projects/areas to be audited, determining who needs to be involved and dealing with the logistics. The audit meeting is a formal meeting with the audit participants to discuss their specific responsibilities in the project, the processes followed and so on.

The audit report details the findings from the audit and includes audit actions that need to be resolved. Once the audit report has been published, the auditor will track the assigned audit actions to completion, and once all actions have been addressed, the audit may then be closed.

Footnotes

This group may vary from a team of auditors in a large organization to a part-time role in a small organization.

The auditor may face a situation where one or more individuals become defensive and will need to reassure individuals that the objective of the audit is not to find fault with individuals, rather the objective is to determine whether the process is fit for purpose and to promote continuous improvement, as well as identifying any quality risks with the project. The culture of an organization has an influence on how open individuals will be during an audit (e.g. individuals may be defensive if there is a blame culture in the organization rather than an emphasis on fixing the process).

It is essential that the audit report is accurate, as otherwise the auditor will lose credibility and become ineffective. Therefore, it is useful to get feedback from the attendees prior to publication of the report, in order to validate the findings. However, in some implementations of software quality assurance, the audit report is issued directly to the attendees without the performance of this step.

Prev Next