Improvement planning and objective evidence

Home / Management Systems Auditing / Improvement planning and objective evidence

During an internal audit of a client’s business continuity management system, an auditee in the company’s communications department who was responsible for internal communications, offered up – when prompted – an improvement opportunity that he said he had identified a month or so earlier regarding the early warning procedure that they were currently following. Naturally, as an auditor, I wanted to see objective evidence of this improvement.

At the time I was auditing, the procedure for notifying staff of an impending, potentially disruptive incident (an approaching sand storm for example) involved sending out a high priority red color-coded alert message. These messages were being sent by email and SMS to all staff in the company, at all levels, and to all locations and offices across the entire country in which they operated to warn management and staff of a coming event.

The communications manager told me that he had a concern that by sending out these messages indiscriminately, staff whom would be unaffected by a localized event may soon become complacent of the little red notice’s importance and potentially just ignore them, hence impacting the effectiveness of the procedure. The improvement was to instead target the sending of such messages to just the relevant offices and staff instead.

Planning Improvements

In any management system audit, such an initiative is a very positive finding since it is evidence that continual improvement is actually occurring, as is required by the standards. In ISO 22301 and others, this requirement basically falls under the “Performance” requirements of clauses 9 and “Improvement” in clause 10. However, it is the next step where, in my experience, many organizations often fall a little short.

When an improvement has been identified, the next steps should obviously involve actions, such as discussions, agreement, approvals, and planning of the proposed change, among other things. And given that, sufficient objective evidence must be retained by the organization as the next audit question is likely going to be something along the lines of, “Please show me the plan!”.

Unfortunately, many organizations with a less mature understanding of the process approach and without that PDCA mindset think that by just fulfilling the minimum stated requirements of the standard that they are fully meeting their obligations and have a conforming management system. But this is not necessarily so.

Auditors are not there to simply tick boxes, but rather, to audit the intention, implementation, and effectiveness of the management system – i.e. they’re auditing the process itself and there must be objective evidence made available to auditors so that they can report factual and objective findings to the organization’s top management.

In the case of my auditee, the communications manager, he was unable to provide anything more than a few inspiring words of intent and show me a couple of vague emails that had been largely ignored by management. Ah-ha, I hear you say! We are now starting to see objective evidence of a lack of management support and commitment 🙂

The Process Approach

Not to get into a discussion about documentation requirements, and it is true that a plan may or may not necessarily be documented, but some level of record keeping (in whichever form) must be available as evidence of decisions and actions planned and taken, along with the results.

A member of staff making the verbal claim that an improvement is being made without then supporting that claim with something more tangible is likely to fall short in an audit and certainly doesn’t give an auditor any confidence.

No matter what is being planned, there should be at least some evidence of the planning process. Standard requirements from clauses 4, 5, 6, and 7 should give us the clues we need to be effective in planning anything here and as an auditor, I need to have confidence that a process is being systematically followed.

What is Objective Evidence?

To clarify what it is that typically constitutes as objective evidence in an audit, it is essentially tangible, first-hand, fact-based information that we can see, hear, smell, or touch. Objective evidence tends to fall into the following three categories:

  1. Documents and records – it is written down and we can see it. This can be in any format, such as electronic (pdf, word files, database entries, etc.) or hardcopy paperwork;
  2. Statement of fact – something that someone tells us verbally. This person must be addressing their own responsibility within the process, of course. A statement of fact must be first hand. For example, rumors cannot be taken as being factual, accurate or true;
  3. Observation – something we can see, such as an emergency exit sign hanging over the fire escape.

To be useable as audit evidence, we must be impartial in the audit process and the evidence that we gather must be objective (as opposed to being subjective). It must be factual, true, and accurate.

Auditors should not make assumptions. For example, in the case that members of staff are required by company policy to lock their desk draws at night – an auditor who believes that this is not actually occurring cannot raise an audit finding by reporting: “I have a feeling that staff are not locking their desk draws in the evening”.

Whereas, if a member of staff tells us, “I don’t lock my draw at night. I can’t, because the company did not give me a key”, this can be taken as a statement of fact. Alternatively, we could visit the office in the evening and test a sample of draws to check if they are locked. This would be a finding based on observation. Or in the case of an electronic locking system, we could review the records of date and time when the draws are being locked and unlocked.

Conclusion

To ensure that you are getting the most out of your internal and external audits, ensure that your management processes identify and include your own requirements for maintaining evidence that is sufficient to be able to prove that business processes are being effectively implemented and controlled. For example, it is generally a good idea to document any improvement plans and keep records of the decisions that were made regarding the change as well as the results.

This is not to say document and record everything! Evidence only needs to be sufficient enough that we can have confidence that things are, or aren’t, as they should be.

So, in establishing document and record processes the organization should be considering the business context, purpose of the management system, objectives, risk, etc. Evidence can also be in many forms, such as email, system logs, video, statements made by staff, notice board announcements, etc, etc. But objective evidence there must be.

Leave a Reply

%d bloggers like this: