EMS survey reveals widespread concerns over certification

As ENDS has reported over the past few years, the importance of certified environmental management systems (EMSs) is gradually increasing. More firms are ploughing significant resources into writing and implementing these systems and employing third parties to audit them and award certificates.

Some 3,000 firms in the UK now hold ISO14001 certificates, and 78 are registered under the EU eco-management and auditing scheme (EMAS).

However, a significant proportion of companies, certification bodies and consultancies are expressing scepticism as to what ISO14001 and EMAS certificates prove about a company's commitment to environmental protection (ENDS Report 327, pp 31-33 ). This has been prompted in part by a steady stream of certified companies being prosecuted for environmental offences - the latest being brick manufacturer Ibstock (see p 64 ).

Empirical studies have so far failed unequivocally to establish that certified EMSs deliver environmental performance improvements. Researchers at the University of Sussex found no discernible benefits from having an EMS when they looked at the emissions of certain pollutants in a Europe-wide study two years ago. For some pollutants, performance was worse (ENDS Report 311, pp 27-29 ).

In a study for the Environment Agency, the Policy Studies Institute found that firms with an EMS had demonstrably better procedures, but these did not result in better outcomes such as lower rates of non-compliance or enforcement activity (ENDS Report 332, pp 3-4 ).

ENDS' survey was aimed at gauging the level of confidence now held in certified EMSs by inviting views from those with the most direct experience of them.

Some 350 people responded, representing a cross-section of companies with ISO14001 certificates, consultancies providing EMS implementation services, organisations providing ISO14001 certification and EMAS verification, and EMAS-registered companies.

Sustained performance?
A key finding is that a third of respondents believe that EMSs do not in themselves deliver sustained environmental performance improvements. Some 17% believe they make a difference around the time of initial implementation but that this is not sustained. A further 16% feel they make no difference over and above what would have been achieved as a result of other drivers, such as legislation. The rest felt that they did deliver sustained improvements.

While it can be difficult to distinguish between performance drivers, it is significant that so many respondents felt unable to attribute improvements specifically to EMSs.

Around one in four participants from companies with ISO14001 certificates believe that EMSs do not inherently afford sustained benefit in terms of environmental performance (see Table 1). Curiously, this view is shared by a third of respondents from companies who are in the process of implementing EMSs. This suggests that they may be being driven by expediency rather than by a real belief that EMSs deliver performance improvement.

Asked whether EMSs result in reduced environmental risks for companies in terms of non-compliance and reputational and financial issues, more than half of the respondents said that performance varies between firms. This view is held particularly strongly by consultants, only a third of whom believe that EMSs normally reduce environmental risk.

Furthermore, most respondents appear to believe that EMSs do not in themselves drive firms to identify resource efficiencies. More than half said that other drivers were more important. Only 28% agreed with the statement that EMSs deliver resource efficiencies which would not otherwise have been identified or pursued.

Both ISO14001 and EMAS require a corporate commitment to comply with applicable legislation. Yet our survey revealed that one in four participants felt that EMSs made little difference to legal compliance - either in terms of the frequency and scope of checks, or performance against permits and other legal requirements.

One respondent commented that it did not matter if EMSs do not by themselves result in such benefits. He argued that they play a role alongside a host of other drivers and should be viewed as part of a package of measures.

But management often turns this argument on its head and asks environmental managers to justify allocation of the substantial resources required to achieve and maintain EMS certification, particularly if other drivers can effect similar performance improvements. Regulators, too, have asked why they should offer benefits to companies with certified EMSs in preference to those without.

Variable picture
A number of respondents were responsible for the environmental performance of large corporations comprising many sites. Most reported that the level of improvement achieved by implementing EMSs varied considerably.

Jonathan Garrett of Smiths engineering group, which is in the process of getting 80 sites certified to ISO14001 worldwide, commented that "improvement in performance is variable....Some businesses make great improvement, others none at all and some in between."

He believes that two of the key factors determining the level of benefit which a site achieves from its EMS are the degree of management commitment at the site and the quality of the staff available to implement it. One site where the managing director had taken a keen interest in results delivered "huge improvements in performance." Other multi-site corporations cited the same factors. He expressed surprise that third party certification had not made the outcomes more consistent.

However, Mr Garrett believes that the decision to obtain ISO14001 certification had acted as a catalyst for change in his company, given a previous lack of focus on the environment. "Overall, for the group as a whole, we are confident that performance has improved and will improve still further in the future."

In December, Rolls-Royce reported a remarkably similar hit-and-miss experience with its certification bodies (see pp 3-4 ). Like Smiths, it believes that the problem lies not with ISO14001 as a tool but in who uses it and how well.

Motivating factors
The survey also examined the drivers for companies implementing certified or verified EMSs. Respondents were asked to choose any of five potential drivers.

The findings suggest that internal motivation is the biggest single driver for certification - pointing to a sincere desire to improve performance on the part of many. This is followed closely by supply chain and competitive pressures.

Regulatory pressure was also said to be significant, but few cited public pressure as a reason.

Conversely, more than half of respondents felt that other factors were more significant in determining companies' environmental reputations than whether they had ISO14001 or EMAS.

This low credibility appears to be translated into purchasing policy. Few respondents would be willing to take ISO14001 certification or EMAS registration at face value when it comes to judging the environmental credentials of their own potential suppliers.

Around half said they would ask for other evidence of performance, while a further 38% said it would make little difference to them whether a supplier had ISO14001 or EMAS (see Table 2).

One environmental consultancy commented that its major clients had recently begun to ask for other evidence of environmental performance from suppliers. However, others suggested that in practice these are still in a minority.

Guy Balcon of plastics converter Huhtamaki UK commented: "We get innumerable questionnaires demanding achievement of various standards without any follow through or demand for specific performance criteria."

Disquiet over certification
The survey also sought views on the way certification and verification of EMSs are carried out. It revealed even more disquiet and attracted the most comments from respondents, the majority of them critical - including some from certification bodies themselves.

Nearly half of the respondents do not believe that certification bodies or verifiers are sufficiently competent. A quarter felt they lacked knowledge of their clients' business operations, 13% said they did not have enough environmental knowledge, and 11% said they did not have enough of either (see Table 3).

In addition, four in ten respondents felt that certification bodies employed different approaches to their work and that this meant that the outcome was neither consistent nor comparable.

This view appears to be shared by some in the certification community itself. One individual from a large mainstream UK certification body who declined to be named commented: "A certified EMS is only as good as the company implementing it. If the top management just want a greenwash or a badge on the wall then there are certification bodies out there that will do that - i.e., give certification based on intent rather than actual evidence."

He added: "Pressure should be put on the UK Accreditation Service to name and shame the poor performing certification bodies rather than dragging us all down with them."

Another participant from a different certification body said he had seen the full spectrum of EMSs - from those where companies had been driven to investigate resource or waste reduction to the "flat and uninspired EMS". However, he argued, "certification bodies can help to push organisations but they cannot force them."

A manager from a company with several ISO14001-certified sites said that certification bodies appeared to "rubber stamp average performance" on some sites. Another commented that "it is possible to get the certificate on the wall and effect very little environmental improvement."

Adam Faiers, a policy officer at South Northants Council who oversaw the selection process for its certification body, revealed that the cost estimates submitted by eight bodies varied from £12,000 to £38,000 - a range which he described as "inordinate." He said that the bids were similar in terms of planned auditing time and auditor skills, but in some cases were inflated by "hidden" fees such as expenses and management fees.

Smiths has employed three certification bodies to certify its 80 sites. On the basis of this experience, Jonathan Garrett believes that "the key to delivering environmental performance improvement through ISO14001 lies with the certification body. If the body is focused on performance and outcomes as opposed to system documentation...then environmental improvement is much more likely."

Indeed, four out of ten respondents felt that certifiers and verifiers do not spend sufficient time assessing companies' performance outcomes as opposed to documented policies and procedures. Some two-thirds of those working for consultancies feel this way.

Respondents were also asked about their experiences of legal compliance assessments by certification bodies and verifiers. One-third said that in their experience this simply involved checking for the existence of documented procedures. Fewer than half of respondents had experienced any additional on-site evaluation of compliance.

Costs and benefits
Respondents were equivocal about whether the benefits of certification or verification outweighed the costs. Half felt that they did. Roughly one in six felt that they did not. The rest said that costs and benefits were evenly matched.

In selecting certification or verification bodies, the two most frequently applied criteria appear to be reputation and previous knowledge of the certification body from quality certification. A quarter of respondents cited each of these factors. One in six based their choice on cost and a similar proportion on whether or not the body had relevant sector knowledge.

Some respondents expressed concern about the trend towards integrated audits for quality, health, safety and environmental systems.

David Symons, associate director of WS Atkins, commented: "There are few expert quality auditors who are also really sound in environmental matters, and this provides real challenges to certification bodies where the client is also pushing hard for an integrated ISO9000/14000 audit."

He concluded: "Some certification audits of my clients' sites have been really exceptional and value added. Others have been a complete waste of time with the auditor starting at Clause 4.1 and working through on a clause by clause basis."

Strong signals
There will be some who will be put on the defensive by the survey findings. Indeed, some participants complained that people with a grievance would be more likely to respond. That said, others feared that people trying to sell a positive image for EMSs would be more likely to participate.

Some will argue that the survey only relates people's perceptions rather than constituting hard evidence. This is undeniable. But the results convey a strong signal about the credibility of EMSs based on the direct experience of professionals working in companies, certification bodies and consultancies. The findings must be viewed in conjunction with the empirical evidence available.

It is also true that many respondents were positive about the benefits of EMSs. Two-thirds felt that EMSs did deliver sustained environmental performance improvement beyond that achieved by other drivers. One half believed that the benefits of certification outweighed the costs. Mr Garrett's views on the catalytic effect of EMSs were shared by others.

Nevertheless, it is inescapable that the survey has revealed a significant lack of confidence in EMSs. Two years ago, UKAS dismissed a complaint by the chief executive of one certification body about the quality of the industry's work as "one emotional overview". Our survey suggests that many others feel the same way as this chief executive, and the scale of discontent - especially in relation to certification - means that it can no longer be glossed over.

Indeed, our survey sample included representatives of just under a third of the UK-based EMS certification bodies accredited by UKAS. Perhaps predictably, their responses are generally more positive than those of the overall pool. For instance, they all feel that the time spent on assessment of organisations is appropriate to their size and complexity.

But in some areas there is clearly a little doubt, as evidenced by the qualitative remarks submitted by some bodies. For instance, many do not believe that the approaches of their counterparts deliver a consistent or comparable outcome. Most also say they would not trust an EMS certificate on its own merits when making purchasing decisions, and would look for other evidence of environmental performance.

At the EMS national forum in December, UKAS publicly aligned itself with those saying that something has gone awry since the initial conception of EMS certification (see pp 3-4 ). Our survey findings suggest that many do not believe that EMSs deliver consistently for the environment. UKAS said it believes that this is because too many bodies have treated certification of EMSs in the same way as quality management.

One UKAS insider is even believed to have described the typical approach as being one of "effluent quality control" - a comment on the preoccupation of some certifiers on documented procedures instead of performance.

UKAS has pledged a programme of action to improve the quality of environmental assessment by certification bodies. It will look specifically at the potential problems posed by the move by certification bodies towards offering integrated audits of quality, safety, health and environmental management systems.

Some in the certification industry are also trying to encourage the crystallisation of a movement to address the criticisms around the efforts of the Association of British Certification Bodies, which plans a first meeting in February.

However, the impetus for change might yet fizzle out in arguments over resources, culpability and perceptions. If it does, the future for certified EMSs - despite their evident power to help companies improve environmental performance if applied and certified with the right mindset - would look bleak.

¹ ENDS would like to thank all those who took part in the survey. Figures illustrating the results are available at:www.endsreport.com/emssurvey