Tag Archives: auditing

Results from the Auditor QuickQuiz

Our auditor quiz is now closed after a month. The questions were based on existing international non-financial auditing standards, Association of Certified Fraud Examiners (ACFE) fraud identification/examination techniques and US Government Auditing Standards for non-financial audits. There were fewer respondents than we had hoped so we can’t extrapolate beyond our dataset. Even so, some notable trends did emerge.

Of those who responded, 47% were EHS auditors and 27% were CSR auditors. We had hoped more CSR auditors would have participated. Other information about the respondents’ backgrounds:

  • 60% had no certification or “other”
  • 50% have 10 years or less auditing experience
  • 50% have 50 or fewer audits
  • 13% have participated in more than 500 audits during their career
  • 63% spend at least 75% of their time conducting audits

There were only 2 “passing” scores – i.e., greater than 70%. The average score was 49% – far lower than was expected.

Knowledge of standard terminology seems to be lacking, further reflected in poor scores for questions that embedded the terminology within them. For instance, only 30% correctly defined “audit criteria” as meaning the audit protocol. This likely led to 53% of respondents incorrectly answering that QA/QC reviews should include assessing the correctness of the “audit criteria used by the auditor.” QA/QC reviews of auditor working papers should look at how an auditor applied the audit criteria, not the inherent accuracy of the criteria (or audit protocol) used by the auditor. Indeed, only 10% correctly identified that none of the answer options are appropriate for QA/QC reviews.

Only 3% considered interviews better than document reviews when asked directly what type of evidence is strongest. Yet when the question was placed in a practical setting, 73% indicated they would rely on interviews over documentation. Only 26% correctly identified the evidence hierarchy (from strongest to weakest).

On a more positive note, 83% answered that they would decline to develop a document that they audited, meaning 17% did not view this as a conflict of interest. Frankly, we were disappointed that there was not a perfect score in identifying this to be an independence issue.

In answering the question listing possible common evidence problems, just over half (53%) correctly indicated that all of the answer options are common evidence problems.

Finally, 2/3 incorrectly answered that initial determinations of significance/materiality should be made after assessing evidence. It is possible that respondents did not read the question carefully and pick up the word initial.

Certainly more responses would have provided a better representation, but we think there are some valuable take aways from our limited data.  Among them – the gap between EHS/CSR auditor knowledge and existing (and theoretically similar) non-financial audit standards may be larger than previously thought.  As the importance – and liabilities – of sustainability/CSR audits grow, increased auditor training and competence seems warranted.

Last Week for Auditor QuickQuiz

Our auditor QuickQuiz will close at the end of the day September 1.  We hope to see more folks will take a few minutes to answer the questions.  It is painless.

Some of the trends we are seeing are:

  • 67% of the respondents have more than 10 years experience, with 75% or more of that experience doing EHS/CSR audits.
  • Only 15% of the respondents had a passing score.
  • There is a gap in knowledge and application of fundamental audit terminology.
  • There is inconsistency in understanding the strength of evidence types, with an over-reliance on interviews over documentation.

Things are likely to improve when we get more responses.

Fraud in Sustainability/CSR

Fraud is increasingly a topic in our conversations. We have had direct experience with EHS fraud in the past. The most recent occurrence was helping a client unravel an embezzlement scheme using waste disposal as the fraud mechanism. It played out a bit like a made-for-TV movie – not the kind of thing I ever expected to see personally, nor in the 21st century.

New pressures and risks are developing around sustainability/CSR reporting. Although still largely voluntary (certain aspects are mandated in the US, UK and Australia for instance), its business importance has grown dramatically in the past 5 years.

Customers demand more transparency and reporting in their supply chains, and many make procurement decisions based on this information. Many institutional and activist investors carefully review sustainability/CSR disclosures and make decisions using that information. It is now common for shareholder resolutions to be filed related to the disclosures, or lack thereof. Major media outlets have sustainability/CSR desks specifically focused on these matters and who pore over the filings and report on them.

We are finding that there is very little consideration given to fraud assessment or monitoring in this context – so is it even meaningful? We think so, and well known fraud and compliance expert Hui Chen agrees. Let’s apply the Fraud Triangle to supplier CSR performance.

  • Motivation. There is much on the line for businesses and their suppliers in terms of CSR results. As pointed out above, sustainability/CSR disclosures and performance may directly impact revenues, reputation and investor activity. No one wants to be on the wrong end of that. Motivation? Check.
  • Rationalization. It isn’t much of a stretch to see how an individual can rationalize using alternative facts due to the business pressures. In some cases, suppliers in developing countries may rationalize their actions further due to their own cultural setting. But let’s not kid ourselves into thinking that the US is immune itself.
  • Opportunity. There is ample opportunity for motivated suppliers to commit fraud. In some instances, CSR auditors are used to review suppliers. But those hiring audit firms many times severely limit the auditors by imposing minimal scope/effort driven primarily by cost. Suppliers know their customers’ auditors are not enabled to conduct a thorough review, and with pre-scheduled site visits, they have plenty of notice to dress the place up for the auditors.

This is only one example of how fraud can enter into the sustainability/CSR picture. If this isn’t included in your company risk assessments, or considered in the context of CSR/sustainability reporting, it should be.

Hui Chen: Applying DOJ’s Compliance Questions to Supplier/Social Responsibility Auditing

By: Hui Chen, former Compliance Counsel Expert, Fraud Section, Department of Justice

Ed. Note:  Hui Chen gained national attention when she made a “noisy withdrawal” from the Department of Justice this past June.  Afterwards, she graciously agreed to write the following for us.  We applaud her deep commitment to integrity and greatly appreciate her making time to pen this article.

In February 2017, the Fraud Section of the Criminal Division of the Department of Justice (“DOJ”) released a document entitled “Evaluation of Corporate Compliance Programs” (“Evaluation Questions”) which makes public the types of questions the Fraud Sections asks in its evaluation of corporate compliance programs in the context of its criminal investigations. The Evaluation Questions immediately gained the attention and interest of anti-fraud compliance professionals as well as global regulators and law enforcement interested in corporate accountability. The Fraud Section, which prosecutes cases involving foreign corruption, financial,  securities  and healthcare fraud, has brought corporate prosecutions with historic fines and penalties and exerts enormous influence in the those areas of compliance. What is often missed in the narrative, however, is the Fraud Section’s leading role in two of the largest environmental criminal prosecutions in history: the Deepwater Horizon and the VW emission scandal. As the Fraud Section’s Compliance Counsel Expert, I had the privilege of being involved in these cases, and they were very much on my mind as I drafted the Evaluation Questions.

Although the Evaluation Questions are set in the context of criminal investigations, one of the intents of the document is to also provide a framework for companies and compliance professionals to design, implement, and test their compliance programs for effectiveness. That framework is every bit as applicable to EHS and sustainability programs as it is for anti-fraud compliance programs.

At its core, the Evaluation Questions center around the following tenets of effectiveness: credibility, measurements, accountability, and continuous improvement. Let’s briefly explore these principles and see how they apply in the context of supplier/social responsibility auditing.

Credibility

The Evaluation Questions probe the credibility of companies’ boards, senior leadership, and compliance and control functions. It specifically names “audit” as one of the “relevant control functions”. It asks whether “compliance and control personnel ha[ve] the appropriate experience and qualifications for their roles and responsibilities.” How companies define that appropriateness tells a lot about the company. For example, companies that define appropriate experience largely in terms of certifications tend to be less sophisticated: they rely on commercial certification bodies to exercise the judgment and evaluation on their behalf. These types of personnel often do not perform impressively when specific questions involving real experience and expertise are posed to them: i.e. “Explain your sample selection methodology”, “How would you handle specific situations”, “What specific red flags do you look for when you are auditing for X”, etc. In this regard, I find Elm’s Auditor QuickQuiz an intriguing and useful concept and tool. My instinct tells me that this quiz may reveal more about auditors’ competency and judgment than most certifications do.

It is important to note that the notion of credibility, as explored by the Evaluation Questions, goes far beyond experience and qualifications. Corporate and professional credibility comes also in the form of visible commitment, demonstrated conduct, soundness of processes, levels of autonomy, strength of empowerment, and responses to risks, all of which are explored throughout the Evaluation Questions.

Applying these questions to supplier/social responsibility auditing, it means companies need to seriously consider factors more than subject matter expertise and cost of the auditors. Companies need to define auditor competency in terms of independence, judgment, field experience, statistical and analytical sophistication, and interpersonal and intercultural skills. Companies should also examine their auditors’ approach closely, asking specific questions about approach, methodology, and plans to identify and prepare for the types of issues that are likely to arise during the audit process.

Measurements

The Evaluation Questions are rooted in various prior guidance issued by the DOJ and other regulatory agencies and international organizations. The document, however, does bring a very significant new element: the demand for evidence of effectiveness in the form of measurements and data.   Evidence of results is, after all, a foundation to credibility. Not only do the Evaluation Questions ask about “information or metrics” the company collects and uses to help detect misconduct, but also “how has the company measured the effectiveness” of activities such as training and policy implementation. There are many “how” questions such as “How has the company assessed whether…policies and procedures have been effectively implemented?” or “How has the company evaluated the usefulness of …policies and procedures.” Companies that are able to answer these how questions in measurable metrics and data are regarded with far more credibility than those who answer with unsubstantiated adjectives.

Measurement and data are concepts that are expected to be second nature for auditors. What is important for companies is to make sure they work with their supplier/social responsibility auditors to define what to  measure and how. Whether you are auditing for manufacturing quality, environmental compliance, or safety, it is important that you sit down with your auditors to define what satisfaction looks like, and identify ways to measure it.

Accountability

Compliance programs cannot succeed with accountability. This is why the Evaluation Questions are focused on the accountability of both individual players and the company’s systems and processes. Accountability is about clearly defined roles and responsibilities, and visible consequences for words and actions. In line with this emphasis, the Evaluation Questions elevate the inquiry from the traditional “tone from the top” to “conduct at the top”  and ask about “concrete” and “specific” actions. There are questions about whether supervisors are held accountable for failures in oversight and how the companies train relationship managers on their responsibilities in managing third party risks. More importantly, there are questions about the accountability of the company: what happens when “compliance raise[s] concerns or objections”? “Were there prior opportunities to detect the misconduct in question, such as audit reports identifying relevant control failures…” In other words, when issues and risks are identified, how has the company been accountable in addressing and remediating them?

As both an in-house compliance officer and as the DOJ Compliance Counsel Expert, I have seen numerous instances where companies have failed to address audit-identified issues adequately. In the eyes of prosecutors, regulators, and other stakeholders such as investors, this failure speaks volumes about the company’s commitment to accountability and raises serious questions about the company’s operational competency. It reminds me of the TV commercial where the bank security guard tells customers, in the midst of a robbery, that his job is only to notify people when there is a robbery, not to do anything about it. That is why the Evaluation Questions include questions on how audit findings and remediation progress are reported to the management and the board, and how the management and board follow up on such reports. Finding the problem is not the goal: fixing it is.

Continuous Improvement   

Even the best compliance program would become an obsolete compliance program should it not continuously update itself. Everything from business and operational realities to company culture to regulatory and legal requirements changes constantly, and only a persistently self-critical program regularly seeking improvements can remain top of its game. The Evaluation Questions recognize the necessity for continuous improvement, not only in its questions in Section 9 on audit, testing, and updates, but also in how its focus on root cause analysis and risk assessments. Every instances of breach, whether it resulted in actual harm or not, is an opportunity for learning and improvement.

This same principle applies in the supplier/social responsibility auditing process. It is important for companies to ask how their auditors are keeping up with ongoing trends, regulations, audit practices/standards and realities, as well as themselves how they are learning from the audit findings in not just short-term remediation, but long-term improvements in how they manage suppliers.

———————-

Hui Chen may be contacted at www.HuiChenEthics.com

Auditor QuickQuiz Update

Our short auditor skills QuickQuiz has only been live for a few days and we have logged responses.  The number of respondents is smaller than anticipated but trends are appearing.

The Good:  Respondents understand follow through with sampling plans, are aware of the Fraud Triangle and know the role body language plays in interviews.

The Bad:  Most importantly, respondents have been unable to identify specific threats to auditor independence and they have demonstrated a lower-than-expected understanding evidence corroboration and hierarchy.  Other areas where knowledge improvements seem necessary are materiality determinations, awareness of basic audit terminology and the scope of a QA/QC review.

Keep those responses coming in, and thank you for taking a few minutes to complete it.

Is 2017 the Time to Think About 2021?

This may sound like a US presidential election campaign, but thankfully it’s not. The EU conflict minerals directive is now final and reporting is required beginning January 1, 2021 – three and a half years from now. If you aren’t familiar with the upcoming EU conflict minerals due diligence and reporting obligations, take a moment to read what we think is a good, plain-language overview.

We have had many conversations about what the EU Directive means for companies right now. Below is a table of the more common discussion points that may be helpful to those grappling with whether to begin program development and implementation now or wait until the deadline is closer.

Pros

Cons

Third party service provider costs are low at this time due to maturity of conflict minerals reporting in US Program implementation costs and effort not yet necessary
Limited incremental costs for EU companies already reporting conflict minerals information to US customers or the SEC Program implementation costs and effort not yet necessary; regulatory uncertainty exists until member states adopt their own supporting mandates.
Flexibility in reporting Reporting format unknown at this time; regulatory uncertainty exists until member states adopt their own supporting mandates.
Potential competitive advantage may be gained with customers and corporate reputation Market awareness may be low currently, so investment now may not show a return
Acquire experience and correct program errors and gaps in advance of legal deadline, reduce risk of fines, penalties, customer pressure and reputational damage Further options and third party information sources likely to develop and improve over time, leading to better reporting at legal deadline

 

Feel free to contact us with any questions.

New Advanced Auditor Training Program for HSE/CSR Auditors

Elm Sustainability Partners and Elm Consulting Group International have launched a new training module for senior-level and experienced health, safety, environmental and social auditors seeking to improve their auditing skills and get updates on timely topics related to non-financial auditing and technology.

It is also relevant to those buying HSE/CSR audit services who are looking to improve the quality of audits they receive.  After this course, buyers can identify specific areas of audit practice improvements to request of their providers.  Alternatively, these buyers may wish to require their external HSE/CSR auditor to complete this training themselves.

A partial list of what is covered includes detailed review and practicum concerning:

  • auditor independence standards and managing impairment threats
  • audit criteria requirements
  • audit and evidence limitations
  • evidence hierarchy, weighting and corroboration
  • fraud, forgery and tampering – including new concerns brought about by technology
  • interviewing skills including fraud examination and FBI techniques
  • discussions of US Department of Justice Criminal Division Evaluation of Compliance Program criteria (2017), the June 1, 2017 US Public Company Accounting Oversight Board (“PCAOB”) auditor reporting standard on Critical Audit Matters and EU Non-financial reporting rule
  • audit QA/QC considerations

Each participant will take a pre-test to establish a knowledge baseline and identify specific areas for improvements.  Exercises are administered throughout and a post-test will conclude the session demonstrating the advanced competencies gained.  HSE/CSR regulatory and other technical topics will not be covered as this is not a regulatory update session.

Elm Principals are BEAC Certified Professional Environmental/Health/Safety Auditors (CPEA), have served on the Board of Directors of The Auditing Roundtable (recently merged into the Institute of Internal Auditors (IIA)) and BEAC, and have trained thousands of internal and external HSE auditors over the past three decades.

Contact us to learn how you and your team can take advantage of this unique program.

New Social Auditor Certification in the Works

We have been vocal in our concerns and criticisms concerning social/CSR auditing.  And we have ourselves been criticized for that. Fair enough.

The Association for Professional Social Compliance Auditors (APSCA) has released for public comment its draft Code of Conduct and Auditor Competency Standards – available here.

We support APSCA and its work towards improving the entire “ecosystem” of CSR auditing.  Anyone with a dog in this hunt should click on the link above and submit comments.  APSCA is keen to obtain input from as wide a range of stakeholders as possible to help become as credible as possible.  Given the breath of subject matter that is being demanded of CSR auditors by buyers of their services, there is a great deal of overlap in APSCA’s draft into environmental health, safety, transportation and other technical areas.

Conflict Minerals is Dead! Long Live Conflict Minerals!

The deadline for filing the CY2016 SEC conflict minerals disclosure has now passed, although there are likely to be a few late filers. It is too early to glean anything from the filings and at least three analyses will be conducted, including the Development International study, which is the most comprehensive of them. We all anxiously await these reports.

The future of the SEC disclosure requirement is murky and there is a chance that this may be the last year of mandated filing in the US. Many clients and others are asking us questions about the future of conflict minerals, and what the past results have been. These are our thoughts.

Looking forward, we do not know what is in store for the SEC rule. There are many moving parts politically and publically. We will know what happens when it happens. I’d like to think there will be adequate advance notice to those impacted, but even that is not assured.

But the review mirror tells a story too. While aspects of the rule’s impact are hotly debated, one thing is indisputable – it resulted in much greater visibility into material sourcing and other companies deep in supply chains. This has allowed some companies to reduce business risk by optimizing their supply chains – concentrating spending power or diversifying their supply base to manage potential disruptions. Companies identified that, unbeknownst to them, entities sanctioned by the US Department of Treasury Office of Foreign Asset Control (OFAC) may have been present in their supply chains. Supplier audits/screening improved in many cases.  Appropriate auditor qualifications in light of global reliance on audit results has also become a major question in the scheme of things.

Of course, the rule brought human rights abuses in the DRC and other countries out of the shadows and into the light of the public. But has the population of the DRC benefitted? Experts continue to argue both sides of the question. Without taking sides, earlier this year we attempted to evaluate one major criticism of the SEC rule – that it directly resulted in hundreds of thousands, if not millions, of jobs lost in the 3TG mining sector. The question we posed ourselves was what impact did the 2008 – 2010 global economic recession have on artisanal and small miner (ASM) job losses which are currently attributed only to Dodd-Frank Section 1502? Did the timing of 1502 coincidentally occur at a time when mining jobs were already in decline because of pre-existing macroeconomic conditions?

Our intent was to rely on existing literature rather than creating original research as this was an unfunded effort on our own part. After a few months, we ran into two insurmountable obstacles:

  • The existing DRC-specific literature we found does not acknowledge or give any consideration to potential impacts of the 2008 – 2010 global economic recession. Yet analyses from The World Bank, the World Economic Forum (WEF) and the International Finance Corporation (IFC) demonstrate that global economic downturns play a major role in commodity prices and mining jobs worldwide, including ASM.
  • The DRC has a uniquely major informal economy which some literature indicated accounts for up to 80% of the country’s total economic activity annually. There is a significant gap in available information on DRC’s informal economy and what is available was sometimes inconsistent with other data on the same matter or irrelevant to our study.

We found only two sources referencing global 3TG price influence on prices paid to DRC ASMs.  Other data supported the position that a very large number of ASM miners in DRC move between multiple jobs based on income potential, so when ore prices were low in the past, miners moved to agriculture or other income sources. There was a meaningful amount of anecdotal information supporting the hypothesis that several factors other than Section 1502 (such as the DRC’s own taxation and mining policies) had a direct effect on DRC ASM job losses within the timeframe of interest, but we were not willing to rely on non-empirical information. We put down our pen (or mouse) and moved on to other things.

So the debate will continue.

There have been developments beyond just the SEC rule. The European Union adopted their own version of a conflict minerals due diligence rule that impacts a different class of companies and goes into effect in 2021. And the application of the OECD Due Diligence Framework is expanding into other materials (such as cobalt) and other geographies. At the moment, that appears to be just the beginning of that trend and that future is unknown as well.

In the end, what can be said about Section 1502 in consideration of it’s possible end? It all depends on your perspective, but it ain’t over till it’s over.  And it ain’t over.

How to Say “DRC Conflict Free” Without an IPSA

As the SEC conflict minerals filing deadline closes in, companies are carefully assessing what to say in their Form SDs and conflict minerals reports, especially in light of the recent statement from the Commission about enforcement of the filings.  Certainly, part of the internal deliberations concern how – or whether – to describe product determinations.  If a company voluntarily chooses to use the words “DRC Conflict Free” in its Conflict Minerals Report, then an Independent Private Sector Audit (IPSA) is required.

But did you know that the words “DRC Conflict Free” can be used without triggering an IPSA?

Without going into the painful explanatory details, issuers who file only a Form SD can use the specific determination wording in the Form SD without needing an IPSA.  As SEC stated in FAQ #19,

An issuer is only required to obtain an IPSA of its Conflict Minerals Report and not of the disclosures contained in the body of its Form SD.

The basic rationale is that when the RCOI results indicate there is no reason to believe that necessary conflict minerals did or may have originated from a covered country,  only a Form SD is required and additional due diligence is not necessary.  Therefore, a Form SD-only filing means that products are “DRC Conflict Free” by virtue of the absence of materials from a Covered Country.

But be careful – this only applies to Form SD language.  We also caution against claiming DRC Conflict Free in a Form SD that includes the CMR exhibit – but the CMR doesn’t mirror the Form SD.

We are happy to answer any questions you may have.  Feel free to give us a call.