Tag Archives: GAGAS

Results from the Auditor QuickQuiz

Our auditor quiz is now closed after a month. The questions were based on existing international non-financial auditing standards, Association of Certified Fraud Examiners (ACFE) fraud identification/examination techniques and US Government Auditing Standards for non-financial audits. There were fewer respondents than we had hoped so we can’t extrapolate beyond our dataset. Even so, some notable trends did emerge.

Of those who responded, 47% were EHS auditors and 27% were CSR auditors. We had hoped more CSR auditors would have participated. Other information about the respondents’ backgrounds:

  • 60% had no certification or “other”
  • 50% have 10 years or less auditing experience
  • 50% have 50 or fewer audits
  • 13% have participated in more than 500 audits during their career
  • 63% spend at least 75% of their time conducting audits

There were only 2 “passing” scores – i.e., greater than 70%. The average score was 49% – far lower than was expected.

Knowledge of standard terminology seems to be lacking, further reflected in poor scores for questions that embedded the terminology within them. For instance, only 30% correctly defined “audit criteria” as meaning the audit protocol. This likely led to 53% of respondents incorrectly answering that QA/QC reviews should include assessing the correctness of the “audit criteria used by the auditor.” QA/QC reviews of auditor working papers should look at how an auditor applied the audit criteria, not the inherent accuracy of the criteria (or audit protocol) used by the auditor. Indeed, only 10% correctly identified that none of the answer options are appropriate for QA/QC reviews.

Only 3% considered interviews better than document reviews when asked directly what type of evidence is strongest. Yet when the question was placed in a practical setting, 73% indicated they would rely on interviews over documentation. Only 26% correctly identified the evidence hierarchy (from strongest to weakest).

On a more positive note, 83% answered that they would decline to develop a document that they audited, meaning 17% did not view this as a conflict of interest. Frankly, we were disappointed that there was not a perfect score in identifying this to be an independence issue.

In answering the question listing possible common evidence problems, just over half (53%) correctly indicated that all of the answer options are common evidence problems.

Finally, 2/3 incorrectly answered that initial determinations of significance/materiality should be made after assessing evidence. It is possible that respondents did not read the question carefully and pick up the word initial.

Certainly more responses would have provided a better representation, but we think there are some valuable take aways from our limited data.  Among them – the gap between EHS/CSR auditor knowledge and existing (and theoretically similar) non-financial audit standards may be larger than previously thought.  As the importance – and liabilities – of sustainability/CSR audits grow, increased auditor training and competence seems warranted.

Ice Cream Parties and Materiality in Conflict Minerals Reporting

There is an important distinction issuers should be aware of between materiality in reporting and significance in the GAGAS Performance Audit standards as those relate to the Independent Private Sector Audit (IPSA) of conflict minerals reports (CMRs).  And an ice cream party will help explain.

Materiality is a critical, but loosely defined, principle in accounting and financial reporting.  It is a threshold relative to the size and particular circumstances of individual companies, above which information is expected to be important to decisions of the users of the information.  Typically, materiality has a financial basis, “users” are considered “reasonable investors” (itself a loosely defined term) and “the information” refers to the total mix of information made available in SEC reporting.  The term “significance” is used in the GAGAS Performance Audit standards and has a similar meaning.

However, in the context of the conflict minerals IPSA objectives, significance relates to the presence, absence and/or relevance of audit evidence corresponding to the specified audit objectives. In their final release of the conflict minerals disclosure requirements, the SEC addressed potential IPSA objectives that were considered but not adopted[1].  Had they been adopted, significance in an IPSA would be more aligned with the general materiality definition for accounting/reporting.  Some of these rejected objectives include whether the

  • descriptions of procedures and controls performed are fairly described in the report;
  • descriptions of the due diligence process are accurate and the results fairly stated;
  • due diligence process conformed to a due diligence standard or was effective;
  • conclusions were accurate.

This distinction between reporting principles and auditing standards is quite important.  Determining significance for purposes of the second IPSA audit objective revolves around whether sufficient and appropriate evidence is available to support the description of due diligence measures undertaken as those are described in the Conflict Minerals Report; it does not involve determining whether the description of those measures diverges from the conflict minerals disclosure requirement or the conclusions made by the issuer.

If an issuer’s CMR  description of due diligence measures performed contains significant omissions or other inadequacies, but the issuer provides the auditor with sufficient and appropriate evidence that the measures described were undertaken, the auditor should not have an adverse opinion or conclusion[2]. Therefore, materiality in the SEC reporting context is not the same as significance in the IPSA context.

We have said in the past – should an issuer’s description of due diligence measures undertaken state that they threw an ice cream party for their sourcing staff, if the issuer provides sufficient and appropriate evidence for the auditor to confirm that the ice cream party was indeed provided for the sourcing staff, then the auditor should not have an adverse IPSA opinion or conclusion for the second objective.  Obviously there is an expectation that an issuer won’t really omit material information from their due diligence description or talk about ice cream parties.

Issuers who are trying to balance their concern about materiality in their SEC filing versus creating an efficient and low cost IPSA could consider this approach:

  • Become deeply familiar with various interpretations of “due diligence” and select an interpretation that meets internal requirements;
  • Carefully consider what aspects of that interpretation are material to the letter and spirit of the law and disclosure requirements.  You may want to involve senior management, the Board’s Audit Committee, Internal Audit staff and/or counsel in this process; then
  • Limit the description of due diligence measures undertaken to those material aspects of due diligence.

For example, if an issuer interprets “due diligence” to be equivalent to OECD Steps 3 and 4, they should assess the most important (i.e., material) aspects of those two steps.  They could decide that relying on CFSI smelter/refiner audits and conducting additional research into countries of origin are the most important aspects of Step 4.  For Step 3, perhaps they view the foundation of risk management is making internal decisions about how to handle the business relationship with high-risk suppliers, and keeping management apprised.  The issuer may then keep their CMR descriptions of measures undertaken to only these material aspects of Steps 3 and 4.  In our opinion (and others whose views are important in this matter), this should satisfy the SEC reporting materiality threshold while concurrently managing the IPSA effort.

Without worrying about the calories and fat of the ice cream.


[1] Conflict Minerals Rule Final Release, Pages 284 – 286.

[2] See also SEC FAQ Numbers 17 & 21 http://www.sec.gov/divisions/corpfin/guidance/conflictminerals-faq.htm – q13

Our Audit Quality Practices and GAGAS Peer Review Update

A Big 4 audit firm recently asked to see our external peer review report under GAGAS (“Yellow Book”) Chapter 3, Quality Control and Assurance. We decided to take this as an opportunity to publicly report on the status of our audit quality assurance practices.

External Peer Review and Report
One of the requirements for audit organizations being qualified to conduct GAGAS Performance Audits is to obtain an external peer review at least once every three years to provide a “reasonable basis” for determining whether the firm’s quality control systems are “suitably designed” and whether the firm is complying with its systems. Under GAGAS Section 3.97, “The first peer review for an audit organization not already subject to a peer review requirement covers a review period ending no later than 3 years from the date an audit organization begins its first audit in accordance with GAGAS”.

We have not previously been subject to peer review under GAGAS; therefore our first peer review is due no later than December 12, 2017 (December 12, 2014 is the date we began our first IPSA).  More than two years ago, we reached out to a handful of well known Yellow Book experts to identify an appropriate firm to conduct a review of a firm of our size and circumstances.

Our peer review is now underway and should be completed before the end of 1Q18.

Relevant Audit Experience, Expertise and Technical Knowledge
Elm Sustainability Principals have:

  • HSE and management systems auditing experience and expertise extending well over 25 years each
  • Each completed more than 500 audit engagements in all team capacities
  • Conducted numerous HSE auditor training sessions for both public and client-specific audiences, with hundreds of trainees
  • Held professional HSE auditor certifications (BEAC) since the inception of that certification program in 1997
  • Served, or currently serve, as President of The Auditing Roundtable and Chair of the Board of Environmental Auditor Certifications (BEAC) over the past 10 years
  • Played a leadership role in the recent acquisition of The Auditing Roundtable and BEAC by the Institute of Internal Auditors (IIA) and the on-going integration activities
  • Conducted the three initial tantalum smelter audits under EICC’s Conflict Free Smelter (CFS) program in 2010
  • Served in advisory roles in numerous conflict minerals studies conducted by third parties
  • Participated in and/or led numerous expert panels on auditing practices, including the US Securities and Exchange Commission’s Conflict Minerals Roundtable in 2011
  • Led the development of The Auditing Roundtable’s IPSA Performance Audit Guidance for non-CPAs

Audit Quality Practices
Quality control procedures we formally implemented in advance of our first IPSA include audit team member qualification requirements and IPSA-specific training, audit documentation requirements, audit documentation/report reviews. These remain in place for future IPSAs. GAGAS Section 3.76 requires 24 hours of continuing education (CE) every two years for auditors and 80 hours of CE every two years for audit leaders. Our existing professional certifications require 40 hours of CE every two years, the management of which will become more formalized as The Auditing Roundtable and BEAC become fully integrated into the IIA. We are in the process of increasing our CE hours for team leaders.

Auditor Independence
Another critical component of our practice is ensuring auditor independence.  We have company-wide structural controls and an auditor-level control to prevent impairment.  At a company level, we recuse ourselves from conducting IPSAs for clients to whom we act as an advisor or consultant related to the IPSA subject matter.  In our view, it is easier to draw this bright line between audit clients and advisory clients than it is to create safeguards for preventing impairments where both services are provided to the same client.  We also endeavor to identify situations where companies may be competitors, then (with approval) disclosing our existing and/or potential business relationship to each in order to vet competitive conflicts of interest.  We have not employed any former employees of conflict minerals clients that could exert inappropriate pressure or interference in an IPSA audit team.  Finally, because our conflict minerals client base is wide and diverse, we are not dependent on income from any single client – whether an IPSA client or consulting/advisory client.  At the individual auditor level, we employ an extensive written auditor independence checklist that requires review and signoff by another Elm Principal.

As always, feel free to contact us with any questions or comments.

This DOESN’T Impact Your Conflict Minerals IPSA

There is more interest in the Independent Private Sector Audit (IPSA) than ever before, and companies should be aware of important considerations in planning their Conflict Minerals Report (CMR) and preparing for IPSAs.  Unfortunately, misinformation continues to be spread in the market which can cause issuers to spend more than is necessary for their IPSA.

A consulting firm just announced it “received an independent auditor’s opinion that [the consultant’s] RCOI and due diligence process design and measures performed are in compliance with Step 2 of the OECD Due Diligence Guidance.”  The firm explained that “[b]y undergoing an audit of the design and performance of our RCOI and due diligence processes, we simplify the auditing requirements of our clients” because, they continued, “it did not make sense for each of our clients to be audited individually against the same process over and over again.”

This marketing ploy might sound good, but the practical impact is minimal – if anything at all – for companies needing an IPSA.

To begin with, the audit described is specifically limited to OECD Step 2 – the equivalent of the SEC Reasonable Country of Origin (RCOI) process. The SEC staff’s answer to Question 18 of their April 7, 2014 FAQs plainly states that RCOI processes, along with associated procedures under a nationally or internationally recognized due diligence framework (i.e., Step 2 of the 5 Step OECD Due Diligence Guidance) are not within the IPSA scope. So the subject of the consulting firm’s audit is not part of the IPSA audit process. For OECD Step 2 to be relevant in the IPSA, an issuer would have to unnecessarily expand the discussion of due diligence measures undertaken to include RCOI activities.

Second, while GAGAS allows auditors to rely on work done by others, there are conditions associated with this reliance. For Performance Audits, these conditions are covered in Sections 6.40 – 6.44.  To sum those up, an IPSA auditor

… should obtain evidence concerning the other auditors’ qualifications and independence and should determine whether the scope, quality, and timing of the audit work performed by the other auditors is adequate for reliance …

Given these conditions, an IPSA auditor may have concerns including:

  • As explained above, the scope of the OECD Step 2 audit is not directly relevant to IPSAs of issuers who thoughtfully craft their CMR to exclude RCOI activities and results from their due diligence description.
  • The OECD Step 2 audit was conducted by a non-US CPA firm whose experience in conflict minerals, the OECD Framework and/or the United States GAGAS audit standards is unclear.
  • There is no indication of the audit criteria or process used, evidence considered in the audit, audit risks and limitations or the independence of the CPA firm.
  • With IPSAs being a new concept, scope and use, practitioners are not likely ready to rely on the work of others. We ourselves are not comfortable with doing so without conducting a significant amount of verification work that could offset the anticipated value of the other audit.

Elm has recommended that companies consider a formal evaluation of their RCOI processes, especially for Form SD-only filers. But we did not intend or anticipate that such a review be a formal audit, nor have any specific use in an IPSA context. Because it doesn’t.

Issuers should be aware of the influence they have on the IPSA effort/cost by virtue of the CMR language they develop. When this is recognized, an issuer can craft a CMR that sets the foundation for an efficient IPSA. If you are not sure about the auditability of your CMR, you can engage an experienced consultant/auditor to conduct a readiness assessment, or train your internal audit staff to do the review and provide on-going support.  Feel free to contact us about either service if you are interested.