The instances below are not the only criticisms that the PCAOB had for each of these firms with respect to their application of audit sampling; reading the PCAOB inspection reports yields further criticisms. Given that PCAOB inspection reports continue to identify these types of deficiencies, why do you think auditors have such difficulty satisfying the audit quality demands of the PCAOB? The PCAOB states that it uses a risk-based approach to selecting engagements to scrutinize; what does that imply about the generalizability of the criticisms to the firms’ overall audit client portfolios?

PCAOB Inspection Reports Regarding Audit Sampling: Continuing Deficiencies

This feature provides examples of audit sampling deficiencies noted by the PCAOB during its inspections of various audit firms.

Audit sampling is the application of an audit procedure to less than 100% of the items within an account balance or class of transactions for the purpose of evaluating some characteristic of the balance or class. Said another way, audit sampling allows the auditor to learn a lot about a population of items of interest without examining every one of those items individually. In 2008, the PCAOB issued its report on its 2004–2007 inspections of domestic annually inspected firms (PCAOB Release No. 2008–008). The report provides a summary of certain common issues related to sampling that were identified during those inspections:

The inspection teams identified deficiencies in firms’ performance of audit sampling, including

1. using sample sizes that were too small to obtain enough evidence to form a conclusion about the account balance or class of transactions being tested,

2. failing to appropriately project the effect of errors identified when testing the items selected to the entire population,

3. failing to select the sample in such a way that it could be expected to be representative of the underlying population, and

4. not appropriately testing all the items in the sample.

Since that time, the PCAOB has continued to criticize audit firms for weaknesses in sampling procedures. For example, in its 2015 inspection of Ernst & Young, the PCAOB stated (PCAOB Release No. 104-2016-142, p. 17):

The Firm selected key items to test whether the revenue recognition criteria were met. For the portion of revenue that was not covered by the Firm’s testing of key items, which was multiple times the Firm’s established level of materiality, the Firm failed to perform sufficient procedures. The Firm selected one contract from this portion of revenue to evaluate whether revenue was appropriately recognized and also tested revenue recognition for a sample of invoices. The Firm, however, had selected these invoices to support its accounts receivable cut-off testing; as a result, the sample was selected from only three months during the year and therefore was not representative of the revenue population.

Similarly, in its 2015 inspection of Deloitte (PCAOB Release No. 104-2016-141, p. 11), the PCAOB stated:

For a category of POC revenue that represented a significant portion of the issuer’s total revenue, the Firm selected for testing two controls that involved reviews of the estimated costs to complete and the status of each project. The Firm, however, failed to identify and test any controls over the accuracy and completeness of certain important data that the control owners used in the operation of these controls.

Also along these same lines, the PCAOB stated the following in its 2015 inspection of KPMG (PCAOB Release No. 104-2016-175, p. 6):

For the locations at which substantive procedures were performed related to revenue and inventory, the Firm designed its substantive procedures – including sample sizes – based on a level of control reliance that was not supported due to the unsupported reduction in the number of locations selected for testing that is discussed above. As a result, the sample sizes the Firm used to test revenue and inventory were too small to provide sufficient evidence.

Rounding out the Big 4, the PCAOB had this criticism of in its 2015 inspection of PwC (PCAOB Release No. 104-2016-140, p. 20):

For part of the sample that the Firm selected for dual-purpose testing, the Firm performed only the procedures that tested the control. For the remainder of the sample, it performed only substantive procedures. In addition, the Firm performed part of its testing as of an interim date and divided its sample between loans outstanding at the interim date and those outstanding at year end; the Firm, however, selected some of the same loans for testing at both dates. As a result, the Firm failed to perform both control and substantive testing for an appropriate sample of items based on the sample size for its dual-purpose test.