Case Study - The Examination In 2010, the SEC performed a routine examination on ACME Investment Management, Inc. It was clear from the document requests that quantitative investment procedures were among this year's examination priorities. The Investment department provided a series of documents showing how their algorithms were reviewed annually by the Chief Risk Officer (CRO) of the firm. On reviewing these documents, the examiners were skeptical as to the depth of this review. The documents only laid out a summary of the algorithms in questions and a risk assessment. For this reason, the examiners scheduled the CRO as the first individual to the interviewed on this topic. During that interview, the CRO confirmed the examiners' view stating that she was asked only to opine on the overall specification of the algorithm and the reasonableness of the risk assessment - and those qualifications were clearly stated in her certification memo which was contained among the materials presented by the Investment Department. Asked if she was satisfied with this process, she replied that the fundamental constraints on her review were that she was not allowed access to source code, test data sets and test evidence and so only the most cursory review for reasonableness was possible. One examiner on a hunch asked for how long had the CRO been performing this review. "This was the first year." she stated. "When was the review performed?" was the follow-up question. Clearly, the examiner wanted to determine if it had been a quick and dirty review for purposes of this examination. The CRO explained, These reviews was done three months ago as you can see from the dates on the files I created to document the reviews and the date of my final certification. We were only aware of this examination a month ago." Not wanting to let this line of inquiry end there, the examiner asked, "Was there any particular event or other trigger that caused ACME to institute this review process." Without hesitation, the CRO replied, l really cannot answer that as I would only be speculating. I was asked to perform these reviews by the Chief Administrative Officer as an independent high level overview. You would have to follow up with him and the folks in the Investment Department." The examiners thanked the CRO for her help and candor and terminated the interview. Before the next interview, another examiner suggested to the chief examiner that they request that ACME's Chief Compliance Officer and one of the firms senior lawyers, who were the permanent liaison persons present all interviews, leave the room for 10 minutes. This examiner, who was preparing to review the internal audit department's work, told his colleagues that four months earlier, the IA team had concluded an audit on the quantitative investment department's procedures and had rated them as "Needing Improvement." (The three possible outcomes were Adequate, Needing Improvement" and "Inadequate.] Based on that information, the examiners decided to postpone interviewing the investment team on this topic and requested all the working papers on the Internal Audit review. In the course of that review, it was determined that Internal Audit had issued a preliminary finding to the quantitative team of "Inadequate" some six weeks before their final report. The examiners' review of IA's working papers seemed to indicate that this was not an unreasonable finding. The Examiners next interviewed the head of the Investment Department and her subordinate, the manager of the quantitative investment team, and the head of the Internal Audit Department. The examiners reviewed their findings to date and outlined their concerns. Both ACME managers defended the process stating that they believed that an "Inadequate rating was too harsh. The head of IA agreed that there was a vigorous internal debate and that in the end they came around to the views of the head of the Investment Department. What arguments did Internal Audit find most compelling in this regard. The head of Investments interrupted to interject that as there had been no errors detected in the meticulous review that followed the preliminary finding. The head of Internal Audit was clearly unhappy with this interruption, but she continued her answer stating that there had been a very thorough examination of all the team's work and no material faults or errors were uncovered. The principal issues upon which their findings were ultimately based were a lack of documentation and a lack of comprehensive evidence of testing and review. The chief examiner ended the interview and announced that they would request all emails and internal correspondence related to the topic between the issuance of the preliminary finding and the final report of the Internal Audit department. At this, the head of the quantitative investment team asserted that a number of those emails were protected by Attorney-Client privilege. The senior counsel for ACME present in the room stated immediately that the firm would waive privilege. 1. Who were the principles? 2. Who were the stakeholders? 3. Comment on the statement of the Head of the Investment Team about the lack of errors in the record - and the head of the Internal Audit department's reaction to this assertion. 4. What do you believe is the most pressing concern of the examiners at this point? What might they be looking for in the relevant emails? 5. Assume they find what they suspect they might find, what would be the likely result? 6. Assume that they do not find what they suspect, what would be the likely result? Case Study - The Examination In 2010, the SEC performed a routine examination on ACME Investment Management, Inc. It was clear from the document requests that quantitative investment procedures were among this year's examination priorities. The Investment department provided a series of documents showing how their algorithms were reviewed annually by the Chief Risk Officer (CRO) of the firm. On reviewing these documents, the examiners were skeptical as to the depth of this review. The documents only laid out a summary of the algorithms in questions and a risk assessment. For this reason, the examiners scheduled the CRO as the first individual to the interviewed on this topic. During that interview, the CRO confirmed the examiners' view stating that she was asked only to opine on the overall specification of the algorithm and the reasonableness of the risk assessment - and those qualifications were clearly stated in her certification memo which was contained among the materials presented by the Investment Department. Asked if she was satisfied with this process, she replied that the fundamental constraints on her review were that she was not allowed access to source code, test data sets and test evidence and so only the most cursory review for reasonableness was possible. One examiner on a hunch asked for how long had the CRO been performing this review. "This was the first year." she stated. "When was the review performed?" was the follow-up question. Clearly, the examiner wanted to determine if it had been a quick and dirty review for purposes of this examination. The CRO explained, These reviews was done three months ago as you can see from the dates on the files I created to document the reviews and the date of my final certification. We were only aware of this examination a month ago." Not wanting to let this line of inquiry end there, the examiner asked, "Was there any particular event or other trigger that caused ACME to institute this review process." Without hesitation, the CRO replied, l really cannot answer that as I would only be speculating. I was asked to perform these reviews by the Chief Administrative Officer as an independent high level overview. You would have to follow up with him and the folks in the Investment Department." The examiners thanked the CRO for her help and candor and terminated the interview. Before the next interview, another examiner suggested to the chief examiner that they request that ACME's Chief Compliance Officer and one of the firms senior lawyers, who were the permanent liaison persons present all interviews, leave the room for 10 minutes. This examiner, who was preparing to review the internal audit department's work, told his colleagues that four months earlier, the IA team had concluded an audit on the quantitative investment department's procedures and had rated them as "Needing Improvement." (The three possible outcomes were Adequate, Needing Improvement" and "Inadequate.] Based on that information, the examiners decided to postpone interviewing the investment team on this topic and requested all the working papers on the Internal Audit review. In the course of that review, it was determined that Internal Audit had issued a preliminary finding to the quantitative team of "Inadequate" some six weeks before their final report. The examiners' review of IA's working papers seemed to indicate that this was not an unreasonable finding. The Examiners next interviewed the head of the Investment Department and her subordinate, the manager of the quantitative investment team, and the head of the Internal Audit Department. The examiners reviewed their findings to date and outlined their concerns. Both ACME managers defended the process stating that they believed that an "Inadequate rating was too harsh. The head of IA agreed that there was a vigorous internal debate and that in the end they came around to the views of the head of the Investment Department. What arguments did Internal Audit find most compelling in this regard. The head of Investments interrupted to interject that as there had been no errors detected in the meticulous review that followed the preliminary finding. The head of Internal Audit was clearly unhappy with this interruption, but she continued her answer stating that there had been a very thorough examination of all the team's work and no material faults or errors were uncovered. The principal issues upon which their findings were ultimately based were a lack of documentation and a lack of comprehensive evidence of testing and review. The chief examiner ended the interview and announced that they would request all emails and internal correspondence related to the topic between the issuance of the preliminary finding and the final report of the Internal Audit department. At this, the head of the quantitative investment team asserted that a number of those emails were protected by Attorney-Client privilege. The senior counsel for ACME present in the room stated immediately that the firm would waive privilege. 1. Who were the principles? 2. Who were the stakeholders? 3. Comment on the statement of the Head of the Investment Team about the lack of errors in the record - and the head of the Internal Audit department's reaction to this assertion. 4. What do you believe is the most pressing concern of the examiners at this point? What might they be looking for in the relevant emails? 5. Assume they find what they suspect they might find, what would be the likely result? 6. Assume that they do not find what they suspect, what would be the likely result