Associate each criticism of a peer-reviewed risk analysis report noted on page 70 of the Norman article Download page 70 of the Norman article with the most appropriate intellectual standard that it violates. Explain why. Worksheet: Intellectual Standards Appraisal Worksheet

70 Risk Analysis and Security Countermeasure Selection - You want to be certain that you are addressing all of the points of view and arguments that will be raised by any opposition. - You want to acquire and address as much evidence as possible and as objectively as possible, regardless of which direction it may point. - You want to eliminate any false assumptions and take into consideration any other presuppositions and assumptions. - You want to deduce correctly what might happen and know what does happen. - You want to draw correct inferences from the evidence. ANALYSIS REQUIRES CRITICAL THINKING One simply cannot analyze any problem without thinking about it carefully. Yet the analysis of some ill-prepared security industry workers who position themselves as security consultants is sadly often lacking in evidence of thought. There is evidence of opinion, of scant observation, of references to past experiences, and conclusions reached that often mostly provide evidence of an agenda that is not fully in the best interests of the client. Observing, offering opinions, and focusing on only part of the problem are not analy sis. Yet this is too often what passes for analysis. I am often asked to review the analysis work of other consultants, contractors, and vendors, in the context of preparing for a project. I am frequently shocked at what passes for risk analysis. In one analysis that reportedly cost the client over $100,000, there was not a single reference given to support either the data or the conclusions. The report was filled entirely with references to "experience" and "we were told by (anonymous") persons of authority," with no validating reference. Upon reading the report, I counted only 22 points of analysis, while that particular case called for thousands. The report, typical of so many, lacked the following elements of critical thinking: - Threat assessment was apparently "off the shelf" and did not appear to have been refreshed for this project. - Focus was primarily on basic vulnerabilities, and those were limited to the obvious, without any evidence of searching for environmentally unique vulnerabilities. - Risk conclusions were drawn directly from the vulnerabilities, without reference to the type of threat actor. - The report was written entirely from an authoritarian single point of view. - There was little reference to research, no reference to interviews, no reference to source of facts (therefore making "facts" suspect), no reference to concepts, and one reference only to an applicable law. - There was no context for interpretations. - No options were offered. - No mitigation effect was offered for countermeasures recommended. - No budget was provided. - No evidence was provided of architectural or aesthetic considerations for countermeasures, nor for their effect on the efficiency of daily operations. - No consideration was made of the effect of recommended countermeasures on the quality of life effects for the customer and employee base. - The qualifications of the consultant were not identified. "The author's insertion