Robert J. Potter is entering his thirteenth year of involvement with ASNT at the national level.  He first became involved with ASNT in 1976 where he served as secretary of the Oklahoma Section.  He credits his involvement with ASNT to his employer who encouraged participation in professional organizations as a blueprint to professional growth.  Potter took this conviction to heart by involvement with the American Society for Quality where he earned certifications as a Certified Quality Engineer, Certified Quality Auditor and Certified Mechanical Inspector; and, with the American Welding Society where he earned a Certified Weld Inspector certification.  His commitment to both community and professional involvement has continued.  He serves on the NDT advisory board for Cowley County Community College and for the National Institute for Aviation Research – National Center for Aviation Training.  Potter was awarded the ASNT Fellow award in 2002 and was the recipient of the Engineering Service Award in 2010 by the Wichita Council of Engineering Societies.

As an ASNT Level III in eight (8) NDT methods, Potter has served on the ASNT Certification Management Council as Chair Elect and is past chair of the Eddy Current and Neutron Radiography committees.  He is also an active member of the ASNT Section Operations Council where he has served as Region 9 Director since 2000.   Potter is employed in the aerospace sector where he works for a company that specializes in nondestructive testing and metal finishing of aerospace parts.  He has 15 years of nuclear power experience and 20 years of aerospace experience.  Potter has a Bachelor of Science in Business Management, is an avid pilot, and enjoys doing research when it is too cold to hunt or fish.

Presentation Title: Methods of Determining Core Competencies of NDT Personnel

The presentation will center on the importance of developing effective certification examinations using psychometric principles.  The discussion will focus on the anatomy of an ASNT examination question with emphasis on question development, attribution and validation.  The foundation of this process serves as the basis of the ANSI 17024 accreditation that ASNT has earned as a certification agency.

In developing certification examination questions, a committee of subject-matter experts work with an experienced psychometrician (testing expert with a background in statistics) to determine the level of performance that corresponds to minimal competence of a specific inspection method.  The need for a new or revised examination questions is driven by statistical analysis that involves how effective the question is serving in determining competence.  This process considers many factors – how many examinees got the question correct, how many examinees that failed the examination got the question correct, how many examinees that passed the examination missed the question….  Examination questions must be attributed to a topical outline (body of knowledge) in an industry accepted document such as ANSI/ASNT CP-105.  The subject matter expert must also classify the question based on six criteria and skills.  The criteria include knowledge (major ideas, recall of information…), comprehension (understanding information, predict consequences…), application (use information, theories, concepts…), analysis (seeing patterns, organization of parts, recognition of hidden meanings…), synthesis (use old ideas to create new ones, relate knowledge from several areas…), and evaluation (compare and discriminate between ideas, make choices based on reasoning…).  An estimate is made as to how many qualified examinees will answer the question correctly.   Once developed, the question is reviewed by other method experts (Level IIIs with certification in the NDT method).  They review the question for sentence structure, clarity, body of knowledge attribution, classification, difficulty level and the correct answer.  If there are different difficulty levels based on the pier review, an average is taken and assigned to the question.  This process is later validated by statistical analysis of which will be discussed.