NICET staff started working with industry volunteers almost 18 months ago to update the Water-Based Systems Layout program. The main purpose was to update exam content to reference newer codes and standards for all levels. Dozens of knowledgeable volunteers participated to update the practice analysis, write test questions, review test questions, verify reference material and participate in score setting. New Level I, II and III exams have been available since October 1, 2018. Level IV exams are expected in early 2019.
The process also provided the perfect opportunity to address industry concerns with previously lower than expected passing rates for the Level III and IV exams. NICET is consulting with highly respected psychometrician Joy Matthews-López, PhD to perform analysis that compares the cut score study based on an ideal candidate (Angoff) with actual candidate data (Beuk).
“The Beuk Compromise (Beuk, 1984) method, as it is often known, makes a great partner to the Angoff (1971) method, and the two are the most commonly used approaches to set a sound, scientifically-based cutscore. The Angoff approach sets a cutscore based on how many questions a panel of experts thinks a competent candidate should answer correctly; the Beuk then checks to see how the Angoff cutscore performs on real examinees as a ‘reality check’ and then recommends an updated cutscore that takes both aspects into account.”
Joy Matthews-López, PhD
President and a Senior Psychometrician
JML Measurement & Testing Services
The actual candidate data is being collected during a pilot testing period between October 1 and December 31, 2018During the pilot testing period, candidates will not immediately receive official results and for data analysis purposes participants are only permitted to test one time during the pilot period After the pilot testing period, NICET will work with our psychometric consultant to review actual test data to establish the final passing score. Once the passing score is set, NICET will notify participants directly. Exam results attained during the pilot testing period may be adjusted after the data is reviewed.
“I would like to thank the many industry volunteers that worked tirelessly to update this program; and their continuing efforts to ensure NICET offers fair, valid, reliable and legally defensible certification tests. Level III in Water-Based Systems Layout is recognized as the national benchmark to distinguish those engineering technicians that are qualified from those that are not-yet qualified for this critical life-safety role. I am certain this extra score-setting step will enhance our efforts to certify competent sprinkler layout technicians.”
Michael A. Clark, CAE
NICET Chief Operating Executive
For questions regarding the pilot test, please contact Sara Clark, Manager of Certification Program Development at 1-888-476-4238 x119 or sclark@nicet.org.