Every year, the Work Psychology Group undertakes a detailed evaluation of the Situational Judgement Test (SJT) used for selection the Foundation Programme, the item development process and item-level analysis including reliability and validity, as well as analysis by applicant groups and recommendations for future improvements.
The report is divided into three main parts:
- The development process for SJT items
- The results and analysis of the operational SJT and initial analysis of the trial items
- Summary and recommendations going forward.
FP 2015 Technical Report
Around 8,000 applicants took the SJT for selection to FP 2015. A total of 140 items were piloted, alongside the ‘live’ 60 items in each test paper on which applicants are scored. The technical report of the FP 2015 SJT confirms the overall reliability of the SJT to be good (0.71).
FP 2014 Technical Report
Nearly 8,000 applicants took the SJT for selection to FP 2014. Around 180 items were developed/ amended from previous years, and 140 items were piloted, alongside the ‘live’ 60 items in each test paper on which applicants are scored. Overall the test papers had a reliability of α=0.69. Applicant scores were distributed across a bell shaped curve (normal distribution) with a slight negative skew. There is a positive correlation of r=0.3 with the EPM score, showing that the two measures are testing different things. In terms of test completion, 98.8% of applicants answered all questions and 99.4% attempted the last question.
FP 2013 Technical Report
The report also looks at the SJT in combination with the EPM, where a significant correlation was found between SJT scores and EPM decile scores, and between SJT scores and total EPM scores, and offers recommendations for improvements for FP 2014. The key findings of the technical report highlight that the FP 2013 SJT is valid, reliable and defensible.