In our previous two articles we looked at the skills and competencies that actuaries will need in the future.
As actuarial work evolves, the profession and employers will increasingly seek individuals who have both technical skills and ‘softer’ skills, such as creativity, collaborative working, critical thinking and a willingness to embrace change.
However, the onus won’t just be on individuals to continually develop such blended competencies throughout their careers; professional bodies such as the IFoA also need to respond to these changes, and show that the members they qualify are entering their profession with an appropriate combination of contemporary skills and competencies. This requires a different approach to qualification – one that assesses qualifiers against an explicit framework of competencies, thus enabling employers and other external stakeholders to immediately identify the competencies they can expect from those who hold these qualifications.
Assessing ‘higher-order’ skills
As work changes, so do professional qualifications, in terms of both their ‘knowledge content’ and the way qualifiers are assessed. Professional bodies need to show that the members they qualify have not only the ‘knowledge content’ of their qualifications, but also the competency to analyse this content and apply it to real-world business situations – so-called ‘higher-order’ professional skills, which are now seen as key indicators of competency for a wide range of professions. The IFoA is responding to such challenges through its Learning Change Programme, initially by reviewing its Associate qualification. In doing so, the IFoA, like all professional bodies, is guided by the principle that standards must be maintained in terms of both quality and rigour.
A common assumption is that ‘higher-order’ professional skills can only be assessed using traditional ‘constructed response’ or essay-style assessment. However, there is now evidence that other forms of assessment may not only be associated with greater objectivity and reliability, but also demonstrate better validity than some current customary measures of professional achievement. Objective-based assessment (OBA) forms the basis of such approaches.
OBA is a well-established method for assessing both competencies and technical knowledge, and provides an exacting and rigorous way to show that candidates have achieved the level of competency that a profession, employers or other external stakeholders expect from someone holding a particular qualification. “OBA is now widely recognised among a number of prestigious professional bodies as being a comprehensive, fair and challenging way to assess their qualifiers, while still maintaining the level of ‘academic rigour’ expected of our professional bodies,” says Sarah Hutchinson, independent chair of the IFoA Board of Examiners.
The IFoA is preparing its move into this assessment space, joining other professional bodies that have recognised the important role OBA plays in their qualifications, such as the UK’s Royal College of Surgeons, Royal College of General Practitioners, and Solicitors Regulation Authority.
“OBAs are widely recognised among professional bodies as a comprehensive, fair and challenging way to assess qualifiers”
What is OBA?
The purpose of OBA is to distinguish between those candidates who possess the necessary knowledge, skills and abilities to practise a particular profession, and those who do not – and to do so more objectively than more subjective ‘long question’ examinations can. OBA uses a variety of assessment types, such as data input, hotspots on graphs, assertion and reasoning questions, and extended true/false questions to assess a candidate’s core knowledge and their ability to analyse and apply it.
Contemporary OBA is usually computer-based and involves sophisticated multiple-answer responses. Many of us may recall this type of assessment from our school days, when they were known as ‘multiple-choice tests’. However, contemporary OBA offers much more than the multiple-choice tests of yesteryear. If there is any perception that OBA is incapable of assessing anything beyond recall or the recognition of knowledge, rest assured; as Professor Darina Scully from the Centre for Assessment Research in Dublin says: “Empirical studies show multiple choice items do indeed have the potential to assess higher-order skills.” Using OBA, qualifying bodies can develop sophisticated, challenging and nuanced assessments that can distinguish a candidate’s level and depth of understanding more reliably than more traditional assessment methods, without ‘watering down’ standards. It is also clear that the probability of passing such assessments through pure ‘guesswork’ alone is extremely small.
How will OBA be used?
The IFoA will be introducing OBA for relevant modules of its Associate qualification in coming years to further improve the objectivity and reliability of its assessments, and to increasingly assess not only candidates’ core knowledge, but also their ability to analyse and apply it. Not all subject areas lend themselves to this type of assessment, so a blend of assessment types will continue, following good assessment practice. OBA will also be underpinned by the principle that standards are maintained across all subjects in terms of their quality and rigour.
The current plan is to introduce OBA for the core Finance and Economic subjects within the Associate qualification in 2023, and then follow in stages with other more technical subjects that lend themselves to this form of assessment. The syllabi for the first assessments carried out in this form will be the same as those that are currently in place for the ‘traditional’ examinations, so no individual will be asked to study new material. The only change will be in the form of the assessment.
What’s next?
To help candidates prepare for this new type of examination in April 2023, sample questions for the Finance and Economics assessments will be available by June 2022. This will allow candidates who are planning to sit these subjects to see the form of these new question types. A full specimen examination paper for each subject will be released at least six months before the first sitting, with a current release date of October 2022.
Qualified members may also be interested in these sample assessments to assure themselves that, while the form of assessment is changing, the level of challenge is certainly not.
What does OBA look like and how does it assess ‘higher-order’ competencies?
Here’s an illustration from the field of medicine. Both questions draw on the same ‘knowledge’ base.
To answer this correctly, one must know that a shuffling gait is characteristic of Parkinson’s disease, and use this knowledge to understand that the presence of rugs poses a significant safety hazard to the client, and hence has the ‘greatest’ implication for their care.
The ‘distractor’ answers are plausible as they are relate to Parkinson’s disease and also have implications for the client’s care.
Based on Scully, Practical Assessment, Research, and Evaluation, vol. 22 (2017)
Try an OBA question for yourself.
Here’s one from the field of statistics:
The other articles in this series can be found at:
Part 1: Ringing the changes
Huw Morris explores how technology is changing the professions – and the skills that will be needed to embrace this new era
https://www.theactuary.com/2021/11/29/ringing-changes-skillset-requirements
Part 2: Softly does it
IFoA Curriculum: how professional work is evolving and non-technical 'soft' skills are increasing
https://www.theactuary.com/ifoa/news/2022/01/28/ifoa-curriculum-how-professional-work-evolving-and-non-technical-soft-skills
Image credit | Shutterstock
