Many readers will be familiar with ‘competency-based selection processes’. They will have been subjected to them during a previous application for a judicial role or a private sector position. For those who have not, the exercise broadly attempts to ensure a good match between a candidate’s skills and the skill requirements of the post.

Julian Stafford

Julian Stafford

The competency-based approach raises its head in three separate areas. First, it is used in the candidate’s self-assessment (which might be used either at the ‘sifting’ stage or on the selection day). Second, it is applied to referee responses. Third, it is used in the competency-based interview on selection day.

Self-assessment

Here is an example of a self-assessment by a candidate under the ‘Skill and ability competence’ which is quoted in Judicial Appointments Commission (JAC) guidance: ‘In my 12 years as a deputy district judge, I have heard a wide range of complex and difficult cases that require independence of mind and resilience. I approached each decision with an open mind and was often complimented on the clarity of my judgments. As well as my judicial experience, my time as a school governor and charity board chair means that I have chaired meetings and hearings with senior business leaders, politicians, government officials, and people from all parts of society. I always deal with people in an inclusive and respectful manner, even when they are rude and tempers are frayed I always act impartially. I am not afraid to speak my mind, especially where I see potential injustice. I stay calm when under pressure. Chairing brings a requirement to move people towards consensus and get people on board with decisions, which requires a combination of diplomacy, a firm hand and patience. I work well with other court staff and I am efficient. My most recent appraisal 2019 stated “He is personable and appears to get on well with colleagues. During my observation, he liaised effectively with the clerk and they were clearly comfortable working together. The day involved a heavy caseload but he managed each case effectively.” Working where I do, I have dealt with parties from a wide range of communities. I am careful to never discriminate and I make my decisions irrespective of the parties’ ethnicity, age, educational background, mental health and so on. I recognise that my decisions can have life-changing impacts and am sensitive and respectful to all.’

To an untrained eye, this person sounds like a decent, competent and experienced member of the judiciary. However, this person would attract a D grade (unappointable!) for this answer. The reason given is that it is full of assertions and lacks specific examples of what they did and how they did it.

The problem here is that another quoted answer which was deemed to deserve an A grade was also full of assertions: ‘Sitting as Deputy District Judge. Difficult defendant, very rude to usher at start. Refused to sit, answer me or accept court’s authority, claiming religious grounds for objection. I explained carefully the purpose of hearing, process and how he could assist. Asked questions and listened to his concerns, checked how he wanted to be addressed, explained why he may be more comfortable sitting down. Was able to ascertain he had some literacy issues and was very nervous. Checked his understanding throughout, read out key parts of documents for him. Established authority by being firm about his outbursts and reiterating what I was doing at each step and what was expected of him. Defendant began to engage in process and was able to present his case. Judgment went against him finally, but he told me that he accepted decision and apologised to the usher.’

Let’s remind ourselves what an assertion is: ‘Something declared or stated positively, often with no support or attempt at proof.’ Let’s also remind ourselves what proof means: ‘1. To establish the truth or validity of (something) by the presentation of argument or evidence. 2. To demonstrate the reality of (something). 3. To show (oneself) to be what is specified or to have a certain characteristic.’

As we can see, the only difference in the ‘good’ example is that the assertions are of factual matters rather than assertions of the person’s own ability. They are still assertions, and are untested and uncorroborated. Neither of them proves anything.

Quite apart from the fact that the evidential value of a self-assessment is non-existent, there are other problems:

1.    The statement or opinion may be incorrect;

2.    If not incorrect, it may be an incomplete or biased representation of the situation put forward as an example;

3.    The person may have misunderstood the dynamics of the situation on which they are commenting;

4.    Even if none of the above applies, it may not be representative of that person’s normal approach to situations such as they have described. Indeed, it may be the only occasion on which they have shown that particular skill or ability.

But not only is a self-assessment of limited use from an evidential perspective, it is also potentially unfair as it disadvantages candidates who do not know how to ‘play the game’. The first deputy district judge referred to above clearly did not know how to play the game although they somehow seem to have managed to hold down a responsible judicial post for many years.

It is very worrying in a judicial context that, on the JAC website, it states ‘self-assessment is a critical part of your application’. Let’s look at what ‘critical’ means: ‘of the greatest importance to the way things might happen’. More critical presumably than the other parts of the selection process aimed at establishing whether the candidate can actually do the job for which they are applying.

Finally, it is axiomatic that, if all candidates complete the self-assessment properly, it would be impossible to differentiate between them as the only difference would be the factual circumstances being reported. All the other things such as giving specific examples, structuring answers using the SOAR method (look it up if you’re interested), providing examples with depth or complexity, avoiding too many examples, keeping the answers free of jargon and technical terms, and avoiding assertions or generalities would be indistinguishable. Therefore, to the extent that the candidates can be differentiated, all this demonstrates is that the ‘better’ candidates have read, understood and acted on the rules of the game. While this does indicate a degree of thorough preparation, it would be difficult to extrapolate this to greater suitability for a judicial role or, at the other extreme, evidence of unappointability (a very serious charge to be levelled at anyone – not least an existing judicial office-holder as above).

Independent assessment (referee)

Competence-based factors even find their way into independent assessments. The guidance regarding referees says: ‘Then they are asked to comment on each competency or skill and ability, providing specific evidence where possible.’

So we have a further trap for the unwary. Not only will the candidate be disadvantaged if they are not up to the mark with how to play the game, they will also be disadvantaged if their referee is not up to speed either. Again the guidance states: ‘Remember it is not about who you know – it is about the quality of the assessment’ (my emphasis).

Back to evidential matters. The JAC guidance goes on: ‘We use independent assessments as a source of evidence to assess the merit of candidates.’ The same problems arise as set out above:

1.    The referee may be the only person the candidate can find who could speak positively about them;

2.    The candidate’s interaction with the referee may not be representative of their normal behaviour;

3.    The referee may be a friend of the candidate and therefore potentially biased; and

4.    Referees are likely to ‘accentuate the positive’ and downplay any perceived shortfalls in ability so as not to impair the candidate’s chances of selection.

Competency-based interview

By the time we get to selection day, the spectre of the competency-based approach has still not disappeared as candidates have to go through a competency-based interview. This is effectively a rehash of self-assessment using different examples. It is an improvement in that the panel can question the candidate on things they have said in a self-assessment or in response to questions, but well-prepared candidates will be able to deal with this quite easily. One thing the interview does require is preparation for questions on skills and abilities, but these questions are confined to digging out from memory in advance further examples of exemplary behaviour in historical situations. I was once asked ‘can you give us an example of when you showed independence of mind?’ For an existing judicial office-holder this is a rather insulting question. I doubt whether I answered it terribly well (unappointable).

Do competency-based approaches have any advantages?

The official line is that the process achieves openness and transparency and ensures objectivity in the interests of equality and diversity. The first point rather implies that there is no other or better way of achieving openness and transparency (whatever that means) and this must be open to doubt. Objectivity is clearly important but a rigorous assessment process testing actual abilities must surely be capable of achieving this too.

Self-assessment can be quite useful for encouraging employees to use their personal insights and introspectively establish which skills and behaviour they need to work on. So it might be useful to use it as an appraisal rather than as a recruitment tool. But for the reasons given above it is highly debatable whether it is useful as a judicial recruitment tool.

Other stated advantages are safety, consistency, lack of bias and clarity. It may also avoid awkward gaps at interview (at least for the well-prepared candidate) as they will be expecting the question. This is also the main disadvantage as it will be difficult to differentiate between well-prepared candidates. Also, past performance in a different role is not necessarily a good predictor of future performance in the new role.

Other stated advantages are that the approach lowers employee turnover by ensuring that employees’ skills are a good fit for the post but, given the rigours of the recruitment process, I would guess that dropout rates within the judiciary are very low.

Generally, the advantages seem greatest in organisations where people are recruited for specific roles, usually within a team of some sort and where they need to fit in. Conversely, the judiciary is a fairly broad church with a variety of backgrounds, experience, character and skills. Also, the skills required to do the job effectively are quite closely defined and can easily be tested by written tests and role-plays. In addition, judicial office-holders (at least the fee-paid ones) are not really employees at all (technically I believe we are ‘workers’ in employment law terms) and are quite distanced from the Ministry of Justice (in more ways than one). We are more in the mould of independent contractors with specific expertise.

Conclusion

I caveat this article with an acknowledgement that I have no history of recruitment or HR generally and am commenting solely on the basis of what I have personally experienced and read about the matter in the public domain. I do however have a reasonable grasp of what constitutes evidence as opposed to assertion.

It is unclear to what extent the competency-based elements of the process influence final selection. Given that someone could receive an ‘unappointable’ D grade for their self-assessment, one suspects this is a possible method of thinning out the candidate pool. My view is that this would be most unfortunate for a candidate who had otherwise objectively demonstrated their suitability. Given the numerous other ways that judicial ability and potential can be evaluated, it seems unfortunate that any reliance is placed on a process which is unverifiable, potentially biased, open to abuse, has a very low evidential value and which favours the well-prepared.

Maybe it is time to take a step back and redesign a process based on the actual abilities a typical judge might need. I would increase the importance of the role-play and make the interview a more searching examination of reactions to scenarios. The referees should merely be a book-end process to make sure the person was not wholly unsuited to the post. And, of course, resources could be reassigned by scrapping competence-based anything and using the time saved to develop more suitable assessment tools.

 

Julian Stafford, a law graduate, is a chartered accountant, accredited mediator and panel member of the First Tier Tribunal

Topics