The Judges’ Competency Framework

The JCF: Is change the same as improvement?

{First published in Our Dogs, September 2017]

The Judges’ Competency Framework is possibly one of the biggest changes the KC has introduced to the show scene for some time. In the business world, it would be significant enough to be managed either as a ‘project’ or a ‘programme’. As I spend much of my day-job helping organisations to set-up and manage large-scale improvements, I thought it might be interesting to review the published JCF documentation to see how it stacks up in relation to established good practice. This is the sort of assessment I often carry out for my clients and it’s very straightforward to look at available documentation and form a view on the viability and likely success of a project.

Generally, programmes are established to manage strategically important changes that require multiple, contributory, projects running over a longer period of time. Projects tend to be more self-contained, shorter timescales and narrower in scope. The JCF would appear to align with two of the KC’s Strategic Objectives; the one focused on regulating canine activities and the one about providing opportunities for education and training. On that basis and the fact the full implementation timescale is four and half years, the JCF probably ought to be managed as a programme.

There is plenty of documented evidence of why programmes (and projects) fail, and these typically include:

  • Ineffective leadership
  • Inadequate user involvement in defining what they want and need
  • Over-ambitious expectations over developing and implementing new, enabling, IT

I use a simple checklist covering “Why, What, Who, When and How” to structure any review of what an organisation claims its programmes or projects will achieve and involve.

The “Why?” question:

Why is the JCF needed? The Press Release says there are “a range of deficiencies in the current process – problems for show societies identifying available and competent judges, open shows being poorly supported, and lack of seminar opportunities and transparency in the approval processes”. A Press Release is, of necessity, concise (although this one runs to 3 pages) but exhibitors would probably also include some of the following commonly discussed bugbears: closed-shop judging lists, CC-swapping, handler-owner-judge registration swapping, appointment-swapping, and judges looking after their mates when awarding CCs and higher awards. Breed Clubs might also include the opacity of the KC’s CC and A2 approval processes, and the fact that their views are, on occasion, overridden with no explanation.

Given those current issues, there’s certainly a need for change and the JCF aims to address some of them.

When it comes to objectives, I would expect to see 2 or 3 specific, measurable, achievable, relevant and timebound statements. That usually means a quantified explanation of what the programme will “increase, reduce or improve” and an associated date by when these will be seen. As far as I can see, in the press release the nearest to my definition of an objective is the Chairman’s comment that the JCF is “all about raising standards of judging”. The 24-page explanation of the JCF also quotes “improving the exhibitor experience”. There’s no mention of how much of an improvement is expected, or by when, or how it will be measured. Presumably, as with all well-managed programmes there is a set of objectives and Key Performance Indicators that can be made public. Otherwise, how will we know the JCF has made any difference? In the absence of any exhibitor satisfaction metrics, we will probably have to rely on show entry numbers as a proxy.

All change programmes set out to achieve a range of benefits as well as having specific objectives. Often, these benefits only become obvious well after the programme has ended but, nevertheless, it is essential that the expected benefits are clearly defined at the start. So, it may be many years after full implementation of the JCF on 1/1/22 before we are able to tell what has been achieved.

The expected benefits we can deduce from the JCF documentation include the following:

  • Increased clarity on the eligibility of individuals to judge particular breeds and classes
  • Increased clarity on what any individual needs to achieve in order to judge a breed or at a particular level
  • Elimination of “bottlenecks” that prevent suitably qualified judges from awarding CCs
  • Less need for prospective judges to travel around gaining experience judging dogs (cost and time savings)
  • It will be easier for people to learn how to judge
  • It will be easier for people to learn about each breed
  • Critiques will be better written
  • Breed assessments will be more objective, transparent and fairer
  • Candidates who fail assessments will be better informed about the reasons
  • There will be administrative cost savings (to the KC and Breed Clubs?) through the use of IT
  • All judges will be able to demonstrate their knowledge of Rules and Regulations (once every 5 years)

There may be others but we will only know if the above are being realised if there is some way to measure them.

It’s usual, in major programmes, also to identify disbenefits. There will inevitably be people (stakeholders) who find themselves disadvantaged as a result of the changes being imposed. A few that have emerged (on social media) include:

  • People will have to pay to be on a judging list, whereas previously there was no cost
  • Some judges simply can’t be bothered to sign-up to the new system and their experience and expertise will be lost, both as judges and potential mentors
  • People who are less confident in using IT may feel disenfranchised by the new system
  • In some breeds, a few people in the roles of BEC, mentor or observer, may have an inordinate amount of power over who will or will not judge
  • Breed specialists, especially long-standing ones, may be less likely to sign-up to the new system than all-rounders (as they judge less frequently) and we will see more all-rounder appointments, particularly at General Championship Shows

Of course, at the moment, we can’t tell the potential scale or impact of these disbenefits.

The “What?” question:

There are two elements that need to be defined for any change programme; its scope and the “deliverables”. Scope defines what the programme covers and what it doesn’t, while deliverables are the things that the programme will have to create in order for the benefits and objectives to be realised.

The scope of the JCF is clear; anyone who wants to judge at a KC licensed show in the UK, at whatever level, will have to become a member of the KC Academy. CC-awarding non-UK-resident judges will not have to join the Academy but a number of scenarios have been defined to explain how the JCF will apply to them.

The JCF programme will have to create numerous deliverables and they will have to be in place ready for the launch on 1/1/19. They include:

  • IT functionality to:
    • Record each judge’s experience and progress against the JCF levels (for one or more breeds)
    • Make records available to show societies as a replacement for Judging Lists
    • Allow for random scrutiny and review
  • New courses:
    • Eye for a dog
    • Critique writing
    • The online presentations of the remaining breeds that haven’t been filmed so far (there are currently 8)
  • Breed Education Coordinator (BEC) resources
    • Role description
    • Guidance documents including running Seminars, Mentoring days and Assessments
    • A BEC for each breed (200+?)
  • Mentor resources
    • Role description
    • Soft skills training
    • Paperwork and/or IT equivalent
    • Mentors for each breed
  • Observer resources
    • Paperwork and/or IT equivalent
    • Observers for each breed
  • Assessment resources
    • A dedicated member of KC staff
    • Multiple choice breed assessment examinations for every breed
  • A dedicated full-time support desk (and person/s, presumably!) at the KC
  • General JCF education and communication resources
    • FAQs
    • Roadshows/events

I’ve no doubt missed some, but it’s obvious that there is a big task to produce all these things, not to mention the Quality Assurance and Control processes that will be required to ensure they are all specified correctly and signed-off as fit for purpose.

The “Who?” question:

There are two main perspectives on the people associated with the success of a change programme; the people running it and everyone else (ubiquitously known, these days, as “stakeholders”).

Usually, you’d appoint a Steering Committee or Programme Board for a change initiative such as the JCF. They would be responsible for the investment decision and leading by example throughout the programme. There’s no mention of this in the material published, so we should perhaps assume that there will be the usual KC Committee structures including the Board, Judges Committee and Training Board involved. In all good programmes there is a single, named person who acts as Senior Responsible Owner and is ultimately accountable for the success of the programme. Do we assume this is Jeff Horswell who has probably been the most visible Board member in explaining the JCF?

Everyone else has, to some extent, an interest in the JCF and/or the power to help shape its success. There are many ways to categorise the JCF’s stakeholders:

  • People who don’t currently judge but who aspire to do so
  • People who currently judge and who aspire to judge more breeds
  • People who judge their own breed and have no aspirations to judge other breeds
  • Exhibitors who have no judging aspirations
  • The Breed Clubs who will have to run educational events
  • The new Breed Education Coordinators
  • People who will act as mentors
  • People who will act as observers and assessors

They will all have different views on the JCF and some people will inevitably end up wearing several hats. The framework has been announced pretty much as a fait accompli and, for any major change programme, that’s always a risky strategy. When those who have to implement a change and those who are going to be affected by it are not involved in its design, it’s inevitably going to be a harder sell. The KC now has the task of selling the JCF to lots of people who didn’t know they wanted, or needed, it.

The role of Breed Clubs will be crucial. They have most of the implementation work to do; running breed appreciation days (at least once every two years), mentoring days and managing “student judging” at their shows. They also have to appoint mentors, observers, a Breed Education Coordinator and sponsor supported entry classes at 2 general open shows every year. For the latter, there will be a cost, albeit typically a small one, to pay for rosettes.

14 breeds have been selected to pilot the JCF levels 2 and 3 and they are expected to run a breed appreciation day in the Autumn of 2017. We are told “This will be a proper pilot scheme and the feedback received from these breeds will be vital in our fine-tuning of the JCF before it is finally rolled out in 2019”.

The role of Breed Education Coordinator has parallels with Breed Health Coordinators. Experience tells us that, in some breeds, these people are highly effective and have the support of their breed clubs, while others struggle to get the support they need. The BEC role looks like it calls for someone with (a) great admin skills, (b) great people skills and (c) loads of time. That probably narrows it down to a very small list in most breeds.

When it comes to current judges’ support for the JCF, a recent poll on Peter Clifton’s Dog Show Exhibitors and Breeders Point of View Facebook Group showed 61% of the 142 respondents would not be signing-up to the KC Academy in order to be included on future judging lists under the JCF. Some, who voted “yes”, said they would do so under protest as they wanted to continue judging. Among the 61% who voted “no”, a number of breed specialists said they simply couldn’t be bothered. Peter estimated that there are nearly 15000 judges on lists currently. If 61% decide not to pay to be on a Judging List, we will be left with around 5500 judges. I don’t know if his figures are correct and we all know how unreliable polls can be, but I think this is a major risk to the JCF programme, particularly in the case of breed specialists who perhaps only judge one breed. It is easy to see how some people are describing the JCF as a charter for more all-rounder judges and the death-knell for the breed specialist. Quite what impact this will have on show entries is hard to tell but, last year, I did an analysis of Dachshund entries at Championship Shows over 10 years which showed that, on average, the more all-rounders with CC appointments, the lower the entries.

It is clear that there is more incentive for all-rounders and those aspiring to judge multiple breeds to pay to be on a Judging List than for breed specialists. They get access to online education for the other breeds they want to judge and, once they have awarded CCs in 30% of a Group’s breeds, they can apply to become a Level 5 (Group) Judge. Once they have awarded CCs in 75% of a Group’s breeds, they may be given approval to award CCs in the rest of that Group’s breeds with no further requirement for education, assessments or observations.

The “When?” question:

The key dates for implementing the JCF have been clearly defined over the next 7 years:

  • The pilot breeds will be running Appreciation Days from “Autumn” 2017
  • The framework will come online in late 2018
  • The JCF will start on 1/1/19 – nobody will be able to judge unless they are at Level 1 or above
  • From 1/1/19 all judges will have to pass the RDSJ exam every 5 years
  • From 1/1/19 all Breed Clubs will have to support classes at 2 general open shows p.a.
  • From 1/1/19 all breeds will have to run a Breed Appreciation Day at least once every 2 years
  • Breed Club lists will remain in place until 31/12/21
  • Full implementation will start on 1/1/22
  • From 1/1/22 only judges at Levels 4, 5 and 6 will be eligible to award CCs, judge Groups and judge BIS, respectively
  • Established judges have until 1/1/24 to pass the RDSJ exam

The “How?” question:

From a programme management perspective, the main points to address are budget, risks and assumptions.

We are told the KC is investing £0.5 million in this and the budget has been approved through to 2021, with the aim of covering costs. This has to pay for:

  • More breed films in the Academy (a figure of £4k per breed has been quoted, which could mean a cost of around £800k to create 200 films), paid for either by the KC or Breed Clubs
  • KC staff to provide the “help desk” to respond to, and check on, queries related to self-input data
  • KC staff to monitor the accuracy of inputs to the IT system
  • Other films and educational resources to be created
  • Training for Mentors
  • Communications and marketing collateral and events
  • The new IT functionality being developed over the next 18 months, required to support the JCF

On the income side, Peter Clifton’s estimate of 15,000 judges paying £26 p.a. would bring in nearly £2 million over 5 years. If his estimate of only 39% of judges pay the Judging Tax, the income would be around £760,000 over 5 years. Either way, most people would probably describe this as a large amount of income for the KC. However, if you deduct the costs, it doesn’t look so attractive.

I’ve mentioned a number of risks already but, to summarise, the major ones are:

  • Significant numbers of breed specialist judges decide not to pay to be on a Judging List and the choice of judges available is both depleted and biased towards all-rounders, resulting in further reductions in show entries, particularly at general championship shows
  • Breed Clubs fail to deliver enough mentors and observers to meet the demand (currently unquantified), causing bottlenecks in the system
  • There are insufficient suitably skilled volunteers with the necessary time available to take on the role of Breed Education Coordinator, which means education events aren’t organised, and mentoring and observations can’t be arranged

There are also some major assumptions underpinning the success of the JCF:

  • Mentors and observers will have the skills and courage to give all candidate judges true assessments of their knowledge and ability
  • Creating more competent judges will also create judges with more integrity
  • Those who “play the game” currently and get “promoted” by their mates, will no longer be able to do so
  • Creating more competent judges will increase show entries

We also have to look at what’s actually different in the JCF compared with the current system.

JCF component Impact on judging competence
Annual KC Academy subscription (to be on a judging list) None
Centralised judging lists None
Removal of the ‘number of dogs judged’ criteria Uncertain but possibly negligible, although in the case of breeds where CC entries can be up to 200 dogs (or more), this could have a negative impact
5 yearly RDSJ Examination Negligible
Online Breed Learning Resources Could have a positive impact
Online Eye for a Dog Course Could have a positive impact
Online critique-writing course Could have a positive impact
Breed Education Coordinator Negligible direct impact but a key enabler of many other elements of the JCF
Breed Appreciation Days Uncertain why these would make any more difference than current Breed Seminars run according to best practice
Mentoring Should have a positive impact
Student judging Could have a positive impact, based on FCI experience
Observed judging Uncertain why these would make any more difference than current A2/A3 ringside assessment observations
Assessments Uncertain why these would make any more difference than current A2/A3 Assessment events run according to best practice

And, of course, none of the JCF components address any exhibitors’ concerns about integrity or unethical behaviour. In reality though, this issue probably only applies in a small number of cases and we have to assume that the vast majority of judges are honest and want to do a good job.

What’s my verdict?

If I was carrying out this assessment for a client, based on the published information, I would be saying:

  • The objectives and benefits are inadequately defined and lack a means of measurement
  • The approach to implementation and change management could result in significant risks of resistance from key users – primarily breed specialist judges and breed clubs
  • The success of the mentoring and observation processes are highly dependent on the BEC role and any failure to fill the role adequately, as well as changes or turnover, will have a serious impact on the implementation of the JCF
  • As a result of the risks and uncertainties over sign-up rates to the JCF and the scale of deliverables that have to be created, the aim to cover costs with a £500k budget seems optimistic

Lessons learnt from the 14 pilot breeds will be essential and we do need to see some data on Academy sign-up rates as this will be a leading indicator of support for the JCF. Only time will tell whether the JCF is just about change or if it really is about improvement.

Advertisements

Comments are closed.

%d bloggers like this: