Hello judges! I’m Alfonso Bueno from Spain. I’m the L4 Testing Manager for the International Judge Program (IJP) and I would like to share with you my thoughts and reasoning on the different aspects of the Level 4 (L4) definition.
To the surprise of no one, I started by taking Judge Foundry’s definition and I made the adaptation I feel was necessary for an international environment and to match the IJP needs and values.
Description
A Level Four (L4) Judge is an expert in tournament policy with experience head-judging medium-sized events (5+judges & 129+ players) and small teams of judges, as well as an active member of the judge community. L4s are also comfortable leading a large team, like a 5-judge team at a Regional Championship. Not only are L4s comfortable with all of the tasks expected of a floor judge at a large event, but they are experts with at least one team and can lead any team while training L2s and L3s on how to do those tasks. L4s are strong communicators and mentors. They train L2s to become L3s and mentor L3s to become L4s. In their local community, an L4 identifies judges who could be advancing, works one-on-one with judges, and encourages mentorship and collaboration within the entire community.
Here we find the definition of what is expected from an L4 judge, which is the base used to define all certification criteria. The elements I find most relevant here are that L4s:
- Have demonstrated the capacity to HJ medium sized competitive events.
- Have demonstrated the capacity to lead teams at large events.
- Are mentors and participate in the certification of other judges.
Advancement
To be promoted to L4, a candidate must complete the following requirements. Unlike the processes for Levels 1-3, these requirements occur in a prescribed order. All advancement processes will be held in English. Accessibility exceptions can be made by the Testing Managers, in coordination with the Diversity, Equity, and Inclusion team.
L4 judges must be able to lead at large competitive events and therefore must be able to operate in an international event. While we are not testing anyone’s English proficiency, the fact a judge is able to collect all the experience and success at this certification process in English is enough to demonstrate they can operate in English well enough.
We’re aware that any testing process will not be identical to the actual judging on the floor, what we ideally should be testing is the performance of that judge in a real event. In order to compensate for people with conditions that make this artificial (but necessary) testing system harder to undertake, we’re appointing a group of experts that will help us understand and adapt to those circumstances.
Step 1: Experience as an L3
The requirements of this step may be completed in any order.
- Work Events as an L3 Judge (no time constraints)
- Work at least six multi-day events
- Lead a team that is assigned each of these tasks at least once
- Pairings
- End of Round
- Deck Checks
- Any Public Events Team (scheduled/ODE/Kickstarter)
- Lead a team with 3 or more judges reporting to them at least 3 times
- Serve as head judge for a headlining event at a multi-day event
- Work at a competitive event with at least 20 other judges
Here we’re looking to ensure a decent amount of experience in the candidate before advancing them to the next steps of the process. We have a limited capacity of human resources and we want to make sure we make good use of the time donated by our volunteers.
All those events in the different bullet points can be the same, technically a candidate could fulfill all these requirements within just the 6 multi day events for bullet 1 (if they are assigned to lead the 4 mentioned teams, each of those teams have 3+ members; they’re assigned to HJ a headline event one of the days; and are assigned to the main event on day one), but this is very unlikely to happen.
At multi day events Team Leading is mostly reserved to judges who are already L4+, therefore that experience leading those tasks is usually acquired at smaller local competitive events. Since sometimes the experience required in bullet 2 is earned at events where the teams are really small, we have bullet 3, to ensure experience in team building, mentoring and taking care of judges.
Bullet 4 refers to any of the big Side Events at a multi-day event, it doesn’t need to be competitive.
Finally, bullet 5 refers to day one of the Main Event in a multi-day event.
It’s important to highlight that this experience doesn’t need to be earned in any specific period of time (unless when using the automatic promotion “path 1” of the Transition Period).
- Write at least three reviews in the previous year, as an L3 (recommendation and advancement reviews are valid)
- Must be from events that the candidate and the other judge both worked
- Must include at least one review of an L4+ judge
- Must include at least one review from a multi-day event
- Each review must contain detailed and actionable feedback.
Reviews are a great tool to help others improve, help them track their evolution over the years; writing them is also a way to improve ourselves, as it requires observations, analysis and presenting the results in a constructive way; and, finally, they can be verified and evaluated in the advancement processes.
We’re aware that reviews are not the only, or even, the best way to give valuable feedback, and we’re working on finding ways to make other kinds of feedback count here, if you have ideas, please reach out.
It’s important to highlight that level up recommendations (not endorsements) and advancements interviews that contain detailed and actionable feedback are as valid as other reviews.
Step 2: Self-Review
The candidate must submit a self-review indicating their preparedness for L4 according to the following requirements. The review will be assessed by an approved L4 panel lead, who may advance them to the L4 panel process, or return the application with feedback for the candidate.
This is one of the most important parts of the process. Getting to know yourself as a judge is a vital touchstone as a judge. When you reach L4 it’s likely you will feel you receive less feedback and mentoring. You already master all the basics and there’s fewer people in the world who also master your strengths to a degree that allows them to mentor you in the same way you used to be mentored when you were a younger judge.
At this step, you need to become your own mentor. Of course, you will receive and can use help from others, but the responsibility of your future improvement is yours.
Step 3: Recommendations
After completing at least the requirements above, candidates for L4 must submit two different recommendation reviews.
In order to reduce the workload on the recommender and reduce the barriers we know getting recommendations is, we’ve split the L4 rec in two halves to make it less burdensome to write. It’s not a problem if a single judge (meeting the criteria to write both recommendations) writes both recommendations.
Finding a senior judge to mentor you and become your recommender is a vital point in the development of a judge. We understand this is hard and may be sometimes frustrating, but developing the network, showing your value and finding someone willing to invest some of their time in your development is a key process in the career of every senior judge.
Tournament recommendation: The candidate can work with an L5 or a judge who has been L4 for at least one year as their mentor until that mentor is comfortable with their assessment that the candidate has the tournament skills for L4. Then the mentor writes a detailed review of the candidate, explaining how the candidate meets the requirements for L4, related to the qualities: Policy Knowledge, Event Skills and in-event Leadership Skills.
The review will be assessed by an approved L4 panel lead, who may advance them to the L4 panel process, or return the review to the mentor with feedback.
This L4 recommendation is meant to cover the judge’s tournament skills. It doesn’t need to be written based on a single tournament observation, but in several of them. The recommender can reach out and ask other judges for their opinions to complete this recommendation.
A suggested template for this recommendation will soon be provided.
Community involvement recommendation: An L4 judge must be able to provide value to the judge community in the way that fits them the most, there are as many ways to do it as judges in the world. The candidate must receive a recommendation explaining how they are involved in a constructive way in the judge community. Usually those recommendations are provided by a Regional Representative, a Program Lead or the leader of an official IJP project. Exceptionally it can be provided by any L4+, subject to the approval of the L4 Testing Manager.
This recommendation must explain how the candidate provides value to the judge community and the activity/es the candidate has done in the past year.
Examples of providing value are, amongst others: actively participate in a project that is beneficial for the regional or international community; actively mentor other judges; participate in the certification of other judges; write educational content; organize conferences; present at conferences, etc.
This L4 recommendation is meant to cover the judge’s community skills. It doesn’t need to be written based on a single project observation, it may include all elements the recommender finds relevant. The recommender can reach out and ask other judges for their opinions to complete this recommendation.
There isn’t a strict list of things that help the community, each judge is encouraged to do it in a way that makes them happy. This recommendation should explain what the judge does and what value it brings to the judge community.
A suggested template for this recommendation will soon be provided.
Step 4: Exam, a Practical Evaluation and a Panel Interview
After being approved for the L4 panel process the candidate must pass a written Exam, a Practical Evaluation and a Panel Interview in no specific order.
The L4 Testing Manager will select an Exam Proctor and leads for the Practical Evaluation and the Panel Interview. The same judge can fill in more than one of those roles.
The Panel Interview lead and L4 Testing Manager then may choose one additional panelist for the Panel Interview.
The L4 Testing Manager, candidate, Exam Proctor, panel leads and panelists will coordinate an appropriate time and venue for the panels and the Exam.
The L4 Testing Manager at the time of writing this article is me, Alfonso Bueno. I will coordinate with the appropriate judges and the candidate the three processes. When possible we’ll follow the candidates preferences to do it all at the same event or at different times. Ideally all three should happen within a few months, but based on the candidates region and circumstances this may vary.
The candidate must pass all three items to be promoted to L4.
Exam
The candidate may take the Advanced Rules/Policy exam. They must complete the exam with a score of 80% or higher. This exam (unless an exception approved by the L4 Testing Manager) must be proctored and it’s timed and open book (CR; MTR; IPG and JAR).
L4 judges are expected to be able to solve the majority of judge calls with a quick check on the rules. In real life tournaments, judges frequently pull out their phones for a quick rules check. That’s totally fine and even encouraged, we’re trying to mimic this in our exams. However, in real life tournaments there’s always a time restriction because the judge can not take forever to make a ruling in an ongoing game, this is why we’re timing the exam.
Finally, the exams are proctored. This means it is administered by another judge who will watch over the tested candidates and solve any doubts they may have.
In cases where candidates put themselves at a disadvantage in taking written tests, the L4 Testing Manager, with the advice of the Diversity, Equity, and Inclusion team can authorize some exceptions.
Practical Evaluation
The Evaluation happens at a multi-day event where the evaluator and the candidate are both on staff. The candidate does not need to be in any specific role at that event, but some roles will make it harder evaluating the required qualities to pass. The Evaluator observes the candidate’s proficiencies, leadership, and success at the event and writes a review, evaluating the candidate. The Evaluator must write an Advancement Review (pass or fail) of the candidate.
The Advancement Review should cover a broad range of skills expected from an L4 Judge, including, but not limited to:
- Pre-event communication
- Preparation for team or head judge tasks
- Communication with other judges, leads, and head judges
- Success at team tasks
- Mentorship
The panel lead will assess the candidate in each of the 3 quality categories: Policy Knowledge, Event Skills and in-event Leadership Skills.
It’s important to notice the Evaluator and the candidate don’t need to be in any specific position in the event, unlike in previous certifications the Evaluator does not need to be in a team led by the candidate. However, the Evaluator must be in a position that allows them to observe the candidate’s performance in the event. The Evaluator is encouraged to ask other potential observers (like the candidate’s HJ, TLs, FJ, SK, etc).
The evaluation is not made based on a single day observation, but on observations and feedback throughout the multi-day event and a short personal interview between the candidate and the evaluator, which should not last more than 30 minutes.
A non-public guide will be provided to the Evaluators. Candidate’s self review and recommendations will be provided to the evaluator before the event.
Panel Interview
In-person panels are highly recommended, but online Panel Interviews may be considered in circumstances where an in-person Panel Interview would be extremely difficult to schedule.
The Panel Interview is a group interview where the panel asks questions of the candidate to assess them in each quality. The questions will vary, depending on the candidate’s known strengths and weaknesses.
The panel lead will assess the candidate in each of the 3 quality categories: Personal Skills, Community Building and out-event Leadership Skills)
This is not the old-L3 panel interview, this is aimed to be much lighter and last for a maximum of 60 minutes. This panel may be run only by the Panel Lead or have one additional panelist. In any case an observer may be added.
This interview will cover the three qualities mentioned above (Personal Skills, Community Building and out-event Leadership Skills), as described below. A non-public guide will be provided to the Panel Lead. Candidate’s self review and recommendations will be provided to the Panel Lead before the panel.
Panel Guidelines
Each category will be evaluated and assessed on this scale.
- Exceeds Expectations
- Meets Expectations
- Area for Improvement
- Deficient
Candidates who are evaluated by the panel lead during the panel to meet expectations in a majority of categories, with no deficiencies, pass the Panel. Once the Practical Evaluation, the Panel Interview and the Exam are passed the candidate is promoted to L4.
L4 panels are additionally regulated by the Level Four Panel and Testing Guide, maintained by existing L4s and L5s. Portions of the requirements are included here as examples. Generally, the L4 process is designed to assess whether the candidate:
- Is capable and comfortable head-judging a competitive event with 150+ players and small teams of judges.
- Is capable and comfortable team-leading a team with 5 judges at a multi-day event
- Is capable and comfortable with each team’s tasks
- Is capable and comfortable mentoring judges to become L3
- Has strong personal skills that enable head-judging, team-leading, mentorship, and participation in the community
These guidelines apply to both the Panel Interview and the Practical Evaluation.
Level Four Judge Qualities
These are omitted from this article for clarity, please refer to the level definition page to see them.
Maintenance
To maintain the L4 certification, a judge must complete the following items each year
- Choose one —
- Pass three out of four advanced set update quizzes throughout the year
- Pass an Advanced Rules Practice and Advanced Policy Practice test
Rules and Policy knowledge are of vital importance for a judge. Judges must keep up to date with rules and policy changes. They can choose to do it set by set or once a year.
2. Choose three, at least one A and one B must be chosen.—
- Lead a total of 4 teams at events (A)
- Head-judge an event with at least five judges (A)
- Work at least six multi-day events (A)
- Serve on one or more L4 advancement panels (B)
- Actively participate in an IJP core project (B)
- Create quality educational content (ex: article, conference presentation, video, etc.) (B)
Judges must also remain active. However, there are several different ways to be an active judge.
A-items are tournament oriented, while B-items are community oriented. Each judge can focus more on the area they prefer, but must have a basic activity level in each of the areas.
3. Write a self review.
I already talked long enough about the self review above, I won’t bore you by repeating it here 😉
4. Maintain membership in the IJP by being up-to-date on their membership dues (annual membership fee is still to be decided based on the number of members and the association’s costs. Our goal is to have it as low as possible.)
Yep… to be a member of the IJP association you must be a member.
Level 5
An L4 can start looking towards L5 when they find themselves very comfortable in all of the L4 roles, mentoring multiple other judges towards L4, and are highly involved in the regional and international community.
Level 4 is a very nice place to be for senior judges, there is no need to become L5 unless you want to display the involvement level it requires. Indeed L4 is where I see myself in the periods of my life when I want to allow myself some free time for my other hobbies.
I believe many senior judges will choose to remain L4 or alternate between L4 and L5 based on their availability in different periods of their lives.
