Assigning Applicants to Reviewers: Vertical vs Horizontal Review

What is Flexible Review in Kira?

The Kira platform now offers the flexibility to review applicants vertically or horizontally.

Vertical reviewing

A vertical review is when reviewers evaluate all competencies for assigned applicants.

Horizontal reviewing

A horizontal review is when reviewers evaluate one (or some) competencies for assigned applicants.

Benefits of Flexible Review

Vertical reviewing allows reviewers to evaluate the responses of an applicant across an entire assessment, and get a full picture of the applicant's soft skills and personality. While this provides a more holistic view of an applicant, there is also a greater likelihood of "halo-effect" bias.

Halo effect bias is a cognitive bias that occurs when an initial positive or negative judgment about a person unconsciously colors the perception of the individual as a whole. In a Kira assessment for example, after being impressed with an applicant’s first response, the ‘halo-effect’ bias may cause a reviewer to rate the applicant’s subsequent responses more generously, even if the quality of the responses didn’t warrant higher ratings.

Horizontal reviewing was created to help avoid this type of bias when assessing applicants. By reviewing horizontally, reviewers become experts in the competency they're assigned to, which improves the accuracy and reliability of their scoring. It will ensure each applicant is evaluated by more reviewers and create an easier onboarding experience for your team, as reviewers need only focus on becoming well-versed at evaluating one competency each.

Which reviewing method should I choose?

There is no right or wrong way to review in Kira. Both vertical and horizontal reviewing are valid approaches. Your decision will depend on your program type, reviewer volumes, question types, and your industry's preferences for applicant evaluation.

Want to use both methods of reviewing? The Kira platform is flexible, and we can accommodate both types of reviewing in one assessment! Experimentation is encouraged. :)

Reviewer assignments

You can assign reviewers to competencies for individual applicants. Click the + button on your applicant list, and a new modal will assist you with assigning reviewers to an applicant's competencies.

Here, you can assign your reviewers to All Competencies (a vertical review) or individual competencies (a horizontal review). Click the arrow beside All Competencies to expose the list of individual competencies.

You can assign as many reviewers as you’d like to your competencies with any distributions you prefer.

Applicant lists

• "In Review" contains all applicants who've been assigned to reviewers but not all evaluations are complete.
• "Reviewed" contains all applicants who've been evaluated by all of their assigned reviewers. (All reviews for these applicants are complete and their Kira scores can now be used in admissions decisions).
• "In Review" is a tab that has been added to the "Assigned to You" view. This is designed to help your reviewers keep track of their assigned applicants who they've partially reviewed.

Review pages

On all applicant review pages, reviewers only have access to the competencies they've been assigned to review. This allows them to focus only on their assigned competencies.

For example, if an assessment has four competencies (Motivation, Engagement, Creativity, and Verbal Communication), and a reviewer is assigned to review Motivation, they will only see Motivation on their review page.

Admin and Staff users can also participate in horizontal reviewing and will be guided to review only their assigned competencies. However, we don't restrict them from viewing the other competencies and responses.

For example, if an assessment has four competencies, and an Admin is assigned to review Motivation, they will see Motivation more prominently on their review pages, and can open the other three (unassigned) competencies if they want to.