Flexible Review

What is Flexible Review in Kira?

The Kira platform now offers the flexibility to review applicants vertically or horizontally.

Vertical reviewing

A vertical review is when reviewers evaluate all competencies for assigned applicants.

Horizontal reviewing

A horizontal review is when reviewers evaluate one (or some) competencies for assigned applicants.

Benefits of Flexible Review

Vertical reviewing allows reviewers to evaluate the responses of an applicant across an entire assessment, and get a full picture of the applicant's soft skills and personality. While this provides a more holistic view of an applicant, there is also a greater likelihood of "halo-effect" bias.

Halo effect bias is a cognitive bias that occurs when an initial positive or negative judgment about a person unconsciously colors the perception of the individual as a whole. In a Kira assessment for example, after being impressed with an applicant’s first response, the ‘halo-effect’ bias may cause a reviewer to rate the applicant’s subsequent responses more generously, even if the quality of the responses didn’t warrant higher ratings.

Horizontal reviewing was created to help avoid this type of bias when assessing applicants. By reviewing horizontally, reviewers become experts in the competency they're assigned to, which improves the accuracy and reliability of their scoring. It will ensure each applicant is evaluated by more reviewers and create an easier onboarding experience for your team, as reviewers need only focus on becoming well-versed at evaluating one competency each.

Which reviewing method should I choose?

There is no right or wrong way to review in Kira. Both vertical and horizontal reviewing are valid approaches. Your decision will depend on your program type, reviewer volumes, question types, and your industry's preferences for applicant evaluation.

Want to use both methods of reviewing? The Kira platform is flexible, and we can accommodate both types of reviewing in one assessment! Experimentation is encouraged. :)

What's new in Kira with Flexible Review?

There are many new capabilities to explore in the platform.

Reviewer assignments

You can now assign reviewers to competencies for individual applicants. Click the + button on your applicant list, and a new modal will assist you with assigning reviewers to an applicant's competencies.

Here, you can assign your reviewers to All Competencies (a vertical review) or individual competencies (a horizontal review). Click the arrow beside All Competencies to expose the list of individual competencies.

You can assign as many reviewers as you’d like to your competencies with any distributions you prefer.

To assign reviewers horizontally to your applicants in bulk, please contact your Client Success Manager. They'll help you create the perfect spreadsheet and upload custom reviewer assignments into your assessment on your behalf.

Applicant lists

The organization of applicants on your assessment page has changed slightly.

  • "In Review" now contains all applicants who've been assigned to reviewers but not all evaluations are complete.
  • "Reviewed" now contains all applicants who've been evaluated by all of their assigned reviewers. (All reviews for these applicants are complete and their Kira scores can now be used in admissions decisions).
  • "In Review" is a new tab that has been added to the "Assigned to You" view. This was a popular feature request which was designed to help your reviewers keep track of their assigned applicants who they've partially reviewed.

Review pages

On all applicant review pages, reviewers only have access to the competencies they've been assigned to review. This allows them to focus only on their assigned competencies.

For example, if an assessment has four competencies (Motivation, Engagement, Creativity, and Verbal Communication), and a reviewer is assigned to review Motivation, they will only see Motivation on their review page.

Admin and Staff users can also participate in horizontal reviewing and will be guided to review only their assigned competencies. However, we don't restrict them from viewing the other competencies and responses.

For example, if an assessment has four competencies, and an Admin is assigned to review Motivation, they will see Motivation more prominently on their review pages, and can open the other three (unassigned) competencies if they want to. 

Other minor changes

  • The "Add Note" function in the "Notes" section on the applicant review pages is now called "Save Note"
  • Reviewers are listed in assignment dropdowns in alphabetical order by last name instead of first name
  • An "Assigned competency" column has been added to the Reviewer Progress table, Reviewer Progress CSV, and Reviewer Details CSV 
  • A "Number of completed reviews" column has been to the Applicant Details CSV for each competency

If you're interested in using horizontal reviewing in Kira, please connect with us directly

We'll be happy to answer any questions you have.