Score Calculations at Kira

Introduction

Welcome to the Kira Talent scoring system designed to streamline and enhance the evaluation process during interviews. At Kira Talent, we are committed to providing a reliable and insightful assessment platform that aids institutions in making informed decisions when selecting applicants.

This documentation will guide you through each component of our scoring system, ensuring you understand the mechanisms behind our calculations and how to effectively use our platform to evaluate applicants.

Key Concepts and Definitions

To ensure clarity and effectiveness in using the Kira Talent scoring system, it's important to understand the following key concepts and definitions:

  1. Competency: A competency refers to a set of skills, knowledge, abilities, or attributes that are critical for success in the program.
  2. Weighting System: Different weights are assigned to each competency based on its importance to the overall fit with the institution. Weighting affects how individual scores impact the overall score, ensuring that more critical competencies have a greater influence on the applicant’s final evaluation. Schools can decide on the weight of each competency, and your dedicated Client Success Manager can also share their knowledge and expertise.
  3. Percentiles: Percentile scoring is a method of evaluating performance by comparing a specific score to a distribution of scores on different scales. For example, if an interview rating is 3 out of 5, it can be converted into a percentile to standardize it across various rating scales such as 3 out of 5, 2 out of 6, etc. This percentile represents the relative standing of a score within the entire range of possible scores, making it easier to compare and interpret results consistently.
  4. Rubric: A rubric is a scoring tool that outlines the criteria used to evaluate a response, typically ranging from 1 to 5, although custom ranges (e.g., 0-4, 1-7) can be configured.
  5. Calculated Scores:
    1. Overall Applicant Score: This is the aggregate score for an applicant, based on all scores from all reviewers. This comprehensive score offers a holistic view of the applicant's suitability, taking into account the importance of each competency.
    2. Average Score by Reviewer: This score represents the average of all the scores given by a single reviewer to one applicant across different competencies. It provides insight into the overall evaluation of the applicant by that particular reviewer. When aggregated for a reviewer across a range of scores for a given interview it helps sheds light on if the reviewer is being lenient or too harsh.
    3. Average Score by Competency: This is the average of all scores from all reviewers for a particular competency in an assessment. It measures how well an applicant performed in specific areas crucial to determining fit, such as communication or teamwork.

Score Calculations

In this section, we delve into the specifics of how scores are calculated within Kira Talent. Learning about how Kira calculates scores will help you gain a deeper understanding of the scores you see on the platform, resulting in interpreting the results more accurately. We will explain the methodologies used to calculate the different types of scores: Overall Applicant Score, Average Score by Reviewer, and Average Score by Competency.

Overall Applicant Score

Description 

The Overall Applicant Score is the final aggregate score that quantifies an applicant's suitability for the role, based on the evaluation across all competencies. This score integrates input from all reviewers, using weights from each competency.

Formula

Example

Imagine an interview with a rating of 1 - 5 and two competencies:

  • Communication (Weight of 1.0, Scale of 1 - 5)
  • Teamwork (Weight of 0.5, Scale of 1 - 5)

A reviewer leaves a score of 5 for Communication, and a score of 2 for Teamwork.

Calculations

To calculate the Overall Applicant Score, follow these steps:

Step 1: Convert Scores to Percentiles - Converts the actual scores for each competency to a 0-1 scale.

Step 2: Calculate Weighted Percentiles - Multiplies the percentiles by the given weights to reflect the importance of each competency.

Step 3: Find the Interview Scale Difference - The difference between the top of the scale and the bottom.

Step 4: Calculate Overall Applicant Score - Adds the weights of all competencies, use the summed weighted percentiles and the total weights to scale the final score to the interview rubric range (1-10).

Average Score by Reviewer

Description

The Average Score by Reviewer reflects the overall evaluation provided by a single reviewer across all competencies for a specific applicant. This score integrates input from all the competencies that only the reviewer has scored on an applicant, using weights from each competency.

Formula

Example

Please see the example under Overall Applicant Score as the calculations are the same, however the scores used are only for the specified reviewer.

Average Score by Competency

Description

The Average Score by Competency calculates the average score an applicant receives for a specific competency across all reviewers. This measure is crucial for assessing how well an applicant performs in particular areas that are critical to the role, allowing for targeted feedback and development.

Formula

Example

Imagine an interview with a competency we are trying to find the average score of, Communication (weight of 1.0, scale of 1-7).

Two reviewers leave scores on the applicant, a 5 and a 6.

Troubleshooting and Support

If you have any questions please feel free to reach out to our Client Success Team directly on the Kira app by using the chat bubble in the bottom right.