AI Talent Assessment: What the Scores Actually Mean

AI talent assessment uses AI to evaluate candidate information or performance and produce scores, classifications or recommendations for hiring or promotion. Examples include automated interview scoring and game-based assessments. The important question is not whether a tool uses AI, but what it measures—and whether the evidence supports using that measurement for the job. The Society for Industrial and Organizational Psychology (SIOP) recommends the same level of scrutiny for AI-based assessments as for traditional employment tests.

A score of 82 is not automatically an 82% chance of succeeding at work. Before using it to reject someone—or treating it as a verdict on your abilities—find out what the number represents.

Where AI enters the assessment

Separate the task from the scoring method. A recorded interview is a format; automated evaluation of its answers is an assessment function. A work sample can be scored by a person, software or both.

A useful way to inspect a system is to trace four stages:

1. Input: What does the candidate provide—written answers, spoken responses, completed tasks or game interactions?
2. Processing: What happens before scoring, such as speech transcription or extracting features from responses?
3. Scoring: Does the system estimate a competency, compare responses with a rubric, or predict an outcome?
4. Decision: Does the employer use the result as supporting evidence, a ranking or a cutoff?

SIOP recommends documenting the pathway from raw data through preprocessing, feature generation and modeling to the use of predicted values or categories. It also warns that changing the algorithm or transcription tool can affect score comparability.

Do not assume every recorded interview receives AI scoring. For example, HireVue’s candidate FAQ says not all its interviews have an assessment added. It describes its video-based assessments as evaluating job-related competencies and considering the context of responses rather than isolated keywords. That is the vendor’s description of its own product, not independent proof of predictive accuracy.

Read the scale before reading the score

The following are illustrative score formats, not results from a particular product:

Displayed result	What it could mean	What to check
82/100	A weighted total of rubric ratings	Which criteria and weights produced it?
82nd percentile	Relative standing in a reference group	Who is in that group, and how current is it?
0.82 predicted probability	A model’s estimated likelihood of a defined outcome	What outcome, over what period, and how closely do estimates match observed outcomes?

These meanings are not interchangeable. A percentile describes relative standing, not the percentage of job requirements met. A rubric total is not a probability merely because it uses a 100-point scale.

Also separate the score from the employer’s decision rule. In a hypothetical workflow, an employer might advance only candidates scoring 80 or above. That threshold is a separate choice requiring justification; the model’s ability to produce numbers does not establish that 79 is unsuitable and 80 is suitable.

For broader assessment design, start with what to measure and how to score it.

What HR needs before relying on the result

Ask for a technical report and interpretation guide, not just a dashboard or an “accuracy” headline. Four questions make the evidence more useful:

• What was validated? Agreement with human ratings, measurement of a named competency and prediction of later job performance are different claims. Ask which one the evidence supports.
• For whom and where? Check the roles, languages, applicant or employee populations, sample sizes and working conditions represented. A study measuring assessment scores and performance at the same time is not a study measuring later performance. Ask whether the participants were applicants or existing employees, and how the findings apply to your intended use.
• Are scores stable and comparable? Ask about reliability, model versions, input requirements and whether changes to format or scoring require new validation.
• What changes when this score enters the workflow? Record the cutoff or ranking rule, reviewer responsibilities, error-handling route and monitoring plan.

These checks follow SIOP’s recommendations on validation, scoring and documentation. Faster processing or more completed assessments may demonstrate operational value, but neither alone demonstrates better hiring decisions.

In the United States, the EEOC’s employment-testing guidance says employers should ensure tests are properly validated for the positions and purposes for which they are used. Vendor documentation can help; it does not transfer the employer’s responsibility to the vendor. Use the AI hiring vendor validation checklist for a fuller review.

What candidates can check or change

Before starting, ask the recruiter:

“What does this assessment measure, is it AI-scored, and how is the result used? What are the rules on notes or AI assistance, and whom should I contact for an accommodation or technical problem?”

Prepare for the actual task: practice relevant work examples, explain your actions and reasoning, and follow the stated tool-use rules. Do not treat a third-party practice score as the employer’s score. If a recording or connection fails, report the problem promptly and ask about reassessment rather than assuming the system will recognize the failure.

Accessibility matters to score interpretation. A result should reflect the skill being assessed, not an unrelated barrier. US Department of Justice guidance explains that hiring tests must measure relevant skills rather than impairments they do not seek to measure, and that reasonable accommodations are required under the ADA unless they create undue hardship.

Validation and legal compliance are separate checks

Rules reference: October 24, 2026. As one local example, NYC Local Law 144 requires covered automated employment decision tools to have a bias audit within one year of use, publicly available audit information and required notices. Enforcement began July 5, 2023. NYC DCWP’s overview describes these duties; its FAQ explains the tool and geographic scope. Not every AI assessment falls within the law.

The NYC audit’s minimum checks concern selection or scoring rates and impact ratios across sex, race/ethnicity and intersectional categories—not prediction of later job performance. A bias-audit summary is therefore not, by itself, evidence that a score predicts job performance. Source: NYC DCWP FAQ.

Request the validation evidence separately, and check the requirements that apply to the employer, location and tool. This article is informational, not legal or employment advice; confirm applicable obligations with qualified counsel.