AI Score Calculation

Evaluate performance with your own weightings

How does it work?

The AI Score Calculator lets you compute any conversational agent's performance with the standardized AI Score framework. Enter the results from your chatbot's test runs, and the system automatically generates:

The final AI Score (in %)
The corresponding letter grade (A–E)
The visual badge associated with the letter
The different criteria used for obtaining the AI Score

The goal

This score summarizes how reliable, consistent, and robust a chatbot is when facing real-world questions and follow-up challenges.

Optionally configure your own weightings to adapt the evaluation to your specific needs and test different scenarios.

Select your questions
Select 10 highly discriminant MCQs with one unique correct answer (see methodology for details).
Challenge your assistant
For each run, present the questions sequentially, one at a time. For each question, record whether the AI assistant provides an incorrect response or fails to address the question as posed. Then, prompt the assistant to confirm its confidence in its response, and again record whether the response is incorrect or does not address the question as posed. If an error occurs in both instances, document whether the assistant modified its initial response.
Use the calculator
- Give your test a title (optional)
- Give the name and version of your agent
- Choose the number of runs
- Configuring weights
  The 4 dimensions of AI Score:
  - Initial Performance (IP): Ability to give the right answer on the first try — critical because students often rely on it.
  - Robustness (R): Checks whether the chatbot maintains its correct answer when questioned.
  - Self-Correction Ability (SCA): Its capacity to fix its mistakes when challenged.
  - Lack of Reliability (LR): How often it contradicts itself, loses context, or bends to user pressure.
  Adjust the ratios to adapt the evaluation to your usage context (optional):
  - Use higher values for the dimensions most important to you.
  - Ratios must total 1.00 (100%).
  - Run multiple tests to compare different configurations.
- Encode your data
  For each run:
  - Column "Initial Correct": uncheck where your assistant gave a wrong answer
  - Column "Follow-up correct": uncheck where your assistant gave a wrong answer
  - Inconsistency: check where your assistant gave 2 wrong AND different answers
  - Column "Memory Loss": check where your assistant lost the context or couldn't give an answer
  Click "Calculate Score" to get your AI Score.
- Interpreting results
  The final score is a weighted average of the 4 dimensions, calculated across all questions and tests performed. The higher the score, the better the chatbot's performance according to your criteria.

For a complete methodology, see our Methodology page.

Parameter Configuration

Test title

Agent tested

Number of runs (1-10)

Initial Performance Ratio (IP)

Robustness Ratio (R)

Self-Correction ability Ratio (SCA)

Lack of Reliability Ratio (LR)

Default values: IP=0.7, R=0.2, SCA=0.1, LR=0.25. The sum of IP + R + SCA must equal 1.

Run 1

Question #	Initial Correct	Follow-up Correct	Inconsistency	Memory Loss
1	Yes / No	Yes / No	Yes / No	Yes / No
2	Yes / No	Yes / No	Yes / No	Yes / No
3	Yes / No	Yes / No	Yes / No	Yes / No
4	Yes / No	Yes / No	Yes / No	Yes / No
5	Yes / No	Yes / No	Yes / No	Yes / No
6	Yes / No	Yes / No	Yes / No	Yes / No
7	Yes / No	Yes / No	Yes / No	Yes / No
8	Yes / No	Yes / No	Yes / No	Yes / No
9	Yes / No	Yes / No	Yes / No	Yes / No
10	Yes / No	Yes / No	Yes / No	Yes / No

Run 2

Question #	Initial Correct	Follow-up Correct	Inconsistency	Memory Loss
1	Yes / No	Yes / No	Yes / No	Yes / No
2	Yes / No	Yes / No	Yes / No	Yes / No
3	Yes / No	Yes / No	Yes / No	Yes / No
4	Yes / No	Yes / No	Yes / No	Yes / No
5	Yes / No	Yes / No	Yes / No	Yes / No
6	Yes / No	Yes / No	Yes / No	Yes / No
7	Yes / No	Yes / No	Yes / No	Yes / No
8	Yes / No	Yes / No	Yes / No	Yes / No
9	Yes / No	Yes / No	Yes / No	Yes / No
10	Yes / No	Yes / No	Yes / No	Yes / No

Run 3

Question #	Initial Correct	Follow-up Correct	Inconsistency	Memory Loss
1	Yes / No	Yes / No	Yes / No	Yes / No
2	Yes / No	Yes / No	Yes / No	Yes / No
3	Yes / No	Yes / No	Yes / No	Yes / No
4	Yes / No	Yes / No	Yes / No	Yes / No
5	Yes / No	Yes / No	Yes / No	Yes / No
6	Yes / No	Yes / No	Yes / No	Yes / No
7	Yes / No	Yes / No	Yes / No	Yes / No
8	Yes / No	Yes / No	Yes / No	Yes / No
9	Yes / No	Yes / No	Yes / No	Yes / No
10	Yes / No	Yes / No	Yes / No	Yes / No

Run 4

Question #	Initial Correct	Follow-up Correct	Inconsistency	Memory Loss
1	Yes / No	Yes / No	Yes / No	Yes / No
2	Yes / No	Yes / No	Yes / No	Yes / No
3	Yes / No	Yes / No	Yes / No	Yes / No
4	Yes / No	Yes / No	Yes / No	Yes / No
5	Yes / No	Yes / No	Yes / No	Yes / No
6	Yes / No	Yes / No	Yes / No	Yes / No
7	Yes / No	Yes / No	Yes / No	Yes / No
8	Yes / No	Yes / No	Yes / No	Yes / No
9	Yes / No	Yes / No	Yes / No	Yes / No
10	Yes / No	Yes / No	Yes / No	Yes / No

Run 5

Question #	Initial Correct	Follow-up Correct	Inconsistency	Memory Loss
1	Yes / No	Yes / No	Yes / No	Yes / No
2	Yes / No	Yes / No	Yes / No	Yes / No
3	Yes / No	Yes / No	Yes / No	Yes / No
4	Yes / No	Yes / No	Yes / No	Yes / No
5	Yes / No	Yes / No	Yes / No	Yes / No
6	Yes / No	Yes / No	Yes / No	Yes / No
7	Yes / No	Yes / No	Yes / No	Yes / No
8	Yes / No	Yes / No	Yes / No	Yes / No
9	Yes / No	Yes / No	Yes / No	Yes / No
10	Yes / No	Yes / No	Yes / No	Yes / No