Guides
Run a check well
- Open a new chat in the AI you want to check. If your app lets you, turn off web search, tools that run code, and memory. These can add outside help to an answer, or carry something over from an earlier chat.
- On HIBN, choose the provider, app, model and level your app shows. If you’re not sure of the provider, the model or the level, choose Unknown. A wrong guess would record the wrong setup with your result. A check with Unknown or Other in its setup can’t be compared with other checks.
- Copy the questions, paste them into the chat and send. The stopwatch is optional. It includes the time you spend copying and switching apps, so it isn’t the AI’s speed.
- Read the reply before you paste it. We store it exactly as pasted. If it holds anything personal, don’t paste it. Turn off memory in your app if you can, and start a new check.
- Paste the AI’s first reply back, in full and unchanged, with its labels. Don’t ask the AI to try again: a second attempt needs a new check.
- If an answer is missing because the AI refused or stopped before the end, the check page asks what happened. Tell us there. If you leave an answer as missing, the check gets no overall score.
Read your result
The score is the average of the question scores, out of 100. Every question has the same weight. Logic puzzles and table-matching questions can be partly correct. Maths questions can’t.
Beside the score you see how many questions were fully correct. A partly correct question adds to the score but not to that count.
A score shows how this AI did on these questions, with the setup you chose, at that time. It isn’t a rating of the AI overall, and it can’t show that an AI was “nerfed”. HIBN has no pass mark.
If there is no score, at least one answer was missing or couldn’t be scored. The other questions still show their outcomes. To get a score, run a new check.
What each outcome means
| Outcome | What it means | Question score |
|---|---|---|
| Correct | The answer matches. | 1 |
| Partly correct | Part of the answer is right. | Between 0 and 1 |
| Incorrect | We found an answer, and it doesn’t match. | 0 |
| Answer format not recognised | We couldn’t find a final answer in the format the question asks for. On a Custom Pack: the AI didn’t give one plain number where a number was expected, or, on a pack you made, it wrote more than the answer. | 0 |
| Refused (you told us) | The answer was missing, and you told us the AI refused. | 0 |
| Not answered (you told us) | The answer was missing, and you told us the AI didn’t answer. | 0 |
| Missing | We couldn’t find the answer in the reply, or its label appeared more than once. | None, so the check has no overall score |
| Couldn’t be scored | We found an answer, and our scoring code couldn’t tell whether it is correct. For maths, this happens when it can’t work out the value of a number. It isn’t counted as incorrect. | None, so the check has no overall score |
Compare over time
To repeat a check, open a result and choose “Check again (same setup)”. It starts a new check with the same questions and the same setup.
We show a change in points only when both checks used the same questions, the same scoring code and the same setup. Nothing in the setup can be Unknown or Other.
One Quick question is worth up to 17 points, and one Deep question up to 10. One answer can move the score by that much. A larger change can also happen by chance.
Keep the conditions the same each time: the same app, model and level, a new chat, and tools and memory off.
Run the check several times over days or weeks. More checks show whether a difference repeats.
A change doesn’t show that the AI was changed, or why the score moved. An unchanged score doesn’t show that nothing changed. The score covers these questions only.
Keep your results
We store your results on our servers. Your browser holds a key that opens them. The key lasts 90 days from the day it is made, and new checks don’t extend it. It is lost sooner if you clear your cookies.
To keep a result for longer, open it and choose “Save my result with Google”. That moves the one result to an account. You can then open it on any device where you sign in.
To download the data linked to your browser or your account, use “Download my data” in My Results. To delete a result, open it and choose “Delete result”.
Build a Custom Pack
A Custom Pack is your own set of questions, with the answer you expect for each. The Pack guide covers how to write questions HIBN can score, and what happens when you publish.