Learn

Pack guide

A Custom Pack is your own set of questions, with the answer you expect for each. HIBN scores an AI’s answers by comparing them with yours.

How HIBN scores a pack

HIBN puts all your questions into one message, under the labels T01, T02 and so on. Whoever runs the pack copies that message into their AI. The message says: “Reply with only the answer, nothing else.”

HIBN finds each answer by its label. The label must start a line. Only heading marks, bold marks or = signs may come before it. It must be alone on the line, or be followed by a colon, a full stop, a closing bracket or a dash. An answer written as T01 Paris, with only a space after the label, is not found and is recorded as “Missing”. So is - T01: Paris, where a list mark comes before the label.

HIBN then compares each answer with your expected answer, using the rule you chose for that question. A match scores 1. An answer that doesn’t match scores 0. No question is partly correct. If HIBN can’t find an answer, the check has no overall score, unless the person running the pack says the AI refused or didn’t answer. That question then scores 0.

HIBN doesn’t confirm that your expected answers are right. If one is wrong, every AI that answers correctly is recorded as incorrect.

The score is the share of questions that matched, out of 100. With 5 questions, each is worth 20 points. With 20 questions, each is worth 5.

Write questions with one possible answer

Ask for one short answer: a number, a word, a name, a date or a letter.
Say exactly how to write it. For example: “Reply with the number only.” “Reply with one word, in lower case.” “Give the date as YYYY-MM-DD.”
Avoid questions with more than one acceptable answer or spelling. “USA” and “United States” don’t match each other.
Avoid answers that change over time, such as the current year. Your expected answer is fixed when you save.
Make each question stand alone. The AI sees them all at once, so one question shouldn’t give away the answer to another.
Write your own questions where you can. An AI may already have seen a puzzle copied from the web.

Choose the right rule

Each question has one matching rule. Spaces at the start and end of an answer are ignored under every rule.

Rule	What matches	What doesn’t match	Use it for
Exact text	The same characters, with the same capitals and punctuation.	`paris` or `Paris.` when you expect `Paris`	Codes, formulas and anything where capitals matter
Text, any capitalisation	The same characters in any mix of capitals.	`Paris.` or `Paris, France` when you expect `Paris`	Words and names
Number	The same value. `1,234` matches `1234`, and `2.50` matches `2.5`.	`50%`, `$5`, `5 kg`, `1/2` or `five`	Counts and calculations

Write a number in digits, with a full stop for decimals. Commas are allowed only as thousands separators, as in 1,234 or 12,345,678. HIBN also reads 1e3 as 1000. HIBN compares about the first 15 digits of a number. For a longer number, use Exact text.

Avoid an expected answer that starts with ** or __, or is made only of = signs. HIBN removes those marks from the start of an AI’s answer before it compares, so a right answer is usually recorded as “Incorrect”. An answer made only of = signs is recorded as “Missing”, and the check then has no overall score.

What stops a match

An AI may add something to a right answer. Each of these stops a match:

A full stop or other punctuation after a word: Paris.
Bold or other formatting: **Paris**
A unit or a symbol with a number: 5 kg, 50%
A minus sign that isn’t the keyboard hyphen: −5 instead of -5
A negative number on the same line as its label, with no colon: T01 -5 is read as 5
A sentence: The answer is Paris.

Under the text rules, an answer with more words or lines than you expected is recorded as “Answer format not recognised” on your own results; anyone else who runs your pack sees “Incorrect”. Under the number rule, anything that isn’t one plain number is recorded as “Answer format not recognised”. Any other answer that doesn’t match is recorded as “Incorrect”. All of these score 0.

To fix this, tell the AI the exact format you want, then run the pack once.

Run it once before you rely on it

Run your pack once against your own AI. On your own result, each question shows the answer you expected and the answer we found. Fix what doesn’t match, and save.

Every save that changes the pack creates a new version. Results from different versions can’t be compared, so settle the pack before you start to track a score over time.

Before you publish

Sign in first. You can publish a pack only if you were signed in with Google when you first saved it. A pack saved while signed out can’t be published later. Sign-in ends 12 hours after you signed in, or after 30 minutes without activity, and typing in the editor doesn’t count as activity. If it has ended when you choose “Save pack”, the editor stops, tells you, and saves nothing until you sign in again in another tab.
The same applies each time you edit a pack. If sign-in has ended, “Save pack” stops and nothing is stored. Your text stays on the page. Sign in again in another tab, then choose “Save pack” again.
A pack made without an account can be lost. Only the browser that made it can open it, and only until that browser’s 90-day key expires. After that, it can’t be opened or deleted from the site.
Everyone can see the title, description and questions, your display name, the version, and the date and time of the last change. They also see how often the pack has been run and, if you copied it from another pack, the “Based on” line.
The expected answers and the matching rules are shown to you and to no other visitor.
The name you enter when you publish is shown as that pack’s author. It can be up to 40 characters, with no links or email addresses.
The title is sent to the AI with the questions, so don’t put hints in it. The description is not sent.
Other people can copy your questions into a pack of their own. Their pack shows “Based on”, with your title and display name. They don’t get your expected answers.
You can’t see other people’s results. A person who runs your pack sees their own result. You see how often the pack has been run.
Don’t include personal information, or anything you don’t have the right to share. Follow the community rules.
A moderator can hide a pack that breaks the rules. Other people then can’t find it or start a check with it.

After you publish

Editing a published pack creates a new version. People who start a check from the pack’s page get the newest version. While the pack stays public, a person who uses “Check again (same setup)” on an old result repeats the version they used before.

“Make private” takes the pack out of the public list. Other people can no longer find it or start a check with it. A check that was already started can still be finished.

After “Delete pack”, nobody can open the pack’s page or start a check with it, and you can’t undo this. Results that people already have can still be opened, and a check that was already started can still be finished. Copies other people made stay, with your title and display name on them. We keep the pack’s questions and expected answers so that those results still work.

What a pack can’t tell you

A pack shows whether an AI’s answers matched yours. It doesn’t show that your answers are right, and no pack can rate an AI in general.

No other visitor can open a pack result. Pack checks are never counted in Analytics.

Limits

A pack has 1 to 20 questions.
The title can be up to 80 characters, and the description up to 500.
Question text can be up to 2,000 characters. A pack holds text only: no images or files.
An expected answer is one line of up to 200 characters.