AI level test: where do you stand?

Right answer: It may be made up: check it in an official source.

• Wrong It's reliable: the assistant checked its sources before answering. No: an assistant produces the most plausible text. Even when it searches the web, it can misread or miscopy a source.
• Right It may be made up: check it in an official source. Correct. An assistant can produce false but very convincing information, with a number or reference that doesn't exist: this is called a hallucination. For anything important, check the original source.
• Wrong It's definitely wrong: an AI knows nothing about law. No: the answer is often right. The problem is that you can't tell whether it is without checking.
• Wrong It's reliable if the answer is long and detailed. No: detail and a confident tone prove nothing. A hallucination can be very precise.

Glossary entry: Hallucination

Right answer: The request or instruction you write to the AI.

• Right The request or instruction you write to the AI. Correct. The prompt is your message: a question or an instruction, possibly with context and examples. The clearer it is, the better the answer.
• Wrong The answer written by the AI. No: the answer is what the AI produces from the prompt, not the prompt itself.
• Wrong The paid version of an assistant. No: a prompt has nothing to do with subscriptions. You write prompts in free and paid versions alike.
• Wrong A program you install on your computer. No: a prompt is plain text, typed straight into the assistant's input box.

Glossary entry: Prompt

Right answer: Because its knowledge stops at a date: the end of its training data.

• Wrong Because assistants refuse to talk about sports. No: nothing stops an assistant from talking about sports. The problem lies in what it was able to learn.
• Wrong Because it forgets everything at the end of each conversation. No: memory from one conversation to the next is a separate issue. Even with a good memory, it can't know a fact it has never seen.
• Right Because its knowledge stops at a date: the end of its training data. Correct. A model learns from texts collected up to a certain date, called the knowledge cutoff. Without web access it doesn't know what happened afterwards, and may make up an answer.

Glossary entry: Knowledge cutoff

Right answer: Your banking password or your card number.

• Wrong A recipe you want to adapt for four people. No risk: a recipe contains nothing personal.
• Wrong A grammar or spelling question. No risk: the question reveals nothing about you.
• Right Your banking password or your card number. Correct. Depending on the service and your settings, conversations may be kept, reviewed or used to improve models. No assistant needs your passwords or card numbers: never share them.
• Wrong The text of a public article you want summarized. Little risk: the text is already public. Just avoid adding personal information to it.

Glossary entry: GDPR and AI

Right answer: The agent can take actions to reach a goal: browsing, filling in a form, using tools, step by step.

• Wrong The agent is always smarter. No: an agent can run on the same model as a chatbot. What changes is what it is allowed to do.
• Wrong The agent works without an internet connection. No: most agents actually need the internet to act on websites and services.
• Wrong The agent is a real person answering on your behalf. No: an AI agent is a program, not a human operator.
• Right The agent can take actions to reach a goal: browsing, filling in a form, using tools, step by step. Correct. A chatbot answers with text; an agent chains actions to complete a task. That's handier, but a mistake also has bigger consequences, which is why you should keep approval of sensitive actions.

Glossary entry: AI agent

Right answer: A chunk of text (a word, part of a word, a punctuation mark) that the model processes.

• Wrong A password that protects your account. No: in AI the word means something else. Don't confuse it with the authentication tokens used in computing.
• Wrong A complete sentence. No: a sentence is usually several tokens.
• Right A chunk of text (a word, part of a word, a punctuation mark) that the model processes. Correct. The model splits text into tokens before reading it and writes its answer token by token. Conversation length and API billing are counted in tokens.
• Wrong A cryptocurrency used to pay for the assistant. No: despite the same word, it has nothing to do with crypto tokens.

Glossary entry: Token

Right answer: The conversation is nearing or exceeding its context window: the amount of text it can take into account at once.

• Wrong It gets tired after a certain number of messages. No: a model doesn't get tired. It handles every answer the same way.
• Wrong Its temperature is set too high. No: temperature affects how varied the answers are, not how much text the model can take into account.
• Wrong The service deleted your first messages to free up space on its servers. No: your messages are still displayed. What's limited is how much the model can read at once.
• Right The conversation is nearing or exceeding its context window: the amount of text it can take into account at once. Correct. Beyond that limit, the oldest parts may be cut or summarized, and even before it, instructions buried in a long exchange are sometimes followed less well. Fix: restate the instructions or start a new conversation with a summary.

Glossary entry: Context window

Right answer: How much randomness goes into word choice: low gives more predictable answers, high gives more varied ones.

• Right How much randomness goes into word choice: low gives more predictable answers, high gives more varied ones. Correct. A low temperature suits data extraction or code, a higher one suits brainstorming or creative writing. It's mostly adjustable through APIs and some advanced tools.
• Wrong How fast the model answers. No: speed depends on the model, the length of the answer and server load, not on temperature.
• Wrong Factual accuracy: the higher it is, the more reliable the model. No: if anything it's the reverse. A high temperature increases variety, which can increase mistakes.
• Wrong The heat of the servers running the model. No: despite its name, this setting isn't physical. It's a mathematical parameter of word choice.

Glossary entry: Temperature

Right answer: That its weights, the parameters learned during training, can be downloaded and run on your own machines, under a license.

• Wrong That it's fully open source, including training data and code. No: most open-weights models don't publish their training data. "Open weights" doesn't mean "fully open source".
• Wrong That it can be used for any purpose, with no conditions at all. No: every model has its license, and some impose conditions (commercial use, attribution, usage restrictions). You need to read it.
• Right That its weights, the parameters learned during training, can be downloaded and run on your own machines, under a license. Correct. You can then run the model at home or on your own servers, adapt it, and keep your data in-house, within the terms of the license.
• Wrong That it's a lightweight model designed for small computers. No: "weights" refers to the model's parameters, not its size. Open-weights models come in all sizes, some of them very large.

Glossary entry: Open-weights model

Right answer: Nothing for sure: the score measures a specific set of tests, sometimes inflated if the questions ended up in the training data.

• Wrong That it will be the best at everything you ask it. No: a benchmark only covers one kind of exercise. A model can shine at math and be weaker at writing in your language.
• Right Nothing for sure: the score measures a specific set of tests, sometimes inflated if the questions ended up in the training data. Correct. A benchmark is a useful clue for comparison, not a guarantee. It can also be skewed by contamination (the model has already seen the questions). The best test is still trying it on your own cases.
• Wrong That it no longer makes factual mistakes. No: no score guarantees error-free answers. Even the best models sometimes hallucinate.
• Wrong That an independent authority has certified it. No: a benchmark is a set of tests, not a certification. Scores are often published by the model's own maker.

Glossary entry: Benchmark

Right answer: Finding the relevant passages in a document collection and giving them to the model along with the question, so it answers based on them.

• Wrong Retraining the model on your documents so it learns them. No: that's fine-tuning. RAG doesn't change the model.
• Wrong A technique that completely eliminates hallucinations. No: RAG reduces them, but the model can still misread a passage or be handed an off-topic one. Always check the cited sources.
• Wrong A compression format that shrinks a model. No: shrinking a model relies on other techniques, such as quantization.
• Right Finding the relevant passages in a document collection and giving them to the model along with the question, so it answers based on them. Correct. The model stays the same; at question time it's given the relevant excerpts (often found using embeddings). That lets it answer about recent or internal documents and cite its sources.

Glossary entry: RAG (retrieval-augmented generation)

Right answer: Connect the assistant to an up-to-date document base it searches for each question (RAG).

• Right Connect the assistant to an up-to-date document base it searches for each question (RAG). Correct. Updating the documents is enough for the answers to follow, and the assistant can show where the information comes from.
• Wrong Fine-tune the model every month with the new procedures. No: fine-tuning is mainly for adjusting style, format or behavior. It's slower and costlier to redo, and doesn't guarantee the model will faithfully reproduce precise facts that change.
• Wrong Raise the temperature so the assistant is more inventive. No: a higher temperature would make answers more varied, and so less faithful to the procedures.
• Wrong Pick the model that ranks highest on benchmarks. No: no model, however well ranked, knows a company's internal documents.

Glossary entry: Fine-tuning

Right answer: A list of numbers (a vector) that represents the meaning of a text: two texts with similar meanings have nearby vectors.

• Wrong Putting a video or widget into a web page. No: that's what "embed" means in web development, but in AI the term means something else.
• Wrong An encryption method that protects data sent to the model. No: an embedding encrypts nothing. It doesn't protect data and shouldn't be treated as a form of anonymization.
• Right A list of numbers (a vector) that represents the meaning of a text: two texts with similar meanings have nearby vectors. Correct. That's what makes search by meaning possible, rather than by keywords: you find a passage about "time off" when searching for "vacation". It's the basis of many RAG systems.
• Wrong A summary of the text written by the AI. No: an embedding isn't human-readable. It's a series of numbers, not a sentence.

Glossary entry: Embedding

Right answer: It's an open standard for connecting AI applications to tools and data sources (files, software, databases) in a common way.

• Wrong It's a language model specialized in code. No: MCP isn't a model. It's a standard way of connecting tools to AI applications.
• Right It's an open standard for connecting AI applications to tools and data sources (files, software, databases) in a common way. Correct. Released by Anthropic in late 2024 as an open standard, it lets you write a connector (an "MCP server") once and use it in every compatible application. Note: each connector grants access, so choose them carefully.
• Wrong It's a European regulation governing AI. No: the European law governing AI is the AI Act. MCP is a technical protocol.
• Wrong It's a benchmark that measures models' memory. No: MCP measures nothing. It defines how an AI application talks to tools.

Glossary entry: Model Context Protocol (MCP)

Right answer: The model writes a structured request (tool name and parameters); the application runs it and sends back the result, which the model uses to continue.

• Wrong The model runs code on the internet by itself. No: the model only produces text. It's the program around it that runs (or doesn't run) the action.
• Wrong A person takes over to carry out the action. No: tool calling is automatic. Human approval can be added, but that's not what defines the mechanism.
• Wrong The model downloads a new version of itself to gain the skill. No: the model doesn't change. The available tools are described to it in the request.
• Right The model writes a structured request (tool name and parameters); the application runs it and sends back the result, which the model uses to continue. Correct. The application decides whether to run the action, so that's where the safeguards go (permissions, approval, limits). It's the basic mechanism behind agents.

Glossary entry: Tool calling (function calling)