ChatGPT makes things up: a standard clause that is not there
Published
What the tool produced
An engineer asks an AI tool for the legal inspection frequency of a component. The tool answers with a clause number, the wording of a requirement and a frequency. It all looks like a standard.
The clause does not exist. This is the failure people describe as "ChatGPT makes things up". The claim is true, but too general. It is worth knowing which invention is dangerous in engineering work.
A construction lawyer calls it a "ghost standard". These are standard designations that follow the real numbering convention, and do not exist.
Why it looked right
The structure is credible. The numbering matches the real standard. The wording is formal. Nothing in the output separates a real clause from an invented one.
So such a clause passes the document review. It is found later, when a testing lab cannot find the method the clause describes. The damage then is a change order or a claim.
The engineering consequence is clear: a legal inspection frequency or a safety requirement that does not exist. People write a procedure from it, and the procedure repeats in every inspection done by it. This is how a recurring fault is born from a wording.
The word "makes up" covers several different failures, and each has its own check.
- A standard clause that does not exist, G3. Check in the official standard.
- A fault code read from memory, G4. Check in the model manual.
- A value silently filled in, G6. Ask: write missing, and compare to the record.
Asking again does not expose the invention. In a study on licensing exams, the answer consistency of one tool was 97 percent. A wrong answer repeated itself.
How the engineer catches it
The rule is simple: the source comes before memory. Open the standard in its official wording and look for the clause. If it is found, copy the wording from it.
A meaning taken from the tool's memory is not acceptable, even when it is correct. Output from a second tool is not an authoritative source either, so there is no point asking one tool to verify another. Without such a rule, it is unclear which correct answer is luck and which is a source.
There is also a close failure, silent completion. The tool fills in a part number, a date or a model that it was not given. The instruction that catches it is short: do not invent, write missing.
The check to run
For every clause the tool cites, take four steps.
- Find the standard in its official wording, not in a summary and not in an unofficial copy.
- Look for the clause number and compare the wording.
- Record in the log the name of the authoritative document and the location in it: page, clause or table.
- Record one of four results: match, gap, not verified, missing.
A clause that is not found is blocked. It is marked not verified, and does not enter the deliverable. If the deliverable depends on it, the deliverable stays a draft.
Note the order of work. An estimate made after you saw the answer is anchored to it. So first write what you expect to find, and only then turn to the tool. This step takes five to twenty minutes. Its purpose is a documented independent judgment, not an accurate one. An estimate that was not written does not count. Keeping to the order separates a real check from the feeling that we checked.
The request to the tool has six parts: role, context, task, constraints, source and the missing-data instruction. A request without constraints and without the missing-data instruction is not a request by the method.
Who is already using the tool
Workers in plants already use these tools, and the output looks right even when it is not. The management question is not whether use is forbidden. The question is who checks the answer before the signature.
An auditor asks who approved the clause and on what basis. A log with the document name and the location is the documented answer.
The gate is the primary control for six of the seven failure types. G3 is one of them.
What the check does not do
The check does not detect hallucinations. It does not tune prompts and does not rank tools. It is a work procedure that stands between the output and the signature.
It also does not make the output correct. It does one thing, and it can be counted: every item in the deliverable is linked to an authoritative source, or blocked. A clause the tool wrote as missing is a valid result, and not a failure.
Self-check
- Was every standard clause in the procedure found in the standard in its official wording?
- Does the log record the document name and the location of the clause in it?
- Did the request to the tool include the instruction: do not invent, write missing?
- Was a clause that was not found marked not verified?
- Was an independent estimate written before the tool was opened?
- Was an item the tool marked as missing handled as a valid result?
- Did anyone ask one tool to verify the output of another?
- Is there an inspection frequency in the procedure that has no source?
For a broader understanding of safe use of these tools: Safe AI for everyone.