Human in the loop: seven ways AI output fails engineers
Published
What the tool produced
A technician photographs a table from a standard. He asks the tool for the value in one row. The tool answers with a clear number, without hesitation.
A refereed benchmark tested nine language models on reading tables from building standards. In two tasks the accuracy was 45.27 and 40.65 percent. The same tables, fed as text, gave 98.94 and 87.27 percent.
The tool knew the content. It failed to read it from the image. This text goes through the seven failure types and the one check that catches them.
Why it looked right
The number arrives in a confident tone, in the right units. Nothing in it signals an error. This is common to all seven failure types. The output looks valid even when it is wrong.
The tool is also not weak. One tool passed two national licensing exams for mechanical engineers in buildings, in all five attempts. The average scores were 80.6 and 81.25 out of 100. The gap is not in ability.
The gap is that no step in the workflow stops the output before the signature. Engineering already has such steps. An inspector signs before a drawing is released. A pressure test comes before handover. AI output has no equivalent step.
Seven failure types
Each single mistake belongs to a pattern. That is how it can be taught, and how it can be counted.
- G1, table from an image: wrong values. Unit tables and data sheets mostly arrive as images.
- G2, analog gauge: the digits are read well, the needle position is read wrong. The reading enters a signed report.
- G3, standard clause: a clause that looks real and does not exist. This is how a requirement nobody wrote is born.
- G4, fault code: the same code gets the opposite meaning at another manufacturer.
- G5, direct calculation: an error in the formula, the arithmetic or the unit conversion.
- G6, silent completion: a part number, a date or an action that was never given is filled in with full confidence.
- G7, right-wrong explanation: a correct result with faulty reasoning.
One example shows the structure. An air conditioning unit shows code E3. At manufacturer A it means a high pressure switch tripped. At manufacturer B it means low pressure. At manufacturer C it means a remote controller communication error.
The same code, opposite meanings. A tool that answers from memory, without the manual for the model, cannot show which table it used.
How the engineer catches it
The axiom is short: a number that was not verified does not exist. Until it is checked, it is a draft. It does not enter a signed document, a submitted document or a decision.
Check every item against an authoritative source. An authoritative source is a manufacturer manual, a standard in its official wording, a signed drawing or a direct measurement. A tool is never an authoritative source. Output from a second tool is not one either.
Three rules set the order of work. The source comes before memory: a meaning taken from the tool's memory is not acceptable, even when it is correct. The tool drafts and does not calculate: run the arithmetic in a spreadsheet. The documentation is part of the deliverable: a deliverable without a check log is not a deliverable.
The check does not rely on suspicion. You cannot suspect every line of every report. It runs on every item and leaves a record.
The check to run
The process has three steps and one gate.
- Before you open the tool, write your own estimate. An estimate written after the answer is anchored to it.
- Ask the tool with a structured request, with the source attached and with the instruction: do not invent, write missing.
- At the gate, check every item against a source and record one of four results: match, gap, not verified, missing.
An independent estimate also catches G7. A correct answer with faulty reasoning passes a source check, and is exposed only against your own reasoning.
A gap is recorded with one of the seven codes. One unverified item holds the whole deliverable as a draft. There is no partial verification.
Every task gets a use level before the work starts. A task that was not tagged counts as level zero, no tool, until someone tags it otherwise. The level is set by the consequence of an error, not by the difficulty. A simple calculation that enters a signed inspection report is level zero.
The working paper behind the classification has not been peer reviewed. Its evidence measures the failures, not the method.
Self-check
- Was every number in the report checked against a manufacturer manual or an official standard?
- Was a photographed table compared with the table as text or with the original document?
- Does every gap carry one of the seven codes?
- Was an independent estimate written before the tool was opened?
- Did the calculation run in a spreadsheet and not inside the tool?
- Did the request to the tool include the instruction: do not invent, write missing?
- Was a task that was not tagged treated as level zero?
- Is there a single unverified item in the deliverable about to go out?
Details of the course where this check is practiced: AI for maintenance management.
Sources: Liang, C. et al. (2026). AECBench: A Hierarchical Benchmark for Knowledge Evaluation of Large Language Models in the AEC Field. Advanced Engineering Informatics, 71: 104314. doi 10.1016/j.aei.2026.104314; Choi, H.; Lee, J.; and Kim, J. (2025). Performance Evaluation of GPT-4o on South Korean National Exams for Building Mechanical Equipment Maintenance. Scientific Reports, 15: 30436. doi 10.1038/s41598-025-16118-x