Error-hunt quiz: find what needs checking in an AI output

Six outputs from an AI tool, each with one thing you cannot sign without a check. Find it, and which source you check it against before you sign.

The outputs here are examples written for the quiz. None of them is a real case, and none names a maker, a client or a person.

How it works

In each item you pick one answer and press check. After the check you see the correct answer, one sentence on why, and the type, from G1 to G7. After the sixth item you get a score out of 6.

Item 1 of 6Example

What the tool was given

A photo of Maker A's data table was uploaded to the tool, with the question: what is the flow on the row of model P-40?

What the tool answered

The maximum flow of model P-40 is 38 m³/h.

What needs checking here, and against which source?

The number is too round to be real. Check the plant's calibration log.

There is nothing to check. The tool answered with a number and the right unit.

The value was read from a photo and may come from a neighbouring row. Check the maker's data sheet as text, on the model's row.

The tool did not cite a standard. Check the piping standard.

Pick an answer to check it.

Item 2 of 6Example

What the tool was given

A photo of a pressure gauge on a feed line was uploaded to the tool. The scale on the gauge is 0 to 10 bar.

What the tool answered

The reading is 6.5 bar.

What needs checking here, and against which source?

The digits on the scale are read well, but the needle position is sometimes misread. Check by re-reading the gauge against its own scale, or against a calibrated gauge.

A scale of 0 to 10 bar is not common. Check the catalogue of gauge makers.

The reading is plausible for a feed line, so it can be recorded.

The bar is not a unit for a pressure gauge. Check the company procedure.

Pick an answer to check it.

Item 3 of 6Example

What the tool was given

An engineer asked the tool how often a repeat pressure test is required for a pressure vessel.

What the tool answered

Under clause 7.4.2 of the test standard, a repeat pressure test is required every 18 months.

What needs checking here, and against which source?

18 months is too long an interval. Check the plant's certification binder.

The clause number is written with two decimal points. Check the standards drafting guide.

The clause is real, because the clause numbering matches the structure of standards.

The clause and the requirement may be invented, in a structure that looks credible. Check the binding text of the standard, at the clause number and the page.

Pick an answer to check it.

Item 4 of 6Example

What the tool was given

An air-conditioning unit from Maker B shows code E3. The tool was asked what the code means.

What the tool answered

E3 is high pressure. Check the compressor and the condenser.

What needs checking here, and against which source?

The recommendation is too long. Check the plant's maintenance procedure.

The meaning was given from memory, and the same code means different things, even opposite things, at different makers. Check the fault-code table in Maker B's service manual, for the model in the field.

E3 is a common code, so the meaning is right. Check against the team's earlier experience.

A fault code cannot be read without a picture of the display. Check a photo of the screen.

Pick an answer to check it.

Item 5 of 6Example

What the tool was given

The tool was asked to convert a flow of 12 litres per second to cubic metres per hour, and the conversion was done inside the answer.

What the tool answered

12 litres per second is 4.32 m³/h.

What needs checking here, and against which source?

m³/h is not a suitable unit for flow. Check the pump data sheet.

The tool did not show the formula. Asking for it is enough, and then the result can be trusted.

The calculation was done inside the model, and the conversion factor is wrong. Check by recalculating in a spreadsheet or by hand, against a unit table.

There is no error. A cubic metre per hour is a small unit, so the number is small.

Pick an answer to check it.

Item 6 of 6Example

What the tool was given

The tool received a technician's note, "Pump P-2 vibrating. Coupling replaced.", and was asked to draft a service report.

What the tool answered

Coupling replaced, part no. 4471-B, on 12 March. Tightening torque checked.

What needs checking here, and against which source?

The part number, the date and the torque check were not given to the tool, and it filled them in with full confidence. Check the signed service sheet and the maker's parts list.

The report is too short. Check the plant's report template.

There is no error. A report more detailed than the note is a better report.

The date has no year. Check the shift log.

Pick an answer to check it.

Which failure types the six items cover

The seven failure types G1 to G7 are defined in the published document. The six items cover six of them, one item per type:

G7, Right answer - wrong reasoning, is not asked here. A correct answer with wrong reasoning passes a check against a source, and shows only against reasoning you wrote yourself before you opened the tool. A multiple-choice quiz cannot test that.

Error hunt: find what needs checking in an AI output

How it works

Item 1 of 6Example

Item 2 of 6Example

Item 3 of 6Example

Item 4 of 6Example

Item 5 of 6Example

Item 6 of 6Example

Which failure types the six items cover