Spaces:
Running on Zero
Running on Zero
Upload examples.py
Browse files- examples.py +96 -0
examples.py
ADDED
|
@@ -0,0 +1,96 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Worked examples shown under the interface: one with an error, one without.
|
| 2 |
+
|
| 3 |
+
The first shows the check finding an error, the second shows it raising no
|
| 4 |
+
false alarm. Each button says where its answer came from, because the field it
|
| 5 |
+
fills is labelled "The AI's answer" — a visitor reads anything placed there as
|
| 6 |
+
something an AI wrote, and a note in this file would never reach them.
|
| 7 |
+
|
| 8 |
+
The texts are written as adjacent string literals inside parentheses. Python
|
| 9 |
+
joins them with nothing in between, so these lines can be rewrapped freely
|
| 10 |
+
without changing a character. Inside triple quotes that is not true: every line
|
| 11 |
+
break becomes part of the text, and would change what the detectors read.
|
| 12 |
+
|
| 13 |
+
1. FIRE — the answer contains an error. From HaluEval (MIT licence,
|
| 14 |
+
https://github.com/RUCAIBox/HaluEval), summarization subset, test split.
|
| 15 |
+
The article is CNN reporting, carried into HaluEval through the CNN/Daily Mail
|
| 16 |
+
dataset. It is quoted here as a short excerpt to demonstrate the tool;
|
| 17 |
+
copyright in the article remains with its publisher.
|
| 18 |
+
⚠️ The answer is NOT a model's natural mistake. HaluEval produced its
|
| 19 |
+
hallucinated summaries deliberately: ChatGPT was instructed to write one, and
|
| 20 |
+
the most plausible were kept. The button label says so.
|
| 21 |
+
|
| 22 |
+
2. RASPBERRY PI — the answer has no errors. Wikipedia lead section, CC BY-SA 4.0,
|
| 23 |
+
https://en.wikipedia.org/w/index.php?title=Raspberry_Pi&oldid=1373778583
|
| 24 |
+
The answer is Gemini 3.6 Flash's, given the instruction "Answer the question
|
| 25 |
+
using only the article below."
|
| 26 |
+
"""
|
| 27 |
+
|
| 28 |
+
# The article says the authorities "didn't know what had caused the fire"; the
|
| 29 |
+
# answer names a cause. The judge and entailment both flag it; text similarity
|
| 30 |
+
# calls it grounded, which is its 36% accuracy in action.
|
| 31 |
+
FIRE_ARTICLE = (
|
| 32 |
+
"A mammoth fire broke out Friday morning in a Kentucky industrial park, "
|
| 33 |
+
"sending plumes of thick smoke over the area as authorities worked to "
|
| 34 |
+
"contain the damage. The blaze began shortly before 7 a.m. at the "
|
| 35 |
+
"General Electric Appliance Park in Louisville, according to Mike Weimer "
|
| 36 |
+
"from the city's emergency management agency. He said that there were no "
|
| 37 |
+
"reports of anyone injured or trapped. Video showed both smoke and "
|
| 38 |
+
"bright orange flames. Firefighters took up positions around the "
|
| 39 |
+
"affected buildings, spraying water from the periphery. Weimer told CNN "
|
| 40 |
+
"that authorities didn't know what had caused the fire, which had gone "
|
| 41 |
+
"to at least four alarms. According to a GE website, its facility in the "
|
| 42 |
+
"Louisville Appliance Park is \"revitalizing manufacturing in the United "
|
| 43 |
+
"States.\" The park is large, such that 34 football fields could fit in "
|
| 44 |
+
"one of its warehouses in the facility."
|
| 45 |
+
)
|
| 46 |
+
|
| 47 |
+
FIRE_ANSWER = (
|
| 48 |
+
"The fire in Louisville's General Electric Appliance Park was caused by "
|
| 49 |
+
"a malfunctioning robot, which caused significant damage but no "
|
| 50 |
+
"injuries. The park's large warehouses were severely impacted, but "
|
| 51 |
+
"authorities are confident that production will continue as usual."
|
| 52 |
+
)
|
| 53 |
+
|
| 54 |
+
# Every claim is supported, so the screen shows what "nothing wrong" looks like.
|
| 55 |
+
RASPBERRY_PI_ARTICLE = (
|
| 56 |
+
"Raspberry Pi ( PY) is a series of small single-board computers (SBCs) "
|
| 57 |
+
"originally developed in the United Kingdom by the Raspberry Pi "
|
| 58 |
+
"Foundation in collaboration with Broadcom. To commercialize the product "
|
| 59 |
+
"and support its growing demand, the Foundation established a commercial "
|
| 60 |
+
"entity, now known as Raspberry Pi Holdings.\n"
|
| 61 |
+
"The Raspberry Pi was originally created to help teach computer science "
|
| 62 |
+
"in schools, but gained popularity for many other uses for its low cost, "
|
| 63 |
+
"high performance, compact size, and flexibility. It is now used in "
|
| 64 |
+
"areas such as industrial automation, robotics, home automation, IoT "
|
| 65 |
+
"devices, and hobbyist projects.\n"
|
| 66 |
+
"The company's products range from simple microcontrollers to computers "
|
| 67 |
+
"that the company markets as being powerful enough to be used as a "
|
| 68 |
+
"general purpose PC. Computers are built around a custom designed system "
|
| 69 |
+
"on a chip and offer features such as HDMI video/audio output, USB "
|
| 70 |
+
"ports, wireless networking, GPIO pins, and up to 16 GB of RAM. Storage "
|
| 71 |
+
"is typically provided via microSD cards.\n"
|
| 72 |
+
"In 2015, the Raspberry Pi surpassed the ZX Spectrum as the best-selling "
|
| 73 |
+
"British computer of all time. As of March 2026, over 75 million units "
|
| 74 |
+
"had been sold."
|
| 75 |
+
)
|
| 76 |
+
|
| 77 |
+
RASPBERRY_PI_ANSWER = (
|
| 78 |
+
"Based on the provided article:\n"
|
| 79 |
+
"\n"
|
| 80 |
+
"Original Purpose: The Raspberry Pi was originally created to help teach "
|
| 81 |
+
"computer science in schools.\n"
|
| 82 |
+
"\n"
|
| 83 |
+
"Units Sold: Over 75 million units had been sold as of March 2026."
|
| 84 |
+
)
|
| 85 |
+
|
| 86 |
+
EXAMPLES = [
|
| 87 |
+
[FIRE_ARTICLE, FIRE_ANSWER],
|
| 88 |
+
[RASPBERRY_PI_ARTICLE, RASPBERRY_PI_ANSWER],
|
| 89 |
+
]
|
| 90 |
+
|
| 91 |
+
# Shown on the buttons instead of the full text. No case identifiers: they mean
|
| 92 |
+
# nothing to a visitor, and the provenance above is where they belong.
|
| 93 |
+
EXAMPLE_LABELS = [
|
| 94 |
+
"Fire at a GE plant — test answer, made-up cause",
|
| 95 |
+
"Raspberry Pi — a real AI answer with no errors",
|
| 96 |
+
]
|