Evkoff commited on
Commit
539053b
·
verified ·
1 Parent(s): faa4ebf

Upload examples.py

Browse files
Files changed (1) hide show
  1. examples.py +96 -0
examples.py ADDED
@@ -0,0 +1,96 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Worked examples shown under the interface: one with an error, one without.
2
+
3
+ The first shows the check finding an error, the second shows it raising no
4
+ false alarm. Each button says where its answer came from, because the field it
5
+ fills is labelled "The AI's answer" — a visitor reads anything placed there as
6
+ something an AI wrote, and a note in this file would never reach them.
7
+
8
+ The texts are written as adjacent string literals inside parentheses. Python
9
+ joins them with nothing in between, so these lines can be rewrapped freely
10
+ without changing a character. Inside triple quotes that is not true: every line
11
+ break becomes part of the text, and would change what the detectors read.
12
+
13
+ 1. FIRE — the answer contains an error. From HaluEval (MIT licence,
14
+ https://github.com/RUCAIBox/HaluEval), summarization subset, test split.
15
+ The article is CNN reporting, carried into HaluEval through the CNN/Daily Mail
16
+ dataset. It is quoted here as a short excerpt to demonstrate the tool;
17
+ copyright in the article remains with its publisher.
18
+ ⚠️ The answer is NOT a model's natural mistake. HaluEval produced its
19
+ hallucinated summaries deliberately: ChatGPT was instructed to write one, and
20
+ the most plausible were kept. The button label says so.
21
+
22
+ 2. RASPBERRY PI — the answer has no errors. Wikipedia lead section, CC BY-SA 4.0,
23
+ https://en.wikipedia.org/w/index.php?title=Raspberry_Pi&oldid=1373778583
24
+ The answer is Gemini 3.6 Flash's, given the instruction "Answer the question
25
+ using only the article below."
26
+ """
27
+
28
+ # The article says the authorities "didn't know what had caused the fire"; the
29
+ # answer names a cause. The judge and entailment both flag it; text similarity
30
+ # calls it grounded, which is its 36% accuracy in action.
31
+ FIRE_ARTICLE = (
32
+ "A mammoth fire broke out Friday morning in a Kentucky industrial park, "
33
+ "sending plumes of thick smoke over the area as authorities worked to "
34
+ "contain the damage. The blaze began shortly before 7 a.m. at the "
35
+ "General Electric Appliance Park in Louisville, according to Mike Weimer "
36
+ "from the city's emergency management agency. He said that there were no "
37
+ "reports of anyone injured or trapped. Video showed both smoke and "
38
+ "bright orange flames. Firefighters took up positions around the "
39
+ "affected buildings, spraying water from the periphery. Weimer told CNN "
40
+ "that authorities didn't know what had caused the fire, which had gone "
41
+ "to at least four alarms. According to a GE website, its facility in the "
42
+ "Louisville Appliance Park is \"revitalizing manufacturing in the United "
43
+ "States.\" The park is large, such that 34 football fields could fit in "
44
+ "one of its warehouses in the facility."
45
+ )
46
+
47
+ FIRE_ANSWER = (
48
+ "The fire in Louisville's General Electric Appliance Park was caused by "
49
+ "a malfunctioning robot, which caused significant damage but no "
50
+ "injuries. The park's large warehouses were severely impacted, but "
51
+ "authorities are confident that production will continue as usual."
52
+ )
53
+
54
+ # Every claim is supported, so the screen shows what "nothing wrong" looks like.
55
+ RASPBERRY_PI_ARTICLE = (
56
+ "Raspberry Pi ( PY) is a series of small single-board computers (SBCs) "
57
+ "originally developed in the United Kingdom by the Raspberry Pi "
58
+ "Foundation in collaboration with Broadcom. To commercialize the product "
59
+ "and support its growing demand, the Foundation established a commercial "
60
+ "entity, now known as Raspberry Pi Holdings.\n"
61
+ "The Raspberry Pi was originally created to help teach computer science "
62
+ "in schools, but gained popularity for many other uses for its low cost, "
63
+ "high performance, compact size, and flexibility. It is now used in "
64
+ "areas such as industrial automation, robotics, home automation, IoT "
65
+ "devices, and hobbyist projects.\n"
66
+ "The company's products range from simple microcontrollers to computers "
67
+ "that the company markets as being powerful enough to be used as a "
68
+ "general purpose PC. Computers are built around a custom designed system "
69
+ "on a chip and offer features such as HDMI video/audio output, USB "
70
+ "ports, wireless networking, GPIO pins, and up to 16 GB of RAM. Storage "
71
+ "is typically provided via microSD cards.\n"
72
+ "In 2015, the Raspberry Pi surpassed the ZX Spectrum as the best-selling "
73
+ "British computer of all time. As of March 2026, over 75 million units "
74
+ "had been sold."
75
+ )
76
+
77
+ RASPBERRY_PI_ANSWER = (
78
+ "Based on the provided article:\n"
79
+ "\n"
80
+ "Original Purpose: The Raspberry Pi was originally created to help teach "
81
+ "computer science in schools.\n"
82
+ "\n"
83
+ "Units Sold: Over 75 million units had been sold as of March 2026."
84
+ )
85
+
86
+ EXAMPLES = [
87
+ [FIRE_ARTICLE, FIRE_ANSWER],
88
+ [RASPBERRY_PI_ARTICLE, RASPBERRY_PI_ANSWER],
89
+ ]
90
+
91
+ # Shown on the buttons instead of the full text. No case identifiers: they mean
92
+ # nothing to a visitor, and the provenance above is where they belong.
93
+ EXAMPLE_LABELS = [
94
+ "Fire at a GE plant — test answer, made-up cause",
95
+ "Raspberry Pi — a real AI answer with no errors",
96
+ ]