RthItalia commited on
Commit
08bee4c
·
verified ·
1 Parent(s): 652680a

Add real test methodology document

Browse files
Files changed (2) hide show
  1. README.md +1 -0
  2. reports/TEST_METHOD.md +38 -0
README.md CHANGED
@@ -62,6 +62,7 @@ Report reali eseguiti su corpus AIO locale (101k+ records, fonti pubbliche reali
62
  Artifact pubblicati:
63
  - `reports/REAL_CASE_TEST_RESULTS_LATEST.md`
64
  - `reports/REAL_CASE_TEST_RESULTS_LATEST.json`
 
65
 
66
  Scope case use testati:
67
  - clima mediterraneo ed energia
 
62
  Artifact pubblicati:
63
  - `reports/REAL_CASE_TEST_RESULTS_LATEST.md`
64
  - `reports/REAL_CASE_TEST_RESULTS_LATEST.json`
65
+ - `reports/TEST_METHOD.md`
66
 
67
  Scope case use testati:
68
  - clima mediterraneo ed energia
reports/TEST_METHOD.md ADDED
@@ -0,0 +1,38 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # AIO Real Test Method
2
+
3
+ Generated on: 2026-02-21
4
+
5
+ ## Goal
6
+ Publish real multi-case retrieval tests on the full local AIO corpus, then expose only results in the public showcase.
7
+
8
+ ## Data Scope
9
+ - Corpus file: `aio_archive/records.json`
10
+ - Corpus size at test time: `101257` records
11
+ - Unique sources at test time: `1867`
12
+ - Source family: public knowledge files already ingested locally (wiki-based IT/EN and related public text sources).
13
+
14
+ ## Engine Configuration
15
+ - Runtime: `python app_oracolo.py` (CLI mode)
16
+ - Embedding backend used in test: `sentence-transformers(local-path): aio_models/paraphrase-multilingual-MiniLM-L12-v2`
17
+ - Retrieval path: AIO native hybrid retrieval (vector + lexical overlap + domain coherence bonus).
18
+
19
+ ## Test Procedure
20
+ 1. Internal benchmark run:
21
+ - command: `AIO_CLI_BENCH=1 python app_oracolo.py`
22
+ 2. Real use-case runs (8 cases):
23
+ - command per case: `AIO_CLI_TEST_QUERY=1 AIO_CLI_QUERY="<query>" python app_oracolo.py`
24
+ 3. Parse output:
25
+ - `hits`, `score`, `domain`, `source`
26
+ 4. Case pass criteria:
27
+ - minimum hits threshold (case-specific),
28
+ - top score threshold,
29
+ - at least one expected term matched in retrieved context.
30
+
31
+ ## Published Artifacts
32
+ - `REAL_CASE_TEST_RESULTS_LATEST.json`
33
+ - `REAL_CASE_TEST_RESULTS_LATEST.md`
34
+ - timestamped copies (`REAL_CASE_TEST_RESULTS_20260221_222608.*`)
35
+
36
+ ## Licensing Note
37
+ - Free for use and study.
38
+ - Commercial use requires written authorization: `info@rthitalia.com`