AnxForever commited on
Commit
24e888e
·
verified ·
1 Parent(s): f9727c4

Add id2label mapping + temperature scaling docs + tags polish

Browse files
Files changed (2) hide show
  1. README.md +10 -3
  2. config.json +8 -0
README.md CHANGED
@@ -9,6 +9,8 @@ tags:
9
  - chinese
10
  - bert
11
  - text-classification
 
 
12
  - academic
13
  base_model: bert-base-chinese
14
  metrics:
@@ -146,6 +148,7 @@ from transformers import AutoTokenizer, AutoModelForSequenceClassification
146
  import torch
147
 
148
  MODEL_ID = "AnxForever/chinese-ai-detector-bert"
 
149
 
150
  tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
151
  model = AutoModelForSequenceClassification.from_pretrained(MODEL_ID)
@@ -156,17 +159,21 @@ inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=256)
156
 
157
  with torch.no_grad():
158
  logits = model(**inputs).logits
159
- probs = torch.softmax(logits, dim=-1)[0]
 
160
 
161
- labels = ["human-written", "AI-generated"]
162
  pred_idx = int(probs.argmax())
163
- print(f"{labels[pred_idx]} (confidence: {probs[pred_idx].item():.2%})")
 
164
  ```
165
 
166
  ### Label mapping
167
  - `0` → human-written (人类撰写)
168
  - `1` → AI-generated (AI 生成)
169
 
 
 
 
170
  ---
171
 
172
  ## 🎯 技术创新 / Contributions
 
9
  - chinese
10
  - bert
11
  - text-classification
12
+ - binary-classification
13
+ - thesis
14
  - academic
15
  base_model: bert-base-chinese
16
  metrics:
 
148
  import torch
149
 
150
  MODEL_ID = "AnxForever/chinese-ai-detector-bert"
151
+ TEMPERATURE = 0.8165 # Temperature scaling, calibrated on 910 samples (ECE=0.0034)
152
 
153
  tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
154
  model = AutoModelForSequenceClassification.from_pretrained(MODEL_ID)
 
159
 
160
  with torch.no_grad():
161
  logits = model(**inputs).logits
162
+ # Apply temperature scaling for calibrated confidence
163
+ probs = torch.softmax(logits / TEMPERATURE, dim=-1)[0]
164
 
 
165
  pred_idx = int(probs.argmax())
166
+ label = model.config.id2label[pred_idx] # "human-written" or "AI-generated"
167
+ print(f"{label} (confidence: {probs[pred_idx].item():.2%})")
168
  ```
169
 
170
  ### Label mapping
171
  - `0` → human-written (人类撰写)
172
  - `1` → AI-generated (AI 生成)
173
 
174
+ > **Note on Temperature Scaling**: `T = 0.8165` was calibrated on a held-out 910-sample set
175
+ > and brings ECE from 0.0121 down to **0.0034**. For uncalibrated probabilities, set `TEMPERATURE = 1.0`.
176
+
177
  ---
178
 
179
  ## 🎯 技术创新 / Contributions
config.json CHANGED
@@ -9,8 +9,16 @@
9
  "hidden_act": "gelu",
10
  "hidden_dropout_prob": 0.1,
11
  "hidden_size": 768,
 
 
 
 
12
  "initializer_range": 0.02,
13
  "intermediate_size": 3072,
 
 
 
 
14
  "layer_norm_eps": 1e-12,
15
  "max_position_embeddings": 512,
16
  "model_type": "bert",
 
9
  "hidden_act": "gelu",
10
  "hidden_dropout_prob": 0.1,
11
  "hidden_size": 768,
12
+ "id2label": {
13
+ "0": "human-written",
14
+ "1": "AI-generated"
15
+ },
16
  "initializer_range": 0.02,
17
  "intermediate_size": 3072,
18
+ "label2id": {
19
+ "AI-generated": 1,
20
+ "human-written": 0
21
+ },
22
  "layer_norm_eps": 1e-12,
23
  "max_position_embeddings": 512,
24
  "model_type": "bert",