LafouCC/PG-Personalization-ckpt

PG-Personalization — checkpoints and eval artifacts

A parameter generator (PG): one shared hypernetwork that maps a user's history to a per-user LoRA on a frozen Qwen2.5-7B-Instruct in a single forward pass. No per-user fine-tuning, no test-time optimization.

Code: https://github.com/…/PG-Personalization. Everything here is produced by, and consumed by, the scripts in that repo.

What is here

path	what	size
p2p/t9_v5o2/	the settled arm. PG trained on the 10-task P2P corpus, CE + gap loss	3.1 GB
p2p/t9_v5o3/	same arm with the gap loss off (--gap_lambda 0) — the loss ablation	3.1 GB
p2p/shared_lora_r8/	G, the shared task adapter the PG is a residual on top of	21 MB
p2p/shared_lora_r8_ctrl/	G-ctrl, the rank-matched SFT baseline	9 MB
amazon2023/t10_a23_v5o3/	PG trained on Amazon-2023 review episodes only (Stage A)	3.1 GB
amazon2023/shared_lora_a23_r8/	that arm's G, refit on the same corpus	9 MB
eval/	every artifact behind the reported numbers — predictions, transfer screens, geometry reports, routing probes	0.4 GB

Only the best checkpoint of each run is published. Intermediate pg_step*.pt, pg_last.pt and the train_state.pt optimizer states (10 GB each, resume-only) are not.

Results

P2P random_test / ood_test, macro over the 10 tasks:

arm	acc	macro-F1	ROUGE-L
t9_v5o2	0.7377 / 0.7003	0.6721 / 0.6109	0.2667 / 0.2656
t9_v5o3 (no gap loss)	0.7137 / 0.6514	0.6403 / 0.5531	0.2686 / 0.2702

Geometry for t9_v5o2: effective rank 6.004 of 8, cross-user cosine 0.572. Against a user-shuffled condition it gains +0.139 accuracy on random.

Amazon-2023 review generation, t10_a23_v5o3, all 3,410 test users:

arm	ROUGE-1	ROUGE-L	METEOR	BLEU
self	0.3168	0.1592	0.1910	0.0190
user_shuffle	0.2851	0.1431	0.1693	0.0118
nocross	0.2874	0.1435	0.1632	0.0115
base+G	0.2853	0.1446	0.1700	0.0129

Three independent ways of deleting this user's condition land within 0.0015 ROUGE-L of each other, and only the real condition moves the number.

Re-inference

The PG runs on a backbone with G merged into the weights. That merged base is ~15 GB and fully reproducible, so it is not published — rebuild it from the stock backbone plus the 10 MB adapter here.

hf download <you>/PG-Personalization-ckpt --local-dir ckpt_hf
hf download Qwen/Qwen2.5-7B-Instruct --local-dir models/Qwen2.5-7B-Instruct

# G -> merged base. merge_id must come out af208c939624f71f56ee9dbc8b47c0e4.
python scripts/merge_shared_lora.py \
  --adapter ckpt_hf/p2p/shared_lora_r8/best \
  --base_model models/Qwen2.5-7B-Instruct \
  --output models/Qwen2.5-7B-Instruct-sharedG-r8

PG_ARCH=v5s torchrun --standalone --nproc_per_node=8 scripts/eval_pg.py \
  --data_root data_p13n/P2P_data \
  --base_model models/Qwen2.5-7B-Instruct-sharedG-r8 \
  --pg_config ckpt_hf/p2p/t9_v5o2/pg_config.json \
  --pg_ckpt   ckpt_hf/p2p/t9_v5o2/pg_best_step16000.pt \
  --eval_split random_test --condition_source raw_hist \
  --max_prompt_len 1024 --max_condition_tokens 4096 --max_new_tokens 1024 \
  --no_repeat_ngram_size 3 \
  --rep_control_tasks longlamp_topic_writing,longlamp_product_review,longlamp_abstract_generation \
  --output preds_random_test.json

Three things that will silently give wrong numbers if you skip them:

1. Use the pg_config.json shipped next to the checkpoint, not one from configs/. PG_ARCH=v5s selects the trunk but not the arm; the two keys that make it v5o2 (decode_head: per_site, self_attn_rope/cross_attn_rope false) change no parameter, so a mismatched config loads without error.
2. Check the merge_id. Each train_meta.json records the base_merge_id its PG was trained against, and each adapter ships the merge_provenance.json of the base built from it. If they disagree, the PG is being scored on the wrong weights. t9_v5o2 and t9_v5o3 both expect af208c93….
3. The P2P corpus is not the public one. longlamp_product_review's history was rebuilt to carry the user's review body instead of the review title; see the dataset repo. Training or scoring on Zhaoxuan/P2P_data will not reproduce these numbers.

For the Amazon arm, swap in amazon2023/shared_lora_a23_r8/best (expects merge_id from its own merge_provenance.json), amazon2023/t10_a23_v5o3/, and scripts/drift_test/eval_pg_temporal.py.

share_ce.pt under p2p/shared_lora_r8/ is the per-row CE of base+G that the gap loss is scored against. It is needed only to retrain, not to infer, and it is indexed by the sampler's global row index — so it is valid only for the exact dataset flags of the original run.

Layout

p2p/
  t9_v5o2/            pg_best_step16000.pt  pg_config.json  train_meta.json
  t9_v5o3/            pg_best_step15000.pt  pg_config.json  train_meta.json
  shared_lora_r8/     best/  best.json  shared_lora.pt  share_ce.pt
                      train_args.json  merge_provenance.json
  shared_lora_r8_ctrl/ best/  best.json  train_args.json  val_history.json
                      merge_provenance.json
amazon2023/
  t10_a23_v5o3/       pg_best_step10000.pt  pg_config.json  train_meta.json
  shared_lora_a23_r8/ best/  best.json  train_args.json  val_history.json
                      merge_provenance.json
eval/
  p2p/t9_v5o2/        preds_{random,ood}_test.json      task metrics, self/shuffle/base
                      transfer_{random,ood}_test.json   is the condition used at all
                      attn_task_layer_*.json            content-vs-position routing
                      liveness_{random,ood}_test/       eff_rank, cross-user cosine
  p2p/t9_v5o3/        same
  p2p/shared_lora_r8/, shared_lora_r8_ctrl/   the base+G floors
  amazon2023/t10_a23_v5o3/
                      self.json shuffle.json nocross.json baseG.json
                      attn_recency.json  liveness/