Instructions to use JoshMe1/11ab9819-b1e0-442b-b8e4-964312311ca4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use JoshMe1/11ab9819-b1e0-442b-b8e4-964312311ca4 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="JoshMe1/11ab9819-b1e0-442b-b8e4-964312311ca4")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("JoshMe1/11ab9819-b1e0-442b-b8e4-964312311ca4")
model = AutoModelForCausalLM.from_pretrained("JoshMe1/11ab9819-b1e0-442b-b8e4-964312311ca4", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use JoshMe1/11ab9819-b1e0-442b-b8e4-964312311ca4 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "JoshMe1/11ab9819-b1e0-442b-b8e4-964312311ca4"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "JoshMe1/11ab9819-b1e0-442b-b8e4-964312311ca4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/JoshMe1/11ab9819-b1e0-442b-b8e4-964312311ca4

SGLang

How to use JoshMe1/11ab9819-b1e0-442b-b8e4-964312311ca4 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "JoshMe1/11ab9819-b1e0-442b-b8e4-964312311ca4" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "JoshMe1/11ab9819-b1e0-442b-b8e4-964312311ca4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "JoshMe1/11ab9819-b1e0-442b-b8e4-964312311ca4" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "JoshMe1/11ab9819-b1e0-442b-b8e4-964312311ca4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use JoshMe1/11ab9819-b1e0-442b-b8e4-964312311ca4 with Docker Model Runner:
```
docker model run hf.co/JoshMe1/11ab9819-b1e0-442b-b8e4-964312311ca4
```

JoshMe1 commited on May 14, 2025

Commit

10b8295

verified ·

1 Parent(s): 763c993

Training in progress, step 900, checkpoint

Browse files

Files changed (4) hide show

last-checkpoint/adapter_model.safetensors +1 -1
last-checkpoint/optimizer.pt +1 -1
last-checkpoint/scheduler.pt +1 -1
last-checkpoint/trainer_state.json +1502 -2

last-checkpoint/adapter_model.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:2832f221e37c6a3eeb8a9283c4222adee164d6d339ef6eaf73a3b12eec12154b
 size 1342238560

 version https://git-lfs.github.com/spec/v1
+oid sha256:db1070c39138d303d014ee04d80d8e9b75566e98b6e55f455c9f6af350c449d5
 size 1342238560

last-checkpoint/optimizer.pt CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:9991802a5016ab59805bfe886a37d8bca3f29ff422db8205e7f63a9d674e12d0
 size 682101844

 version https://git-lfs.github.com/spec/v1
+oid sha256:b6a91d80bf6853bf5303bd48ab60dd8aaec58e331157f7790b89def661e052db
 size 682101844

last-checkpoint/scheduler.pt CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:246ca084214dc27b5762653dde5a6251678dabf4b1880f62d528a3b22fc7e637
 size 1064

 version https://git-lfs.github.com/spec/v1
+oid sha256:0fe463bd95a539e2431433bae454f0af2aa8fc5e311c1d9add289b543af3fd1e
 size 1064

last-checkpoint/trainer_state.json CHANGED Viewed

@@ -1,9 +1,9 @@
 {
   "best_metric": null,
   "best_model_checkpoint": null,
-  "epoch": 0.058164897484368185,
   "eval_steps": 500,
-  "global_step": 800,
   "is_hyper_param_search": false,
   "is_local_process_zero": true,
   "is_world_process_zero": true,
@@ -12007,6 +12007,1506 @@
       "rewards/margins": 11.669793128967285,
       "rewards/rejected": -12.351922988891602,
       "step": 800
     }
   ],
   "logging_steps": 1,

 {
   "best_metric": null,
   "best_model_checkpoint": null,
+  "epoch": 0.06543550966991421,
   "eval_steps": 500,
+  "global_step": 900,
   "is_hyper_param_search": false,
   "is_local_process_zero": true,
   "is_world_process_zero": true,
       "rewards/margins": 11.669793128967285,
       "rewards/rejected": -12.351922988891602,
       "step": 800
+    },
+    {
+      "epoch": 0.05823760360622365,
+      "grad_norm": 0.00048234008136205375,
+      "learning_rate": 1.1585823820323843e-07,
+      "logits/chosen": -0.9287376999855042,
+      "logits/rejected": -0.9042397141456604,
+      "logps/chosen": -38.81523132324219,
+      "logps/rejected": -222.84970092773438,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.6534671187400818,
+      "rewards/margins": 14.314355850219727,
+      "rewards/rejected": -14.967823028564453,
+      "step": 801
+    },
+    {
+      "epoch": 0.0583103097280791,
+      "grad_norm": 0.002243852009996772,
+      "learning_rate": 1.1474337861210543e-07,
+      "logits/chosen": -0.8509057760238647,
+      "logits/rejected": -0.9152532815933228,
+      "logps/chosen": -26.429950714111328,
+      "logps/rejected": -204.90780639648438,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": 0.11170566827058792,
+      "rewards/margins": 13.521089553833008,
+      "rewards/rejected": -13.409383773803711,
+      "step": 802
+    },
+    {
+      "epoch": 0.058383015849934564,
+      "grad_norm": 0.0032597933895885944,
+      "learning_rate": 1.1363321325132447e-07,
+      "logits/chosen": -0.9146807193756104,
+      "logits/rejected": -0.8599616289138794,
+      "logps/chosen": -31.674413681030273,
+      "logps/rejected": -211.09364318847656,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.3697732090950012,
+      "rewards/margins": 13.578716278076172,
+      "rewards/rejected": -13.948490142822266,
+      "step": 803
+    },
+    {
+      "epoch": 0.058455721971790026,
+      "grad_norm": 0.01021889504045248,
+      "learning_rate": 1.1252775564791023e-07,
+      "logits/chosen": -0.9616813063621521,
+      "logits/rejected": -0.8500020503997803,
+      "logps/chosen": -34.90496063232422,
+      "logps/rejected": -212.93182373046875,
+      "loss": 0.0001,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.4326817989349365,
+      "rewards/margins": 12.21169376373291,
+      "rewards/rejected": -12.64437484741211,
+      "step": 804
+    },
+    {
+      "epoch": 0.05852842809364549,
+      "grad_norm": 0.001362229697406292,
+      "learning_rate": 1.1142701927151454e-07,
+      "logits/chosen": -0.9080723524093628,
+      "logits/rejected": -0.8339279890060425,
+      "logps/chosen": -29.448078155517578,
+      "logps/rejected": -211.43698120117188,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.2312290072441101,
+      "rewards/margins": 13.829044342041016,
+      "rewards/rejected": -14.060272216796875,
+      "step": 805
+    },
+    {
+      "epoch": 0.05860113421550094,
+      "grad_norm": 0.004242802504450083,
+      "learning_rate": 1.1033101753426282e-07,
+      "logits/chosen": -0.879481315612793,
+      "logits/rejected": -0.8717973232269287,
+      "logps/chosen": -38.24034118652344,
+      "logps/rejected": -212.5565185546875,
+      "loss": 0.0001,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.9230182766914368,
+      "rewards/margins": 13.294149398803711,
+      "rewards/rejected": -14.217167854309082,
+      "step": 806
+    },
+    {
+      "epoch": 0.058673840337356405,
+      "grad_norm": 0.04330221191048622,
+      "learning_rate": 1.0923976379059058e-07,
+      "logits/chosen": -0.9548594355583191,
+      "logits/rejected": -0.9365129470825195,
+      "logps/chosen": -35.17864990234375,
+      "logps/rejected": -214.065185546875,
+      "loss": 0.0002,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.6930069923400879,
+      "rewards/margins": 13.262189865112305,
+      "rewards/rejected": -13.95519733428955,
+      "step": 807
+    },
+    {
+      "epoch": 0.05874654645921187,
+      "grad_norm": 0.009361770935356617,
+      "learning_rate": 1.0815327133708013e-07,
+      "logits/chosen": -0.8962414264678955,
+      "logits/rejected": -0.8303819894790649,
+      "logps/chosen": -34.687705993652344,
+      "logps/rejected": -208.5323486328125,
+      "loss": 0.0001,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.5704749226570129,
+      "rewards/margins": 11.914556503295898,
+      "rewards/rejected": -12.485031127929688,
+      "step": 808
+    },
+    {
+      "epoch": 0.05881925258106733,
+      "grad_norm": 0.0003296321665402502,
+      "learning_rate": 1.07071553412299e-07,
+      "logits/chosen": -0.8962920904159546,
+      "logits/rejected": -0.8763982057571411,
+      "logps/chosen": -35.05188751220703,
+      "logps/rejected": -221.94332885742188,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.7444876432418823,
+      "rewards/margins": 15.131348609924316,
+      "rewards/rejected": -15.875836372375488,
+      "step": 809
+    },
+    {
+      "epoch": 0.058891958702922784,
+      "grad_norm": 0.002917770529165864,
+      "learning_rate": 1.0599462319663904e-07,
+      "logits/chosen": -0.8972420692443848,
+      "logits/rejected": -0.8533855676651001,
+      "logps/chosen": -37.614925384521484,
+      "logps/rejected": -216.18870544433594,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.5269683003425598,
+      "rewards/margins": 13.62031364440918,
+      "rewards/rejected": -14.147282600402832,
+      "step": 810
+    },
+    {
+      "epoch": 0.058964664824778246,
+      "grad_norm": 0.013271240517497063,
+      "learning_rate": 1.0492249381215478e-07,
+      "logits/chosen": -0.9354763031005859,
+      "logits/rejected": -0.850922703742981,
+      "logps/chosen": -31.526691436767578,
+      "logps/rejected": -211.07916259765625,
+      "loss": 0.0002,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.21681064367294312,
+      "rewards/margins": 14.02525520324707,
+      "rewards/rejected": -14.2420654296875,
+      "step": 811
+    },
+    {
+      "epoch": 0.05903737094663371,
+      "grad_norm": 0.00019169447477906942,
+      "learning_rate": 1.038551783224047e-07,
+      "logits/chosen": -0.8981155753135681,
+      "logits/rejected": -0.9153207540512085,
+      "logps/chosen": -26.782726287841797,
+      "logps/rejected": -218.28050231933594,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.27815234661102295,
+      "rewards/margins": 14.644554138183594,
+      "rewards/rejected": -14.922707557678223,
+      "step": 812
+    },
+    {
+      "epoch": 0.05911007706848917,
+      "grad_norm": 0.00047361120232380927,
+      "learning_rate": 1.0279268973229088e-07,
+      "logits/chosen": -0.9290183782577515,
+      "logits/rejected": -0.9105781316757202,
+      "logps/chosen": -32.31972885131836,
+      "logps/rejected": -218.17587280273438,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.4362805187702179,
+      "rewards/margins": 14.452142715454102,
+      "rewards/rejected": -14.888423919677734,
+      "step": 813
+    },
+    {
+      "epoch": 0.059182783190344625,
+      "grad_norm": 0.0064282529056072235,
+      "learning_rate": 1.0173504098790186e-07,
+      "logits/chosen": -0.7782235145568848,
+      "logits/rejected": -0.7675794363021851,
+      "logps/chosen": -36.16889953613281,
+      "logps/rejected": -212.2026824951172,
+      "loss": 0.0001,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.42383480072021484,
+      "rewards/margins": 13.097190856933594,
+      "rewards/rejected": -13.521025657653809,
+      "step": 814
+    },
+    {
+      "epoch": 0.05925548931220009,
+      "grad_norm": 0.003953695762902498,
+      "learning_rate": 1.0068224497635369e-07,
+      "logits/chosen": -0.9118982553482056,
+      "logits/rejected": -0.8721450567245483,
+      "logps/chosen": -37.163360595703125,
+      "logps/rejected": -210.713623046875,
+      "loss": 0.0001,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.5497003793716431,
+      "rewards/margins": 12.541065216064453,
+      "rewards/rejected": -13.090764999389648,
+      "step": 815
+    },
+    {
+      "epoch": 0.05932819543405555,
+      "grad_norm": 0.009218167513608932,
+      "learning_rate": 9.963431452563331e-08,
+      "logits/chosen": -0.9565808773040771,
+      "logits/rejected": -0.9447869658470154,
+      "logps/chosen": -32.80500030517578,
+      "logps/rejected": -213.85301208496094,
+      "loss": 0.0001,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.3509596288204193,
+      "rewards/margins": 13.14822006225586,
+      "rewards/rejected": -13.49917984008789,
+      "step": 816
+    },
+    {
+      "epoch": 0.05940090155591101,
+      "grad_norm": 0.0003960712347179651,
+      "learning_rate": 9.859126240444282e-08,
+      "logits/chosen": -0.9183976650238037,
+      "logits/rejected": -0.8882307410240173,
+      "logps/chosen": -31.947101593017578,
+      "logps/rejected": -210.08815002441406,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.35045450925827026,
+      "rewards/margins": 13.729374885559082,
+      "rewards/rejected": -14.079830169677734,
+      "step": 817
+    },
+    {
+      "epoch": 0.059473607677766466,
+      "grad_norm": 0.011877443641424179,
+      "learning_rate": 9.755310132204297e-08,
+      "logits/chosen": -0.9917686581611633,
+      "logits/rejected": -0.8296844959259033,
+      "logps/chosen": -40.909507751464844,
+      "logps/rejected": -210.73818969726562,
+      "loss": 0.0001,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.7189110517501831,
+      "rewards/margins": 12.688089370727539,
+      "rewards/rejected": -13.407000541687012,
+      "step": 818
+    },
+    {
+      "epoch": 0.05954631379962193,
+      "grad_norm": 0.006271880120038986,
+      "learning_rate": 9.651984392809914e-08,
+      "logits/chosen": -0.8329911828041077,
+      "logits/rejected": -0.8136376142501831,
+      "logps/chosen": -35.143775939941406,
+      "logps/rejected": -204.38739013671875,
+      "loss": 0.0001,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.4343762993812561,
+      "rewards/margins": 13.196807861328125,
+      "rewards/rejected": -13.631185531616211,
+      "step": 819
+    },
+    {
+      "epoch": 0.05961901992147739,
+      "grad_norm": 0.0015030940994620323,
+      "learning_rate": 9.549150281252632e-08,
+      "logits/chosen": -0.8846481442451477,
+      "logits/rejected": -0.9285573363304138,
+      "logps/chosen": -35.889747619628906,
+      "logps/rejected": -213.53724670410156,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.49309369921684265,
+      "rewards/margins": 13.477800369262695,
+      "rewards/rejected": -13.970893859863281,
+      "step": 820
+    },
+    {
+      "epoch": 0.05969172604333285,
+      "grad_norm": 0.0029090552125126123,
+      "learning_rate": 9.446809050533678e-08,
+      "logits/chosen": -0.8734315633773804,
+      "logits/rejected": -0.8346052169799805,
+      "logps/chosen": -42.2322998046875,
+      "logps/rejected": -208.023681640625,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.7172924876213074,
+      "rewards/margins": 13.537714004516602,
+      "rewards/rejected": -14.255006790161133,
+      "step": 821
+    },
+    {
+      "epoch": 0.05976443216518831,
+      "grad_norm": 0.002488103462383151,
+      "learning_rate": 9.344961947648622e-08,
+      "logits/chosen": -0.9814305305480957,
+      "logits/rejected": -0.841217041015625,
+      "logps/chosen": -30.454376220703125,
+      "logps/rejected": -207.29879760742188,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.2472071498632431,
+      "rewards/margins": 12.674690246582031,
+      "rewards/rejected": -12.921897888183594,
+      "step": 822
+    },
+    {
+      "epoch": 0.05983713828704377,
+      "grad_norm": 0.002519387984648347,
+      "learning_rate": 9.243610213572284e-08,
+      "logits/chosen": -0.9013311862945557,
+      "logits/rejected": -0.8516749739646912,
+      "logps/chosen": -31.57420539855957,
+      "logps/rejected": -213.44784545898438,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.23320429027080536,
+      "rewards/margins": 13.484821319580078,
+      "rewards/rejected": -13.718025207519531,
+      "step": 823
+    },
+    {
+      "epoch": 0.05990984440889923,
+      "grad_norm": 0.04156282916665077,
+      "learning_rate": 9.142755083243575e-08,
+      "logits/chosen": -0.9548500776290894,
+      "logits/rejected": -0.8840470314025879,
+      "logps/chosen": -35.11345672607422,
+      "logps/rejected": -199.01788330078125,
+      "loss": 0.0005,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.41137468814849854,
+      "rewards/margins": 11.84408187866211,
+      "rewards/rejected": -12.255456924438477,
+      "step": 824
+    },
+    {
+      "epoch": 0.05998255053075469,
+      "grad_norm": 0.003949024248868227,
+      "learning_rate": 9.042397785550404e-08,
+      "logits/chosen": -0.8854993581771851,
+      "logits/rejected": -0.7998782396316528,
+      "logps/chosen": -34.553802490234375,
+      "logps/rejected": -208.09466552734375,
+      "loss": 0.0001,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.7249510884284973,
+      "rewards/margins": 13.005220413208008,
+      "rewards/rejected": -13.730171203613281,
+      "step": 825
+    },
+    {
+      "epoch": 0.06005525665261015,
+      "grad_norm": 0.00026854078168980777,
+      "learning_rate": 8.942539543314798e-08,
+      "logits/chosen": -0.843517005443573,
+      "logits/rejected": -0.9094403982162476,
+      "logps/chosen": -42.31779479980469,
+      "logps/rejected": -213.2200164794922,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.9826798439025879,
+      "rewards/margins": 13.873092651367188,
+      "rewards/rejected": -14.855772972106934,
+      "step": 826
+    },
+    {
+      "epoch": 0.06012796277446561,
+      "grad_norm": 0.0023174339439719915,
+      "learning_rate": 8.843181573277903e-08,
+      "logits/chosen": -1.0254926681518555,
+      "logits/rejected": -0.9478312730789185,
+      "logps/chosen": -28.260574340820312,
+      "logps/rejected": -208.76150512695312,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": 0.12190592288970947,
+      "rewards/margins": 12.439874649047852,
+      "rewards/rejected": -12.317968368530273,
+      "step": 827
+    },
+    {
+      "epoch": 0.06020066889632107,
+      "grad_norm": 0.0013523934176191688,
+      "learning_rate": 8.744325086085247e-08,
+      "logits/chosen": -0.9789197444915771,
+      "logits/rejected": -0.8955248594284058,
+      "logps/chosen": -28.94500160217285,
+      "logps/rejected": -212.00112915039062,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.0602029412984848,
+      "rewards/margins": 14.344179153442383,
+      "rewards/rejected": -14.40438175201416,
+      "step": 828
+    },
+    {
+      "epoch": 0.060273375018176534,
+      "grad_norm": 0.0018464912427589297,
+      "learning_rate": 8.645971286271903e-08,
+      "logits/chosen": -1.004629373550415,
+      "logits/rejected": -0.9347493648529053,
+      "logps/chosen": -32.38956069946289,
+      "logps/rejected": -208.96865844726562,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.5599470734596252,
+      "rewards/margins": 13.54710578918457,
+      "rewards/rejected": -14.107051849365234,
+      "step": 829
+    },
+    {
+      "epoch": 0.06034608114003199,
+      "grad_norm": 0.005679186433553696,
+      "learning_rate": 8.548121372247919e-08,
+      "logits/chosen": -0.9179498553276062,
+      "logits/rejected": -0.8190484046936035,
+      "logps/chosen": -27.20296859741211,
+      "logps/rejected": -216.39215087890625,
+      "loss": 0.0001,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.13396206498146057,
+      "rewards/margins": 13.646373748779297,
+      "rewards/rejected": -13.780335426330566,
+      "step": 830
+    },
+    {
+      "epoch": 0.06041878726188745,
+      "grad_norm": 0.0004011060227639973,
+      "learning_rate": 8.450776536283593e-08,
+      "logits/chosen": -0.9138544201850891,
+      "logits/rejected": -0.8859550952911377,
+      "logps/chosen": -38.815269470214844,
+      "logps/rejected": -215.1049041748047,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.5459855198860168,
+      "rewards/margins": 14.472180366516113,
+      "rewards/rejected": -15.018165588378906,
+      "step": 831
+    },
+    {
+      "epoch": 0.06049149338374291,
+      "grad_norm": 0.002478076843544841,
+      "learning_rate": 8.353937964495028e-08,
+      "logits/chosen": -0.9330829977989197,
+      "logits/rejected": -0.8788319230079651,
+      "logps/chosen": -51.765167236328125,
+      "logps/rejected": -226.31045532226562,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -1.417904019355774,
+      "rewards/margins": 14.171648979187012,
+      "rewards/rejected": -15.589553833007812,
+      "step": 832
+    },
+    {
+      "epoch": 0.060564199505598375,
+      "grad_norm": 0.003257776377722621,
+      "learning_rate": 8.257606836829677e-08,
+      "logits/chosen": -0.8976417779922485,
+      "logits/rejected": -0.8416788578033447,
+      "logps/chosen": -34.378883361816406,
+      "logps/rejected": -211.79849243164062,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.29891371726989746,
+      "rewards/margins": 12.301176071166992,
+      "rewards/rejected": -12.600090026855469,
+      "step": 833
+    },
+    {
+      "epoch": 0.06063690562745383,
+      "grad_norm": 0.0030980699229985476,
+      "learning_rate": 8.161784327051919e-08,
+      "logits/chosen": -0.8809930086135864,
+      "logits/rejected": -0.9168920516967773,
+      "logps/chosen": -41.41750717163086,
+      "logps/rejected": -215.60800170898438,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.5576534867286682,
+      "rewards/margins": 13.368885040283203,
+      "rewards/rejected": -13.926538467407227,
+      "step": 834
+    },
+    {
+      "epoch": 0.06070961174930929,
+      "grad_norm": 0.01704149879515171,
+      "learning_rate": 8.066471602728803e-08,
+      "logits/chosen": -0.9516491889953613,
+      "logits/rejected": -0.8884276151657104,
+      "logps/chosen": -37.34577560424805,
+      "logps/rejected": -209.8350067138672,
+      "loss": 0.0002,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.736299991607666,
+      "rewards/margins": 12.734248161315918,
+      "rewards/rejected": -13.470548629760742,
+      "step": 835
+    },
+    {
+      "epoch": 0.060782317871164754,
+      "grad_norm": 0.004049696959555149,
+      "learning_rate": 7.971669825215787e-08,
+      "logits/chosen": -0.9258505702018738,
+      "logits/rejected": -0.8807904720306396,
+      "logps/chosen": -25.67070198059082,
+      "logps/rejected": -200.052978515625,
+      "loss": 0.0001,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.007871553301811218,
+      "rewards/margins": 12.438920974731445,
+      "rewards/rejected": -12.446792602539062,
+      "step": 836
+    },
+    {
+      "epoch": 0.06085502399302021,
+      "grad_norm": 0.003331885440275073,
+      "learning_rate": 7.877380149642626e-08,
+      "logits/chosen": -0.9025241732597351,
+      "logits/rejected": -0.8243002891540527,
+      "logps/chosen": -40.865509033203125,
+      "logps/rejected": -214.27951049804688,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.8534423112869263,
+      "rewards/margins": 13.077730178833008,
+      "rewards/rejected": -13.931172370910645,
+      "step": 837
+    },
+    {
+      "epoch": 0.06092773011487567,
+      "grad_norm": 0.0002625905617605895,
+      "learning_rate": 7.783603724899257e-08,
+      "logits/chosen": -0.8887792825698853,
+      "logits/rejected": -0.8686505556106567,
+      "logps/chosen": -37.729591369628906,
+      "logps/rejected": -218.04788208007812,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.7940793037414551,
+      "rewards/margins": 14.703927993774414,
+      "rewards/rejected": -15.498005867004395,
+      "step": 838
+    },
+    {
+      "epoch": 0.06100043623673113,
+      "grad_norm": 0.011811570264399052,
+      "learning_rate": 7.690341693621805e-08,
+      "logits/chosen": -0.9245173931121826,
+      "logits/rejected": -0.868511438369751,
+      "logps/chosen": -33.1500129699707,
+      "logps/rejected": -206.24111938476562,
+      "loss": 0.0002,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.374135285615921,
+      "rewards/margins": 12.267215728759766,
+      "rewards/rejected": -12.641351699829102,
+      "step": 839
+    },
+    {
+      "epoch": 0.061073142358586595,
+      "grad_norm": 0.002337161684408784,
+      "learning_rate": 7.597595192178702e-08,
+      "logits/chosen": -0.9276552200317383,
+      "logits/rejected": -0.8024710416793823,
+      "logps/chosen": -34.53630828857422,
+      "logps/rejected": -215.49301147460938,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.5310240983963013,
+      "rewards/margins": 13.375576972961426,
+      "rewards/rejected": -13.906600952148438,
+      "step": 840
+    },
+    {
+      "epoch": 0.06114584848044205,
+      "grad_norm": 0.01176636852324009,
+      "learning_rate": 7.505365350656811e-08,
+      "logits/chosen": -0.9031511545181274,
+      "logits/rejected": -0.8692529201507568,
+      "logps/chosen": -35.75494384765625,
+      "logps/rejected": -213.64572143554688,
+      "loss": 0.0002,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.5334094762802124,
+      "rewards/margins": 13.190857887268066,
+      "rewards/rejected": -13.724267959594727,
+      "step": 841
+    },
+    {
+      "epoch": 0.06121855460229751,
+      "grad_norm": 0.000319530488923192,
+      "learning_rate": 7.413653292847616e-08,
+      "logits/chosen": -0.8106807470321655,
+      "logits/rejected": -0.8871402740478516,
+      "logps/chosen": -38.77924346923828,
+      "logps/rejected": -208.69039916992188,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.6083173751831055,
+      "rewards/margins": 13.612259864807129,
+      "rewards/rejected": -14.220577239990234,
+      "step": 842
+    },
+    {
+      "epoch": 0.061291260724152974,
+      "grad_norm": 0.002737750532105565,
+      "learning_rate": 7.322460136233621e-08,
+      "logits/chosen": -0.9447481036186218,
+      "logits/rejected": -0.892672061920166,
+      "logps/chosen": -33.469573974609375,
+      "logps/rejected": -218.03538513183594,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.5679317712783813,
+      "rewards/margins": 13.78137493133545,
+      "rewards/rejected": -14.349306106567383,
+      "step": 843
+    },
+    {
+      "epoch": 0.061363966846008436,
+      "grad_norm": 0.005921626929193735,
+      "learning_rate": 7.23178699197467e-08,
+      "logits/chosen": -0.8690047264099121,
+      "logits/rejected": -0.8909629583358765,
+      "logps/chosen": -41.96760559082031,
+      "logps/rejected": -212.00982666015625,
+      "loss": 0.0001,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -1.1911203861236572,
+      "rewards/margins": 13.252840042114258,
+      "rewards/rejected": -14.443960189819336,
+      "step": 844
+    },
+    {
+      "epoch": 0.06143667296786389,
+      "grad_norm": 0.0036112836096435785,
+      "learning_rate": 7.141634964894388e-08,
+      "logits/chosen": -0.8395127058029175,
+      "logits/rejected": -0.8488901853561401,
+      "logps/chosen": -38.618865966796875,
+      "logps/rejected": -209.74017333984375,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.7918221354484558,
+      "rewards/margins": 12.778753280639648,
+      "rewards/rejected": -13.570574760437012,
+      "step": 845
+    },
+    {
+      "epoch": 0.06150937908971935,
+      "grad_norm": 0.010566781274974346,
+      "learning_rate": 7.052005153466778e-08,
+      "logits/chosen": -0.8986787796020508,
+      "logits/rejected": -0.8492857813835144,
+      "logps/chosen": -33.52727508544922,
+      "logps/rejected": -211.01170349121094,
+      "loss": 0.0001,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.25529420375823975,
+      "rewards/margins": 12.790441513061523,
+      "rewards/rejected": -13.045736312866211,
+      "step": 846
+    },
+    {
+      "epoch": 0.061582085211574815,
+      "grad_norm": 0.004384850617498159,
+      "learning_rate": 6.962898649802822e-08,
+      "logits/chosen": -0.8685309886932373,
+      "logits/rejected": -0.8487534523010254,
+      "logps/chosen": -39.79358673095703,
+      "logps/rejected": -219.06317138671875,
+      "loss": 0.0001,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.5773910880088806,
+      "rewards/margins": 13.647058486938477,
+      "rewards/rejected": -14.224449157714844,
+      "step": 847
+    },
+    {
+      "epoch": 0.06165479133343028,
+      "grad_norm": 0.00011927230661967769,
+      "learning_rate": 6.874316539637126e-08,
+      "logits/chosen": -0.9644160270690918,
+      "logits/rejected": -0.906198263168335,
+      "logps/chosen": -39.05009460449219,
+      "logps/rejected": -217.9404296875,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.7079715132713318,
+      "rewards/margins": 14.837966918945312,
+      "rewards/rejected": -15.545938491821289,
+      "step": 848
+    },
+    {
+      "epoch": 0.06172749745528573,
+      "grad_norm": 0.0005644657649099827,
+      "learning_rate": 6.786259902314767e-08,
+      "logits/chosen": -0.9547232389450073,
+      "logits/rejected": -0.8872819542884827,
+      "logps/chosen": -32.344215393066406,
+      "logps/rejected": -214.14027404785156,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.10444006323814392,
+      "rewards/margins": 14.206622123718262,
+      "rewards/rejected": -14.31106185913086,
+      "step": 849
+    },
+    {
+      "epoch": 0.061800203577141194,
+      "grad_norm": 0.006048460491001606,
+      "learning_rate": 6.698729810778064e-08,
+      "logits/chosen": -0.8742987513542175,
+      "logits/rejected": -0.857744574546814,
+      "logps/chosen": -36.89727783203125,
+      "logps/rejected": -205.7833251953125,
+      "loss": 0.0001,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.40647146105766296,
+      "rewards/margins": 13.201678276062012,
+      "rewards/rejected": -13.608149528503418,
+      "step": 850
+    },
+    {
+      "epoch": 0.061872909698996656,
+      "grad_norm": 0.0003668160643428564,
+      "learning_rate": 6.611727331553585e-08,
+      "logits/chosen": -0.9632246494293213,
+      "logits/rejected": -0.9273592233657837,
+      "logps/chosen": -27.67076873779297,
+      "logps/rejected": -213.115478515625,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.2091766595840454,
+      "rewards/margins": 13.761649131774902,
+      "rewards/rejected": -13.9708251953125,
+      "step": 851
+    },
+    {
+      "epoch": 0.06194561582085212,
+      "grad_norm": 0.010656672529876232,
+      "learning_rate": 6.52525352473905e-08,
+      "logits/chosen": -0.9768116474151611,
+      "logits/rejected": -0.93428635597229,
+      "logps/chosen": -32.60914993286133,
+      "logps/rejected": -216.2017822265625,
+      "loss": 0.0001,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.2603747844696045,
+      "rewards/margins": 14.2334566116333,
+      "rewards/rejected": -14.493831634521484,
+      "step": 852
+    },
+    {
+      "epoch": 0.06201832194270757,
+      "grad_norm": 0.0002745226083789021,
+      "learning_rate": 6.439309443990531e-08,
+      "logits/chosen": -0.964214563369751,
+      "logits/rejected": -0.8941053152084351,
+      "logps/chosen": -33.27979278564453,
+      "logps/rejected": -215.39382934570312,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.17558620870113373,
+      "rewards/margins": 14.90461540222168,
+      "rewards/rejected": -15.0802001953125,
+      "step": 853
+    },
+    {
+      "epoch": 0.062091028064563035,
+      "grad_norm": 0.0005914793000556529,
+      "learning_rate": 6.353896136509524e-08,
+      "logits/chosen": -0.8854947686195374,
+      "logits/rejected": -0.8601844310760498,
+      "logps/chosen": -38.55859375,
+      "logps/rejected": -207.59486389160156,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.44349443912506104,
+      "rewards/margins": 13.547721862792969,
+      "rewards/rejected": -13.991215705871582,
+      "step": 854
+    },
+    {
+      "epoch": 0.0621637341864185,
+      "grad_norm": 0.05004807561635971,
+      "learning_rate": 6.269014643030212e-08,
+      "logits/chosen": -0.9444171786308289,
+      "logits/rejected": -0.8744308948516846,
+      "logps/chosen": -28.180923461914062,
+      "logps/rejected": -210.5736083984375,
+      "loss": 0.0001,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.016716942191123962,
+      "rewards/margins": 12.589879989624023,
+      "rewards/rejected": -12.606595993041992,
+      "step": 855
+    },
+    {
+      "epoch": 0.06223644030827396,
+      "grad_norm": 0.002197504276409745,
+      "learning_rate": 6.184665997806831e-08,
+      "logits/chosen": -0.9766485095024109,
+      "logits/rejected": -0.9080944657325745,
+      "logps/chosen": -32.427574157714844,
+      "logps/rejected": -210.64768981933594,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.010407708585262299,
+      "rewards/margins": 14.489545822143555,
+      "rewards/rejected": -14.499954223632812,
+      "step": 856
+    },
+    {
+      "epoch": 0.062309146430129414,
+      "grad_norm": 0.001485436107032001,
+      "learning_rate": 6.100851228600973e-08,
+      "logits/chosen": -0.9215723872184753,
+      "logits/rejected": -0.847144365310669,
+      "logps/chosen": -35.548492431640625,
+      "logps/rejected": -210.3443603515625,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.527469277381897,
+      "rewards/margins": 13.315184593200684,
+      "rewards/rejected": -13.84265422821045,
+      "step": 857
+    },
+    {
+      "epoch": 0.062381852551984876,
+      "grad_norm": 0.002220870926976204,
+      "learning_rate": 6.017571356669182e-08,
+      "logits/chosen": -0.9269384741783142,
+      "logits/rejected": -0.8386276960372925,
+      "logps/chosen": -31.025964736938477,
+      "logps/rejected": -202.34542846679688,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.42351794242858887,
+      "rewards/margins": 12.572928428649902,
+      "rewards/rejected": -12.99644660949707,
+      "step": 858
+    },
+    {
+      "epoch": 0.06245455867384034,
+      "grad_norm": 0.007875983603298664,
+      "learning_rate": 5.9348273967503916e-08,
+      "logits/chosen": -0.9482343196868896,
+      "logits/rejected": -0.9351887702941895,
+      "logps/chosen": -32.64423751831055,
+      "logps/rejected": -213.67555236816406,
+      "loss": 0.0001,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.29133155941963196,
+      "rewards/margins": 14.106221199035645,
+      "rewards/rejected": -14.397552490234375,
+      "step": 859
+    },
+    {
+      "epoch": 0.0625272647956958,
+      "grad_norm": 0.0005645692581310868,
+      "learning_rate": 5.8526203570536504e-08,
+      "logits/chosen": -0.9474825859069824,
+      "logits/rejected": -0.8204978108406067,
+      "logps/chosen": -29.63552474975586,
+      "logps/rejected": -214.58128356933594,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.02821112424135208,
+      "rewards/margins": 14.28672981262207,
+      "rewards/rejected": -14.314940452575684,
+      "step": 860
+    },
+    {
+      "epoch": 0.06259997091755126,
+      "grad_norm": 0.004658716730773449,
+      "learning_rate": 5.770951239245803e-08,
+      "logits/chosen": -0.917488694190979,
+      "logits/rejected": -0.8856992721557617,
+      "logps/chosen": -38.66998291015625,
+      "logps/rejected": -213.24978637695312,
+      "loss": 0.0001,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.7889727354049683,
+      "rewards/margins": 13.10611343383789,
+      "rewards/rejected": -13.895085334777832,
+      "step": 861
+    },
+    {
+      "epoch": 0.06267267703940672,
+      "grad_norm": 0.003891513915732503,
+      "learning_rate": 5.689821038439263e-08,
+      "logits/chosen": -0.8568642735481262,
+      "logits/rejected": -0.9043086767196655,
+      "logps/chosen": -37.63838577270508,
+      "logps/rejected": -209.49623107910156,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.8549795150756836,
+      "rewards/margins": 12.97883415222168,
+      "rewards/rejected": -13.833813667297363,
+      "step": 862
+    },
+    {
+      "epoch": 0.06274538316126217,
+      "grad_norm": 0.004181155003607273,
+      "learning_rate": 5.609230743179938e-08,
+      "logits/chosen": -0.9312776923179626,
+      "logits/rejected": -0.8475666642189026,
+      "logps/chosen": -35.78960418701172,
+      "logps/rejected": -212.63760375976562,
+      "loss": 0.0001,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.5629060864448547,
+      "rewards/margins": 12.891237258911133,
+      "rewards/rejected": -13.454143524169922,
+      "step": 863
+    },
+    {
+      "epoch": 0.06281808928311763,
+      "grad_norm": 0.0028015945572406054,
+      "learning_rate": 5.529181335435124e-08,
+      "logits/chosen": -0.8663648366928101,
+      "logits/rejected": -0.8609052896499634,
+      "logps/chosen": -31.67678451538086,
+      "logps/rejected": -207.86477661132812,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.23127928376197815,
+      "rewards/margins": 13.272483825683594,
+      "rewards/rejected": -13.503763198852539,
+      "step": 864
+    },
+    {
+      "epoch": 0.0628907954049731,
+      "grad_norm": 0.005344074685126543,
+      "learning_rate": 5.44967379058161e-08,
+      "logits/chosen": -0.8992750644683838,
+      "logits/rejected": -0.9029591083526611,
+      "logps/chosen": -40.26115417480469,
+      "logps/rejected": -214.38668823242188,
+      "loss": 0.0001,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.843727171421051,
+      "rewards/margins": 13.465059280395508,
+      "rewards/rejected": -14.308786392211914,
+      "step": 865
+    },
+    {
+      "epoch": 0.06296350152682856,
+      "grad_norm": 0.004881809465587139,
+      "learning_rate": 5.37070907739372e-08,
+      "logits/chosen": -0.9818276762962341,
+      "logits/rejected": -0.8388216495513916,
+      "logps/chosen": -34.20466232299805,
+      "logps/rejected": -217.65570068359375,
+      "loss": 0.0001,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.4291444718837738,
+      "rewards/margins": 11.864551544189453,
+      "rewards/rejected": -12.293695449829102,
+      "step": 866
+    },
+    {
+      "epoch": 0.06303620764868402,
+      "grad_norm": 0.008066492155194283,
+      "learning_rate": 5.292288158031594e-08,
+      "logits/chosen": -1.0302809476852417,
+      "logits/rejected": -0.8648408651351929,
+      "logps/chosen": -34.25592041015625,
+      "logps/rejected": -219.7183837890625,
+      "loss": 0.0001,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.44313353300094604,
+      "rewards/margins": 13.755355834960938,
+      "rewards/rejected": -14.19848918914795,
+      "step": 867
+    },
+    {
+      "epoch": 0.06310891377053948,
+      "grad_norm": 0.005934158340096474,
+      "learning_rate": 5.2144119880293544e-08,
+      "logits/chosen": -0.8914825916290283,
+      "logits/rejected": -0.820566713809967,
+      "logps/chosen": -26.389057159423828,
+      "logps/rejected": -212.1099853515625,
+      "loss": 0.0001,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.26406508684158325,
+      "rewards/margins": 13.135456085205078,
+      "rewards/rejected": -13.399520874023438,
+      "step": 868
+    },
+    {
+      "epoch": 0.06318161989239494,
+      "grad_norm": 0.015468381345272064,
+      "learning_rate": 5.137081516283581e-08,
+      "logits/chosen": -0.9736705422401428,
+      "logits/rejected": -0.7925209999084473,
+      "logps/chosen": -33.48400115966797,
+      "logps/rejected": -206.9025115966797,
+      "loss": 0.0002,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.24499306082725525,
+      "rewards/margins": 12.329083442687988,
+      "rewards/rejected": -12.574075698852539,
+      "step": 869
+    },
+    {
+      "epoch": 0.0632543260142504,
+      "grad_norm": 0.005471958313137293,
+      "learning_rate": 5.060297685041659e-08,
+      "logits/chosen": -0.983143150806427,
+      "logits/rejected": -0.9158498644828796,
+      "logps/chosen": -31.21188735961914,
+      "logps/rejected": -204.966064453125,
+      "loss": 0.0001,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.3806591331958771,
+      "rewards/margins": 12.467314720153809,
+      "rewards/rejected": -12.847973823547363,
+      "step": 870
+    },
+    {
+      "epoch": 0.06332703213610585,
+      "grad_norm": 0.011555798351764679,
+      "learning_rate": 4.984061429890324e-08,
+      "logits/chosen": -0.9071904420852661,
+      "logits/rejected": -0.8518509864807129,
+      "logps/chosen": -33.272377014160156,
+      "logps/rejected": -211.21652221679688,
+      "loss": 0.0001,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.43156248331069946,
+      "rewards/margins": 13.359749794006348,
+      "rewards/rejected": -13.791312217712402,
+      "step": 871
+    },
+    {
+      "epoch": 0.06339973825796132,
+      "grad_norm": 0.005020745564252138,
+      "learning_rate": 4.908373679744315e-08,
+      "logits/chosen": -1.003391146659851,
+      "logits/rejected": -0.8913395404815674,
+      "logps/chosen": -34.14504623413086,
+      "logps/rejected": -206.21377563476562,
+      "loss": 0.0001,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.3934352993965149,
+      "rewards/margins": 12.375293731689453,
+      "rewards/rejected": -12.768728256225586,
+      "step": 872
+    },
+    {
+      "epoch": 0.06347244437981678,
+      "grad_norm": 0.003745915135368705,
+      "learning_rate": 4.833235356834958e-08,
+      "logits/chosen": -0.9667826294898987,
+      "logits/rejected": -0.9095417261123657,
+      "logps/chosen": -36.59027862548828,
+      "logps/rejected": -221.0901336669922,
+      "loss": 0.0001,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.1525346040725708,
+      "rewards/margins": 14.664800643920898,
+      "rewards/rejected": -14.81733512878418,
+      "step": 873
+    },
+    {
+      "epoch": 0.06354515050167224,
+      "grad_norm": 0.0011890361784026027,
+      "learning_rate": 4.758647376699032e-08,
+      "logits/chosen": -0.9169249534606934,
+      "logits/rejected": -0.8539060354232788,
+      "logps/chosen": -37.6380500793457,
+      "logps/rejected": -212.51040649414062,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.7034934759140015,
+      "rewards/margins": 13.775259017944336,
+      "rewards/rejected": -14.478752136230469,
+      "step": 874
+    },
+    {
+      "epoch": 0.0636178566235277,
+      "grad_norm": 0.000775835185777396,
+      "learning_rate": 4.684610648167503e-08,
+      "logits/chosen": -0.8775843381881714,
+      "logits/rejected": -0.8956321477890015,
+      "logps/chosen": -37.35170364379883,
+      "logps/rejected": -215.11346435546875,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.784580647945404,
+      "rewards/margins": 13.627195358276367,
+      "rewards/rejected": -14.411775588989258,
+      "step": 875
+    },
+    {
+      "epoch": 0.06369056274538316,
+      "grad_norm": 0.005996357183903456,
+      "learning_rate": 4.611126073354571e-08,
+      "logits/chosen": -0.860283613204956,
+      "logits/rejected": -0.8795108795166016,
+      "logps/chosen": -38.77946472167969,
+      "logps/rejected": -214.42984008789062,
+      "loss": 0.0001,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.796995997428894,
+      "rewards/margins": 13.180766105651855,
+      "rewards/rejected": -13.977762222290039,
+      "step": 876
+    },
+    {
+      "epoch": 0.06376326886723863,
+      "grad_norm": 0.008098367601633072,
+      "learning_rate": 4.5381945476465735e-08,
+      "logits/chosen": -0.9040993452072144,
+      "logits/rejected": -0.849277675151825,
+      "logps/chosen": -30.66649627685547,
+      "logps/rejected": -213.02574157714844,
+      "loss": 0.0001,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": 0.07089868187904358,
+      "rewards/margins": 12.305122375488281,
+      "rewards/rejected": -12.234224319458008,
+      "step": 877
+    },
+    {
+      "epoch": 0.06383597498909409,
+      "grad_norm": 0.014966917224228382,
+      "learning_rate": 4.465816959691149e-08,
+      "logits/chosen": -1.044053316116333,
+      "logits/rejected": -0.8688252568244934,
+      "logps/chosen": -39.691871643066406,
+      "logps/rejected": -213.3199462890625,
+      "loss": 0.0002,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.5965285897254944,
+      "rewards/margins": 10.999326705932617,
+      "rewards/rejected": -11.595854759216309,
+      "step": 878
+    },
+    {
+      "epoch": 0.06390868111094954,
+      "grad_norm": 0.00389337120577693,
+      "learning_rate": 4.393994191386352e-08,
+      "logits/chosen": -0.8819083571434021,
+      "logits/rejected": -0.8748650550842285,
+      "logps/chosen": -38.830711364746094,
+      "logps/rejected": -212.04507446289062,
+      "loss": 0.0001,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.9213253855705261,
+      "rewards/margins": 12.269974708557129,
+      "rewards/rejected": -13.191300392150879,
+      "step": 879
+    },
+    {
+      "epoch": 0.063981387232805,
+      "grad_norm": 8.128311310429126e-05,
+      "learning_rate": 4.322727117869951e-08,
+      "logits/chosen": -0.9429582357406616,
+      "logits/rejected": -0.8674257397651672,
+      "logps/chosen": -34.725860595703125,
+      "logps/rejected": -214.15640258789062,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.5518949031829834,
+      "rewards/margins": 15.001272201538086,
+      "rewards/rejected": -15.553168296813965,
+      "step": 880
+    },
+    {
+      "epoch": 0.06405409335466046,
+      "grad_norm": 0.0008476789225824177,
+      "learning_rate": 4.2520166075087624e-08,
+      "logits/chosen": -0.9395104646682739,
+      "logits/rejected": -0.8900717496871948,
+      "logps/chosen": -27.76080894470215,
+      "logps/rejected": -209.03936767578125,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.15514400601387024,
+      "rewards/margins": 13.661928176879883,
+      "rewards/rejected": -13.817071914672852,
+      "step": 881
+    },
+    {
+      "epoch": 0.06412679947651592,
+      "grad_norm": 0.0025844499468803406,
+      "learning_rate": 4.181863521888018e-08,
+      "logits/chosen": -0.995636522769928,
+      "logits/rejected": -0.8438848257064819,
+      "logps/chosen": -38.35039520263672,
+      "logps/rejected": -220.33677673339844,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.35153821110725403,
+      "rewards/margins": 14.15424919128418,
+      "rewards/rejected": -14.505786895751953,
+      "step": 882
+    },
+    {
+      "epoch": 0.06419950559837138,
+      "grad_norm": 0.001683843438513577,
+      "learning_rate": 4.112268715800943e-08,
+      "logits/chosen": -0.9168758392333984,
+      "logits/rejected": -0.8300659656524658,
+      "logps/chosen": -35.08553695678711,
+      "logps/rejected": -212.10479736328125,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.6151477694511414,
+      "rewards/margins": 13.720037460327148,
+      "rewards/rejected": -14.335184097290039,
+      "step": 883
+    },
+    {
+      "epoch": 0.06427221172022685,
+      "grad_norm": 0.0002436938666505739,
+      "learning_rate": 4.043233037238281e-08,
+      "logits/chosen": -0.9856297969818115,
+      "logits/rejected": -0.8883318901062012,
+      "logps/chosen": -33.0865364074707,
+      "logps/rejected": -215.97259521484375,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.3266187310218811,
+      "rewards/margins": 14.306659698486328,
+      "rewards/rejected": -14.633277893066406,
+      "step": 884
+    },
+    {
+      "epoch": 0.06434491784208231,
+      "grad_norm": 0.00038230197969824076,
+      "learning_rate": 3.974757327377981e-08,
+      "logits/chosen": -0.898405909538269,
+      "logits/rejected": -0.8910610675811768,
+      "logps/chosen": -36.913841247558594,
+      "logps/rejected": -210.2689971923828,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.7214539051055908,
+      "rewards/margins": 14.273005485534668,
+      "rewards/rejected": -14.99445915222168,
+      "step": 885
+    },
+    {
+      "epoch": 0.06441762396393777,
+      "grad_norm": 0.008573402650654316,
+      "learning_rate": 3.9068424205749794e-08,
+      "logits/chosen": -0.9435018301010132,
+      "logits/rejected": -0.8671293258666992,
+      "logps/chosen": -35.00514221191406,
+      "logps/rejected": -218.28509521484375,
+      "loss": 0.0001,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.4492579996585846,
+      "rewards/margins": 14.105907440185547,
+      "rewards/rejected": -14.555166244506836,
+      "step": 886
+    },
+    {
+      "epoch": 0.06449033008579322,
+      "grad_norm": 0.016668081283569336,
+      "learning_rate": 3.839489144350955e-08,
+      "logits/chosen": -0.9791244864463806,
+      "logits/rejected": -0.9481874704360962,
+      "logps/chosen": -36.63296127319336,
+      "logps/rejected": -209.4439697265625,
+      "loss": 0.0002,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.7272722721099854,
+      "rewards/margins": 13.731389999389648,
+      "rewards/rejected": -14.458662986755371,
+      "step": 887
+    },
+    {
+      "epoch": 0.06456303620764868,
+      "grad_norm": 0.00035227026091888547,
+      "learning_rate": 3.7726983193843485e-08,
+      "logits/chosen": -0.9785500168800354,
+      "logits/rejected": -0.9569947719573975,
+      "logps/chosen": -29.544147491455078,
+      "logps/rejected": -215.77798461914062,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.13926872611045837,
+      "rewards/margins": 14.384265899658203,
+      "rewards/rejected": -14.523533821105957,
+      "step": 888
+    },
+    {
+      "epoch": 0.06463574232950414,
+      "grad_norm": 0.0024689617566764355,
+      "learning_rate": 3.706470759500263e-08,
+      "logits/chosen": -0.9616410136222839,
+      "logits/rejected": -0.922447681427002,
+      "logps/chosen": -30.809356689453125,
+      "logps/rejected": -206.8951873779297,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.022951841354370117,
+      "rewards/margins": 12.676582336425781,
+      "rewards/rejected": -12.699533462524414,
+      "step": 889
+    },
+    {
+      "epoch": 0.0647084484513596,
+      "grad_norm": 0.00011276199074927717,
+      "learning_rate": 3.6408072716606345e-08,
+      "logits/chosen": -0.8249759674072266,
+      "logits/rejected": -0.8695921301841736,
+      "logps/chosen": -36.34074401855469,
+      "logps/rejected": -218.12167358398438,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.6106380820274353,
+      "rewards/margins": 14.810791969299316,
+      "rewards/rejected": -15.421430587768555,
+      "step": 890
+    },
+    {
+      "epoch": 0.06478115457321507,
+      "grad_norm": 0.005838706623762846,
+      "learning_rate": 3.5757086559543236e-08,
+      "logits/chosen": -1.0293669700622559,
+      "logits/rejected": -0.9434897899627686,
+      "logps/chosen": -26.52021026611328,
+      "logps/rejected": -210.0387725830078,
+      "loss": 0.0001,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.1490568071603775,
+      "rewards/margins": 12.865198135375977,
+      "rewards/rejected": -13.01425552368164,
+      "step": 891
+    },
+    {
+      "epoch": 0.06485386069507053,
+      "grad_norm": 0.004648380447179079,
+      "learning_rate": 3.5111757055874326e-08,
+      "logits/chosen": -0.8273119926452637,
+      "logits/rejected": -0.886111855506897,
+      "logps/chosen": -35.672080993652344,
+      "logps/rejected": -213.85891723632812,
+      "loss": 0.0001,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.6130844950675964,
+      "rewards/margins": 13.616848945617676,
+      "rewards/rejected": -14.229933738708496,
+      "step": 892
+    },
+    {
+      "epoch": 0.06492656681692599,
+      "grad_norm": 0.002144254045560956,
+      "learning_rate": 3.447209206873591e-08,
+      "logits/chosen": -0.9729441404342651,
+      "logits/rejected": -0.8645864725112915,
+      "logps/chosen": -38.90720748901367,
+      "logps/rejected": -212.21334838867188,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.7815188765525818,
+      "rewards/margins": 12.436084747314453,
+      "rewards/rejected": -13.21760368347168,
+      "step": 893
+    },
+    {
+      "epoch": 0.06499927293878145,
+      "grad_norm": 0.0015325326239690185,
+      "learning_rate": 3.3838099392243915e-08,
+      "logits/chosen": -0.8757805824279785,
+      "logits/rejected": -0.8313013315200806,
+      "logps/chosen": -35.981407165527344,
+      "logps/rejected": -215.0237579345703,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.7735263109207153,
+      "rewards/margins": 13.947293281555176,
+      "rewards/rejected": -14.720819473266602,
+      "step": 894
+    },
+    {
+      "epoch": 0.0650719790606369,
+      "grad_norm": 0.008555007167160511,
+      "learning_rate": 3.3209786751399184e-08,
+      "logits/chosen": -0.9840516448020935,
+      "logits/rejected": -0.8736527562141418,
+      "logps/chosen": -33.093833923339844,
+      "logps/rejected": -219.80372619628906,
+      "loss": 0.0001,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.3639523386955261,
+      "rewards/margins": 13.325874328613281,
+      "rewards/rejected": -13.689826965332031,
+      "step": 895
+    },
+    {
+      "epoch": 0.06514468518249236,
+      "grad_norm": 0.0007325622718781233,
+      "learning_rate": 3.258716180199278e-08,
+      "logits/chosen": -0.9212465286254883,
+      "logits/rejected": -0.9177766442298889,
+      "logps/chosen": -41.11238098144531,
+      "logps/rejected": -215.87216186523438,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.6314110159873962,
+      "rewards/margins": 13.714380264282227,
+      "rewards/rejected": -14.34579086303711,
+      "step": 896
+    },
+    {
+      "epoch": 0.06521739130434782,
+      "grad_norm": 0.00017639659927226603,
+      "learning_rate": 3.1970232130513367e-08,
+      "logits/chosen": -0.9255086779594421,
+      "logits/rejected": -0.8803011178970337,
+      "logps/chosen": -32.20260238647461,
+      "logps/rejected": -210.572509765625,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.6896236538887024,
+      "rewards/margins": 14.428664207458496,
+      "rewards/rejected": -15.118288040161133,
+      "step": 897
+    },
+    {
+      "epoch": 0.06529009742620329,
+      "grad_norm": 0.006236107088625431,
+      "learning_rate": 3.135900525405427e-08,
+      "logits/chosen": -0.8336039781570435,
+      "logits/rejected": -0.845130443572998,
+      "logps/chosen": -37.38412094116211,
+      "logps/rejected": -210.4228973388672,
+      "loss": 0.0001,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.3377266526222229,
+      "rewards/margins": 12.897287368774414,
+      "rewards/rejected": -13.235013961791992,
+      "step": 898
+    },
+    {
+      "epoch": 0.06536280354805875,
+      "grad_norm": 0.0024036304093897343,
+      "learning_rate": 3.0753488620222036e-08,
+      "logits/chosen": -0.8953197002410889,
+      "logits/rejected": -0.8082447052001953,
+      "logps/chosen": -34.03400421142578,
+      "logps/rejected": -204.01040649414062,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.45378509163856506,
+      "rewards/margins": 13.715612411499023,
+      "rewards/rejected": -14.169397354125977,
+      "step": 899
+    },
+    {
+      "epoch": 0.06543550966991421,
+      "grad_norm": 0.0010786674683913589,
+      "learning_rate": 3.015368960704584e-08,
+      "logits/chosen": -0.9218509197235107,
+      "logits/rejected": -0.8579545021057129,
+      "logps/chosen": -32.49974822998047,
+      "logps/rejected": -215.10537719726562,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.18748772144317627,
+      "rewards/margins": 14.115120887756348,
+      "rewards/rejected": -14.302607536315918,
+      "step": 900
     }
   ],
   "logging_steps": 1,