Instructions to use JoshMe1/11ab9819-b1e0-442b-b8e4-964312311ca4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use JoshMe1/11ab9819-b1e0-442b-b8e4-964312311ca4 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="JoshMe1/11ab9819-b1e0-442b-b8e4-964312311ca4")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("JoshMe1/11ab9819-b1e0-442b-b8e4-964312311ca4")
model = AutoModelForCausalLM.from_pretrained("JoshMe1/11ab9819-b1e0-442b-b8e4-964312311ca4", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use JoshMe1/11ab9819-b1e0-442b-b8e4-964312311ca4 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "JoshMe1/11ab9819-b1e0-442b-b8e4-964312311ca4"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "JoshMe1/11ab9819-b1e0-442b-b8e4-964312311ca4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/JoshMe1/11ab9819-b1e0-442b-b8e4-964312311ca4

SGLang

How to use JoshMe1/11ab9819-b1e0-442b-b8e4-964312311ca4 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "JoshMe1/11ab9819-b1e0-442b-b8e4-964312311ca4" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "JoshMe1/11ab9819-b1e0-442b-b8e4-964312311ca4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "JoshMe1/11ab9819-b1e0-442b-b8e4-964312311ca4" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "JoshMe1/11ab9819-b1e0-442b-b8e4-964312311ca4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use JoshMe1/11ab9819-b1e0-442b-b8e4-964312311ca4 with Docker Model Runner:
```
docker model run hf.co/JoshMe1/11ab9819-b1e0-442b-b8e4-964312311ca4
```

JoshMe1 commited on May 14, 2025

Commit

763c993

verified ·

1 Parent(s): f09dcb4

Training in progress, step 800, checkpoint

Browse files

Files changed (4) hide show

last-checkpoint/adapter_model.safetensors +1 -1
last-checkpoint/optimizer.pt +1 -1
last-checkpoint/scheduler.pt +1 -1
last-checkpoint/trainer_state.json +1502 -2

last-checkpoint/adapter_model.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:fc388cd49ae9782945b985af61ba1c98bb80164242320e8efe81a46a64f6e089
 size 1342238560

 version https://git-lfs.github.com/spec/v1
+oid sha256:2832f221e37c6a3eeb8a9283c4222adee164d6d339ef6eaf73a3b12eec12154b
 size 1342238560

last-checkpoint/optimizer.pt CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:fbf4951be84e703330158f285f191e22bad19fb2b0e5dcd2b14b957f627f8d2d
 size 682101844

 version https://git-lfs.github.com/spec/v1
+oid sha256:9991802a5016ab59805bfe886a37d8bca3f29ff422db8205e7f63a9d674e12d0
 size 682101844

last-checkpoint/scheduler.pt CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:0627b9793181d3f7cfa52d173f7b800ff718d34a000378ae574d8dca34e2e8f6
 size 1064

 version https://git-lfs.github.com/spec/v1
+oid sha256:246ca084214dc27b5762653dde5a6251678dabf4b1880f62d528a3b22fc7e637
 size 1064

last-checkpoint/trainer_state.json CHANGED Viewed

@@ -1,9 +1,9 @@
 {
   "best_metric": null,
   "best_model_checkpoint": null,
-  "epoch": 0.05089428529882216,
   "eval_steps": 500,
-  "global_step": 700,
   "is_hyper_param_search": false,
   "is_local_process_zero": true,
   "is_world_process_zero": true,
@@ -10507,6 +10507,1506 @@
       "rewards/margins": 14.29006576538086,
       "rewards/rejected": -14.86207389831543,
       "step": 700
     }
   ],
   "logging_steps": 1,

 {
   "best_metric": null,
   "best_model_checkpoint": null,
+  "epoch": 0.058164897484368185,
   "eval_steps": 500,
+  "global_step": 800,
   "is_hyper_param_search": false,
   "is_local_process_zero": true,
   "is_world_process_zero": true,
       "rewards/margins": 14.29006576538086,
       "rewards/rejected": -14.86207389831543,
       "step": 700
+    },
+    {
+      "epoch": 0.05096699142067762,
+      "grad_norm": 0.00016962472000159323,
+      "learning_rate": 2.484900266848824e-07,
+      "logits/chosen": -0.9526031017303467,
+      "logits/rejected": -0.9217356443405151,
+      "logps/chosen": -36.252708435058594,
+      "logps/rejected": -220.39859008789062,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.40994641184806824,
+      "rewards/margins": 14.971365928649902,
+      "rewards/rejected": -15.381312370300293,
+      "step": 701
+    },
+    {
+      "epoch": 0.05103969754253308,
+      "grad_norm": 0.0001438941981177777,
+      "learning_rate": 2.469831179394182e-07,
+      "logits/chosen": -0.8770197629928589,
+      "logits/rejected": -0.8578864336013794,
+      "logps/chosen": -36.073951721191406,
+      "logps/rejected": -220.23309326171875,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.5476633310317993,
+      "rewards/margins": 14.92567253112793,
+      "rewards/rejected": -15.473337173461914,
+      "step": 702
+    },
+    {
+      "epoch": 0.051112403664388545,
+      "grad_norm": 0.0002866745926439762,
+      "learning_rate": 2.4547929212481435e-07,
+      "logits/chosen": -0.8754183650016785,
+      "logits/rejected": -0.8728746771812439,
+      "logps/chosen": -34.195640563964844,
+      "logps/rejected": -215.84222412109375,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.1705656349658966,
+      "rewards/margins": 14.685879707336426,
+      "rewards/rejected": -14.8564453125,
+      "step": 703
+    },
+    {
+      "epoch": 0.051185109786244,
+      "grad_norm": 0.019100969657301903,
+      "learning_rate": 2.439785675647143e-07,
+      "logits/chosen": -0.9529399871826172,
+      "logits/rejected": -0.9230093359947205,
+      "logps/chosen": -30.495248794555664,
+      "logps/rejected": -216.925048828125,
+      "loss": 0.0002,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.41198036074638367,
+      "rewards/margins": 14.4360990524292,
+      "rewards/rejected": -14.848078727722168,
+      "step": 704
+    },
+    {
+      "epoch": 0.05125781590809946,
+      "grad_norm": 0.002025824738666415,
+      "learning_rate": 2.424809625449729e-07,
+      "logits/chosen": -0.9094721078872681,
+      "logits/rejected": -0.8607329726219177,
+      "logps/chosen": -36.99269485473633,
+      "logps/rejected": -210.490966796875,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.4871370792388916,
+      "rewards/margins": 12.869680404663086,
+      "rewards/rejected": -13.356817245483398,
+      "step": 705
+    },
+    {
+      "epoch": 0.051330522029954924,
+      "grad_norm": 0.0006507533835247159,
+      "learning_rate": 2.4098649531343494e-07,
+      "logits/chosen": -0.9695981740951538,
+      "logits/rejected": -0.9127803444862366,
+      "logps/chosen": -32.33372497558594,
+      "logps/rejected": -214.67095947265625,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.34919363260269165,
+      "rewards/margins": 14.252132415771484,
+      "rewards/rejected": -14.601325988769531,
+      "step": 706
+    },
+    {
+      "epoch": 0.05140322815181038,
+      "grad_norm": 0.004732044879347086,
+      "learning_rate": 2.39495184079712e-07,
+      "logits/chosen": -0.9018921852111816,
+      "logits/rejected": -0.8292985558509827,
+      "logps/chosen": -37.659297943115234,
+      "logps/rejected": -217.0321044921875,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.576061487197876,
+      "rewards/margins": 14.48731517791748,
+      "rewards/rejected": -15.063377380371094,
+      "step": 707
+    },
+    {
+      "epoch": 0.05147593427366584,
+      "grad_norm": 0.0028744235169142485,
+      "learning_rate": 2.380070470149605e-07,
+      "logits/chosen": -0.913748562335968,
+      "logits/rejected": -0.9386864304542542,
+      "logps/chosen": -36.09499740600586,
+      "logps/rejected": -205.8506622314453,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.8420953154563904,
+      "rewards/margins": 13.169838905334473,
+      "rewards/rejected": -14.011934280395508,
+      "step": 708
+    },
+    {
+      "epoch": 0.0515486403955213,
+      "grad_norm": 0.000163277582032606,
+      "learning_rate": 2.365221022516612e-07,
+      "logits/chosen": -0.8805249929428101,
+      "logits/rejected": -0.8968266248703003,
+      "logps/chosen": -33.87440490722656,
+      "logps/rejected": -219.28244018554688,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.4786233901977539,
+      "rewards/margins": 14.947343826293945,
+      "rewards/rejected": -15.425966262817383,
+      "step": 709
+    },
+    {
+      "epoch": 0.051621346517376765,
+      "grad_norm": 0.01185124833136797,
+      "learning_rate": 2.350403678833976e-07,
+      "logits/chosen": -0.9778631329536438,
+      "logits/rejected": -0.9422690272331238,
+      "logps/chosen": -40.79098129272461,
+      "logps/rejected": -204.85809326171875,
+      "loss": 0.0002,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.660285472869873,
+      "rewards/margins": 11.561246871948242,
+      "rewards/rejected": -12.221532821655273,
+      "step": 710
+    },
+    {
+      "epoch": 0.05169405263923222,
+      "grad_norm": 0.019607333466410637,
+      "learning_rate": 2.3356186196463496e-07,
+      "logits/chosen": -0.9117095470428467,
+      "logits/rejected": -0.8398680686950684,
+      "logps/chosen": -34.003517150878906,
+      "logps/rejected": -205.01531982421875,
+      "loss": 0.0002,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.5753993988037109,
+      "rewards/margins": 12.464205741882324,
+      "rewards/rejected": -13.039605140686035,
+      "step": 711
+    },
+    {
+      "epoch": 0.05176675876108768,
+      "grad_norm": 0.0005566957988776267,
+      "learning_rate": 2.3208660251050156e-07,
+      "logits/chosen": -0.9298917651176453,
+      "logits/rejected": -0.8853734731674194,
+      "logps/chosen": -37.82965850830078,
+      "logps/rejected": -217.22219848632812,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.6585885286331177,
+      "rewards/margins": 13.603300094604492,
+      "rewards/rejected": -14.26188850402832,
+      "step": 712
+    },
+    {
+      "epoch": 0.051839464882943144,
+      "grad_norm": 0.004182242788374424,
+      "learning_rate": 2.3061460749656841e-07,
+      "logits/chosen": -0.8634892702102661,
+      "logits/rejected": -0.8328802585601807,
+      "logps/chosen": -32.050048828125,
+      "logps/rejected": -218.27481079101562,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.33811381459236145,
+      "rewards/margins": 13.200772285461426,
+      "rewards/rejected": -13.538886070251465,
+      "step": 713
+    },
+    {
+      "epoch": 0.051912171004798606,
+      "grad_norm": 0.005979419220238924,
+      "learning_rate": 2.2914589485863012e-07,
+      "logits/chosen": -0.9607076644897461,
+      "logits/rejected": -0.8761736750602722,
+      "logps/chosen": -29.904586791992188,
+      "logps/rejected": -215.92449951171875,
+      "loss": 0.0001,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.4739420711994171,
+      "rewards/margins": 12.991318702697754,
+      "rewards/rejected": -13.46526050567627,
+      "step": 714
+    },
+    {
+      "epoch": 0.05198487712665406,
+      "grad_norm": 0.00045053259236738086,
+      "learning_rate": 2.2768048249248644e-07,
+      "logits/chosen": -0.8738826513290405,
+      "logits/rejected": -0.9393714070320129,
+      "logps/chosen": -34.599090576171875,
+      "logps/rejected": -212.1854248046875,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.49326005578041077,
+      "rewards/margins": 14.172708511352539,
+      "rewards/rejected": -14.66596794128418,
+      "step": 715
+    },
+    {
+      "epoch": 0.05205758324850952,
+      "grad_norm": 0.0076771085150539875,
+      "learning_rate": 2.262183882537249e-07,
+      "logits/chosen": -0.8194682598114014,
+      "logits/rejected": -0.909551739692688,
+      "logps/chosen": -34.328060150146484,
+      "logps/rejected": -207.3561248779297,
+      "loss": 0.0001,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.2956991493701935,
+      "rewards/margins": 12.504569053649902,
+      "rewards/rejected": -12.800268173217773,
+      "step": 716
+    },
+    {
+      "epoch": 0.052130289370364985,
+      "grad_norm": 0.0008858201326802373,
+      "learning_rate": 2.247596299575022e-07,
+      "logits/chosen": -0.8622158765792847,
+      "logits/rejected": -0.8925617933273315,
+      "logps/chosen": -36.37249755859375,
+      "logps/rejected": -210.2269287109375,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.21173694729804993,
+      "rewards/margins": 13.570272445678711,
+      "rewards/rejected": -13.78200912475586,
+      "step": 717
+    },
+    {
+      "epoch": 0.05220299549222045,
+      "grad_norm": 0.004414277616888285,
+      "learning_rate": 2.23304225378328e-07,
+      "logits/chosen": -0.9940564632415771,
+      "logits/rejected": -0.8593904972076416,
+      "logps/chosen": -30.40024757385254,
+      "logps/rejected": -211.74139404296875,
+      "loss": 0.0001,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.23217126727104187,
+      "rewards/margins": 12.863700866699219,
+      "rewards/rejected": -13.095871925354004,
+      "step": 718
+    },
+    {
+      "epoch": 0.0522757016140759,
+      "grad_norm": 0.0007747263880446553,
+      "learning_rate": 2.218521922498476e-07,
+      "logits/chosen": -0.9582018256187439,
+      "logits/rejected": -0.9187099933624268,
+      "logps/chosen": -35.134254455566406,
+      "logps/rejected": -197.65711975097656,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.3801981210708618,
+      "rewards/margins": 12.766691207885742,
+      "rewards/rejected": -13.146888732910156,
+      "step": 719
+    },
+    {
+      "epoch": 0.052348407735931364,
+      "grad_norm": 0.004136708099395037,
+      "learning_rate": 2.2040354826462664e-07,
+      "logits/chosen": -0.9472717046737671,
+      "logits/rejected": -0.7926496863365173,
+      "logps/chosen": -30.805456161499023,
+      "logps/rejected": -209.50125122070312,
+      "loss": 0.0001,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": 0.05050273239612579,
+      "rewards/margins": 12.728914260864258,
+      "rewards/rejected": -12.678410530090332,
+      "step": 720
+    },
+    {
+      "epoch": 0.052421113857786826,
+      "grad_norm": 9.901488374453038e-05,
+      "learning_rate": 2.1895831107393482e-07,
+      "logits/chosen": -0.9380213022232056,
+      "logits/rejected": -0.931056022644043,
+      "logps/chosen": -30.888587951660156,
+      "logps/rejected": -218.29710388183594,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.3042806386947632,
+      "rewards/margins": 14.971654891967773,
+      "rewards/rejected": -15.275935173034668,
+      "step": 721
+    },
+    {
+      "epoch": 0.05249381997964229,
+      "grad_norm": 0.00036812620237469673,
+      "learning_rate": 2.1751649828753106e-07,
+      "logits/chosen": -0.8787797093391418,
+      "logits/rejected": -0.923054575920105,
+      "logps/chosen": -35.658294677734375,
+      "logps/rejected": -209.59725952148438,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.585743248462677,
+      "rewards/margins": 14.103273391723633,
+      "rewards/rejected": -14.689016342163086,
+      "step": 722
+    },
+    {
+      "epoch": 0.05256652610149774,
+      "grad_norm": 0.007818679325282574,
+      "learning_rate": 2.160781274734495e-07,
+      "logits/chosen": -0.8768399357795715,
+      "logits/rejected": -0.8340914845466614,
+      "logps/chosen": -32.2601318359375,
+      "logps/rejected": -211.66159057617188,
+      "loss": 0.0001,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.616025447845459,
+      "rewards/margins": 13.163147926330566,
+      "rewards/rejected": -13.779172897338867,
+      "step": 723
+    },
+    {
+      "epoch": 0.052639232223353205,
+      "grad_norm": 0.013885089196264744,
+      "learning_rate": 2.146432161577842e-07,
+      "logits/chosen": -0.9050265550613403,
+      "logits/rejected": -0.8358237147331238,
+      "logps/chosen": -40.09930419921875,
+      "logps/rejected": -208.87554931640625,
+      "loss": 0.0002,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.6807125210762024,
+      "rewards/margins": 12.531813621520996,
+      "rewards/rejected": -13.212526321411133,
+      "step": 724
+    },
+    {
+      "epoch": 0.05271193834520867,
+      "grad_norm": 0.00024435605155304074,
+      "learning_rate": 2.1321178182447709e-07,
+      "logits/chosen": -0.950187087059021,
+      "logits/rejected": -0.9424600601196289,
+      "logps/chosen": -40.063987731933594,
+      "logps/rejected": -214.86441040039062,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.8616589307785034,
+      "rewards/margins": 14.418479919433594,
+      "rewards/rejected": -15.280138969421387,
+      "step": 725
+    },
+    {
+      "epoch": 0.05278464446706413,
+      "grad_norm": 0.0031122479122132063,
+      "learning_rate": 2.117838419151034e-07,
+      "logits/chosen": -0.9120379686355591,
+      "logits/rejected": -0.9193058609962463,
+      "logps/chosen": -36.915550231933594,
+      "logps/rejected": -212.9740753173828,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.30296823382377625,
+      "rewards/margins": 13.31496524810791,
+      "rewards/rejected": -13.61793327331543,
+      "step": 726
+    },
+    {
+      "epoch": 0.052857350588919584,
+      "grad_norm": 0.0017582379514351487,
+      "learning_rate": 2.1035941382866067e-07,
+      "logits/chosen": -0.9938796162605286,
+      "logits/rejected": -0.882668137550354,
+      "logps/chosen": -28.024808883666992,
+      "logps/rejected": -203.543212890625,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.15884625911712646,
+      "rewards/margins": 12.8031644821167,
+      "rewards/rejected": -12.962010383605957,
+      "step": 727
+    },
+    {
+      "epoch": 0.052930056710775046,
+      "grad_norm": 0.0002076653327094391,
+      "learning_rate": 2.0893851492135532e-07,
+      "logits/chosen": -0.9264384508132935,
+      "logits/rejected": -0.9043711423873901,
+      "logps/chosen": -41.037567138671875,
+      "logps/rejected": -213.83316040039062,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -1.0929769277572632,
+      "rewards/margins": 14.023452758789062,
+      "rewards/rejected": -15.116429328918457,
+      "step": 728
+    },
+    {
+      "epoch": 0.05300276283263051,
+      "grad_norm": 0.0037191887386143208,
+      "learning_rate": 2.0752116250639224e-07,
+      "logits/chosen": -0.9239534735679626,
+      "logits/rejected": -0.8673202395439148,
+      "logps/chosen": -30.643512725830078,
+      "logps/rejected": -217.33995056152344,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.34290575981140137,
+      "rewards/margins": 13.718175888061523,
+      "rewards/rejected": -14.06108283996582,
+      "step": 729
+    },
+    {
+      "epoch": 0.05307546895448597,
+      "grad_norm": 0.0008325205999426544,
+      "learning_rate": 2.0610737385376348e-07,
+      "logits/chosen": -0.8706562519073486,
+      "logits/rejected": -0.8877383470535278,
+      "logps/chosen": -41.10457992553711,
+      "logps/rejected": -212.40354919433594,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.8662700057029724,
+      "rewards/margins": 14.034844398498535,
+      "rewards/rejected": -14.901114463806152,
+      "step": 730
+    },
+    {
+      "epoch": 0.053148175076341425,
+      "grad_norm": 0.003004083875566721,
+      "learning_rate": 2.0469716619003723e-07,
+      "logits/chosen": -0.9021568894386292,
+      "logits/rejected": -0.8215603828430176,
+      "logps/chosen": -28.95623779296875,
+      "logps/rejected": -213.17466735839844,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.12656313180923462,
+      "rewards/margins": 14.443097114562988,
+      "rewards/rejected": -14.569660186767578,
+      "step": 731
+    },
+    {
+      "epoch": 0.05322088119819689,
+      "grad_norm": 0.0017041056416928768,
+      "learning_rate": 2.0329055669814933e-07,
+      "logits/chosen": -1.0070698261260986,
+      "logits/rejected": -0.891922116279602,
+      "logps/chosen": -30.297428131103516,
+      "logps/rejected": -210.99365234375,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.461770236492157,
+      "rewards/margins": 13.198616027832031,
+      "rewards/rejected": -13.66038703918457,
+      "step": 732
+    },
+    {
+      "epoch": 0.05329358732005235,
+      "grad_norm": 0.004877008497714996,
+      "learning_rate": 2.0188756251719203e-07,
+      "logits/chosen": -1.0166568756103516,
+      "logits/rejected": -0.9349803924560547,
+      "logps/chosen": -35.411537170410156,
+      "logps/rejected": -214.6336669921875,
+      "loss": 0.0001,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.6040887832641602,
+      "rewards/margins": 13.180540084838867,
+      "rewards/rejected": -13.784629821777344,
+      "step": 733
+    },
+    {
+      "epoch": 0.05336629344190781,
+      "grad_norm": 0.001830337685532868,
+      "learning_rate": 2.0048820074220711e-07,
+      "logits/chosen": -0.8823634386062622,
+      "logits/rejected": -0.7977582216262817,
+      "logps/chosen": -27.982765197753906,
+      "logps/rejected": -199.20285034179688,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.13698601722717285,
+      "rewards/margins": 12.950643539428711,
+      "rewards/rejected": -13.087629318237305,
+      "step": 734
+    },
+    {
+      "epoch": 0.053438999563763266,
+      "grad_norm": 0.0004967046552337706,
+      "learning_rate": 1.990924884239758e-07,
+      "logits/chosen": -0.909273087978363,
+      "logits/rejected": -0.8297127485275269,
+      "logps/chosen": -35.87456130981445,
+      "logps/rejected": -210.33042907714844,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.3381763696670532,
+      "rewards/margins": 13.830402374267578,
+      "rewards/rejected": -14.168578147888184,
+      "step": 735
+    },
+    {
+      "epoch": 0.05351170568561873,
+      "grad_norm": 0.0003948746307287365,
+      "learning_rate": 1.9770044256881258e-07,
+      "logits/chosen": -0.9478728771209717,
+      "logits/rejected": -0.8719102144241333,
+      "logps/chosen": -34.23468017578125,
+      "logps/rejected": -210.3759765625,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.6024740934371948,
+      "rewards/margins": 13.7274169921875,
+      "rewards/rejected": -14.329891204833984,
+      "step": 736
+    },
+    {
+      "epoch": 0.05358441180747419,
+      "grad_norm": 0.0023601241409778595,
+      "learning_rate": 1.9631208013835676e-07,
+      "logits/chosen": -0.9740806818008423,
+      "logits/rejected": -0.9044991731643677,
+      "logps/chosen": -41.01429748535156,
+      "logps/rejected": -209.04383850097656,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.5482434034347534,
+      "rewards/margins": 13.41301155090332,
+      "rewards/rejected": -13.96125602722168,
+      "step": 737
+    },
+    {
+      "epoch": 0.05365711792932965,
+      "grad_norm": 0.0020352157298475504,
+      "learning_rate": 1.9492741804936618e-07,
+      "logits/chosen": -0.8628606796264648,
+      "logits/rejected": -0.7685221433639526,
+      "logps/chosen": -40.9301643371582,
+      "logps/rejected": -220.81875610351562,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.6600879430770874,
+      "rewards/margins": 13.031375885009766,
+      "rewards/rejected": -13.691463470458984,
+      "step": 738
+    },
+    {
+      "epoch": 0.05372982405118511,
+      "grad_norm": 0.0009932757820934057,
+      "learning_rate": 1.9354647317351187e-07,
+      "logits/chosen": -0.865092396736145,
+      "logits/rejected": -0.831990122795105,
+      "logps/chosen": -38.46998596191406,
+      "logps/rejected": -223.56800842285156,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.4226331412792206,
+      "rewards/margins": 13.693107604980469,
+      "rewards/rejected": -14.115740776062012,
+      "step": 739
+    },
+    {
+      "epoch": 0.05380253017304057,
+      "grad_norm": 0.0022304505109786987,
+      "learning_rate": 1.9216926233717084e-07,
+      "logits/chosen": -0.9461253881454468,
+      "logits/rejected": -0.9268235564231873,
+      "logps/chosen": -29.67384910583496,
+      "logps/rejected": -204.46505737304688,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.3023516833782196,
+      "rewards/margins": 13.467647552490234,
+      "rewards/rejected": -13.770000457763672,
+      "step": 740
+    },
+    {
+      "epoch": 0.05387523629489603,
+      "grad_norm": 0.003809142392128706,
+      "learning_rate": 1.90795802321223e-07,
+      "logits/chosen": -0.8494369387626648,
+      "logits/rejected": -0.9125988483428955,
+      "logps/chosen": -32.53688049316406,
+      "logps/rejected": -214.2470703125,
+      "loss": 0.0001,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.42231428623199463,
+      "rewards/margins": 13.579572677612305,
+      "rewards/rejected": -14.001887321472168,
+      "step": 741
+    },
+    {
+      "epoch": 0.05394794241675149,
+      "grad_norm": 0.0017715893918648362,
+      "learning_rate": 1.8942610986084484e-07,
+      "logits/chosen": -1.00579833984375,
+      "logits/rejected": -0.9136815071105957,
+      "logps/chosen": -30.22631072998047,
+      "logps/rejected": -202.64682006835938,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.23207417130470276,
+      "rewards/margins": 12.629378318786621,
+      "rewards/rejected": -12.86145305633545,
+      "step": 742
+    },
+    {
+      "epoch": 0.05402064853860695,
+      "grad_norm": 0.0003675991320051253,
+      "learning_rate": 1.8806020164530701e-07,
+      "logits/chosen": -0.8189449310302734,
+      "logits/rejected": -0.8303515911102295,
+      "logps/chosen": -39.320465087890625,
+      "logps/rejected": -218.82669067382812,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.5943813323974609,
+      "rewards/margins": 14.660420417785645,
+      "rewards/rejected": -15.254801750183105,
+      "step": 743
+    },
+    {
+      "epoch": 0.05409335466046241,
+      "grad_norm": 0.01240138616412878,
+      "learning_rate": 1.8669809431776988e-07,
+      "logits/chosen": -1.0041861534118652,
+      "logits/rejected": -0.8971571922302246,
+      "logps/chosen": -28.070751190185547,
+      "logps/rejected": -209.87245178222656,
+      "loss": 0.0002,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.11745762825012207,
+      "rewards/margins": 12.85346794128418,
+      "rewards/rejected": -12.970926284790039,
+      "step": 744
+    },
+    {
+      "epoch": 0.05416606078231787,
+      "grad_norm": 0.0025072619318962097,
+      "learning_rate": 1.8533980447508135e-07,
+      "logits/chosen": -0.9117828607559204,
+      "logits/rejected": -0.8178045153617859,
+      "logps/chosen": -35.66990661621094,
+      "logps/rejected": -205.7391357421875,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.38360828161239624,
+      "rewards/margins": 12.453069686889648,
+      "rewards/rejected": -12.836677551269531,
+      "step": 745
+    },
+    {
+      "epoch": 0.054238766904173334,
+      "grad_norm": 0.0012046655174344778,
+      "learning_rate": 1.8398534866757455e-07,
+      "logits/chosen": -0.8388931155204773,
+      "logits/rejected": -0.8091910481452942,
+      "logps/chosen": -32.421051025390625,
+      "logps/rejected": -217.2982940673828,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.3685145080089569,
+      "rewards/margins": 14.191152572631836,
+      "rewards/rejected": -14.559667587280273,
+      "step": 746
+    },
+    {
+      "epoch": 0.05431147302602879,
+      "grad_norm": 0.008580892346799374,
+      "learning_rate": 1.8263474339886624e-07,
+      "logits/chosen": -1.011499047279358,
+      "logits/rejected": -0.9227244853973389,
+      "logps/chosen": -33.962013244628906,
+      "logps/rejected": -208.20162963867188,
+      "loss": 0.0001,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.379577100276947,
+      "rewards/margins": 12.571484565734863,
+      "rewards/rejected": -12.951061248779297,
+      "step": 747
+    },
+    {
+      "epoch": 0.05438417914788425,
+      "grad_norm": 0.0072589050978422165,
+      "learning_rate": 1.812880051256551e-07,
+      "logits/chosen": -0.9485992789268494,
+      "logits/rejected": -0.9038834571838379,
+      "logps/chosen": -30.243196487426758,
+      "logps/rejected": -204.98263549804688,
+      "loss": 0.0001,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.45122310519218445,
+      "rewards/margins": 12.676835060119629,
+      "rewards/rejected": -13.128058433532715,
+      "step": 748
+    },
+    {
+      "epoch": 0.05445688526973971,
+      "grad_norm": 0.0006042845197953284,
+      "learning_rate": 1.7994515025752216e-07,
+      "logits/chosen": -0.8617667555809021,
+      "logits/rejected": -0.9311653971672058,
+      "logps/chosen": -42.464134216308594,
+      "logps/rejected": -213.90817260742188,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.7296816110610962,
+      "rewards/margins": 13.28490924835205,
+      "rewards/rejected": -14.014591217041016,
+      "step": 749
+    },
+    {
+      "epoch": 0.054529591391595175,
+      "grad_norm": 0.008451288565993309,
+      "learning_rate": 1.7860619515673032e-07,
+      "logits/chosen": -0.8343260288238525,
+      "logits/rejected": -0.9304801821708679,
+      "logps/chosen": -36.72785186767578,
+      "logps/rejected": -206.89028930664062,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.651132345199585,
+      "rewards/margins": 12.767133712768555,
+      "rewards/rejected": -13.418266296386719,
+      "step": 750
+    },
+    {
+      "epoch": 0.05460229751345063,
+      "grad_norm": 0.0019508537370711565,
+      "learning_rate": 1.7727115613802464e-07,
+      "logits/chosen": -0.9755252599716187,
+      "logits/rejected": -0.9243016242980957,
+      "logps/chosen": -31.149749755859375,
+      "logps/rejected": -215.17953491210938,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.1917463093996048,
+      "rewards/margins": 13.494813919067383,
+      "rewards/rejected": -13.686559677124023,
+      "step": 751
+    },
+    {
+      "epoch": 0.05467500363530609,
+      "grad_norm": 0.013230530545115471,
+      "learning_rate": 1.7594004946843454e-07,
+      "logits/chosen": -0.8957071304321289,
+      "logits/rejected": -0.8952853083610535,
+      "logps/chosen": -37.80425262451172,
+      "logps/rejected": -213.73846435546875,
+      "loss": 0.0002,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.5001394152641296,
+      "rewards/margins": 14.075218200683594,
+      "rewards/rejected": -14.575357437133789,
+      "step": 752
+    },
+    {
+      "epoch": 0.054747709757161554,
+      "grad_norm": 0.010208229534327984,
+      "learning_rate": 1.7461289136707457e-07,
+      "logits/chosen": -0.9123841524124146,
+      "logits/rejected": -0.8014664053916931,
+      "logps/chosen": -32.373016357421875,
+      "logps/rejected": -211.44223022460938,
+      "loss": 0.0001,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.5280441045761108,
+      "rewards/margins": 13.349374771118164,
+      "rewards/rejected": -13.877418518066406,
+      "step": 753
+    },
+    {
+      "epoch": 0.054820415879017016,
+      "grad_norm": 0.005788872018456459,
+      "learning_rate": 1.7328969800494726e-07,
+      "logits/chosen": -0.951117753982544,
+      "logits/rejected": -0.9061521291732788,
+      "logps/chosen": -26.266429901123047,
+      "logps/rejected": -200.56845092773438,
+      "loss": 0.0001,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.18065418303012848,
+      "rewards/margins": 12.40198040008545,
+      "rewards/rejected": -12.582633972167969,
+      "step": 754
+    },
+    {
+      "epoch": 0.05489312200087247,
+      "grad_norm": 0.003130243858322501,
+      "learning_rate": 1.7197048550474641e-07,
+      "logits/chosen": -0.9353317618370056,
+      "logits/rejected": -0.8683757781982422,
+      "logps/chosen": -31.96941566467285,
+      "logps/rejected": -217.26443481445312,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.1149619072675705,
+      "rewards/margins": 14.614456176757812,
+      "rewards/rejected": -14.729418754577637,
+      "step": 755
+    },
+    {
+      "epoch": 0.05496582812272793,
+      "grad_norm": 0.0008835052140057087,
+      "learning_rate": 1.7065526994065972e-07,
+      "logits/chosen": -0.9726742506027222,
+      "logits/rejected": -0.8771126866340637,
+      "logps/chosen": -43.78021240234375,
+      "logps/rejected": -221.291748046875,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -1.2411898374557495,
+      "rewards/margins": 13.419130325317383,
+      "rewards/rejected": -14.660320281982422,
+      "step": 756
+    },
+    {
+      "epoch": 0.055038534244583395,
+      "grad_norm": 0.00016662158304825425,
+      "learning_rate": 1.6934406733817413e-07,
+      "logits/chosen": -0.8647419214248657,
+      "logits/rejected": -0.8561787605285645,
+      "logps/chosen": -32.77350616455078,
+      "logps/rejected": -215.56143188476562,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.3826359212398529,
+      "rewards/margins": 14.84445571899414,
+      "rewards/rejected": -15.227091789245605,
+      "step": 757
+    },
+    {
+      "epoch": 0.05511124036643886,
+      "grad_norm": 0.003062991425395012,
+      "learning_rate": 1.6803689367387918e-07,
+      "logits/chosen": -0.909042477607727,
+      "logits/rejected": -0.8357021808624268,
+      "logps/chosen": -39.56062316894531,
+      "logps/rejected": -203.7021026611328,
+      "loss": 0.0001,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.46654921770095825,
+      "rewards/margins": 12.062686920166016,
+      "rewards/rejected": -12.52923583984375,
+      "step": 758
+    },
+    {
+      "epoch": 0.05518394648829431,
+      "grad_norm": 0.04407789930701256,
+      "learning_rate": 1.667337648752738e-07,
+      "logits/chosen": -0.9962819814682007,
+      "logits/rejected": -0.8614116907119751,
+      "logps/chosen": -37.062164306640625,
+      "logps/rejected": -201.96527099609375,
+      "loss": 0.0006,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.6556903123855591,
+      "rewards/margins": 11.274649620056152,
+      "rewards/rejected": -11.930339813232422,
+      "step": 759
+    },
+    {
+      "epoch": 0.055256652610149774,
+      "grad_norm": 0.008902635425329208,
+      "learning_rate": 1.6543469682057104e-07,
+      "logits/chosen": -0.967126190662384,
+      "logits/rejected": -0.8661681413650513,
+      "logps/chosen": -24.940128326416016,
+      "logps/rejected": -202.6082000732422,
+      "loss": 0.0001,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.12640663981437683,
+      "rewards/margins": 11.179431915283203,
+      "rewards/rejected": -11.305837631225586,
+      "step": 760
+    },
+    {
+      "epoch": 0.055329358732005236,
+      "grad_norm": 0.001392183592543006,
+      "learning_rate": 1.6413970533850497e-07,
+      "logits/chosen": -0.9138966202735901,
+      "logits/rejected": -0.8320655822753906,
+      "logps/chosen": -32.464195251464844,
+      "logps/rejected": -213.43788146972656,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.3245856463909149,
+      "rewards/margins": 13.77605152130127,
+      "rewards/rejected": -14.100637435913086,
+      "step": 761
+    },
+    {
+      "epoch": 0.0554020648538607,
+      "grad_norm": 0.0030789838638156652,
+      "learning_rate": 1.6284880620813846e-07,
+      "logits/chosen": -0.9797265529632568,
+      "logits/rejected": -0.8685609102249146,
+      "logps/chosen": -28.6915283203125,
+      "logps/rejected": -204.46298217773438,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": 0.10937747359275818,
+      "rewards/margins": 12.886858940124512,
+      "rewards/rejected": -12.777481079101562,
+      "step": 762
+    },
+    {
+      "epoch": 0.05547477097571615,
+      "grad_norm": 0.00035047787241637707,
+      "learning_rate": 1.6156201515866968e-07,
+      "logits/chosen": -0.9586347341537476,
+      "logits/rejected": -0.9675611257553101,
+      "logps/chosen": -40.65100860595703,
+      "logps/rejected": -220.50894165039062,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.5836011171340942,
+      "rewards/margins": 14.78853988647461,
+      "rewards/rejected": -15.372140884399414,
+      "step": 763
+    },
+    {
+      "epoch": 0.055547477097571615,
+      "grad_norm": 0.021464752033352852,
+      "learning_rate": 1.6027934786924185e-07,
+      "logits/chosen": -0.96466064453125,
+      "logits/rejected": -0.8601709604263306,
+      "logps/chosen": -31.03888511657715,
+      "logps/rejected": -202.77810668945312,
+      "loss": 0.0003,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.11492705345153809,
+      "rewards/margins": 12.924112319946289,
+      "rewards/rejected": -13.039039611816406,
+      "step": 764
+    },
+    {
+      "epoch": 0.05562018321942708,
+      "grad_norm": 0.00018579527386464179,
+      "learning_rate": 1.590008199687508e-07,
+      "logits/chosen": -0.87689208984375,
+      "logits/rejected": -0.8554306030273438,
+      "logps/chosen": -35.217952728271484,
+      "logps/rejected": -208.33447265625,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.5234267711639404,
+      "rewards/margins": 14.254180908203125,
+      "rewards/rejected": -14.777606964111328,
+      "step": 765
+    },
+    {
+      "epoch": 0.05569288934128254,
+      "grad_norm": 0.006527530495077372,
+      "learning_rate": 1.5772644703565564e-07,
+      "logits/chosen": -0.934159517288208,
+      "logits/rejected": -0.8871695399284363,
+      "logps/chosen": -38.746253967285156,
+      "logps/rejected": -223.29010009765625,
+      "loss": 0.0001,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.7018031477928162,
+      "rewards/margins": 12.746376991271973,
+      "rewards/rejected": -13.448179244995117,
+      "step": 766
+    },
+    {
+      "epoch": 0.055765595463137994,
+      "grad_norm": 0.010887491516768932,
+      "learning_rate": 1.5645624459778855e-07,
+      "logits/chosen": -0.8821107149124146,
+      "logits/rejected": -0.8527417182922363,
+      "logps/chosen": -37.610591888427734,
+      "logps/rejected": -212.83218383789062,
+      "loss": 0.0001,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.8018917441368103,
+      "rewards/margins": 13.662994384765625,
+      "rewards/rejected": -14.464885711669922,
+      "step": 767
+    },
+    {
+      "epoch": 0.055838301584993456,
+      "grad_norm": 0.001788746565580368,
+      "learning_rate": 1.551902281321651e-07,
+      "logits/chosen": -0.9478468894958496,
+      "logits/rejected": -0.9769448637962341,
+      "logps/chosen": -32.67488098144531,
+      "logps/rejected": -209.9729461669922,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.40322765707969666,
+      "rewards/margins": 14.15975570678711,
+      "rewards/rejected": -14.562983512878418,
+      "step": 768
+    },
+    {
+      "epoch": 0.05591100770684892,
+      "grad_norm": 0.008931302465498447,
+      "learning_rate": 1.5392841306479666e-07,
+      "logits/chosen": -0.9474010467529297,
+      "logits/rejected": -0.8884421586990356,
+      "logps/chosen": -34.60919189453125,
+      "logps/rejected": -211.77670288085938,
+      "loss": 0.0001,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.359250545501709,
+      "rewards/margins": 12.960168838500977,
+      "rewards/rejected": -13.319419860839844,
+      "step": 769
+    },
+    {
+      "epoch": 0.05598371382870438,
+      "grad_norm": 0.0068222894333302975,
+      "learning_rate": 1.5267081477050131e-07,
+      "logits/chosen": -0.891141414642334,
+      "logits/rejected": -0.9084587097167969,
+      "logps/chosen": -41.72639083862305,
+      "logps/rejected": -210.28097534179688,
+      "loss": 0.0001,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.5299360156059265,
+      "rewards/margins": 13.119318962097168,
+      "rewards/rejected": -13.64925479888916,
+      "step": 770
+    },
+    {
+      "epoch": 0.056056419950559835,
+      "grad_norm": 9.367970051243901e-05,
+      "learning_rate": 1.5141744857271778e-07,
+      "logits/chosen": -0.913861095905304,
+      "logits/rejected": -0.8445248603820801,
+      "logps/chosen": -33.63587951660156,
+      "logps/rejected": -225.3160858154297,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.6350897550582886,
+      "rewards/margins": 15.599540710449219,
+      "rewards/rejected": -16.234630584716797,
+      "step": 771
+    },
+    {
+      "epoch": 0.0561291260724153,
+      "grad_norm": 0.00025686310254968703,
+      "learning_rate": 1.5016832974331723e-07,
+      "logits/chosen": -0.9397752285003662,
+      "logits/rejected": -0.8799442052841187,
+      "logps/chosen": -34.88556671142578,
+      "logps/rejected": -216.75021362304688,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.5901429653167725,
+      "rewards/margins": 14.332117080688477,
+      "rewards/rejected": -14.922258377075195,
+      "step": 772
+    },
+    {
+      "epoch": 0.05620183219427076,
+      "grad_norm": 0.0031295663211494684,
+      "learning_rate": 1.489234735024188e-07,
+      "logits/chosen": -0.9795909523963928,
+      "logits/rejected": -0.8580805063247681,
+      "logps/chosen": -33.576663970947266,
+      "logps/rejected": -205.09866333007812,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.19408807158470154,
+      "rewards/margins": 13.219499588012695,
+      "rewards/rejected": -13.41358757019043,
+      "step": 773
+    },
+    {
+      "epoch": 0.056274538316126214,
+      "grad_norm": 0.0016190169844776392,
+      "learning_rate": 1.4768289501820263e-07,
+      "logits/chosen": -0.9007223844528198,
+      "logits/rejected": -0.8394637703895569,
+      "logps/chosen": -34.12432861328125,
+      "logps/rejected": -207.40000915527344,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.3453722894191742,
+      "rewards/margins": 12.839717864990234,
+      "rewards/rejected": -13.185091018676758,
+      "step": 774
+    },
+    {
+      "epoch": 0.056347244437981676,
+      "grad_norm": 0.00919532310217619,
+      "learning_rate": 1.4644660940672627e-07,
+      "logits/chosen": -0.9666440486907959,
+      "logits/rejected": -0.8888868093490601,
+      "logps/chosen": -35.73973846435547,
+      "logps/rejected": -201.26190185546875,
+      "loss": 0.0001,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.6386173367500305,
+      "rewards/margins": 11.141023635864258,
+      "rewards/rejected": -11.779642105102539,
+      "step": 775
+    },
+    {
+      "epoch": 0.05641995055983714,
+      "grad_norm": 0.0018441978609189391,
+      "learning_rate": 1.4521463173173965e-07,
+      "logits/chosen": -0.8597419261932373,
+      "logits/rejected": -0.8423570394515991,
+      "logps/chosen": -35.59410858154297,
+      "logps/rejected": -211.53579711914062,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.631655752658844,
+      "rewards/margins": 13.613819122314453,
+      "rewards/rejected": -14.245475769042969,
+      "step": 776
+    },
+    {
+      "epoch": 0.0564926566816926,
+      "grad_norm": 0.002046742243692279,
+      "learning_rate": 1.439869770045018e-07,
+      "logits/chosen": -0.9244673252105713,
+      "logits/rejected": -0.8332409858703613,
+      "logps/chosen": -31.977285385131836,
+      "logps/rejected": -212.346923828125,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.3353448212146759,
+      "rewards/margins": 12.832916259765625,
+      "rewards/rejected": -13.168261528015137,
+      "step": 777
+    },
+    {
+      "epoch": 0.056565362803548055,
+      "grad_norm": 0.0026525191497057676,
+      "learning_rate": 1.4276366018359842e-07,
+      "logits/chosen": -0.9291462898254395,
+      "logits/rejected": -0.8785039186477661,
+      "logps/chosen": -34.03281784057617,
+      "logps/rejected": -210.6335906982422,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.37746644020080566,
+      "rewards/margins": 13.856457710266113,
+      "rewards/rejected": -14.233924865722656,
+      "step": 778
+    },
+    {
+      "epoch": 0.05663806892540352,
+      "grad_norm": 0.0008529649348929524,
+      "learning_rate": 1.4154469617475862e-07,
+      "logits/chosen": -0.9258161783218384,
+      "logits/rejected": -0.8843492269515991,
+      "logps/chosen": -31.135189056396484,
+      "logps/rejected": -214.30499267578125,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.17117196321487427,
+      "rewards/margins": 13.937726974487305,
+      "rewards/rejected": -14.10890007019043,
+      "step": 779
+    },
+    {
+      "epoch": 0.05671077504725898,
+      "grad_norm": 0.0008481541299261153,
+      "learning_rate": 1.4033009983067452e-07,
+      "logits/chosen": -0.8543636798858643,
+      "logits/rejected": -0.9436533451080322,
+      "logps/chosen": -35.73031997680664,
+      "logps/rejected": -207.31524658203125,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.6781507134437561,
+      "rewards/margins": 13.80234146118164,
+      "rewards/rejected": -14.48049259185791,
+      "step": 780
+    },
+    {
+      "epoch": 0.05678348116911444,
+      "grad_norm": 0.0035167636815458536,
+      "learning_rate": 1.391198859508189e-07,
+      "logits/chosen": -0.9763043522834778,
+      "logits/rejected": -0.9388880729675293,
+      "logps/chosen": -38.95911407470703,
+      "logps/rejected": -223.01495361328125,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.8786532282829285,
+      "rewards/margins": 14.221936225891113,
+      "rewards/rejected": -15.100589752197266,
+      "step": 781
+    },
+    {
+      "epoch": 0.056856187290969896,
+      "grad_norm": 0.0006581367924809456,
+      "learning_rate": 1.3791406928126635e-07,
+      "logits/chosen": -0.8602852821350098,
+      "logits/rejected": -0.8619078993797302,
+      "logps/chosen": -42.86963653564453,
+      "logps/rejected": -219.94992065429688,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.9679461717605591,
+      "rewards/margins": 14.3308687210083,
+      "rewards/rejected": -15.29881477355957,
+      "step": 782
+    },
+    {
+      "epoch": 0.05692889341282536,
+      "grad_norm": 0.0033537691924721003,
+      "learning_rate": 1.3671266451451207e-07,
+      "logits/chosen": -0.9410476684570312,
+      "logits/rejected": -0.819915771484375,
+      "logps/chosen": -32.9018669128418,
+      "logps/rejected": -202.12039184570312,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.16996833682060242,
+      "rewards/margins": 12.449856758117676,
+      "rewards/rejected": -12.61982536315918,
+      "step": 783
+    },
+    {
+      "epoch": 0.05700159953468082,
+      "grad_norm": 0.009281881153583527,
+      "learning_rate": 1.3551568628929432e-07,
+      "logits/chosen": -0.9356061220169067,
+      "logits/rejected": -0.8372440934181213,
+      "logps/chosen": -37.815330505371094,
+      "logps/rejected": -205.18292236328125,
+      "loss": 0.0001,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.39503559470176697,
+      "rewards/margins": 12.175322532653809,
+      "rewards/rejected": -12.570358276367188,
+      "step": 784
+    },
+    {
+      "epoch": 0.05707430565653628,
+      "grad_norm": 0.014241136610507965,
+      "learning_rate": 1.3432314919041477e-07,
+      "logits/chosen": -0.9060748219490051,
+      "logits/rejected": -0.8896147012710571,
+      "logps/chosen": -36.553627014160156,
+      "logps/rejected": -205.4870147705078,
+      "loss": 0.0001,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.6540037393569946,
+      "rewards/margins": 12.849777221679688,
+      "rewards/rejected": -13.503782272338867,
+      "step": 785
+    },
+    {
+      "epoch": 0.05714701177839174,
+      "grad_norm": 0.00046840065624564886,
+      "learning_rate": 1.3313506774856175e-07,
+      "logits/chosen": -0.8884440064430237,
+      "logits/rejected": -0.9113964438438416,
+      "logps/chosen": -38.097328186035156,
+      "logps/rejected": -211.51486206054688,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.6025850772857666,
+      "rewards/margins": 13.68801498413086,
+      "rewards/rejected": -14.29059886932373,
+      "step": 786
+    },
+    {
+      "epoch": 0.0572197179002472,
+      "grad_norm": 0.00023676814453210682,
+      "learning_rate": 1.3195145644013283e-07,
+      "logits/chosen": -0.8364005088806152,
+      "logits/rejected": -0.8652154207229614,
+      "logps/chosen": -41.36793518066406,
+      "logps/rejected": -218.6131134033203,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.7971158027648926,
+      "rewards/margins": 14.284409523010254,
+      "rewards/rejected": -15.081525802612305,
+      "step": 787
+    },
+    {
+      "epoch": 0.05729242402210266,
+      "grad_norm": 0.0007886055973358452,
+      "learning_rate": 1.3077232968705805e-07,
+      "logits/chosen": -0.8077453374862671,
+      "logits/rejected": -0.8728396892547607,
+      "logps/chosen": -34.755226135253906,
+      "logps/rejected": -217.66848754882812,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.26428288221359253,
+      "rewards/margins": 14.841293334960938,
+      "rewards/rejected": -15.105575561523438,
+      "step": 788
+    },
+    {
+      "epoch": 0.057365130143958123,
+      "grad_norm": 0.00012120642350055277,
+      "learning_rate": 1.29597701856625e-07,
+      "logits/chosen": -0.8904929161071777,
+      "logits/rejected": -0.8933730125427246,
+      "logps/chosen": -42.038604736328125,
+      "logps/rejected": -223.13511657714844,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.9371490478515625,
+      "rewards/margins": 15.016273498535156,
+      "rewards/rejected": -15.953420639038086,
+      "step": 789
+    },
+    {
+      "epoch": 0.05743783626581358,
+      "grad_norm": 0.006652898620814085,
+      "learning_rate": 1.284275872613028e-07,
+      "logits/chosen": -0.9450567960739136,
+      "logits/rejected": -0.8114190697669983,
+      "logps/chosen": -32.27555465698242,
+      "logps/rejected": -207.89077758789062,
+      "loss": 0.0001,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.42420023679733276,
+      "rewards/margins": 13.019515037536621,
+      "rewards/rejected": -13.443716049194336,
+      "step": 790
+    },
+    {
+      "epoch": 0.05751054238766904,
+      "grad_norm": 0.004501700401306152,
+      "learning_rate": 1.2726200015856893e-07,
+      "logits/chosen": -0.9678090810775757,
+      "logits/rejected": -0.9044240117073059,
+      "logps/chosen": -30.702655792236328,
+      "logps/rejected": -215.24301147460938,
+      "loss": 0.0001,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.24800333380699158,
+      "rewards/margins": 14.26740550994873,
+      "rewards/rejected": -14.515409469604492,
+      "step": 791
+    },
+    {
+      "epoch": 0.0575832485095245,
+      "grad_norm": 0.00010200103133684024,
+      "learning_rate": 1.2610095475073413e-07,
+      "logits/chosen": -0.9053033590316772,
+      "logits/rejected": -0.9651377201080322,
+      "logps/chosen": -38.61802291870117,
+      "logps/rejected": -219.66586303710938,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.5994962453842163,
+      "rewards/margins": 14.6649751663208,
+      "rewards/rejected": -15.264471054077148,
+      "step": 792
+    },
+    {
+      "epoch": 0.057655954631379965,
+      "grad_norm": 0.0016042115166783333,
+      "learning_rate": 1.249444651847702e-07,
+      "logits/chosen": -0.9351927042007446,
+      "logits/rejected": -0.8913071155548096,
+      "logps/chosen": -38.65888214111328,
+      "logps/rejected": -211.62966918945312,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.8786839842796326,
+      "rewards/margins": 13.222261428833008,
+      "rewards/rejected": -14.100945472717285,
+      "step": 793
+    },
+    {
+      "epoch": 0.05772866075323542,
+      "grad_norm": 0.000392068934161216,
+      "learning_rate": 1.2379254555213786e-07,
+      "logits/chosen": -0.9008168578147888,
+      "logits/rejected": -0.7671511769294739,
+      "logps/chosen": -31.751562118530273,
+      "logps/rejected": -218.06961059570312,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.26216575503349304,
+      "rewards/margins": 14.60870361328125,
+      "rewards/rejected": -14.870868682861328,
+      "step": 794
+    },
+    {
+      "epoch": 0.05780136687509088,
+      "grad_norm": 0.0012484568869695067,
+      "learning_rate": 1.22645209888614e-07,
+      "logits/chosen": -0.9315560460090637,
+      "logits/rejected": -0.8799965381622314,
+      "logps/chosen": -36.277191162109375,
+      "logps/rejected": -210.5481719970703,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.527661919593811,
+      "rewards/margins": 13.524513244628906,
+      "rewards/rejected": -14.052175521850586,
+      "step": 795
+    },
+    {
+      "epoch": 0.057874072996946344,
+      "grad_norm": 0.0004414519644342363,
+      "learning_rate": 1.2150247217412185e-07,
+      "logits/chosen": -0.9453125,
+      "logits/rejected": -0.9239727854728699,
+      "logps/chosen": -28.21062469482422,
+      "logps/rejected": -213.19349670410156,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.16052374243736267,
+      "rewards/margins": 14.973607063293457,
+      "rewards/rejected": -15.134130477905273,
+      "step": 796
+    },
+    {
+      "epoch": 0.057946779118801806,
+      "grad_norm": 0.0011588726192712784,
+      "learning_rate": 1.203643463325596e-07,
+      "logits/chosen": -0.9374755024909973,
+      "logits/rejected": -0.8596886396408081,
+      "logps/chosen": -39.49070739746094,
+      "logps/rejected": -212.61399841308594,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.9445662498474121,
+      "rewards/margins": 13.794650077819824,
+      "rewards/rejected": -14.739216804504395,
+      "step": 797
+    },
+    {
+      "epoch": 0.05801948524065726,
+      "grad_norm": 0.001562570221722126,
+      "learning_rate": 1.192308462316317e-07,
+      "logits/chosen": -0.9465897679328918,
+      "logits/rejected": -0.8477259278297424,
+      "logps/chosen": -39.65185546875,
+      "logps/rejected": -216.62901306152344,
+      "loss": 0.0,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.4916737377643585,
+      "rewards/margins": 14.508772850036621,
+      "rewards/rejected": -15.000446319580078,
+      "step": 798
+    },
+    {
+      "epoch": 0.05809219136251272,
+      "grad_norm": 0.00827076192945242,
+      "learning_rate": 1.1810198568267903e-07,
+      "logits/chosen": -0.9219031929969788,
+      "logits/rejected": -0.9003797769546509,
+      "logps/chosen": -33.286399841308594,
+      "logps/rejected": -209.7589111328125,
+      "loss": 0.0001,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.6317225694656372,
+      "rewards/margins": 13.434264183044434,
+      "rewards/rejected": -14.065986633300781,
+      "step": 799
+    },
+    {
+      "epoch": 0.058164897484368185,
+      "grad_norm": 0.005054752808064222,
+      "learning_rate": 1.1697777844051104e-07,
+      "logits/chosen": -0.9649146795272827,
+      "logits/rejected": -0.8858678340911865,
+      "logps/chosen": -39.712615966796875,
+      "logps/rejected": -208.3673858642578,
+      "loss": 0.0001,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": -0.6821295022964478,
+      "rewards/margins": 11.669793128967285,
+      "rewards/rejected": -12.351922988891602,
+      "step": 800
     }
   ],
   "logging_steps": 1,