Instructions to use watermelonhjg/Qwen2.5-3B-Instruct-CN-Math-Zero with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use watermelonhjg/Qwen2.5-3B-Instruct-CN-Math-Zero with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="watermelonhjg/Qwen2.5-3B-Instruct-CN-Math-Zero")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("watermelonhjg/Qwen2.5-3B-Instruct-CN-Math-Zero")
model = AutoModelForCausalLM.from_pretrained("watermelonhjg/Qwen2.5-3B-Instruct-CN-Math-Zero", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use watermelonhjg/Qwen2.5-3B-Instruct-CN-Math-Zero with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "watermelonhjg/Qwen2.5-3B-Instruct-CN-Math-Zero"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "watermelonhjg/Qwen2.5-3B-Instruct-CN-Math-Zero",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/watermelonhjg/Qwen2.5-3B-Instruct-CN-Math-Zero

SGLang

How to use watermelonhjg/Qwen2.5-3B-Instruct-CN-Math-Zero with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "watermelonhjg/Qwen2.5-3B-Instruct-CN-Math-Zero" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "watermelonhjg/Qwen2.5-3B-Instruct-CN-Math-Zero",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "watermelonhjg/Qwen2.5-3B-Instruct-CN-Math-Zero" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "watermelonhjg/Qwen2.5-3B-Instruct-CN-Math-Zero",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use watermelonhjg/Qwen2.5-3B-Instruct-CN-Math-Zero with Docker Model Runner:
```
docker model run hf.co/watermelonhjg/Qwen2.5-3B-Instruct-CN-Math-Zero
```

Qwen2.5-3B-Instruct-CN-Math-Zero / trainer_state.json

watermelonhjg

Upload folder using huggingface_hub

8594e3b verified over 1 year ago

Raw

History Blame Contribute Delete

96.4 kB

	{
	"best_metric": null,
	"best_model_checkpoint": null,
	"epoch": 3.0,
	"eval_steps": 10,
	"global_step": 2250,
	"is_hyper_param_search": false,
	"is_local_process_zero": true,
	"is_world_process_zero": true,
	"log_history": [
	{
	"completion_length": 358.0412570953369,
	"epoch": 0.013333333333333334,
	"grad_norm": 0.1626528948545456,
	"kl": 0.0003269195556640625,
	"learning_rate": 1.3333333333333334e-07,
	"loss": 0.0023,
	"reward": 0.16000000322237612,
	"reward_std": 0.25307216234505175,
	"rewards/accuracy_reward": 0.09875000147148967,
	"rewards/format_reward": 0.061250001285225154,
	"step": 10
	},
	{
	"completion_length": 365.58875579833983,
	"epoch": 0.02666666666666667,
	"grad_norm": 0.0887719914317131,
	"kl": 0.0004455804824829102,
	"learning_rate": 2.6666666666666667e-07,
	"loss": 0.0173,
	"reward": 0.1725000019185245,
	"reward_std": 0.20696474984288216,
	"rewards/accuracy_reward": 0.11375000216066837,
	"rewards/format_reward": 0.058750000689178704,
	"step": 20
	},
	{
	"completion_length": 364.56750640869143,
	"epoch": 0.04,
	"grad_norm": 0.4593224823474884,
	"kl": 0.003619217872619629,
	"learning_rate": 4e-07,
	"loss": 0.0084,
	"reward": 0.2150000055320561,
	"reward_std": 0.31464808210730555,
	"rewards/accuracy_reward": 0.12000000262632966,
	"rewards/format_reward": 0.09500000132247806,
	"step": 30
	},
	{
	"completion_length": 380.52500648498534,
	"epoch": 0.05333333333333334,
	"grad_norm": 0.3159937262535095,
	"kl": 0.04821491241455078,
	"learning_rate": 5.333333333333333e-07,
	"loss": 0.0439,
	"reward": 0.323750005569309,
	"reward_std": 0.38860970810055734,
	"rewards/accuracy_reward": 0.13250000309199095,
	"rewards/format_reward": 0.19125000201165676,
	"step": 40
	},
	{
	"completion_length": 322.1050077438355,
	"epoch": 0.06666666666666667,
	"grad_norm": 0.6419838070869446,
	"kl": 0.013416862487792969,
	"learning_rate": 6.666666666666666e-07,
	"loss": 0.0551,
	"reward": 0.31375000393018126,
	"reward_std": 0.3664026964455843,
	"rewards/accuracy_reward": 0.11875000335276127,
	"rewards/format_reward": 0.19500000402331352,
	"step": 50
	},
	{
	"completion_length": 313.8512550354004,
	"epoch": 0.08,
	"grad_norm": 0.43794259428977966,
	"kl": 0.031152725219726562,
	"learning_rate": 8e-07,
	"loss": 0.044,
	"reward": 0.5187500116415322,
	"reward_std": 0.3655149843543768,
	"rewards/accuracy_reward": 0.14000000162050127,
	"rewards/format_reward": 0.37875000573694706,
	"step": 60
	},
	{
	"completion_length": 334.03500576019286,
	"epoch": 0.09333333333333334,
	"grad_norm": 0.43774211406707764,
	"kl": 0.034784889221191405,
	"learning_rate": 9.333333333333333e-07,
	"loss": 0.0763,
	"reward": 0.4912500069476664,
	"reward_std": 0.418181811645627,
	"rewards/accuracy_reward": 0.09625000236555933,
	"rewards/format_reward": 0.3950000065378845,
	"step": 70
	},
	{
	"completion_length": 292.93375568389894,
	"epoch": 0.10666666666666667,
	"grad_norm": 1.4061037302017212,
	"kl": 0.04006805419921875,
	"learning_rate": 1.0666666666666667e-06,
	"loss": 0.0895,
	"reward": 0.6825000144541263,
	"reward_std": 0.40880861394107343,
	"rewards/accuracy_reward": 0.09875000203028321,
	"rewards/format_reward": 0.5837500099092722,
	"step": 80
	},
	{
	"completion_length": 328.5087568283081,
	"epoch": 0.12,
	"grad_norm": 0.4465954303741455,
	"kl": 0.0293853759765625,
	"learning_rate": 1.2000000000000002e-06,
	"loss": 0.1086,
	"reward": 0.6250000076368452,
	"reward_std": 0.42218363620340826,
	"rewards/accuracy_reward": 0.12625000178813933,
	"rewards/format_reward": 0.4987500081770122,
	"step": 90
	},
	{
	"completion_length": 271.25250511169435,
	"epoch": 0.13333333333333333,
	"grad_norm": 0.4239254891872406,
	"kl": 0.0445556640625,
	"learning_rate": 1.3333333333333332e-06,
	"loss": 0.0794,
	"reward": 0.8187500163912773,
	"reward_std": 0.4009550239890814,
	"rewards/accuracy_reward": 0.12000000113621354,
	"rewards/format_reward": 0.6987500078976154,
	"step": 100
	},
	{
	"completion_length": 238.7037540435791,
	"epoch": 0.14666666666666667,
	"grad_norm": 0.45924320816993713,
	"kl": 0.045458984375,
	"learning_rate": 1.4666666666666667e-06,
	"loss": 0.0688,
	"reward": 0.9087500125169754,
	"reward_std": 0.2800120744854212,
	"rewards/accuracy_reward": 0.0662500012665987,
	"rewards/format_reward": 0.8425000071525574,
	"step": 110
	},
	{
	"completion_length": 251.6537540435791,
	"epoch": 0.16,
	"grad_norm": 0.4579772353172302,
	"kl": 0.0559661865234375,
	"learning_rate": 1.6e-06,
	"loss": 0.0552,
	"reward": 0.9850000143051147,
	"reward_std": 0.28530918546020984,
	"rewards/accuracy_reward": 0.11875000353902579,
	"rewards/format_reward": 0.8662500090897083,
	"step": 120
	},
	{
	"completion_length": 285.78625469207765,
	"epoch": 0.17333333333333334,
	"grad_norm": 0.4548390209674835,
	"kl": 0.0518890380859375,
	"learning_rate": 1.7333333333333332e-06,
	"loss": 0.0473,
	"reward": 0.9662500187754631,
	"reward_std": 0.3944980699568987,
	"rewards/accuracy_reward": 0.14875000305473804,
	"rewards/format_reward": 0.8175000086426735,
	"step": 130
	},
	{
	"completion_length": 268.450004196167,
	"epoch": 0.18666666666666668,
	"grad_norm": 0.358254611492157,
	"kl": 0.082470703125,
	"learning_rate": 1.8666666666666667e-06,
	"loss": 0.0507,
	"reward": 1.0162500128149987,
	"reward_std": 0.38786814287304877,
	"rewards/accuracy_reward": 0.17125000292435288,
	"rewards/format_reward": 0.8450000129640103,
	"step": 140
	},
	{
	"completion_length": 304.1125057220459,
	"epoch": 0.2,
	"grad_norm": 4.261535167694092,
	"kl": 0.107891845703125,
	"learning_rate": 2e-06,
	"loss": 0.0398,
	"reward": 1.1762500151991844,
	"reward_std": 0.46659310162067413,
	"rewards/accuracy_reward": 0.2962500025518239,
	"rewards/format_reward": 0.8800000116229058,
	"step": 150
	},
	{
	"completion_length": 284.4462554931641,
	"epoch": 0.21333333333333335,
	"grad_norm": 0.29285335540771484,
	"kl": 0.086627197265625,
	"learning_rate": 2.1333333333333334e-06,
	"loss": 0.0141,
	"reward": 1.3200000196695327,
	"reward_std": 0.4418893948197365,
	"rewards/accuracy_reward": 0.37750000674277545,
	"rewards/format_reward": 0.9425000041723252,
	"step": 160
	},
	{
	"completion_length": 279.2050067901611,
	"epoch": 0.22666666666666666,
	"grad_norm": 0.47264882922172546,
	"kl": 0.100860595703125,
	"learning_rate": 2.266666666666667e-06,
	"loss": 0.051,
	"reward": 1.3650000289082527,
	"reward_std": 0.4196435324847698,
	"rewards/accuracy_reward": 0.4200000060722232,
	"rewards/format_reward": 0.9450000047683715,
	"step": 170
	},
	{
	"completion_length": 266.97375679016113,
	"epoch": 0.24,
	"grad_norm": 0.31524142622947693,
	"kl": 0.1145751953125,
	"learning_rate": 2.4000000000000003e-06,
	"loss": 0.0426,
	"reward": 1.4912500247359275,
	"reward_std": 0.38923392072319984,
	"rewards/accuracy_reward": 0.543750009778887,
	"rewards/format_reward": 0.9475000031292439,
	"step": 180
	},
	{
	"completion_length": 340.537505531311,
	"epoch": 0.25333333333333335,
	"grad_norm": 0.28264713287353516,
	"kl": 0.1277587890625,
	"learning_rate": 2.5333333333333334e-06,
	"loss": 0.0855,
	"reward": 1.2712500244379044,
	"reward_std": 0.49123715460300443,
	"rewards/accuracy_reward": 0.4312500088475645,
	"rewards/format_reward": 0.840000007301569,
	"step": 190
	},
	{
	"completion_length": 298.52000389099123,
	"epoch": 0.26666666666666666,
	"grad_norm": 0.4394519627094269,
	"kl": 0.16021728515625,
	"learning_rate": 2.6666666666666664e-06,
	"loss": 0.1097,
	"reward": 1.2837500274181366,
	"reward_std": 0.5194122649729251,
	"rewards/accuracy_reward": 0.43875000802800057,
	"rewards/format_reward": 0.845000010728836,
	"step": 200
	},
	{
	"completion_length": 337.09625549316405,
	"epoch": 0.28,
	"grad_norm": 0.4397446811199188,
	"kl": 0.286328125,
	"learning_rate": 2.8000000000000003e-06,
	"loss": 0.1787,
	"reward": 1.057500022649765,
	"reward_std": 0.6535363413393498,
	"rewards/accuracy_reward": 0.3850000096485019,
	"rewards/format_reward": 0.6725000105798244,
	"step": 210
	},
	{
	"completion_length": 195.2412536621094,
	"epoch": 0.29333333333333333,
	"grad_norm": 1.9711108207702637,
	"kl": 3.52977294921875,
	"learning_rate": 2.9333333333333333e-06,
	"loss": 0.0607,
	"reward": 1.2225000165402888,
	"reward_std": 0.504400571808219,
	"rewards/accuracy_reward": 0.41500000692903993,
	"rewards/format_reward": 0.8075000129640102,
	"step": 220
	},
	{
	"completion_length": 197.4412525177002,
	"epoch": 0.30666666666666664,
	"grad_norm": 2256.19873046875,
	"kl": 17.518798828125,
	"learning_rate": 2.9999548717196514e-06,
	"loss": 0.5765,
	"reward": 1.3587500154972076,
	"reward_std": 0.4323975473642349,
	"rewards/accuracy_reward": 0.4712500068359077,
	"rewards/format_reward": 0.8875000089406967,
	"step": 230
	},
	{
	"completion_length": 222.5012535095215,
	"epoch": 0.32,
	"grad_norm": 0.370374858379364,
	"kl": 0.28663330078125,
	"learning_rate": 2.9995938617691924e-06,
	"loss": 0.0185,
	"reward": 1.3312500312924385,
	"reward_std": 0.5138044867664575,
	"rewards/accuracy_reward": 0.45125000588595865,
	"rewards/format_reward": 0.8800000086426735,
	"step": 240
	},
	{
	"completion_length": 213.4362533569336,
	"epoch": 0.3333333333333333,
	"grad_norm": 0.3446196913719177,
	"kl": 0.21429443359375,
	"learning_rate": 2.9988719287563454e-06,
	"loss": 0.0129,
	"reward": 1.4750000223517419,
	"reward_std": 0.3717566329985857,
	"rewards/accuracy_reward": 0.5112500058487057,
	"rewards/format_reward": 0.9637500002980233,
	"step": 250
	},
	{
	"completion_length": 252.42875595092772,
	"epoch": 0.3466666666666667,
	"grad_norm": 2.4441771507263184,
	"kl": 1.01444091796875,
	"learning_rate": 2.9977892464363372e-06,
	"loss": 0.0758,
	"reward": 1.4012500166893005,
	"reward_std": 0.4755144312977791,
	"rewards/accuracy_reward": 0.4700000075623393,
	"rewards/format_reward": 0.9312500089406968,
	"step": 260
	},
	{
	"completion_length": 247.25250587463378,
	"epoch": 0.36,
	"grad_norm": 2.5208985805511475,
	"kl": 0.5568603515625,
	"learning_rate": 2.9963460753897363e-06,
	"loss": 0.0401,
	"reward": 1.3375000193715096,
	"reward_std": 0.4566458873450756,
	"rewards/accuracy_reward": 0.4337500057183206,
	"rewards/format_reward": 0.9037500128149987,
	"step": 270
	},
	{
	"completion_length": 246.50625495910646,
	"epoch": 0.37333333333333335,
	"grad_norm": 96.51844024658203,
	"kl": 1.2575439453125,
	"learning_rate": 2.9945427629597305e-06,
	"loss": 0.0839,
	"reward": 1.376250022649765,
	"reward_std": 0.4465434730052948,
	"rewards/accuracy_reward": 0.4450000057928264,
	"rewards/format_reward": 0.9312500059604645,
	"step": 280
	},
	{
	"completion_length": 286.57250423431395,
	"epoch": 0.38666666666666666,
	"grad_norm": 0.3689205050468445,
	"kl": 0.414300537109375,
	"learning_rate": 2.992379743168532e-06,
	"loss": 0.0462,
	"reward": 1.4450000196695327,
	"reward_std": 0.4324021231383085,
	"rewards/accuracy_reward": 0.503750005364418,
	"rewards/format_reward": 0.941250005364418,
	"step": 290
	},
	{
	"completion_length": 332.27125587463377,
	"epoch": 0.4,
	"grad_norm": 0.4576380252838135,
	"kl": 0.3354248046875,
	"learning_rate": 2.989857536612915e-06,
	"loss": 0.0378,
	"reward": 1.4400000303983689,
	"reward_std": 0.39175261445343496,
	"rewards/accuracy_reward": 0.4987500077113509,
	"rewards/format_reward": 0.941250005364418,
	"step": 300
	},
	{
	"completion_length": 318.8962543487549,
	"epoch": 0.41333333333333333,
	"grad_norm": 0.44957035779953003,
	"kl": 1.1047119140625,
	"learning_rate": 2.9869767503389175e-06,
	"loss": 0.0515,
	"reward": 1.4150000244379044,
	"reward_std": 0.4395812951028347,
	"rewards/accuracy_reward": 0.4762500121258199,
	"rewards/format_reward": 0.9387500062584877,
	"step": 310
	},
	{
	"completion_length": 293.3062553405762,
	"epoch": 0.4266666666666667,
	"grad_norm": 0.47710099816322327,
	"kl": 8.84039306640625,
	"learning_rate": 2.983738077695741e-06,
	"loss": 0.4032,
	"reward": 1.4200000286102294,
	"reward_std": 0.40100472904741763,
	"rewards/accuracy_reward": 0.4850000117905438,
	"rewards/format_reward": 0.9350000098347664,
	"step": 320
	},
	{
	"completion_length": 311.5937572479248,
	"epoch": 0.44,
	"grad_norm": 4.005710124969482,
	"kl": 9.49320068359375,
	"learning_rate": 2.980142298168869e-06,
	"loss": 0.5626,
	"reward": 1.4375000283122064,
	"reward_std": 0.46121844090521336,
	"rewards/accuracy_reward": 0.5175000098533928,
	"rewards/format_reward": 0.9200000077486038,
	"step": 330
	},
	{
	"completion_length": 313.74875526428224,
	"epoch": 0.4533333333333333,
	"grad_norm": 10.983561515808105,
	"kl": 0.34615478515625,
	"learning_rate": 2.976190277192465e-06,
	"loss": 0.0495,
	"reward": 1.4325000122189522,
	"reward_std": 0.4127622898668051,
	"rewards/accuracy_reward": 0.5262500100769103,
	"rewards/format_reward": 0.9062500059604645,
	"step": 340
	},
	{
	"completion_length": 317.42875480651855,
	"epoch": 0.4666666666666667,
	"grad_norm": 1.4480901956558228,
	"kl": 16.34658203125,
	"learning_rate": 2.971882965941077e-06,
	"loss": 0.7464,
	"reward": 1.4687500238418578,
	"reward_std": 0.44693985804915426,
	"rewards/accuracy_reward": 0.5800000098533928,
	"rewards/format_reward": 0.8887500122189522,
	"step": 350
	},
	{
	"completion_length": 345.25625648498533,
	"epoch": 0.48,
	"grad_norm": 0.3819202184677124,
	"kl": 0.4673583984375,
	"learning_rate": 2.9672214011007086e-06,
	"loss": 0.0278,
	"reward": 1.3925000250339508,
	"reward_std": 0.36666983254253865,
	"rewards/accuracy_reward": 0.45375000443309543,
	"rewards/format_reward": 0.9387500062584877,
	"step": 360
	},
	{
	"completion_length": 308.32375564575193,
	"epoch": 0.49333333333333335,
	"grad_norm": 0.812126874923706,
	"kl": 0.8212890625,
	"learning_rate": 2.9622067046193085e-06,
	"loss": 0.0635,
	"reward": 1.4100000277161597,
	"reward_std": 0.420729149505496,
	"rewards/accuracy_reward": 0.46875001015141604,
	"rewards/format_reward": 0.9412500023841858,
	"step": 370
	},
	{
	"completion_length": 298.4312526702881,
	"epoch": 0.5066666666666667,
	"grad_norm": 1.2376179695129395,
	"kl": 1.26268310546875,
	"learning_rate": 2.956840083436741e-06,
	"loss": 0.0954,
	"reward": 1.471250019967556,
	"reward_std": 0.4468890752643347,
	"rewards/accuracy_reward": 0.5375000098720193,
	"rewards/format_reward": 0.933750006556511,
	"step": 380
	},
	{
	"completion_length": 277.4425052642822,
	"epoch": 0.52,
	"grad_norm": 1.022598147392273,
	"kl": 2.3487060546875,
	"learning_rate": 2.951122829194296e-06,
	"loss": 0.1583,
	"reward": 1.3462500274181366,
	"reward_std": 0.5475293599069119,
	"rewards/accuracy_reward": 0.4762500089593232,
	"rewards/format_reward": 0.870000010728836,
	"step": 390
	},
	{
	"completion_length": 311.357506942749,
	"epoch": 0.5333333333333333,
	"grad_norm": 0.5792402625083923,
	"kl": 0.7629638671875,
	"learning_rate": 2.9450563179238205e-06,
	"loss": 0.1346,
	"reward": 1.3450000122189523,
	"reward_std": 0.5392270684242249,
	"rewards/accuracy_reward": 0.4387500088661909,
	"rewards/format_reward": 0.9062500074505806,
	"step": 400
	},
	{
	"completion_length": 268.91125507354735,
	"epoch": 0.5466666666666666,
	"grad_norm": 2.355006694793701,
	"kl": 10.3116455078125,
	"learning_rate": 2.938642009716531e-06,
	"loss": 0.7061,
	"reward": 1.411250014603138,
	"reward_std": 0.44688397236168387,
	"rewards/accuracy_reward": 0.48625000976026056,
	"rewards/format_reward": 0.9250000059604645,
	"step": 410
	},
	{
	"completion_length": 330.4012561798096,
	"epoch": 0.56,
	"grad_norm": 1.2017070055007935,
	"kl": 2.034033203125,
	"learning_rate": 2.9318814483715983e-06,
	"loss": 0.3712,
	"reward": 1.37750001847744,
	"reward_std": 0.5567759402096272,
	"rewards/accuracy_reward": 0.5162500075995922,
	"rewards/format_reward": 0.8612500131130219,
	"step": 420
	},
	{
	"completion_length": 262.88875579833984,
	"epoch": 0.5733333333333334,
	"grad_norm": 8.315053939819336,
	"kl": 1.03427734375,
	"learning_rate": 2.924776261024586e-06,
	"loss": 0.1372,
	"reward": 1.5275000169873238,
	"reward_std": 0.4006519988179207,
	"rewards/accuracy_reward": 0.5787500092759729,
	"rewards/format_reward": 0.948750002682209,
	"step": 430
	},
	{
	"completion_length": 250.29125347137452,
	"epoch": 0.5866666666666667,
	"grad_norm": 15.071596145629883,
	"kl": 1.954150390625,
	"learning_rate": 2.917328157755832e-06,
	"loss": 0.1571,
	"reward": 1.378750017285347,
	"reward_std": 0.42997160777449606,
	"rewards/accuracy_reward": 0.5175000073388218,
	"rewards/format_reward": 0.8612500071525574,
	"step": 440
	},
	{
	"completion_length": 267.2262548446655,
	"epoch": 0.6,
	"grad_norm": 4.3944549560546875,
	"kl": 3.891845703125,
	"learning_rate": 2.9095389311788626e-06,
	"loss": 0.2468,
	"reward": 1.2350000157952308,
	"reward_std": 0.5935393430292606,
	"rewards/accuracy_reward": 0.48000000631436707,
	"rewards/format_reward": 0.7550000131130219,
	"step": 450
	},
	{
	"completion_length": 232.88375339508056,
	"epoch": 0.6133333333333333,
	"grad_norm": 10.471932411193848,
	"kl": 1.11915283203125,
	"learning_rate": 2.9014104560089465e-06,
	"loss": 0.1208,
	"reward": 1.4125000298023225,
	"reward_std": 0.4672528047114611,
	"rewards/accuracy_reward": 0.5075000061653554,
	"rewards/format_reward": 0.9050000071525574,
	"step": 460
	},
	{
	"completion_length": 266.50000495910643,
	"epoch": 0.6266666666666667,
	"grad_norm": 0.3841674029827118,
	"kl": 1.0602783203125,
	"learning_rate": 2.8929446886118866e-06,
	"loss": 0.2335,
	"reward": 1.4612500220537186,
	"reward_std": 0.42485330663621423,
	"rewards/accuracy_reward": 0.541250011883676,
	"rewards/format_reward": 0.9200000077486038,
	"step": 470
	},
	{
	"completion_length": 256.0362522125244,
	"epoch": 0.64,
	"grad_norm": 0.425624281167984,
	"kl": 0.7212890625,
	"learning_rate": 2.8841436665331635e-06,
	"loss": 0.1887,
	"reward": 1.3937500238418579,
	"reward_std": 0.48963438235223294,
	"rewards/accuracy_reward": 0.4762500075623393,
	"rewards/format_reward": 0.9175000041723251,
	"step": 480
	},
	{
	"completion_length": 278.662505531311,
	"epoch": 0.6533333333333333,
	"grad_norm": 0.6774723529815674,
	"kl": 0.314697265625,
	"learning_rate": 2.875009508007535e-06,
	"loss": 0.2242,
	"reward": 1.3637500166893006,
	"reward_std": 0.5403582151979208,
	"rewards/accuracy_reward": 0.5012500101700426,
	"rewards/format_reward": 0.8625000104308128,
	"step": 490
	},
	{
	"completion_length": 295.65125541687013,
	"epoch": 0.6666666666666666,
	"grad_norm": 1.2344285249710083,
	"kl": 0.48665771484375,
	"learning_rate": 2.86554441144922e-06,
	"loss": 0.2865,
	"reward": 1.4550000220537185,
	"reward_std": 0.4966724131256342,
	"rewards/accuracy_reward": 0.5925000066868961,
	"rewards/format_reward": 0.8625000059604645,
	"step": 500
	},
	{
	"completion_length": 309.08625679016114,
	"epoch": 0.68,
	"grad_norm": 16.80157470703125,
	"kl": 2.29215087890625,
	"learning_rate": 2.855750654922781e-06,
	"loss": 0.4345,
	"reward": 1.4012500315904617,
	"reward_std": 0.5136609964072705,
	"rewards/accuracy_reward": 0.556250006146729,
	"rewards/format_reward": 0.8450000122189522,
	"step": 510
	},
	{
	"completion_length": 320.7975061416626,
	"epoch": 0.6933333333333334,
	"grad_norm": 6.8107590675354,
	"kl": 2.5089599609375,
	"learning_rate": 2.84563059559484e-06,
	"loss": 0.5524,
	"reward": 1.2937500305473804,
	"reward_std": 0.5724672272801399,
	"rewards/accuracy_reward": 0.49500000951811673,
	"rewards/format_reward": 0.7987500131130219,
	"step": 520
	},
	{
	"completion_length": 312.92125396728517,
	"epoch": 0.7066666666666667,
	"grad_norm": 1.1629796028137207,
	"kl": 1.2917724609375,
	"learning_rate": 2.8351866691667543e-06,
	"loss": 0.3865,
	"reward": 1.1650000177323818,
	"reward_std": 0.5869733650237322,
	"rewards/accuracy_reward": 0.39125000657513737,
	"rewards/format_reward": 0.7737500131130218,
	"step": 530
	},
	{
	"completion_length": 264.66250495910646,
	"epoch": 0.72,
	"grad_norm": 1.581681251525879,
	"kl": 1.5130859375,
	"learning_rate": 2.8244213892883906e-06,
	"loss": 0.3496,
	"reward": 1.4375000178813935,
	"reward_std": 0.45811997428536416,
	"rewards/accuracy_reward": 0.5450000078417361,
	"rewards/format_reward": 0.8925000086426735,
	"step": 540
	},
	{
	"completion_length": 299.5112560272217,
	"epoch": 0.7333333333333333,
	"grad_norm": 1.8020122051239014,
	"kl": 1.855810546875,
	"learning_rate": 2.8133373469531365e-06,
	"loss": 0.4243,
	"reward": 1.37500002682209,
	"reward_std": 0.4744249366223812,
	"rewards/accuracy_reward": 0.5300000084564089,
	"rewards/format_reward": 0.845000010728836,
	"step": 550
	},
	{
	"completion_length": 279.5200065612793,
	"epoch": 0.7466666666666667,
	"grad_norm": 1.1065877676010132,
	"kl": 1.2305419921875,
	"learning_rate": 2.801937209874301e-06,
	"loss": 0.3489,
	"reward": 1.3450000271201135,
	"reward_std": 0.48592774122953414,
	"rewards/accuracy_reward": 0.4600000066682696,
	"rewards/format_reward": 0.8850000105798245,
	"step": 560
	},
	{
	"completion_length": 262.961252784729,
	"epoch": 0.76,
	"grad_norm": 2.5658416748046875,
	"kl": 0.5228271484375,
	"learning_rate": 2.7902237218430485e-06,
	"loss": 0.2554,
	"reward": 1.4400000259280206,
	"reward_std": 0.437805675342679,
	"rewards/accuracy_reward": 0.5325000083073974,
	"rewards/format_reward": 0.90750000923872,
	"step": 570
	},
	{
	"completion_length": 393.1900066375732,
	"epoch": 0.7733333333333333,
	"grad_norm": 3.0309977531433105,
	"kl": 0.73193359375,
	"learning_rate": 2.778199702068017e-06,
	"loss": 0.4131,
	"reward": 1.1987500235438346,
	"reward_std": 0.603440997377038,
	"rewards/accuracy_reward": 0.46500000907108185,
	"rewards/format_reward": 0.7337500136345625,
	"step": 580
	},
	{
	"completion_length": 356.66125717163084,
	"epoch": 0.7866666666666666,
	"grad_norm": 4.907287120819092,
	"kl": 15.196728515625,
	"learning_rate": 2.7658680444967964e-06,
	"loss": 1.5553,
	"reward": 1.2862500250339508,
	"reward_std": 0.5923882402479649,
	"rewards/accuracy_reward": 0.5062500080093741,
	"rewards/format_reward": 0.7800000138580799,
	"step": 590
	},
	{
	"completion_length": 308.2250047683716,
	"epoch": 0.8,
	"grad_norm": 3.880446434020996,
	"kl": 1.6797119140625,
	"learning_rate": 2.753231717119405e-06,
	"loss": 0.3899,
	"reward": 1.3175000190734862,
	"reward_std": 0.4703429743647575,
	"rewards/accuracy_reward": 0.47125000776723025,
	"rewards/format_reward": 0.8462500080466271,
	"step": 600
	},
	{
	"completion_length": 272.85125598907473,
	"epoch": 0.8133333333333334,
	"grad_norm": 7.690130233764648,
	"kl": 2.55137939453125,
	"learning_rate": 2.740293761253956e-06,
	"loss": 0.4518,
	"reward": 1.413750021159649,
	"reward_std": 0.4325053282082081,
	"rewards/accuracy_reward": 0.531250010151416,
	"rewards/format_reward": 0.8825000062584877,
	"step": 610
	},
	{
	"completion_length": 297.4387529373169,
	"epoch": 0.8266666666666667,
	"grad_norm": 2.0755786895751953,
	"kl": 2.9087890625,
	"learning_rate": 2.7270572908146716e-06,
	"loss": 0.4945,
	"reward": 1.3737500190734864,
	"reward_std": 0.48490689508616924,
	"rewards/accuracy_reward": 0.5175000118091703,
	"rewards/format_reward": 0.8562500081956387,
	"step": 620
	},
	{
	"completion_length": 288.5512563705444,
	"epoch": 0.84,
	"grad_norm": 2.110800266265869,
	"kl": 4.269287109375,
	"learning_rate": 2.713525491562421e-06,
	"loss": 0.6393,
	"reward": 1.3537500187754632,
	"reward_std": 0.5092445306479931,
	"rewards/accuracy_reward": 0.4800000081770122,
	"rewards/format_reward": 0.8737500131130218,
	"step": 630
	},
	{
	"completion_length": 273.556254196167,
	"epoch": 0.8533333333333334,
	"grad_norm": 2.306751251220703,
	"kl": 2.8123046875,
	"learning_rate": 2.699701620337974e-06,
	"loss": 0.4281,
	"reward": 1.3900000244379043,
	"reward_std": 0.48922200947999955,
	"rewards/accuracy_reward": 0.4987500078044832,
	"rewards/format_reward": 0.8912500098347664,
	"step": 640
	},
	{
	"completion_length": 273.3600038528442,
	"epoch": 0.8666666666666667,
	"grad_norm": 3.8477585315704346,
	"kl": 4.17413330078125,
	"learning_rate": 2.685589004278139e-06,
	"loss": 0.4829,
	"reward": 1.2900000154972076,
	"reward_std": 0.5038297042250633,
	"rewards/accuracy_reward": 0.4262500065378845,
	"rewards/format_reward": 0.8637500092387199,
	"step": 650
	},
	{
	"completion_length": 209.00375442504884,
	"epoch": 0.88,
	"grad_norm": 0.8722585439682007,
	"kl": 2.2757568359375,
	"learning_rate": 2.671191040014989e-06,
	"loss": 0.2218,
	"reward": 1.4350000306963921,
	"reward_std": 0.4456099320203066,
	"rewards/accuracy_reward": 0.5400000099092722,
	"rewards/format_reward": 0.8950000107288361,
	"step": 660
	},
	{
	"completion_length": 210.77125377655028,
	"epoch": 0.8933333333333333,
	"grad_norm": 19.34569549560547,
	"kl": 2.20546875,
	"learning_rate": 2.656511192858356e-06,
	"loss": 0.1753,
	"reward": 1.536250014603138,
	"reward_std": 0.34286502450704576,
	"rewards/accuracy_reward": 0.6000000078231096,
	"rewards/format_reward": 0.9362500041723252,
	"step": 670
	},
	{
	"completion_length": 215.156254196167,
	"epoch": 0.9066666666666666,
	"grad_norm": 1.1418198347091675,
	"kl": 1.42802734375,
	"learning_rate": 2.641552995961801e-06,
	"loss": 0.1849,
	"reward": 1.4125000208616256,
	"reward_std": 0.4128339193761349,
	"rewards/accuracy_reward": 0.4987500081770122,
	"rewards/format_reward": 0.9137500122189521,
	"step": 680
	},
	{
	"completion_length": 203.80875244140626,
	"epoch": 0.92,
	"grad_norm": 1.1534955501556396,
	"kl": 1.554296875,
	"learning_rate": 2.626320049472249e-06,
	"loss": 0.1974,
	"reward": 1.4050000324845313,
	"reward_std": 0.46511752642691134,
	"rewards/accuracy_reward": 0.4950000086799264,
	"rewards/format_reward": 0.910000005364418,
	"step": 690
	},
	{
	"completion_length": 229.41625385284425,
	"epoch": 0.9333333333333333,
	"grad_norm": 0.6829231977462769,
	"kl": 1.338916015625,
	"learning_rate": 2.610816019663507e-06,
	"loss": 0.1411,
	"reward": 1.4262500196695327,
	"reward_std": 0.4427020225673914,
	"rewards/accuracy_reward": 0.5125000110827387,
	"rewards/format_reward": 0.9137500077486038,
	"step": 700
	},
	{
	"completion_length": 227.5175048828125,
	"epoch": 0.9466666666666667,
	"grad_norm": 3.9809014797210693,
	"kl": 0.76220703125,
	"learning_rate": 2.595044638053862e-06,
	"loss": 0.0753,
	"reward": 1.5462500333786011,
	"reward_std": 0.43644947446882726,
	"rewards/accuracy_reward": 0.5962500106543303,
	"rewards/format_reward": 0.9500000029802322,
	"step": 710
	},
	{
	"completion_length": 211.47125434875488,
	"epoch": 0.96,
	"grad_norm": 1.0556763410568237,
	"kl": 0.74736328125,
	"learning_rate": 2.5790097005079765e-06,
	"loss": 0.0787,
	"reward": 1.537500023841858,
	"reward_std": 0.36086434237658976,
	"rewards/accuracy_reward": 0.58500001039356,
	"rewards/format_reward": 0.9525000035762787,
	"step": 720
	},
	{
	"completion_length": 275.242505645752,
	"epoch": 0.9733333333333334,
	"grad_norm": 0.5037354826927185,
	"kl": 1.3829833984375,
	"learning_rate": 2.5627150663233e-06,
	"loss": 0.2147,
	"reward": 1.410000029206276,
	"reward_std": 0.4641982387751341,
	"rewards/accuracy_reward": 0.518750009406358,
	"rewards/format_reward": 0.8912500098347664,
	"step": 730
	},
	{
	"completion_length": 230.93375358581542,
	"epoch": 0.9866666666666667,
	"grad_norm": 0.47338035702705383,
	"kl": 0.36016845703125,
	"learning_rate": 2.5461646573012073e-06,
	"loss": 0.0876,
	"reward": 1.560000017285347,
	"reward_std": 0.3064346365630627,
	"rewards/accuracy_reward": 0.5987500073388219,
	"rewards/format_reward": 0.9612500011920929,
	"step": 740
	},
	{
	"completion_length": 253.68500442504882,
	"epoch": 1.0,
	"grad_norm": 1.1472781896591187,
	"kl": 0.9532470703125,
	"learning_rate": 2.529362456803101e-06,
	"loss": 0.1619,
	"reward": 1.4637500196695328,
	"reward_std": 0.4234457302838564,
	"rewards/accuracy_reward": 0.5312500048428774,
	"rewards/format_reward": 0.9325000002980233,
	"step": 750
	},
	{
	"completion_length": 231.0400037765503,
	"epoch": 1.0133333333333334,
	"grad_norm": 0.6581693887710571,
	"kl": 0.7316162109375,
	"learning_rate": 2.5123125087916918e-06,
	"loss": 0.1485,
	"reward": 1.4725000202655791,
	"reward_std": 0.39589033350348474,
	"rewards/accuracy_reward": 0.5262500112876296,
	"rewards/format_reward": 0.9462500065565109,
	"step": 760
	},
	{
	"completion_length": 258.4175052642822,
	"epoch": 1.0266666666666666,
	"grad_norm": 0.7054517269134521,
	"kl": 0.7291259765625,
	"learning_rate": 2.495018916857696e-06,
	"loss": 0.2479,
	"reward": 1.4350000232458116,
	"reward_std": 0.46736107058823106,
	"rewards/accuracy_reward": 0.5350000127218664,
	"rewards/format_reward": 0.9000000074505806,
	"step": 770
	},
	{
	"completion_length": 262.02250385284424,
	"epoch": 1.04,
	"grad_norm": 0.44452065229415894,
	"kl": 0.6681884765625,
	"learning_rate": 2.477485843232183e-06,
	"loss": 0.2143,
	"reward": 1.4725000143051148,
	"reward_std": 0.4894147712737322,
	"rewards/accuracy_reward": 0.5600000077858567,
	"rewards/format_reward": 0.9125000074505806,
	"step": 780
	},
	{
	"completion_length": 225.12500190734863,
	"epoch": 1.0533333333333332,
	"grad_norm": 0.4003940224647522,
	"kl": 0.526318359375,
	"learning_rate": 2.4597175077848023e-06,
	"loss": 0.0642,
	"reward": 1.5487500220537185,
	"reward_std": 0.3219704765826464,
	"rewards/accuracy_reward": 0.5812500060535968,
	"rewards/format_reward": 0.9674999997019768,
	"step": 790
	},
	{
	"completion_length": 271.36250534057615,
	"epoch": 1.0666666666666667,
	"grad_norm": 2.0866036415100098,
	"kl": 0.4521728515625,
	"learning_rate": 2.441718187008148e-06,
	"loss": 0.1265,
	"reward": 1.448750016093254,
	"reward_std": 0.4373233333230019,
	"rewards/accuracy_reward": 0.5062500108033419,
	"rewards/format_reward": 0.9425000041723252,
	"step": 800
	},
	{
	"completion_length": 251.02625427246093,
	"epoch": 1.08,
	"grad_norm": 0.5189586877822876,
	"kl": 0.80755615234375,
	"learning_rate": 2.4234922129884873e-06,
	"loss": 0.1842,
	"reward": 1.4925000175833703,
	"reward_std": 0.41841145791113377,
	"rewards/accuracy_reward": 0.5675000082701445,
	"rewards/format_reward": 0.9250000044703484,
	"step": 810
	},
	{
	"completion_length": 252.2237548828125,
	"epoch": 1.0933333333333333,
	"grad_norm": 0.30723270773887634,
	"kl": 0.4115234375,
	"learning_rate": 2.405043972363114e-06,
	"loss": 0.1354,
	"reward": 1.4812500208616257,
	"reward_std": 0.39995415285229685,
	"rewards/accuracy_reward": 0.5387500130571425,
	"rewards/format_reward": 0.9425000056624413,
	"step": 820
	},
	{
	"completion_length": 259.47125205993655,
	"epoch": 1.1066666666666667,
	"grad_norm": 0.6361457705497742,
	"kl": 0.4533203125,
	"learning_rate": 2.386377905264567e-06,
	"loss": 0.1431,
	"reward": 1.4937500223517417,
	"reward_std": 0.4016253810375929,
	"rewards/accuracy_reward": 0.5575000120326876,
	"rewards/format_reward": 0.9362500071525574,
	"step": 830
	},
	{
	"completion_length": 273.63625564575193,
	"epoch": 1.12,
	"grad_norm": 1.0046416521072388,
	"kl": 0.51337890625,
	"learning_rate": 2.36749850425198e-06,
	"loss": 0.1486,
	"reward": 1.4950000181794167,
	"reward_std": 0.39630253836512563,
	"rewards/accuracy_reward": 0.5575000086799264,
	"rewards/format_reward": 0.9375000059604645,
	"step": 840
	},
	{
	"completion_length": 274.59875602722167,
	"epoch": 1.1333333333333333,
	"grad_norm": 1.476180076599121,
	"kl": 0.6783447265625,
	"learning_rate": 2.348410313229808e-06,
	"loss": 0.1884,
	"reward": 1.431250025331974,
	"reward_std": 0.4796358771622181,
	"rewards/accuracy_reward": 0.5237500093877315,
	"rewards/format_reward": 0.907500010728836,
	"step": 850
	},
	{
	"completion_length": 247.44750480651857,
	"epoch": 1.1466666666666667,
	"grad_norm": 0.2607942819595337,
	"kl": 0.76661376953125,
	"learning_rate": 2.329117926354199e-06,
	"loss": 0.1561,
	"reward": 1.5350000128149985,
	"reward_std": 0.44854437448084356,
	"rewards/accuracy_reward": 0.6200000122189522,
	"rewards/format_reward": 0.9150000125169754,
	"step": 860
	},
	{
	"completion_length": 233.2700038909912,
	"epoch": 1.16,
	"grad_norm": 1.2619364261627197,
	"kl": 0.8174560546875,
	"learning_rate": 2.3096259869272697e-06,
	"loss": 0.1501,
	"reward": 1.5450000151991845,
	"reward_std": 0.35568486750125883,
	"rewards/accuracy_reward": 0.6012500108219683,
	"rewards/format_reward": 0.9437500029802323,
	"step": 870
	},
	{
	"completion_length": 294.38750610351565,
	"epoch": 1.1733333333333333,
	"grad_norm": 2.265449047088623,
	"kl": 1.6038818359375,
	"learning_rate": 2.2899391862795513e-06,
	"loss": 0.2953,
	"reward": 1.3987500235438346,
	"reward_std": 0.4723483145236969,
	"rewards/accuracy_reward": 0.5225000081583857,
	"rewards/format_reward": 0.8762500047683716,
	"step": 880
	},
	{
	"completion_length": 282.3900045394897,
	"epoch": 1.1866666666666668,
	"grad_norm": 1.2791324853897095,
	"kl": 1.3260498046875,
	"learning_rate": 2.2700622626408814e-06,
	"loss": 0.2613,
	"reward": 1.4525000251829625,
	"reward_std": 0.46175364293158055,
	"rewards/accuracy_reward": 0.5725000085309148,
	"rewards/format_reward": 0.8800000064074993,
	"step": 890
	},
	{
	"completion_length": 292.1812559127808,
	"epoch": 1.2,
	"grad_norm": 2.6686275005340576,
	"kl": 1.6286865234375,
	"learning_rate": 2.25e-06,
	"loss": 0.2693,
	"reward": 1.4675000235438347,
	"reward_std": 0.42986451014876365,
	"rewards/accuracy_reward": 0.5775000100024045,
	"rewards/format_reward": 0.8900000095367432,
	"step": 900
	},
	{
	"completion_length": 258.4387538909912,
	"epoch": 1.2133333333333334,
	"grad_norm": 0.4539574384689331,
	"kl": 0.7174560546875,
	"learning_rate": 2.2297572269531398e-06,
	"loss": 0.1006,
	"reward": 1.4200000211596489,
	"reward_std": 0.4356360357254744,
	"rewards/accuracy_reward": 0.5112500078976154,
	"rewards/format_reward": 0.9087500035762787,
	"step": 910
	},
	{
	"completion_length": 269.2125053405762,
	"epoch": 1.2266666666666666,
	"grad_norm": 1.3635307550430298,
	"kl": 0.868798828125,
	"learning_rate": 2.2093388155418754e-06,
	"loss": 0.1843,
	"reward": 1.4000000208616257,
	"reward_std": 0.45293766520917417,
	"rewards/accuracy_reward": 0.5125000071711838,
	"rewards/format_reward": 0.8875000134110451,
	"step": 920
	},
	{
	"completion_length": 259.2087543487549,
	"epoch": 1.24,
	"grad_norm": 0.48847752809524536,
	"kl": 0.7795654296875,
	"learning_rate": 2.1887496800805174e-06,
	"loss": 0.1366,
	"reward": 1.4850000232458114,
	"reward_std": 0.41869538016617297,
	"rewards/accuracy_reward": 0.5825000096112489,
	"rewards/format_reward": 0.9025000095367431,
	"step": 930
	},
	{
	"completion_length": 269.03500480651854,
	"epoch": 1.2533333333333334,
	"grad_norm": 0.4667813777923584,
	"kl": 0.8,
	"learning_rate": 2.167994775973334e-06,
	"loss": 0.1864,
	"reward": 1.4250000298023224,
	"reward_std": 0.4630339309573174,
	"rewards/accuracy_reward": 0.5162500089034439,
	"rewards/format_reward": 0.9087500050663948,
	"step": 940
	},
	{
	"completion_length": 244.85000133514404,
	"epoch": 1.2666666666666666,
	"grad_norm": 0.40062400698661804,
	"kl": 0.45528564453125,
	"learning_rate": 2.1470790985218807e-06,
	"loss": 0.1361,
	"reward": 1.5050000175833702,
	"reward_std": 0.3423164799809456,
	"rewards/accuracy_reward": 0.5662500064820051,
	"rewards/format_reward": 0.9387500077486038,
	"step": 950
	},
	{
	"completion_length": 222.37750396728515,
	"epoch": 1.28,
	"grad_norm": 0.6321477293968201,
	"kl": 0.36839599609375,
	"learning_rate": 2.126007681722727e-06,
	"loss": 0.0723,
	"reward": 1.59375002682209,
	"reward_std": 0.3785114776343107,
	"rewards/accuracy_reward": 0.635000008251518,
	"rewards/format_reward": 0.9587499991059303,
	"step": 960
	},
	{
	"completion_length": 251.10250453948976,
	"epoch": 1.2933333333333334,
	"grad_norm": 0.9746308326721191,
	"kl": 0.741796875,
	"learning_rate": 2.1047855970558753e-06,
	"loss": 0.1903,
	"reward": 1.4962500169873238,
	"reward_std": 0.4037658181041479,
	"rewards/accuracy_reward": 0.5725000067614019,
	"rewards/format_reward": 0.9237500071525574,
	"step": 970
	},
	{
	"completion_length": 267.5337539672852,
	"epoch": 1.3066666666666666,
	"grad_norm": 0.694874107837677,
	"kl": 0.81298828125,
	"learning_rate": 2.0834179522641508e-06,
	"loss": 0.175,
	"reward": 1.436250016093254,
	"reward_std": 0.48411559909582136,
	"rewards/accuracy_reward": 0.5475000135600567,
	"rewards/format_reward": 0.8887500107288361,
	"step": 980
	},
	{
	"completion_length": 249.49375286102295,
	"epoch": 1.32,
	"grad_norm": 1.1866592168807983,
	"kl": 0.6787109375,
	"learning_rate": 2.061909890123868e-06,
	"loss": 0.1094,
	"reward": 1.5050000250339508,
	"reward_std": 0.38125632815063,
	"rewards/accuracy_reward": 0.5500000072643161,
	"rewards/format_reward": 0.955000002682209,
	"step": 990
	},
	{
	"completion_length": 212.09500427246093,
	"epoch": 1.3333333333333333,
	"grad_norm": 0.23463357985019684,
	"kl": 0.49178466796875,
	"learning_rate": 2.040266587207066e-06,
	"loss": 0.06,
	"reward": 1.6800000190734863,
	"reward_std": 0.24786825627088546,
	"rewards/accuracy_reward": 0.695000005979091,
	"rewards/format_reward": 0.9850000008940697,
	"step": 1000
	},
	{
	"completion_length": 242.02375335693358,
	"epoch": 1.3466666666666667,
	"grad_norm": 0.6274837255477905,
	"kl": 0.54989013671875,
	"learning_rate": 2.018493252635605e-06,
	"loss": 0.1312,
	"reward": 1.4687500163912772,
	"reward_std": 0.4060431461781263,
	"rewards/accuracy_reward": 0.5275000087916851,
	"rewards/format_reward": 0.9412499994039536,
	"step": 1010
	},
	{
	"completion_length": 245.67000522613526,
	"epoch": 1.3599999999999999,
	"grad_norm": 0.6642753481864929,
	"kl": 1.0931396484375,
	"learning_rate": 1.9965951268274372e-06,
	"loss": 0.1868,
	"reward": 1.407500022649765,
	"reward_std": 0.43992825597524643,
	"rewards/accuracy_reward": 0.5087500089779496,
	"rewards/format_reward": 0.8987500041723251,
	"step": 1020
	},
	{
	"completion_length": 278.58875465393066,
	"epoch": 1.3733333333333333,
	"grad_norm": 0.6922321915626526,
	"kl": 0.83028564453125,
	"learning_rate": 1.9745774802353347e-06,
	"loss": 0.2212,
	"reward": 1.403750029206276,
	"reward_std": 0.4952257383614779,
	"rewards/accuracy_reward": 0.5325000079348683,
	"rewards/format_reward": 0.871250006556511,
	"step": 1030
	},
	{
	"completion_length": 248.9375057220459,
	"epoch": 1.3866666666666667,
	"grad_norm": 0.7095142602920532,
	"kl": 0.58486328125,
	"learning_rate": 1.9524456120783984e-06,
	"loss": 0.1207,
	"reward": 1.4600000217556954,
	"reward_std": 0.40168970376253127,
	"rewards/accuracy_reward": 0.5100000099278986,
	"rewards/format_reward": 0.9500000044703484,
	"step": 1040
	},
	{
	"completion_length": 205.45750255584716,
	"epoch": 1.4,
	"grad_norm": 0.5801204442977905,
	"kl": 0.292529296875,
	"learning_rate": 1.9302048490666355e-06,
	"loss": 0.0814,
	"reward": 1.6212500214576722,
	"reward_std": 0.3047562278807163,
	"rewards/accuracy_reward": 0.6475000135600567,
	"rewards/format_reward": 0.9737500011920929,
	"step": 1050
	},
	{
	"completion_length": 273.5137548446655,
	"epoch": 1.4133333333333333,
	"grad_norm": 0.6409977078437805,
	"kl": 0.58953857421875,
	"learning_rate": 1.9078605441189272e-06,
	"loss": 0.1281,
	"reward": 1.563750021159649,
	"reward_std": 0.356473108753562,
	"rewards/accuracy_reward": 0.620000010356307,
	"rewards/format_reward": 0.9437500007450581,
	"step": 1060
	},
	{
	"completion_length": 294.0987562179565,
	"epoch": 1.4266666666666667,
	"grad_norm": 0.7412673234939575,
	"kl": 1.6057373046875,
	"learning_rate": 1.8854180750746833e-06,
	"loss": 0.2387,
	"reward": 1.4962500244379044,
	"reward_std": 0.4668436422944069,
	"rewards/accuracy_reward": 0.5862500118091702,
	"rewards/format_reward": 0.9100000038743019,
	"step": 1070
	},
	{
	"completion_length": 293.7837539672852,
	"epoch": 1.44,
	"grad_norm": 0.8722511529922485,
	"kl": 0.612548828125,
	"learning_rate": 1.8628828433995015e-06,
	"loss": 0.1585,
	"reward": 1.4412500128149985,
	"reward_std": 0.44985799603164195,
	"rewards/accuracy_reward": 0.528750010021031,
	"rewards/format_reward": 0.9125000044703484,
	"step": 1080
	},
	{
	"completion_length": 252.32750244140624,
	"epoch": 1.4533333333333334,
	"grad_norm": 0.5518996119499207,
	"kl": 0.4095947265625,
	"learning_rate": 1.8402602728851404e-06,
	"loss": 0.1381,
	"reward": 1.5250000193715096,
	"reward_std": 0.4259300407022238,
	"rewards/accuracy_reward": 0.5825000069104135,
	"rewards/format_reward": 0.942500002682209,
	"step": 1090
	},
	{
	"completion_length": 241.40625343322753,
	"epoch": 1.4666666666666668,
	"grad_norm": 0.4469586908817291,
	"kl": 0.302294921875,
	"learning_rate": 1.8175558083441164e-06,
	"loss": 0.0911,
	"reward": 1.506250025331974,
	"reward_std": 0.4210752282291651,
	"rewards/accuracy_reward": 0.5437500099651515,
	"rewards/format_reward": 0.9625000029802322,
	"step": 1100
	},
	{
	"completion_length": 286.4412563323975,
	"epoch": 1.48,
	"grad_norm": 0.7064781785011292,
	"kl": 0.4929443359375,
	"learning_rate": 1.7947749142992453e-06,
	"loss": 0.2215,
	"reward": 1.4950000211596488,
	"reward_std": 0.51452008895576,
	"rewards/accuracy_reward": 0.5887500088661909,
	"rewards/format_reward": 0.9062500074505806,
	"step": 1110
	},
	{
	"completion_length": 285.3000036239624,
	"epoch": 1.4933333333333334,
	"grad_norm": 0.34774771332740784,
	"kl": 0.623095703125,
	"learning_rate": 1.7719230736684376e-06,
	"loss": 0.1511,
	"reward": 1.4975000232458116,
	"reward_std": 0.43432362116873263,
	"rewards/accuracy_reward": 0.567500009946525,
	"rewards/format_reward": 0.9300000041723251,
	"step": 1120
	},
	{
	"completion_length": 286.05500450134275,
	"epoch": 1.5066666666666668,
	"grad_norm": 0.33899247646331787,
	"kl": 0.45242919921875,
	"learning_rate": 1.7490057864450664e-06,
	"loss": 0.1266,
	"reward": 1.588750024139881,
	"reward_std": 0.38815081119537354,
	"rewards/accuracy_reward": 0.6287500103004277,
	"rewards/format_reward": 0.9600000023841858,
	"step": 1130
	},
	{
	"completion_length": 244.75125102996827,
	"epoch": 1.52,
	"grad_norm": 0.8289720416069031,
	"kl": 0.41170654296875,
	"learning_rate": 1.7260285683742248e-06,
	"loss": 0.1008,
	"reward": 1.5962500244379043,
	"reward_std": 0.3575746387243271,
	"rewards/accuracy_reward": 0.6387500097043812,
	"rewards/format_reward": 0.9575000002980232,
	"step": 1140
	},
	{
	"completion_length": 295.0500051498413,
	"epoch": 1.5333333333333332,
	"grad_norm": 0.7848290205001831,
	"kl": 0.7958251953125,
	"learning_rate": 1.702996949625197e-06,
	"loss": 0.1871,
	"reward": 1.458750021457672,
	"reward_std": 0.44078024849295616,
	"rewards/accuracy_reward": 0.556250006519258,
	"rewards/format_reward": 0.9025000110268593,
	"step": 1150
	},
	{
	"completion_length": 257.19375495910646,
	"epoch": 1.5466666666666666,
	"grad_norm": 1.2715580463409424,
	"kl": 0.8576416015625,
	"learning_rate": 1.6799164734604496e-06,
	"loss": 0.2015,
	"reward": 1.556250013411045,
	"reward_std": 0.5148981977254152,
	"rewards/accuracy_reward": 0.6575000089593231,
	"rewards/format_reward": 0.8987500116229057,
	"step": 1160
	},
	{
	"completion_length": 288.4437545776367,
	"epoch": 1.56,
	"grad_norm": 1.1853872537612915,
	"kl": 1.4738037109375,
	"learning_rate": 1.6567926949014804e-06,
	"loss": 0.2435,
	"reward": 1.4325000256299973,
	"reward_std": 0.45326643362641333,
	"rewards/accuracy_reward": 0.5550000136718154,
	"rewards/format_reward": 0.8775000154972077,
	"step": 1170
	},
	{
	"completion_length": 250.51375427246094,
	"epoch": 1.5733333333333333,
	"grad_norm": 0.7030949592590332,
	"kl": 0.99638671875,
	"learning_rate": 1.6336311793918296e-06,
	"loss": 0.1342,
	"reward": 1.4425000190734862,
	"reward_std": 0.3800163000822067,
	"rewards/accuracy_reward": 0.49750001104548575,
	"rewards/format_reward": 0.9450000047683715,
	"step": 1180
	},
	{
	"completion_length": 239.85750503540038,
	"epoch": 1.5866666666666667,
	"grad_norm": 0.744570791721344,
	"kl": 0.98948974609375,
	"learning_rate": 1.6104375014575872e-06,
	"loss": 0.1446,
	"reward": 1.5650000274181366,
	"reward_std": 0.404941875860095,
	"rewards/accuracy_reward": 0.6237500123679638,
	"rewards/format_reward": 0.9412500113248825,
	"step": 1190
	},
	{
	"completion_length": 242.6050043106079,
	"epoch": 1.6,
	"grad_norm": 0.6778237819671631,
	"kl": 59.4015869140625,
	"learning_rate": 1.5872172433657137e-06,
	"loss": 3.6695,
	"reward": 1.4675000220537187,
	"reward_std": 0.4261274352669716,
	"rewards/accuracy_reward": 0.5350000061094761,
	"rewards/format_reward": 0.9325000092387199,
	"step": 1200
	},
	{
	"completion_length": 254.56750507354735,
	"epoch": 1.6133333333333333,
	"grad_norm": 1.1174402236938477,
	"kl": 1.0924072265625,
	"learning_rate": 1.563975993780496e-06,
	"loss": 0.2037,
	"reward": 1.426250010728836,
	"reward_std": 0.4171265188604593,
	"rewards/accuracy_reward": 0.5037500091828406,
	"rewards/format_reward": 0.9225000068545341,
	"step": 1210
	},
	{
	"completion_length": 250.00875434875488,
	"epoch": 1.6266666666666667,
	"grad_norm": 0.1975303441286087,
	"kl": 0.5531982421875,
	"learning_rate": 1.5407193464184646e-06,
	"loss": 0.0767,
	"reward": 1.5937500238418578,
	"reward_std": 0.293658309802413,
	"rewards/accuracy_reward": 0.6125000089406967,
	"rewards/format_reward": 0.9812499985098839,
	"step": 1220
	},
	{
	"completion_length": 250.21375293731688,
	"epoch": 1.6400000000000001,
	"grad_norm": 0.6114612817764282,
	"kl": 0.53138427734375,
	"learning_rate": 1.5174528987020958e-06,
	"loss": 0.1333,
	"reward": 1.557500022649765,
	"reward_std": 0.36959521323442457,
	"rewards/accuracy_reward": 0.6162500135600567,
	"rewards/format_reward": 0.9412500083446502,
	"step": 1230
	},
	{
	"completion_length": 296.5837543487549,
	"epoch": 1.6533333333333333,
	"grad_norm": 0.6539386510848999,
	"kl": 0.7985107421875,
	"learning_rate": 1.4941822504126198e-06,
	"loss": 0.1803,
	"reward": 1.4350000217556953,
	"reward_std": 0.4217760156840086,
	"rewards/accuracy_reward": 0.5200000086799264,
	"rewards/format_reward": 0.9150000095367432,
	"step": 1240
	},
	{
	"completion_length": 257.9250057220459,
	"epoch": 1.6666666666666665,
	"grad_norm": 0.6114557385444641,
	"kl": 44.12962646484375,
	"learning_rate": 1.4709130023422637e-06,
	"loss": 0.8266,
	"reward": 1.5075000166893004,
	"reward_std": 0.3939127091318369,
	"rewards/accuracy_reward": 0.5762500106357038,
	"rewards/format_reward": 0.9312500074505806,
	"step": 1250
	},
	{
	"completion_length": 265.36750450134275,
	"epoch": 1.6800000000000002,
	"grad_norm": 0.49496781826019287,
	"kl": 0.77027587890625,
	"learning_rate": 1.4476507549462489e-06,
	"loss": 0.1636,
	"reward": 1.4825000166893005,
	"reward_std": 0.443204028531909,
	"rewards/accuracy_reward": 0.5437500104308128,
	"rewards/format_reward": 0.9387500062584877,
	"step": 1260
	},
	{
	"completion_length": 281.16000480651854,
	"epoch": 1.6933333333333334,
	"grad_norm": 0.7085089087486267,
	"kl": 0.9700439453125,
	"learning_rate": 1.4244011069948701e-06,
	"loss": 0.2137,
	"reward": 1.413750021159649,
	"reward_std": 0.4564998596906662,
	"rewards/accuracy_reward": 0.5200000060722232,
	"rewards/format_reward": 0.8937500104308128,
	"step": 1270
	},
	{
	"completion_length": 287.8412548065186,
	"epoch": 1.7066666666666666,
	"grad_norm": 1.8162543773651123,
	"kl": 1.462109375,
	"learning_rate": 1.401169654225982e-06,
	"loss": 0.2375,
	"reward": 1.4162500217556953,
	"reward_std": 0.49586384519934656,
	"rewards/accuracy_reward": 0.5150000074878335,
	"rewards/format_reward": 0.9012500092387199,
	"step": 1280
	},
	{
	"completion_length": 279.57375583648684,
	"epoch": 1.72,
	"grad_norm": 0.674432098865509,
	"kl": 0.92662353515625,
	"learning_rate": 1.3779619879982127e-06,
	"loss": 0.1772,
	"reward": 1.5175000190734864,
	"reward_std": 0.4668840833008289,
	"rewards/accuracy_reward": 0.5912500144913793,
	"rewards/format_reward": 0.9262500107288361,
	"step": 1290
	},
	{
	"completion_length": 289.03125457763673,
	"epoch": 1.7333333333333334,
	"grad_norm": 0.7836817502975464,
	"kl": 0.88634033203125,
	"learning_rate": 1.3547836939452313e-06,
	"loss": 0.1244,
	"reward": 1.4987500250339507,
	"reward_std": 0.3810611065477133,
	"rewards/accuracy_reward": 0.5412500070407986,
	"rewards/format_reward": 0.9575000047683716,
	"step": 1300
	},
	{
	"completion_length": 284.0875045776367,
	"epoch": 1.7466666666666666,
	"grad_norm": 0.7931172847747803,
	"kl": 1.0406494140625,
	"learning_rate": 1.3316403506313982e-06,
	"loss": 0.1187,
	"reward": 1.5062500283122062,
	"reward_std": 0.3796513009816408,
	"rewards/accuracy_reward": 0.560000008251518,
	"rewards/format_reward": 0.9462500020861626,
	"step": 1310
	},
	{
	"completion_length": 267.00625438690184,
	"epoch": 1.76,
	"grad_norm": 0.7942720651626587,
	"kl": 0.7413818359375,
	"learning_rate": 1.308537528209108e-06,
	"loss": 0.0855,
	"reward": 1.5075000256299973,
	"reward_std": 0.43465386927127836,
	"rewards/accuracy_reward": 0.568750012665987,
	"rewards/format_reward": 0.9387500032782554,
	"step": 1320
	},
	{
	"completion_length": 275.38625450134276,
	"epoch": 1.7733333333333334,
	"grad_norm": 1.8723665475845337,
	"kl": 1.5499755859375,
	"learning_rate": 1.2854807870781687e-06,
	"loss": 0.1942,
	"reward": 1.4487500324845315,
	"reward_std": 0.42911659814417363,
	"rewards/accuracy_reward": 0.5437500085681677,
	"rewards/format_reward": 0.9050000056624412,
	"step": 1330
	},
	{
	"completion_length": 276.2012542724609,
	"epoch": 1.7866666666666666,
	"grad_norm": 0.5324939489364624,
	"kl": 1.421630859375,
	"learning_rate": 1.2624756765475158e-06,
	"loss": 0.1505,
	"reward": 1.3537500247359275,
	"reward_std": 0.43583590127527716,
	"rewards/accuracy_reward": 0.462500009406358,
	"rewards/format_reward": 0.8912500083446503,
	"step": 1340
	},
	{
	"completion_length": 270.83750343322754,
	"epoch": 1.8,
	"grad_norm": 2.056743621826172,
	"kl": 1.954345703125,
	"learning_rate": 1.2395277334996047e-06,
	"loss": 0.2111,
	"reward": 1.5400000274181367,
	"reward_std": 0.45903370566666124,
	"rewards/accuracy_reward": 0.60125000923872,
	"rewards/format_reward": 0.9387500077486038,
	"step": 1350
	},
	{
	"completion_length": 279.6050037384033,
	"epoch": 1.8133333333333335,
	"grad_norm": 0.9626988768577576,
	"kl": 1.0287353515625,
	"learning_rate": 1.2166424810577898e-06,
	"loss": 0.1235,
	"reward": 1.4925000071525574,
	"reward_std": 0.3699305009096861,
	"rewards/accuracy_reward": 0.5512500081211329,
	"rewards/format_reward": 0.9412500083446502,
	"step": 1360
	},
	{
	"completion_length": 325.78000526428224,
	"epoch": 1.8266666666666667,
	"grad_norm": 1.466002345085144,
	"kl": 1.82490234375,
	"learning_rate": 1.1938254272570166e-06,
	"loss": 0.2711,
	"reward": 1.445000024139881,
	"reward_std": 0.5365722481161356,
	"rewards/accuracy_reward": 0.5700000077486038,
	"rewards/format_reward": 0.8750000096857548,
	"step": 1370
	},
	{
	"completion_length": 284.211255645752,
	"epoch": 1.8399999999999999,
	"grad_norm": 0.9256098866462708,
	"kl": 1.1132080078125,
	"learning_rate": 1.1710820637181448e-06,
	"loss": 0.1941,
	"reward": 1.490000031888485,
	"reward_std": 0.47110406346619127,
	"rewards/accuracy_reward": 0.58000000808388,
	"rewards/format_reward": 0.9100000098347664,
	"step": 1380
	},
	{
	"completion_length": 248.72625560760497,
	"epoch": 1.8533333333333335,
	"grad_norm": 0.4293259084224701,
	"kl": 0.5682373046875,
	"learning_rate": 1.1484178643262232e-06,
	"loss": 0.0911,
	"reward": 1.6337500184774398,
	"reward_std": 0.3414465494453907,
	"rewards/accuracy_reward": 0.6662500083446503,
	"rewards/format_reward": 0.967500002682209,
	"step": 1390
	},
	{
	"completion_length": 240.6875026702881,
	"epoch": 1.8666666666666667,
	"grad_norm": 0.5720169544219971,
	"kl": 0.82125244140625,
	"learning_rate": 1.1258382839130282e-06,
	"loss": 0.1097,
	"reward": 1.570000022649765,
	"reward_std": 0.4139075022190809,
	"rewards/accuracy_reward": 0.6237500090152025,
	"rewards/format_reward": 0.9462500035762786,
	"step": 1400
	},
	{
	"completion_length": 257.53500480651854,
	"epoch": 1.88,
	"grad_norm": 1.1818897724151611,
	"kl": 0.95081787109375,
	"learning_rate": 1.103348756944197e-06,
	"loss": 0.1008,
	"reward": 1.4837500154972076,
	"reward_std": 0.42501535080373287,
	"rewards/accuracy_reward": 0.5600000080652535,
	"rewards/format_reward": 0.9237500101327896,
	"step": 1410
	},
	{
	"completion_length": 247.4950050354004,
	"epoch": 1.8933333333333333,
	"grad_norm": 0.6974407434463501,
	"kl": 1.0361083984375,
	"learning_rate": 1.0809546962112535e-06,
	"loss": 0.1492,
	"reward": 1.4875000149011612,
	"reward_std": 0.3567169703543186,
	"rewards/accuracy_reward": 0.538750009983778,
	"rewards/format_reward": 0.9487500041723251,
	"step": 1420
	},
	{
	"completion_length": 268.65250663757325,
	"epoch": 1.9066666666666667,
	"grad_norm": 1.1433619260787964,
	"kl": 1.185693359375,
	"learning_rate": 1.0586614915288572e-06,
	"loss": 0.1517,
	"reward": 1.4925000205636025,
	"reward_std": 0.4326970729976892,
	"rewards/accuracy_reward": 0.5587500042282045,
	"rewards/format_reward": 0.9337500005960464,
	"step": 1430
	},
	{
	"completion_length": 255.78125648498536,
	"epoch": 1.92,
	"grad_norm": 0.6243150234222412,
	"kl": 1.2357177734375,
	"learning_rate": 1.036474508437579e-06,
	"loss": 0.1554,
	"reward": 1.4700000181794166,
	"reward_std": 0.44454205557703974,
	"rewards/accuracy_reward": 0.5537500033155084,
	"rewards/format_reward": 0.9162500083446503,
	"step": 1440
	},
	{
	"completion_length": 234.58750495910644,
	"epoch": 1.9333333333333333,
	"grad_norm": 0.6096332669258118,
	"kl": 0.67691650390625,
	"learning_rate": 1.0143990869125186e-06,
	"loss": 0.099,
	"reward": 1.5912500202655793,
	"reward_std": 0.36347288005053996,
	"rewards/accuracy_reward": 0.6275000140070915,
	"rewards/format_reward": 0.9637500032782554,
	"step": 1450
	},
	{
	"completion_length": 256.7262538909912,
	"epoch": 1.9466666666666668,
	"grad_norm": 1.1025887727737427,
	"kl": 1.1599609375,
	"learning_rate": 9.924405400780786e-07,
	"loss": 0.1453,
	"reward": 1.4675000190734864,
	"reward_std": 0.3775463242083788,
	"rewards/accuracy_reward": 0.5312500094994903,
	"rewards/format_reward": 0.9362500011920929,
	"step": 1460
	},
	{
	"completion_length": 268.33000297546386,
	"epoch": 1.96,
	"grad_norm": 1.1707123517990112,
	"kl": 1.133642578125,
	"learning_rate": 9.70604152929197e-07,
	"loss": 0.1489,
	"reward": 1.4312500163912774,
	"reward_std": 0.4398480519652367,
	"rewards/accuracy_reward": 0.5075000053271651,
	"rewards/format_reward": 0.9237500041723251,
	"step": 1470
	},
	{
	"completion_length": 257.52125358581543,
	"epoch": 1.9733333333333334,
	"grad_norm": 0.9173078536987305,
	"kl": 1.48629150390625,
	"learning_rate": 9.488951810593526e-07,
	"loss": 0.1856,
	"reward": 1.4750000312924385,
	"reward_std": 0.4462425637990236,
	"rewards/accuracy_reward": 0.5600000127218664,
	"rewards/format_reward": 0.9150000050663948,
	"step": 1480
	},
	{
	"completion_length": 266.10000228881836,
	"epoch": 1.9866666666666668,
	"grad_norm": 0.7360009551048279,
	"kl": 1.1963623046875,
	"learning_rate": 9.273188493956475e-07,
	"loss": 0.1318,
	"reward": 1.5275000244379044,
	"reward_std": 0.38836930617690085,
	"rewards/accuracy_reward": 0.6137500053271652,
	"rewards/format_reward": 0.9137500077486038,
	"step": 1490
	},
	{
	"completion_length": 254.0337522506714,
	"epoch": 2.0,
	"grad_norm": 0.8448986411094666,
	"kl": 1.1837646484375,
	"learning_rate": 9.058803509412648e-07,
	"loss": 0.1552,
	"reward": 1.5100000187754632,
	"reward_std": 0.4034050587564707,
	"rewards/accuracy_reward": 0.5787500125356019,
	"rewards/format_reward": 0.9312500029802322,
	"step": 1500
	},
	{
	"completion_length": 251.92000274658204,
	"epoch": 2.013333333333333,
	"grad_norm": 0.3304300904273987,
	"kl": 0.811279296875,
	"learning_rate": 8.84584845525618e-07,
	"loss": 0.1094,
	"reward": 1.5350000232458114,
	"reward_std": 0.37311027348041537,
	"rewards/accuracy_reward": 0.5937500129453838,
	"rewards/format_reward": 0.941250005364418,
	"step": 1510
	},
	{
	"completion_length": 257.3825050354004,
	"epoch": 2.026666666666667,
	"grad_norm": 2.269232988357544,
	"kl": 1.01397705078125,
	"learning_rate": 8.63437458562477e-07,
	"loss": 0.1731,
	"reward": 1.5512500256299973,
	"reward_std": 0.4206295557320118,
	"rewards/accuracy_reward": 0.6212500111199916,
	"rewards/format_reward": 0.9300000056624412,
	"step": 1520
	},
	{
	"completion_length": 246.79625415802002,
	"epoch": 2.04,
	"grad_norm": 2.4657490253448486,
	"kl": 1.84619140625,
	"learning_rate": 8.424432798163837e-07,
	"loss": 0.1957,
	"reward": 1.462500013411045,
	"reward_std": 0.46618823148310184,
	"rewards/accuracy_reward": 0.5862500076182187,
	"rewards/format_reward": 0.87625000923872,
	"step": 1530
	},
	{
	"completion_length": 244.07625579833984,
	"epoch": 2.0533333333333332,
	"grad_norm": 0.878600537776947,
	"kl": 1.319287109375,
	"learning_rate": 8.216073621776437e-07,
	"loss": 0.097,
	"reward": 1.4925000190734863,
	"reward_std": 0.4461408320814371,
	"rewards/accuracy_reward": 0.5762500118464231,
	"rewards/format_reward": 0.916250005364418,
	"step": 1540
	},
	{
	"completion_length": 231.71000556945802,
	"epoch": 2.066666666666667,
	"grad_norm": 0.6690750122070312,
	"kl": 0.91011962890625,
	"learning_rate": 8.009347204461922e-07,
	"loss": 0.0719,
	"reward": 1.5675000205636025,
	"reward_std": 0.3554057668894529,
	"rewards/accuracy_reward": 0.6100000100210309,
	"rewards/format_reward": 0.9575000047683716,
	"step": 1550
	},
	{
	"completion_length": 230.32750396728517,
	"epoch": 2.08,
	"grad_norm": 0.9904080033302307,
	"kl": 0.50635986328125,
	"learning_rate": 7.804303301246311e-07,
	"loss": 0.0681,
	"reward": 1.675000011920929,
	"reward_std": 0.30205987617373464,
	"rewards/accuracy_reward": 0.7025000074878335,
	"rewards/format_reward": 0.9725000008940696,
	"step": 1560
	},
	{
	"completion_length": 257.5050031661987,
	"epoch": 2.0933333333333333,
	"grad_norm": 0.5869216918945312,
	"kl": 1.6803466796875,
	"learning_rate": 7.600991262207221e-07,
	"loss": 0.1555,
	"reward": 1.542500025033951,
	"reward_std": 0.3918322518467903,
	"rewards/accuracy_reward": 0.6312500072643161,
	"rewards/format_reward": 0.9112500101327896,
	"step": 1570
	},
	{
	"completion_length": 257.85625553131104,
	"epoch": 2.1066666666666665,
	"grad_norm": 0.7470917105674744,
	"kl": 1.1965576171875,
	"learning_rate": 7.399460020596266e-07,
	"loss": 0.1217,
	"reward": 1.4912500128149986,
	"reward_std": 0.44596712924540044,
	"rewards/accuracy_reward": 0.5762500122189522,
	"rewards/format_reward": 0.9150000065565109,
	"step": 1580
	},
	{
	"completion_length": 249.50250511169435,
	"epoch": 2.12,
	"grad_norm": 1.4397474527359009,
	"kl": 1.5828369140625,
	"learning_rate": 7.19975808106177e-07,
	"loss": 0.1932,
	"reward": 1.5000000178813935,
	"reward_std": 0.44045890420675277,
	"rewards/accuracy_reward": 0.5787500116042793,
	"rewards/format_reward": 0.921250007301569,
	"step": 1590
	},
	{
	"completion_length": 255.7712547302246,
	"epoch": 2.1333333333333333,
	"grad_norm": 0.47938272356987,
	"kl": 1.07216796875,
	"learning_rate": 7.001933507974634e-07,
	"loss": 0.1435,
	"reward": 1.4987500235438347,
	"reward_std": 0.4268573712557554,
	"rewards/accuracy_reward": 0.5737500079907477,
	"rewards/format_reward": 0.9250000044703484,
	"step": 1600
	},
	{
	"completion_length": 245.55125331878662,
	"epoch": 2.1466666666666665,
	"grad_norm": 0.8688609600067139,
	"kl": 1.22481689453125,
	"learning_rate": 6.806033913860195e-07,
	"loss": 0.1412,
	"reward": 1.4875000298023224,
	"reward_std": 0.41953446678817274,
	"rewards/accuracy_reward": 0.5675000123679638,
	"rewards/format_reward": 0.9200000070035458,
	"step": 1610
	},
	{
	"completion_length": 250.85125579833985,
	"epoch": 2.16,
	"grad_norm": 0.5293228030204773,
	"kl": 0.846044921875,
	"learning_rate": 6.6121064479388e-07,
	"loss": 0.1231,
	"reward": 1.5712500154972076,
	"reward_std": 0.38107405565679076,
	"rewards/accuracy_reward": 0.6225000062957406,
	"rewards/format_reward": 0.9487500011920929,
	"step": 1620
	},
	{
	"completion_length": 252.32750434875487,
	"epoch": 2.1733333333333333,
	"grad_norm": 1.0724272727966309,
	"kl": 1.2016357421875,
	"learning_rate": 6.420197784777925e-07,
	"loss": 0.1292,
	"reward": 1.4837500169873237,
	"reward_std": 0.4388523455709219,
	"rewards/accuracy_reward": 0.5450000113807618,
	"rewards/format_reward": 0.9387500062584877,
	"step": 1630
	},
	{
	"completion_length": 262.9075046539307,
	"epoch": 2.1866666666666665,
	"grad_norm": 0.852165699005127,
	"kl": 1.47001953125,
	"learning_rate": 6.230354113058505e-07,
	"loss": 0.1207,
	"reward": 1.438750022649765,
	"reward_std": 0.47429373003542424,
	"rewards/accuracy_reward": 0.5275000119581819,
	"rewards/format_reward": 0.9112500056624413,
	"step": 1640
	},
	{
	"completion_length": 237.82875328063966,
	"epoch": 2.2,
	"grad_norm": 0.7865124344825745,
	"kl": 0.8052978515625,
	"learning_rate": 6.04262112445821e-07,
	"loss": 0.0906,
	"reward": 1.5237500220537186,
	"reward_std": 0.35442090816795824,
	"rewards/accuracy_reward": 0.5787500074133277,
	"rewards/format_reward": 0.9450000032782555,
	"step": 1650
	},
	{
	"completion_length": 244.2200050354004,
	"epoch": 2.2133333333333334,
	"grad_norm": 2.7681257724761963,
	"kl": 1.0701416015625,
	"learning_rate": 5.857044002654356e-07,
	"loss": 0.1363,
	"reward": 1.5400000184774398,
	"reward_std": 0.41701370403170585,
	"rewards/accuracy_reward": 0.6250000096857548,
	"rewards/format_reward": 0.9150000095367432,
	"step": 1660
	},
	{
	"completion_length": 236.56875324249268,
	"epoch": 2.2266666666666666,
	"grad_norm": 1.139227032661438,
	"kl": 1.362841796875,
	"learning_rate": 5.673667412449069e-07,
	"loss": 0.1759,
	"reward": 1.4887500166893006,
	"reward_std": 0.46174298636615274,
	"rewards/accuracy_reward": 0.5825000075623393,
	"rewards/format_reward": 0.9062500104308129,
	"step": 1670
	},
	{
	"completion_length": 251.44250373840333,
	"epoch": 2.24,
	"grad_norm": 1.1506483554840088,
	"kl": 3.351171875,
	"learning_rate": 5.492535489019345e-07,
	"loss": 0.3264,
	"reward": 1.5837500229477883,
	"reward_std": 0.4318976990878582,
	"rewards/accuracy_reward": 0.6537500128149987,
	"rewards/format_reward": 0.9300000041723251,
	"step": 1680
	},
	{
	"completion_length": 237.26500434875487,
	"epoch": 2.2533333333333334,
	"grad_norm": 0.24002552032470703,
	"kl": 0.6318603515625,
	"learning_rate": 5.313691827294569e-07,
	"loss": 0.0718,
	"reward": 1.6012500166893004,
	"reward_std": 0.3187932658940554,
	"rewards/accuracy_reward": 0.6325000114738941,
	"rewards/format_reward": 0.9687500029802323,
	"step": 1690
	},
	{
	"completion_length": 239.81500282287598,
	"epoch": 2.2666666666666666,
	"grad_norm": 0.3545837998390198,
	"kl": 0.5182861328125,
	"learning_rate": 5.137179471464047e-07,
	"loss": 0.0914,
	"reward": 1.5900000303983688,
	"reward_std": 0.36133957989513876,
	"rewards/accuracy_reward": 0.6212500061839819,
	"rewards/format_reward": 0.9687500029802323,
	"step": 1700
	},
	{
	"completion_length": 239.17750453948975,
	"epoch": 2.2800000000000002,
	"grad_norm": 0.3821852505207062,
	"kl": 0.6372802734375,
	"learning_rate": 4.963040904617131e-07,
	"loss": 0.0939,
	"reward": 1.532500021159649,
	"reward_std": 0.3489324226975441,
	"rewards/accuracy_reward": 0.580000011343509,
	"rewards/format_reward": 0.9525000095367432,
	"step": 1710
	},
	{
	"completion_length": 251.4825029373169,
	"epoch": 2.2933333333333334,
	"grad_norm": 1.0452297925949097,
	"kl": 1.1371826171875,
	"learning_rate": 4.791318038518345e-07,
	"loss": 0.1565,
	"reward": 1.4887500151991844,
	"reward_std": 0.4074371732771397,
	"rewards/accuracy_reward": 0.5575000095181167,
	"rewards/format_reward": 0.9312500029802322,
	"step": 1720
	},
	{
	"completion_length": 252.94750308990479,
	"epoch": 2.3066666666666666,
	"grad_norm": 1.300804853439331,
	"kl": 1.0220703125,
	"learning_rate": 4.6220522035200607e-07,
	"loss": 0.1473,
	"reward": 1.4225000247359276,
	"reward_std": 0.4227703019976616,
	"rewards/accuracy_reward": 0.4975000057369471,
	"rewards/format_reward": 0.9250000044703484,
	"step": 1730
	},
	{
	"completion_length": 245.8537540435791,
	"epoch": 2.32,
	"grad_norm": 1.0373808145523071,
	"kl": 1.1215576171875,
	"learning_rate": 4.4552841386150737e-07,
	"loss": 0.1625,
	"reward": 1.570000010728836,
	"reward_std": 0.44271881096065047,
	"rewards/accuracy_reward": 0.6412500072270632,
	"rewards/format_reward": 0.9287500068545341,
	"step": 1740
	},
	{
	"completion_length": 242.66250534057616,
	"epoch": 2.3333333333333335,
	"grad_norm": 1.0271825790405273,
	"kl": 1.0448974609375,
	"learning_rate": 4.291053981631517e-07,
	"loss": 0.1839,
	"reward": 1.5500000163912773,
	"reward_std": 0.4699274588376284,
	"rewards/accuracy_reward": 0.6150000074878335,
	"rewards/format_reward": 0.9350000068545341,
	"step": 1750
	},
	{
	"completion_length": 243.92625598907472,
	"epoch": 2.3466666666666667,
	"grad_norm": 0.7303493022918701,
	"kl": 1.138134765625,
	"learning_rate": 4.129401259572467e-07,
	"loss": 0.1799,
	"reward": 1.595000022649765,
	"reward_std": 0.41913925893604753,
	"rewards/accuracy_reward": 0.6700000133365392,
	"rewards/format_reward": 0.9250000059604645,
	"step": 1760
	},
	{
	"completion_length": 246.7412540435791,
	"epoch": 2.36,
	"grad_norm": 0.9631237983703613,
	"kl": 0.8330322265625,
	"learning_rate": 3.9703648791025716e-07,
	"loss": 0.1262,
	"reward": 1.5162500217556953,
	"reward_std": 0.3487443562597036,
	"rewards/accuracy_reward": 0.5662500067614019,
	"rewards/format_reward": 0.95,
	"step": 1770
	},
	{
	"completion_length": 232.69125442504884,
	"epoch": 2.3733333333333335,
	"grad_norm": 0.3894151449203491,
	"kl": 0.7792236328125,
	"learning_rate": 3.813983117183973e-07,
	"loss": 0.1313,
	"reward": 1.5537500232458115,
	"reward_std": 0.36217943094670774,
	"rewards/accuracy_reward": 0.6062500105239451,
	"rewards/format_reward": 0.947500005364418,
	"step": 1780
	},
	{
	"completion_length": 248.34000549316406,
	"epoch": 2.3866666666666667,
	"grad_norm": 1.5399941205978394,
	"kl": 0.61976318359375,
	"learning_rate": 3.660293611863782e-07,
	"loss": 0.1232,
	"reward": 1.5750000208616257,
	"reward_std": 0.3620978184044361,
	"rewards/accuracy_reward": 0.6150000086054206,
	"rewards/format_reward": 0.9600000008940697,
	"step": 1790
	},
	{
	"completion_length": 256.740007019043,
	"epoch": 2.4,
	"grad_norm": 1.343291163444519,
	"kl": 0.779443359375,
	"learning_rate": 3.5093333532153313e-07,
	"loss": 0.1479,
	"reward": 1.5312500238418578,
	"reward_std": 0.4155931018292904,
	"rewards/accuracy_reward": 0.6075000097043812,
	"rewards/format_reward": 0.9237500041723251,
	"step": 1800
	},
	{
	"completion_length": 254.71500473022462,
	"epoch": 2.413333333333333,
	"grad_norm": 0.8221819400787354,
	"kl": 0.804345703125,
	"learning_rate": 3.361138674435386e-07,
	"loss": 0.1537,
	"reward": 1.520000022649765,
	"reward_std": 0.39584226049482824,
	"rewards/accuracy_reward": 0.5900000093504787,
	"rewards/format_reward": 0.9300000071525574,
	"step": 1810
	},
	{
	"completion_length": 225.73875350952147,
	"epoch": 2.4266666666666667,
	"grad_norm": 0.5089777112007141,
	"kl": 0.951416015625,
	"learning_rate": 3.215745243099449e-07,
	"loss": 0.0989,
	"reward": 1.5050000190734862,
	"reward_std": 0.34333288110792637,
	"rewards/accuracy_reward": 0.5612500077113509,
	"rewards/format_reward": 0.9437500059604644,
	"step": 1820
	},
	{
	"completion_length": 244.07000427246095,
	"epoch": 2.44,
	"grad_norm": 0.4870164096355438,
	"kl": 0.99180908203125,
	"learning_rate": 3.073188052577282e-07,
	"loss": 0.1526,
	"reward": 1.5562500178813934,
	"reward_std": 0.424020304530859,
	"rewards/accuracy_reward": 0.6175000119954348,
	"rewards/format_reward": 0.9387500047683716,
	"step": 1830
	},
	{
	"completion_length": 233.82125263214112,
	"epoch": 2.453333333333333,
	"grad_norm": 1.4273000955581665,
	"kl": 1.02783203125,
	"learning_rate": 2.93350141361067e-07,
	"loss": 0.1579,
	"reward": 1.5900000289082528,
	"reward_std": 0.4301666900515556,
	"rewards/accuracy_reward": 0.6575000090524554,
	"rewards/format_reward": 0.9325000047683716,
	"step": 1840
	},
	{
	"completion_length": 230.63875503540038,
	"epoch": 2.466666666666667,
	"grad_norm": 0.8823533654212952,
	"kl": 0.8214599609375,
	"learning_rate": 2.796718946055488e-07,
	"loss": 0.1264,
	"reward": 1.5850000232458115,
	"reward_std": 0.3670020330697298,
	"rewards/accuracy_reward": 0.6350000068545342,
	"rewards/format_reward": 0.9500000029802322,
	"step": 1850
	},
	{
	"completion_length": 238.70250186920165,
	"epoch": 2.48,
	"grad_norm": 0.6716477274894714,
	"kl": 0.7898681640625,
	"learning_rate": 2.6628735707900655e-07,
	"loss": 0.119,
	"reward": 1.6300000220537185,
	"reward_std": 0.34950714409351347,
	"rewards/accuracy_reward": 0.6787500105798244,
	"rewards/format_reward": 0.9512500062584877,
	"step": 1860
	},
	{
	"completion_length": 255.3150053024292,
	"epoch": 2.493333333333333,
	"grad_norm": 0.72442227602005,
	"kl": 0.9836181640625,
	"learning_rate": 2.531997501791779e-07,
	"loss": 0.1863,
	"reward": 1.5250000268220902,
	"reward_std": 0.4403968006372452,
	"rewards/accuracy_reward": 0.5950000144541263,
	"rewards/format_reward": 0.9300000086426735,
	"step": 1870
	},
	{
	"completion_length": 269.0000047683716,
	"epoch": 2.506666666666667,
	"grad_norm": 0.5673468112945557,
	"kl": 0.9330322265625,
	"learning_rate": 2.4041222383837535e-07,
	"loss": 0.1553,
	"reward": 1.537500023841858,
	"reward_std": 0.4138102397322655,
	"rewards/accuracy_reward": 0.6012500094249844,
	"rewards/format_reward": 0.936250002682209,
	"step": 1880
	},
	{
	"completion_length": 285.29625511169434,
	"epoch": 2.52,
	"grad_norm": 0.7273412942886353,
	"kl": 1.2927001953125,
	"learning_rate": 2.2792785576536108e-07,
	"loss": 0.2201,
	"reward": 1.441250018775463,
	"reward_std": 0.47595637403428553,
	"rewards/accuracy_reward": 0.5312500111758709,
	"rewards/format_reward": 0.9100000083446502,
	"step": 1890
	},
	{
	"completion_length": 254.1075038909912,
	"epoch": 2.533333333333333,
	"grad_norm": 0.752768874168396,
	"kl": 1.1075439453125,
	"learning_rate": 2.1574965070460045e-07,
	"loss": 0.152,
	"reward": 1.6112500235438347,
	"reward_std": 0.40718725696206093,
	"rewards/accuracy_reward": 0.6700000101700425,
	"rewards/format_reward": 0.9412500068545342,
	"step": 1900
	},
	{
	"completion_length": 234.2762550354004,
	"epoch": 2.546666666666667,
	"grad_norm": 0.5253682136535645,
	"kl": 0.95108642578125,
	"learning_rate": 2.0388053971307929e-07,
	"loss": 0.1638,
	"reward": 1.605000016093254,
	"reward_std": 0.39718882702291014,
	"rewards/accuracy_reward": 0.6737500067800284,
	"rewards/format_reward": 0.9312500044703483,
	"step": 1910
	},
	{
	"completion_length": 255.07750434875487,
	"epoch": 2.56,
	"grad_norm": 0.5825310945510864,
	"kl": 1.0739501953125,
	"learning_rate": 1.9232337945485655e-07,
	"loss": 0.1466,
	"reward": 1.5462500244379043,
	"reward_std": 0.4311613071709871,
	"rewards/accuracy_reward": 0.6100000113248825,
	"rewards/format_reward": 0.9362500086426735,
	"step": 1920
	},
	{
	"completion_length": 262.95500526428225,
	"epoch": 2.5733333333333333,
	"grad_norm": 1.1008692979812622,
	"kl": 8.222119140625,
	"learning_rate": 1.810809515135184e-07,
	"loss": 0.4634,
	"reward": 1.4800000235438346,
	"reward_std": 0.4289998158812523,
	"rewards/accuracy_reward": 0.5450000076554715,
	"rewards/format_reward": 0.9350000068545341,
	"step": 1930
	},
	{
	"completion_length": 239.78375244140625,
	"epoch": 2.586666666666667,
	"grad_norm": 0.9321441054344177,
	"kl": 1.1059326171875,
	"learning_rate": 1.701559617227084e-07,
	"loss": 0.1856,
	"reward": 1.5887500256299973,
	"reward_std": 0.413610565662384,
	"rewards/accuracy_reward": 0.6537500060163438,
	"rewards/format_reward": 0.9350000038743019,
	"step": 1940
	},
	{
	"completion_length": 267.56750259399416,
	"epoch": 2.6,
	"grad_norm": 0.4667798578739166,
	"kl": 0.897509765625,
	"learning_rate": 1.5955103951488177e-07,
	"loss": 0.1291,
	"reward": 1.512500023841858,
	"reward_std": 0.36861380077898503,
	"rewards/accuracy_reward": 0.5650000072084367,
	"rewards/format_reward": 0.9475000098347663,
	"step": 1950
	},
	{
	"completion_length": 244.6812545776367,
	"epoch": 2.6133333333333333,
	"grad_norm": 1.1859592199325562,
	"kl": 0.7323974609375,
	"learning_rate": 1.4926873728845668e-07,
	"loss": 0.132,
	"reward": 1.6525000289082528,
	"reward_std": 0.3647048894315958,
	"rewards/accuracy_reward": 0.7000000123865903,
	"rewards/format_reward": 0.9525000050663948,
	"step": 1960
	},
	{
	"completion_length": 250.45750370025635,
	"epoch": 2.626666666666667,
	"grad_norm": 0.6404736042022705,
	"kl": 0.98271484375,
	"learning_rate": 1.3931152979349926e-07,
	"loss": 0.1389,
	"reward": 1.5887500122189522,
	"reward_std": 0.4178105805069208,
	"rewards/accuracy_reward": 0.6537500105798244,
	"rewards/format_reward": 0.9350000023841858,
	"step": 1970
	},
	{
	"completion_length": 241.34000396728516,
	"epoch": 2.64,
	"grad_norm": 1.1606582403182983,
	"kl": 0.8234375,
	"learning_rate": 1.2968181353609853e-07,
	"loss": 0.1034,
	"reward": 1.6037500262260438,
	"reward_std": 0.3661252219229937,
	"rewards/accuracy_reward": 0.6487500038929284,
	"rewards/format_reward": 0.9550000041723251,
	"step": 1980
	},
	{
	"completion_length": 258.2050045013428,
	"epoch": 2.6533333333333333,
	"grad_norm": 0.9993649125099182,
	"kl": 0.7095947265625,
	"learning_rate": 1.2038190620157685e-07,
	"loss": 0.0885,
	"reward": 1.6100000202655793,
	"reward_std": 0.3543590843677521,
	"rewards/accuracy_reward": 0.6512500140815973,
	"rewards/format_reward": 0.9587500020861626,
	"step": 1990
	},
	{
	"completion_length": 248.90000267028807,
	"epoch": 2.6666666666666665,
	"grad_norm": 0.7114527821540833,
	"kl": 486.95928955078125,
	"learning_rate": 1.114140460966645e-07,
	"loss": 20.4763,
	"reward": 1.6000000208616256,
	"reward_std": 0.3433480467647314,
	"rewards/accuracy_reward": 0.6437500057741999,
	"rewards/format_reward": 0.95625,
	"step": 2000
	},
	{
	"completion_length": 242.95375537872314,
	"epoch": 2.68,
	"grad_norm": 1.368294596672058,
	"kl": 0.9123046875,
	"learning_rate": 1.0278039161078634e-07,
	"loss": 0.1081,
	"reward": 1.583750031888485,
	"reward_std": 0.3553517021238804,
	"rewards/accuracy_reward": 0.6350000066682696,
	"rewards/format_reward": 0.9487500056624413,
	"step": 2010
	},
	{
	"completion_length": 267.4425052642822,
	"epoch": 2.6933333333333334,
	"grad_norm": 0.9436002969741821,
	"kl": 1.2000244140625,
	"learning_rate": 9.4483020696578e-08,
	"loss": 0.1847,
	"reward": 1.4937500163912774,
	"reward_std": 0.42551035098731516,
	"rewards/accuracy_reward": 0.5700000072829425,
	"rewards/format_reward": 0.9237500086426735,
	"step": 2020
	},
	{
	"completion_length": 261.7937522888184,
	"epoch": 2.7066666666666666,
	"grad_norm": 0.832007884979248,
	"kl": 1.18310546875,
	"learning_rate": 8.652393036976158e-08,
	"loss": 0.1359,
	"reward": 1.4962500289082528,
	"reward_std": 0.4743993539363146,
	"rewards/accuracy_reward": 0.5837500118650496,
	"rewards/format_reward": 0.9125000044703484,
	"step": 2030
	},
	{
	"completion_length": 248.56500606536866,
	"epoch": 2.7199999999999998,
	"grad_norm": 0.9683050513267517,
	"kl": 1.044775390625,
	"learning_rate": 7.89050362285062e-08,
	"loss": 0.1735,
	"reward": 1.5050000220537185,
	"reward_std": 0.4250973217189312,
	"rewards/accuracy_reward": 0.5762500053271651,
	"rewards/format_reward": 0.928750005364418,
	"step": 2040
	},
	{
	"completion_length": 261.2425033569336,
	"epoch": 2.7333333333333334,
	"grad_norm": 0.8732206225395203,
	"kl": 2.1559326171875,
	"learning_rate": 7.162817199237703e-08,
	"loss": 0.228,
	"reward": 1.4837500289082528,
	"reward_std": 0.4267508018761873,
	"rewards/accuracy_reward": 0.5775000069290399,
	"rewards/format_reward": 0.9062500104308129,
	"step": 2050
	},
	{
	"completion_length": 264.16000480651854,
	"epoch": 2.7466666666666666,
	"grad_norm": 0.6869509220123291,
	"kl": 0.93458251953125,
	"learning_rate": 6.469508906099792e-08,
	"loss": 0.1569,
	"reward": 1.5300000235438347,
	"reward_std": 0.43696386478841304,
	"rewards/accuracy_reward": 0.5937500102445483,
	"rewards/format_reward": 0.9362500056624412,
	"step": 2060
	},
	{
	"completion_length": 238.19875507354737,
	"epoch": 2.76,
	"grad_norm": 1.0900628566741943,
	"kl": 0.9701171875,
	"learning_rate": 5.810745609252166e-08,
	"loss": 0.116,
	"reward": 1.491250030696392,
	"reward_std": 0.35836701430380347,
	"rewards/accuracy_reward": 0.5612500054761768,
	"rewards/format_reward": 0.9300000026822091,
	"step": 2070
	},
	{
	"completion_length": 249.6000057220459,
	"epoch": 2.7733333333333334,
	"grad_norm": 0.7228848338127136,
	"kl": 0.80611572265625,
	"learning_rate": 5.186685860201718e-08,
	"loss": 0.1248,
	"reward": 1.57750001847744,
	"reward_std": 0.37631080821156504,
	"rewards/accuracy_reward": 0.6237500101327896,
	"rewards/format_reward": 0.9537500008940697,
	"step": 2080
	},
	{
	"completion_length": 251.9662546157837,
	"epoch": 2.7866666666666666,
	"grad_norm": 0.471282422542572,
	"kl": 1.272216796875,
	"learning_rate": 4.59747985798662e-08,
	"loss": 0.1567,
	"reward": 1.493750023841858,
	"reward_std": 0.44336883127689364,
	"rewards/accuracy_reward": 0.5787500059232116,
	"rewards/format_reward": 0.9150000050663948,
	"step": 2090
	},
	{
	"completion_length": 268.16250438690184,
	"epoch": 2.8,
	"grad_norm": 0.7542273998260498,
	"kl": 1.019189453125,
	"learning_rate": 4.0432694130264294e-08,
	"loss": 0.1766,
	"reward": 1.512500025331974,
	"reward_std": 0.3997300285845995,
	"rewards/accuracy_reward": 0.5850000084377825,
	"rewards/format_reward": 0.9275000020861626,
	"step": 2100
	},
	{
	"completion_length": 258.7000057220459,
	"epoch": 2.8133333333333335,
	"grad_norm": 0.652788519859314,
	"kl": 1.01083984375,
	"learning_rate": 3.524187912991056e-08,
	"loss": 0.0999,
	"reward": 1.5262500286102294,
	"reward_std": 0.3544511809945107,
	"rewards/accuracy_reward": 0.5775000077672303,
	"rewards/format_reward": 0.9487500041723251,
	"step": 2110
	},
	{
	"completion_length": 257.93125343322754,
	"epoch": 2.8266666666666667,
	"grad_norm": 0.9080987572669983,
	"kl": 1.49254150390625,
	"learning_rate": 3.040360290696909e-08,
	"loss": 0.2127,
	"reward": 1.4812500149011611,
	"reward_std": 0.42118182219564915,
	"rewards/accuracy_reward": 0.5700000080280005,
	"rewards/format_reward": 0.9112500056624413,
	"step": 2120
	},
	{
	"completion_length": 234.36625385284424,
	"epoch": 2.84,
	"grad_norm": 0.9428286552429199,
	"kl": 0.8838134765625,
	"learning_rate": 2.5919029940380145e-08,
	"loss": 0.1446,
	"reward": 1.5850000232458115,
	"reward_std": 0.3889846485108137,
	"rewards/accuracy_reward": 0.652500010933727,
	"rewards/format_reward": 0.9325000062584877,
	"step": 2130
	},
	{
	"completion_length": 253.12500400543212,
	"epoch": 2.8533333333333335,
	"grad_norm": 1.143280029296875,
	"kl": 0.8545166015625,
	"learning_rate": 2.178923957959289e-08,
	"loss": 0.1153,
	"reward": 1.526250024139881,
	"reward_std": 0.39961728677153585,
	"rewards/accuracy_reward": 0.5787500070407987,
	"rewards/format_reward": 0.947500005364418,
	"step": 2140
	},
	{
	"completion_length": 244.91625385284425,
	"epoch": 2.8666666666666667,
	"grad_norm": 0.6661178469657898,
	"kl": 0.988916015625,
	"learning_rate": 1.8015225784786483e-08,
	"loss": 0.1399,
	"reward": 1.551250022649765,
	"reward_std": 0.3881247241050005,
	"rewards/accuracy_reward": 0.6075000107288361,
	"rewards/format_reward": 0.9437500059604644,
	"step": 2150
	},
	{
	"completion_length": 232.5712547302246,
	"epoch": 2.88,
	"grad_norm": 0.9537074565887451,
	"kl": 0.7067138671875,
	"learning_rate": 1.4597896887644457e-08,
	"loss": 0.1309,
	"reward": 1.641250017285347,
	"reward_std": 0.3669580578804016,
	"rewards/accuracy_reward": 0.6787500075995923,
	"rewards/format_reward": 0.9625000029802322,
	"step": 2160
	},
	{
	"completion_length": 228.0150047302246,
	"epoch": 2.8933333333333335,
	"grad_norm": 1.5120928287506104,
	"kl": 0.9121337890625,
	"learning_rate": 1.1538075372735435e-08,
	"loss": 0.1243,
	"reward": 1.5912500202655793,
	"reward_std": 0.36697540059685707,
	"rewards/accuracy_reward": 0.6437500088475645,
	"rewards/format_reward": 0.9475000023841857,
	"step": 2170
	},
	{
	"completion_length": 248.69500427246095,
	"epoch": 2.9066666666666667,
	"grad_norm": 13.778345108032227,
	"kl": 1.008544921875,
	"learning_rate": 8.836497679557964e-09,
	"loss": 0.1507,
	"reward": 1.547500017285347,
	"reward_std": 0.39245944768190383,
	"rewards/accuracy_reward": 0.600000009033829,
	"rewards/format_reward": 0.9475000008940697,
	"step": 2180
	},
	{
	"completion_length": 239.8487533569336,
	"epoch": 2.92,
	"grad_norm": 0.7719865441322327,
	"kl": 0.8395263671875,
	"learning_rate": 6.493814025293476e-09,
	"loss": 0.1101,
	"reward": 1.5650000244379043,
	"reward_std": 0.36453715413808824,
	"rewards/accuracy_reward": 0.616250010766089,
	"rewards/format_reward": 0.948750002682209,
	"step": 2190
	},
	{
	"completion_length": 241.00000209808348,
	"epoch": 2.9333333333333336,
	"grad_norm": 0.8857264518737793,
	"kl": 1.47626953125,
	"learning_rate": 4.510588248311964e-09,
	"loss": 0.1851,
	"reward": 1.4975000217556953,
	"reward_std": 0.3946533836424351,
	"rewards/accuracy_reward": 0.5750000107102096,
	"rewards/format_reward": 0.9225000023841858,
	"step": 2200
	},
	{
	"completion_length": 259.2575029373169,
	"epoch": 2.9466666666666668,
	"grad_norm": 1.154828667640686,
	"kl": 0.94993896484375,
	"learning_rate": 2.8872976724670375e-09,
	"loss": 0.1104,
	"reward": 1.6000000178813933,
	"reward_std": 0.37620634213089943,
	"rewards/accuracy_reward": 0.6500000108033419,
	"rewards/format_reward": 0.9500000044703484,
	"step": 2210
	},
	{
	"completion_length": 269.49375610351564,
	"epoch": 2.96,
	"grad_norm": 0.6843157410621643,
	"kl": 1.0091552734375,
	"learning_rate": 1.624332992213151e-09,
	"loss": 0.1773,
	"reward": 1.4937500327825546,
	"reward_std": 0.4581430654972792,
	"rewards/accuracy_reward": 0.5700000094249844,
	"rewards/format_reward": 0.9237500056624413,
	"step": 2220
	},
	{
	"completion_length": 259.83625545501707,
	"epoch": 2.9733333333333336,
	"grad_norm": 0.854248583316803,
	"kl": 0.9797119140625,
	"learning_rate": 7.219981785733243e-10,
	"loss": 0.1681,
	"reward": 1.557500024139881,
	"reward_std": 0.4305118963122368,
	"rewards/accuracy_reward": 0.6300000097602606,
	"rewards/format_reward": 0.927500008046627,
	"step": 2230
	},
	{
	"completion_length": 244.36375274658204,
	"epoch": 2.986666666666667,
	"grad_norm": 0.7409846186637878,
	"kl": 1.03876953125,
	"learning_rate": 1.8051040597882872e-10,
	"loss": 0.1227,
	"reward": 1.4725000217556954,
	"reward_std": 0.3398312862962484,
	"rewards/accuracy_reward": 0.5237500051036477,
	"rewards/format_reward": 0.9487500056624413,
	"step": 2240
	},
	{
	"completion_length": 253.62000465393066,
	"epoch": 3.0,
	"grad_norm": 1.5809358358383179,
	"kl": 1.09482421875,
	"learning_rate": 0.0,
	"loss": 0.1685,
	"reward": 1.4925000250339509,
	"reward_std": 0.3981329433619976,
	"rewards/accuracy_reward": 0.5537500067614018,
	"rewards/format_reward": 0.9387500017881394,
	"step": 2250
	},
	{
	"epoch": 3.0,
	"step": 2250,
	"total_flos": 0.0,
	"train_loss": 0.283822166296343,
	"train_runtime": 92192.5552,
	"train_samples_per_second": 0.195,
	"train_steps_per_second": 0.024
	}
	],
	"logging_steps": 10,
	"max_steps": 2250,
	"num_input_tokens_seen": 0,
	"num_train_epochs": 3,
	"save_steps": 500,
	"stateful_callbacks": {
	"TrainerControl": {
	"args": {
	"should_epoch_stop": false,
	"should_evaluate": false,
	"should_log": false,
	"should_save": true,
	"should_training_stop": true
	},
	"attributes": {}
	}
	},
	"total_flos": 0.0,
	"train_batch_size": 4,
	"trial_name": null,
	"trial_params": null
	}