| [2024-08-26 11:33:17,763][15329] Saving configuration to /home/ai24/condaprojects/droid/d0/train_dir/default_experiment/config.json... |
| [2024-08-26 11:33:17,764][15329] Rollout worker 0 uses device cpu |
| [2024-08-26 11:33:17,764][15329] Rollout worker 1 uses device cpu |
| [2024-08-26 11:33:17,764][15329] Rollout worker 2 uses device cpu |
| [2024-08-26 11:33:17,764][15329] Rollout worker 3 uses device cpu |
| [2024-08-26 11:33:17,764][15329] Rollout worker 4 uses device cpu |
| [2024-08-26 11:33:17,764][15329] Rollout worker 5 uses device cpu |
| [2024-08-26 11:33:17,764][15329] Rollout worker 6 uses device cpu |
| [2024-08-26 11:33:17,764][15329] Rollout worker 7 uses device cpu |
| [2024-08-26 11:33:17,798][15329] Using GPUs [0] for process 0 (actually maps to GPUs [0]) |
| [2024-08-26 11:33:17,799][15329] InferenceWorker_p0-w0: min num requests: 2 |
| [2024-08-26 11:33:17,810][15329] Starting all processes... |
| [2024-08-26 11:33:17,810][15329] Starting process learner_proc0 |
| [2024-08-26 11:33:18,683][15329] Starting all processes... |
| [2024-08-26 11:33:18,685][15453] Using GPUs [0] for process 0 (actually maps to GPUs [0]) |
| [2024-08-26 11:33:18,685][15453] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0 |
| [2024-08-26 11:33:18,693][15329] Starting process inference_proc0-0 |
| [2024-08-26 11:33:18,693][15329] Starting process rollout_proc0 |
| [2024-08-26 11:33:18,693][15329] Starting process rollout_proc1 |
| [2024-08-26 11:33:18,693][15329] Starting process rollout_proc2 |
| [2024-08-26 11:33:18,693][15329] Starting process rollout_proc3 |
| [2024-08-26 11:33:18,695][15329] Starting process rollout_proc4 |
| [2024-08-26 11:33:18,695][15329] Starting process rollout_proc5 |
| [2024-08-26 11:33:18,698][15329] Starting process rollout_proc6 |
| [2024-08-26 11:33:18,698][15329] Starting process rollout_proc7 |
| [2024-08-26 11:33:18,739][15453] Num visible devices: 1 |
| [2024-08-26 11:33:18,848][15453] Starting seed is not provided |
| [2024-08-26 11:33:18,848][15453] Using GPUs [0] for process 0 (actually maps to GPUs [0]) |
| [2024-08-26 11:33:18,848][15453] Initializing actor-critic model on device cuda:0 |
| [2024-08-26 11:33:18,848][15453] RunningMeanStd input shape: (3, 72, 128) |
| [2024-08-26 11:33:18,850][15453] RunningMeanStd input shape: (1,) |
| [2024-08-26 11:33:18,860][15453] ConvEncoder: input_channels=3 |
| [2024-08-26 11:33:18,999][15453] Conv encoder output size: 512 |
| [2024-08-26 11:33:18,999][15453] Policy head output size: 512 |
| [2024-08-26 11:33:19,016][15453] Created Actor Critic model with architecture: |
| [2024-08-26 11:33:19,016][15453] ActorCriticSharedWeights( |
| (obs_normalizer): ObservationNormalizer( |
| (running_mean_std): RunningMeanStdDictInPlace( |
| (running_mean_std): ModuleDict( |
| (obs): RunningMeanStdInPlace() |
| ) |
| ) |
| ) |
| (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) |
| (encoder): VizdoomEncoder( |
| (basic_encoder): ConvEncoder( |
| (enc): RecursiveScriptModule( |
| original_name=ConvEncoderImpl |
| (conv_head): RecursiveScriptModule( |
| original_name=Sequential |
| (0): RecursiveScriptModule(original_name=Conv2d) |
| (1): RecursiveScriptModule(original_name=ELU) |
| (2): RecursiveScriptModule(original_name=Conv2d) |
| (3): RecursiveScriptModule(original_name=ELU) |
| (4): RecursiveScriptModule(original_name=Conv2d) |
| (5): RecursiveScriptModule(original_name=ELU) |
| ) |
| (mlp_layers): RecursiveScriptModule( |
| original_name=Sequential |
| (0): RecursiveScriptModule(original_name=Linear) |
| (1): RecursiveScriptModule(original_name=ELU) |
| ) |
| ) |
| ) |
| ) |
| (core): ModelCoreRNN( |
| (core): GRU(512, 512) |
| ) |
| (decoder): MlpDecoder( |
| (mlp): Identity() |
| ) |
| (critic_linear): Linear(in_features=512, out_features=1, bias=True) |
| (action_parameterization): ActionParameterizationDefault( |
| (distribution_linear): Linear(in_features=512, out_features=5, bias=True) |
| ) |
| ) |
| [2024-08-26 11:33:19,217][15453] Using optimizer <class 'torch.optim.adam.Adam'> |
| [2024-08-26 11:33:19,609][15507] Worker 6 uses CPU cores [24, 25, 26, 27] |
| [2024-08-26 11:33:19,648][15501] Worker 1 uses CPU cores [4, 5, 6, 7] |
| [2024-08-26 11:33:19,658][15500] Using GPUs [0] for process 0 (actually maps to GPUs [0]) |
| [2024-08-26 11:33:19,658][15500] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0 |
| [2024-08-26 11:33:19,659][15502] Worker 0 uses CPU cores [0, 1, 2, 3] |
| [2024-08-26 11:33:19,664][15503] Worker 2 uses CPU cores [8, 9, 10, 11] |
| [2024-08-26 11:33:19,671][15506] Worker 3 uses CPU cores [12, 13, 14, 15] |
| [2024-08-26 11:33:19,674][15500] Num visible devices: 1 |
| [2024-08-26 11:33:19,684][15505] Worker 4 uses CPU cores [16, 17, 18, 19] |
| [2024-08-26 11:33:19,705][15453] No checkpoints found |
| [2024-08-26 11:33:19,706][15453] Did not load from checkpoint, starting from scratch! |
| [2024-08-26 11:33:19,706][15453] Initialized policy 0 weights for model version 0 |
| [2024-08-26 11:33:19,730][15508] Worker 7 uses CPU cores [28, 29, 30, 31] |
| [2024-08-26 11:33:19,745][15504] Worker 5 uses CPU cores [20, 21, 22, 23] |
| [2024-08-26 11:33:19,829][15453] LearnerWorker_p0 finished initialization! |
| [2024-08-26 11:33:19,829][15453] Using GPUs [0] for process 0 (actually maps to GPUs [0]) |
| [2024-08-26 11:33:19,989][15500] RunningMeanStd input shape: (3, 72, 128) |
| [2024-08-26 11:33:19,989][15500] RunningMeanStd input shape: (1,) |
| [2024-08-26 11:33:19,994][15500] ConvEncoder: input_channels=3 |
| [2024-08-26 11:33:20,033][15500] Conv encoder output size: 512 |
| [2024-08-26 11:33:20,033][15500] Policy head output size: 512 |
| [2024-08-26 11:33:20,058][15329] Inference worker 0-0 is ready! |
| [2024-08-26 11:33:20,058][15329] All inference workers are ready! Signal rollout workers to start! |
| [2024-08-26 11:33:20,073][15507] Doom resolution: 160x120, resize resolution: (128, 72) |
| [2024-08-26 11:33:20,073][15508] Doom resolution: 160x120, resize resolution: (128, 72) |
| [2024-08-26 11:33:20,073][15502] Doom resolution: 160x120, resize resolution: (128, 72) |
| [2024-08-26 11:33:20,074][15504] Doom resolution: 160x120, resize resolution: (128, 72) |
| [2024-08-26 11:33:20,074][15505] Doom resolution: 160x120, resize resolution: (128, 72) |
| [2024-08-26 11:33:20,075][15503] Doom resolution: 160x120, resize resolution: (128, 72) |
| [2024-08-26 11:33:20,076][15506] Doom resolution: 160x120, resize resolution: (128, 72) |
| [2024-08-26 11:33:20,078][15501] Doom resolution: 160x120, resize resolution: (128, 72) |
| [2024-08-26 11:33:20,268][15502] Decorrelating experience for 0 frames... |
| [2024-08-26 11:33:20,268][15504] Decorrelating experience for 0 frames... |
| [2024-08-26 11:33:20,371][15502] Decorrelating experience for 32 frames... |
| [2024-08-26 11:33:20,371][15504] Decorrelating experience for 32 frames... |
| [2024-08-26 11:33:20,501][15507] Decorrelating experience for 0 frames... |
| [2024-08-26 11:33:20,505][15502] Decorrelating experience for 64 frames... |
| [2024-08-26 11:33:20,506][15504] Decorrelating experience for 64 frames... |
| [2024-08-26 11:33:20,610][15507] Decorrelating experience for 32 frames... |
| [2024-08-26 11:33:20,631][15502] Decorrelating experience for 96 frames... |
| [2024-08-26 11:33:20,648][15506] Decorrelating experience for 0 frames... |
| [2024-08-26 11:33:20,755][15507] Decorrelating experience for 64 frames... |
| [2024-08-26 11:33:20,765][15506] Decorrelating experience for 32 frames... |
| [2024-08-26 11:33:20,799][15504] Decorrelating experience for 96 frames... |
| [2024-08-26 11:33:20,898][15507] Decorrelating experience for 96 frames... |
| [2024-08-26 11:33:20,903][15506] Decorrelating experience for 64 frames... |
| [2024-08-26 11:33:21,035][15506] Decorrelating experience for 96 frames... |
| [2024-08-26 11:33:21,231][15329] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 0. Throughput: 0: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) |
| [2024-08-26 11:33:21,231][15329] Avg episode reward: [(0, '1.600')] |
| [2024-08-26 11:33:21,521][15453] Signal inference workers to stop experience collection... |
| [2024-08-26 11:33:21,524][15500] InferenceWorker_p0-w0: stopping experience collection |
| [2024-08-26 11:33:22,065][15453] Signal inference workers to resume experience collection... |
| [2024-08-26 11:33:22,065][15500] InferenceWorker_p0-w0: resuming experience collection |
| [2024-08-26 11:33:23,837][15500] Updated weights for policy 0, policy_version 10 (0.0080) |
| [2024-08-26 11:33:25,952][15500] Updated weights for policy 0, policy_version 20 (0.0006) |
| [2024-08-26 11:33:26,231][15329] Fps is (10 sec: 17203.2, 60 sec: 17203.2, 300 sec: 17203.2). Total num frames: 86016. Throughput: 0: 4256.4. Samples: 21282. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) |
| [2024-08-26 11:33:26,231][15329] Avg episode reward: [(0, '4.481')] |
| [2024-08-26 11:33:28,097][15500] Updated weights for policy 0, policy_version 30 (0.0007) |
| [2024-08-26 11:33:30,188][15500] Updated weights for policy 0, policy_version 40 (0.0006) |
| [2024-08-26 11:33:31,231][15329] Fps is (10 sec: 18022.4, 60 sec: 18022.4, 300 sec: 18022.4). Total num frames: 180224. Throughput: 0: 3564.0. Samples: 35640. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) |
| [2024-08-26 11:33:31,231][15329] Avg episode reward: [(0, '4.371')] |
| [2024-08-26 11:33:31,253][15453] Saving new best policy, reward=4.371! |
| [2024-08-26 11:33:32,242][15500] Updated weights for policy 0, policy_version 50 (0.0005) |
| [2024-08-26 11:33:34,342][15500] Updated weights for policy 0, policy_version 60 (0.0005) |
| [2024-08-26 11:33:36,231][15329] Fps is (10 sec: 19251.2, 60 sec: 18568.6, 300 sec: 18568.6). Total num frames: 278528. Throughput: 0: 4354.8. Samples: 65322. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) |
| [2024-08-26 11:33:36,231][15329] Avg episode reward: [(0, '4.412')] |
| [2024-08-26 11:33:36,240][15453] Saving new best policy, reward=4.412! |
| [2024-08-26 11:33:36,538][15500] Updated weights for policy 0, policy_version 70 (0.0007) |
| [2024-08-26 11:33:37,794][15329] Heartbeat connected on Batcher_0 |
| [2024-08-26 11:33:37,796][15329] Heartbeat connected on LearnerWorker_p0 |
| [2024-08-26 11:33:37,800][15329] Heartbeat connected on RolloutWorker_w0 |
| [2024-08-26 11:33:37,800][15329] Heartbeat connected on InferenceWorker_p0-w0 |
| [2024-08-26 11:33:37,805][15329] Heartbeat connected on RolloutWorker_w3 |
| [2024-08-26 11:33:37,807][15329] Heartbeat connected on RolloutWorker_w5 |
| [2024-08-26 11:33:37,809][15329] Heartbeat connected on RolloutWorker_w6 |
| [2024-08-26 11:33:38,555][15500] Updated weights for policy 0, policy_version 80 (0.0006) |
| [2024-08-26 11:33:40,603][15500] Updated weights for policy 0, policy_version 90 (0.0005) |
| [2024-08-26 11:33:41,231][15329] Fps is (10 sec: 19660.8, 60 sec: 18841.6, 300 sec: 18841.6). Total num frames: 376832. Throughput: 0: 4734.1. Samples: 94682. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) |
| [2024-08-26 11:33:41,231][15329] Avg episode reward: [(0, '4.394')] |
| [2024-08-26 11:33:42,719][15500] Updated weights for policy 0, policy_version 100 (0.0005) |
| [2024-08-26 11:33:44,779][15500] Updated weights for policy 0, policy_version 110 (0.0005) |
| [2024-08-26 11:33:46,231][15329] Fps is (10 sec: 19660.8, 60 sec: 19005.5, 300 sec: 19005.5). Total num frames: 475136. Throughput: 0: 4372.3. Samples: 109308. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) |
| [2024-08-26 11:33:46,231][15329] Avg episode reward: [(0, '4.216')] |
| [2024-08-26 11:33:46,942][15500] Updated weights for policy 0, policy_version 120 (0.0005) |
| [2024-08-26 11:33:49,043][15500] Updated weights for policy 0, policy_version 130 (0.0005) |
| [2024-08-26 11:33:51,200][15500] Updated weights for policy 0, policy_version 140 (0.0006) |
| [2024-08-26 11:33:51,231][15329] Fps is (10 sec: 19660.6, 60 sec: 19114.6, 300 sec: 19114.6). Total num frames: 573440. Throughput: 0: 4612.5. Samples: 138374. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) |
| [2024-08-26 11:33:51,231][15329] Avg episode reward: [(0, '4.462')] |
| [2024-08-26 11:33:51,238][15453] Saving new best policy, reward=4.462! |
| [2024-08-26 11:33:53,316][15500] Updated weights for policy 0, policy_version 150 (0.0006) |
| [2024-08-26 11:33:55,512][15500] Updated weights for policy 0, policy_version 160 (0.0006) |
| [2024-08-26 11:33:56,231][15329] Fps is (10 sec: 19251.2, 60 sec: 19075.7, 300 sec: 19075.7). Total num frames: 667648. Throughput: 0: 4770.6. Samples: 166970. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) |
| [2024-08-26 11:33:56,231][15329] Avg episode reward: [(0, '4.716')] |
| [2024-08-26 11:33:56,240][15453] Saving new best policy, reward=4.716! |
| [2024-08-26 11:33:57,675][15500] Updated weights for policy 0, policy_version 170 (0.0005) |
| [2024-08-26 11:33:59,746][15500] Updated weights for policy 0, policy_version 180 (0.0005) |
| [2024-08-26 11:34:01,231][15329] Fps is (10 sec: 18841.8, 60 sec: 19046.4, 300 sec: 19046.4). Total num frames: 761856. Throughput: 0: 4533.1. Samples: 181322. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) |
| [2024-08-26 11:34:01,231][15329] Avg episode reward: [(0, '4.403')] |
| [2024-08-26 11:34:01,917][15500] Updated weights for policy 0, policy_version 190 (0.0006) |
| [2024-08-26 11:34:04,063][15500] Updated weights for policy 0, policy_version 200 (0.0006) |
| [2024-08-26 11:34:06,215][15500] Updated weights for policy 0, policy_version 210 (0.0005) |
| [2024-08-26 11:34:06,231][15329] Fps is (10 sec: 19251.2, 60 sec: 19114.7, 300 sec: 19114.7). Total num frames: 860160. Throughput: 0: 4671.0. Samples: 210194. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) |
| [2024-08-26 11:34:06,231][15329] Avg episode reward: [(0, '4.441')] |
| [2024-08-26 11:34:08,388][15500] Updated weights for policy 0, policy_version 220 (0.0006) |
| [2024-08-26 11:34:10,511][15500] Updated weights for policy 0, policy_version 230 (0.0006) |
| [2024-08-26 11:34:11,231][15329] Fps is (10 sec: 19250.9, 60 sec: 19087.3, 300 sec: 19087.3). Total num frames: 954368. Throughput: 0: 4829.7. Samples: 238620. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) |
| [2024-08-26 11:34:11,231][15329] Avg episode reward: [(0, '4.489')] |
| [2024-08-26 11:34:12,593][15500] Updated weights for policy 0, policy_version 240 (0.0006) |
| [2024-08-26 11:34:14,724][15500] Updated weights for policy 0, policy_version 250 (0.0006) |
| [2024-08-26 11:34:16,231][15329] Fps is (10 sec: 19251.2, 60 sec: 19139.5, 300 sec: 19139.5). Total num frames: 1052672. Throughput: 0: 4835.6. Samples: 253244. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) |
| [2024-08-26 11:34:16,231][15329] Avg episode reward: [(0, '4.133')] |
| [2024-08-26 11:34:16,855][15500] Updated weights for policy 0, policy_version 260 (0.0005) |
| [2024-08-26 11:34:18,969][15500] Updated weights for policy 0, policy_version 270 (0.0006) |
| [2024-08-26 11:34:21,066][15500] Updated weights for policy 0, policy_version 280 (0.0006) |
| [2024-08-26 11:34:21,231][15329] Fps is (10 sec: 19251.5, 60 sec: 19114.7, 300 sec: 19114.7). Total num frames: 1146880. Throughput: 0: 4817.9. Samples: 282126. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) |
| [2024-08-26 11:34:21,231][15329] Avg episode reward: [(0, '4.360')] |
| [2024-08-26 11:34:23,128][15500] Updated weights for policy 0, policy_version 290 (0.0006) |
| [2024-08-26 11:34:25,248][15500] Updated weights for policy 0, policy_version 300 (0.0006) |
| [2024-08-26 11:34:26,231][15329] Fps is (10 sec: 19251.0, 60 sec: 19319.4, 300 sec: 19156.7). Total num frames: 1245184. Throughput: 0: 4818.5. Samples: 311516. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) |
| [2024-08-26 11:34:26,231][15329] Avg episode reward: [(0, '4.594')] |
| [2024-08-26 11:34:27,381][15500] Updated weights for policy 0, policy_version 310 (0.0006) |
| [2024-08-26 11:34:29,535][15500] Updated weights for policy 0, policy_version 320 (0.0005) |
| [2024-08-26 11:34:31,231][15329] Fps is (10 sec: 19251.2, 60 sec: 19319.5, 300 sec: 19134.2). Total num frames: 1339392. Throughput: 0: 4813.2. Samples: 325902. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) |
| [2024-08-26 11:34:31,231][15329] Avg episode reward: [(0, '4.642')] |
| [2024-08-26 11:34:31,724][15500] Updated weights for policy 0, policy_version 330 (0.0006) |
| [2024-08-26 11:34:33,751][15500] Updated weights for policy 0, policy_version 340 (0.0006) |
| [2024-08-26 11:34:35,759][15500] Updated weights for policy 0, policy_version 350 (0.0006) |
| [2024-08-26 11:34:36,231][15329] Fps is (10 sec: 19661.0, 60 sec: 19387.7, 300 sec: 19223.9). Total num frames: 1441792. Throughput: 0: 4820.8. Samples: 355310. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) |
| [2024-08-26 11:34:36,231][15329] Avg episode reward: [(0, '4.352')] |
| [2024-08-26 11:34:37,844][15500] Updated weights for policy 0, policy_version 360 (0.0005) |
| [2024-08-26 11:34:39,956][15500] Updated weights for policy 0, policy_version 370 (0.0006) |
| [2024-08-26 11:34:41,231][15329] Fps is (10 sec: 20070.4, 60 sec: 19387.7, 300 sec: 19251.2). Total num frames: 1540096. Throughput: 0: 4839.6. Samples: 384752. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) |
| [2024-08-26 11:34:41,231][15329] Avg episode reward: [(0, '4.384')] |
| [2024-08-26 11:34:42,041][15500] Updated weights for policy 0, policy_version 380 (0.0005) |
| [2024-08-26 11:34:44,370][15500] Updated weights for policy 0, policy_version 390 (0.0005) |
| [2024-08-26 11:34:46,231][15329] Fps is (10 sec: 19250.8, 60 sec: 19319.4, 300 sec: 19227.1). Total num frames: 1634304. Throughput: 0: 4840.1. Samples: 399128. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) |
| [2024-08-26 11:34:46,231][15329] Avg episode reward: [(0, '4.588')] |
| [2024-08-26 11:34:46,339][15500] Updated weights for policy 0, policy_version 400 (0.0006) |
| [2024-08-26 11:34:48,430][15500] Updated weights for policy 0, policy_version 410 (0.0005) |
| [2024-08-26 11:34:50,504][15500] Updated weights for policy 0, policy_version 420 (0.0005) |
| [2024-08-26 11:34:51,231][15329] Fps is (10 sec: 19251.2, 60 sec: 19319.5, 300 sec: 19251.2). Total num frames: 1732608. Throughput: 0: 4847.9. Samples: 428350. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) |
| [2024-08-26 11:34:51,231][15329] Avg episode reward: [(0, '4.458')] |
| [2024-08-26 11:34:52,632][15500] Updated weights for policy 0, policy_version 430 (0.0006) |
| [2024-08-26 11:34:54,827][15500] Updated weights for policy 0, policy_version 440 (0.0006) |
| [2024-08-26 11:34:56,231][15329] Fps is (10 sec: 19251.6, 60 sec: 19319.5, 300 sec: 19229.6). Total num frames: 1826816. Throughput: 0: 4845.9. Samples: 456686. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) |
| [2024-08-26 11:34:56,231][15329] Avg episode reward: [(0, '4.352')] |
| [2024-08-26 11:34:57,014][15500] Updated weights for policy 0, policy_version 450 (0.0005) |
| [2024-08-26 11:34:59,112][15500] Updated weights for policy 0, policy_version 460 (0.0005) |
| [2024-08-26 11:35:01,231][15329] Fps is (10 sec: 18841.3, 60 sec: 19319.4, 300 sec: 19210.2). Total num frames: 1921024. Throughput: 0: 4844.0. Samples: 471226. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) |
| [2024-08-26 11:35:01,231][15329] Avg episode reward: [(0, '4.442')] |
| [2024-08-26 11:35:01,271][15500] Updated weights for policy 0, policy_version 470 (0.0007) |
| [2024-08-26 11:35:03,357][15500] Updated weights for policy 0, policy_version 480 (0.0005) |
| [2024-08-26 11:35:05,576][15500] Updated weights for policy 0, policy_version 490 (0.0006) |
| [2024-08-26 11:35:06,231][15329] Fps is (10 sec: 19251.3, 60 sec: 19319.5, 300 sec: 19231.7). Total num frames: 2019328. Throughput: 0: 4833.7. Samples: 499644. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) |
| [2024-08-26 11:35:06,231][15329] Avg episode reward: [(0, '4.392')] |
| [2024-08-26 11:35:07,729][15500] Updated weights for policy 0, policy_version 500 (0.0006) |
| [2024-08-26 11:35:09,850][15500] Updated weights for policy 0, policy_version 510 (0.0005) |
| [2024-08-26 11:35:11,231][15329] Fps is (10 sec: 19251.5, 60 sec: 19319.5, 300 sec: 19214.0). Total num frames: 2113536. Throughput: 0: 4816.2. Samples: 528246. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) |
| [2024-08-26 11:35:11,231][15329] Avg episode reward: [(0, '4.462')] |
| [2024-08-26 11:35:12,017][15500] Updated weights for policy 0, policy_version 520 (0.0005) |
| [2024-08-26 11:35:14,196][15500] Updated weights for policy 0, policy_version 530 (0.0006) |
| [2024-08-26 11:35:16,231][15329] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19197.8). Total num frames: 2207744. Throughput: 0: 4813.6. Samples: 542514. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) |
| [2024-08-26 11:35:16,231][15329] Avg episode reward: [(0, '4.444')] |
| [2024-08-26 11:35:16,239][15453] Saving /home/ai24/condaprojects/droid/d0/train_dir/default_experiment/checkpoint_p0/checkpoint_000000539_2207744.pth... |
| [2024-08-26 11:35:16,343][15500] Updated weights for policy 0, policy_version 540 (0.0005) |
| [2024-08-26 11:35:18,495][15500] Updated weights for policy 0, policy_version 550 (0.0006) |
| [2024-08-26 11:35:20,671][15500] Updated weights for policy 0, policy_version 560 (0.0005) |
| [2024-08-26 11:35:21,231][15329] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19182.9). Total num frames: 2301952. Throughput: 0: 4790.4. Samples: 570878. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) |
| [2024-08-26 11:35:21,231][15329] Avg episode reward: [(0, '4.518')] |
| [2024-08-26 11:35:22,903][15500] Updated weights for policy 0, policy_version 570 (0.0007) |
| [2024-08-26 11:35:24,951][15500] Updated weights for policy 0, policy_version 580 (0.0006) |
| [2024-08-26 11:35:26,231][15329] Fps is (10 sec: 19251.2, 60 sec: 19251.2, 300 sec: 19202.0). Total num frames: 2400256. Throughput: 0: 4776.0. Samples: 599674. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) |
| [2024-08-26 11:35:26,238][15329] Avg episode reward: [(0, '4.595')] |
| [2024-08-26 11:35:27,065][15500] Updated weights for policy 0, policy_version 590 (0.0006) |
| [2024-08-26 11:35:29,240][15500] Updated weights for policy 0, policy_version 600 (0.0006) |
| [2024-08-26 11:35:31,231][15329] Fps is (10 sec: 19251.2, 60 sec: 19251.2, 300 sec: 19188.2). Total num frames: 2494464. Throughput: 0: 4772.2. Samples: 613874. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) |
| [2024-08-26 11:35:31,231][15329] Avg episode reward: [(0, '4.264')] |
| [2024-08-26 11:35:31,456][15500] Updated weights for policy 0, policy_version 610 (0.0006) |
| [2024-08-26 11:35:33,727][15500] Updated weights for policy 0, policy_version 620 (0.0006) |
| [2024-08-26 11:35:35,874][15500] Updated weights for policy 0, policy_version 630 (0.0005) |
| [2024-08-26 11:35:36,231][15329] Fps is (10 sec: 18432.1, 60 sec: 19046.4, 300 sec: 19145.0). Total num frames: 2584576. Throughput: 0: 4739.8. Samples: 641640. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) |
| [2024-08-26 11:35:36,231][15329] Avg episode reward: [(0, '4.461')] |
| [2024-08-26 11:35:38,152][15500] Updated weights for policy 0, policy_version 640 (0.0006) |
| [2024-08-26 11:35:40,451][15500] Updated weights for policy 0, policy_version 650 (0.0006) |
| [2024-08-26 11:35:41,231][15329] Fps is (10 sec: 18022.4, 60 sec: 18909.9, 300 sec: 19104.9). Total num frames: 2674688. Throughput: 0: 4707.6. Samples: 668528. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) |
| [2024-08-26 11:35:41,231][15329] Avg episode reward: [(0, '4.467')] |
| [2024-08-26 11:35:42,687][15500] Updated weights for policy 0, policy_version 660 (0.0006) |
| [2024-08-26 11:35:44,871][15500] Updated weights for policy 0, policy_version 670 (0.0005) |
| [2024-08-26 11:35:46,231][15329] Fps is (10 sec: 18431.9, 60 sec: 18909.9, 300 sec: 19095.8). Total num frames: 2768896. Throughput: 0: 4694.3. Samples: 682468. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) |
| [2024-08-26 11:35:46,231][15329] Avg episode reward: [(0, '4.642')] |
| [2024-08-26 11:35:46,972][15500] Updated weights for policy 0, policy_version 680 (0.0005) |
| [2024-08-26 11:35:49,135][15500] Updated weights for policy 0, policy_version 690 (0.0006) |
| [2024-08-26 11:35:51,231][15329] Fps is (10 sec: 18841.6, 60 sec: 18841.6, 300 sec: 19087.4). Total num frames: 2863104. Throughput: 0: 4700.4. Samples: 711160. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) |
| [2024-08-26 11:35:51,231][15329] Avg episode reward: [(0, '4.307')] |
| [2024-08-26 11:35:51,291][15500] Updated weights for policy 0, policy_version 700 (0.0005) |
| [2024-08-26 11:35:53,356][15500] Updated weights for policy 0, policy_version 710 (0.0006) |
| [2024-08-26 11:35:55,472][15500] Updated weights for policy 0, policy_version 720 (0.0006) |
| [2024-08-26 11:35:56,231][15329] Fps is (10 sec: 19251.3, 60 sec: 18909.9, 300 sec: 19105.9). Total num frames: 2961408. Throughput: 0: 4712.4. Samples: 740304. Policy #0 lag: (min: 0.0, avg: 0.1, max: 1.0) |
| [2024-08-26 11:35:56,231][15329] Avg episode reward: [(0, '4.334')] |
| [2024-08-26 11:35:57,719][15500] Updated weights for policy 0, policy_version 730 (0.0005) |
| [2024-08-26 11:35:59,712][15500] Updated weights for policy 0, policy_version 740 (0.0006) |
| [2024-08-26 11:36:01,231][15329] Fps is (10 sec: 19660.8, 60 sec: 18978.2, 300 sec: 19123.2). Total num frames: 3059712. Throughput: 0: 4719.1. Samples: 754874. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) |
| [2024-08-26 11:36:01,231][15329] Avg episode reward: [(0, '4.345')] |
| [2024-08-26 11:36:01,829][15500] Updated weights for policy 0, policy_version 750 (0.0006) |
| [2024-08-26 11:36:03,879][15500] Updated weights for policy 0, policy_version 760 (0.0005) |
| [2024-08-26 11:36:05,989][15500] Updated weights for policy 0, policy_version 770 (0.0006) |
| [2024-08-26 11:36:06,231][15329] Fps is (10 sec: 19660.8, 60 sec: 18978.1, 300 sec: 19139.5). Total num frames: 3158016. Throughput: 0: 4746.6. Samples: 784474. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) |
| [2024-08-26 11:36:06,231][15329] Avg episode reward: [(0, '4.396')] |
| [2024-08-26 11:36:08,282][15500] Updated weights for policy 0, policy_version 780 (0.0006) |
| [2024-08-26 11:36:10,266][15500] Updated weights for policy 0, policy_version 790 (0.0006) |
| [2024-08-26 11:36:11,231][15329] Fps is (10 sec: 19251.2, 60 sec: 18978.1, 300 sec: 19130.7). Total num frames: 3252224. Throughput: 0: 4738.2. Samples: 812892. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) |
| [2024-08-26 11:36:11,231][15329] Avg episode reward: [(0, '4.277')] |
| [2024-08-26 11:36:12,408][15500] Updated weights for policy 0, policy_version 800 (0.0006) |
| [2024-08-26 11:36:14,518][15500] Updated weights for policy 0, policy_version 810 (0.0006) |
| [2024-08-26 11:36:16,232][15329] Fps is (10 sec: 18840.5, 60 sec: 18977.9, 300 sec: 19122.4). Total num frames: 3346432. Throughput: 0: 4748.4. Samples: 827554. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) |
| [2024-08-26 11:36:16,239][15329] Avg episode reward: [(0, '4.404')] |
| [2024-08-26 11:36:16,716][15500] Updated weights for policy 0, policy_version 820 (0.0006) |
| [2024-08-26 11:36:18,847][15500] Updated weights for policy 0, policy_version 830 (0.0005) |
| [2024-08-26 11:36:20,939][15500] Updated weights for policy 0, policy_version 840 (0.0006) |
| [2024-08-26 11:36:21,231][15329] Fps is (10 sec: 19250.8, 60 sec: 19046.3, 300 sec: 19137.4). Total num frames: 3444736. Throughput: 0: 4768.7. Samples: 856234. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) |
| [2024-08-26 11:36:21,231][15329] Avg episode reward: [(0, '4.099')] |
| [2024-08-26 11:36:23,051][15500] Updated weights for policy 0, policy_version 850 (0.0006) |
| [2024-08-26 11:36:25,103][15500] Updated weights for policy 0, policy_version 860 (0.0005) |
| [2024-08-26 11:36:26,231][15329] Fps is (10 sec: 19662.0, 60 sec: 19046.4, 300 sec: 19151.6). Total num frames: 3543040. Throughput: 0: 4824.7. Samples: 885638. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) |
| [2024-08-26 11:36:26,231][15329] Avg episode reward: [(0, '4.497')] |
| [2024-08-26 11:36:27,213][15500] Updated weights for policy 0, policy_version 870 (0.0006) |
| [2024-08-26 11:36:29,346][15500] Updated weights for policy 0, policy_version 880 (0.0005) |
| [2024-08-26 11:36:31,231][15329] Fps is (10 sec: 19251.6, 60 sec: 19046.4, 300 sec: 19143.4). Total num frames: 3637248. Throughput: 0: 4836.7. Samples: 900120. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) |
| [2024-08-26 11:36:31,231][15329] Avg episode reward: [(0, '4.661')] |
| [2024-08-26 11:36:31,505][15500] Updated weights for policy 0, policy_version 890 (0.0005) |
| [2024-08-26 11:36:33,651][15500] Updated weights for policy 0, policy_version 900 (0.0006) |
| [2024-08-26 11:36:35,723][15500] Updated weights for policy 0, policy_version 910 (0.0006) |
| [2024-08-26 11:36:36,231][15329] Fps is (10 sec: 19251.2, 60 sec: 19182.9, 300 sec: 19156.7). Total num frames: 3735552. Throughput: 0: 4840.5. Samples: 928982. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) |
| [2024-08-26 11:36:36,231][15329] Avg episode reward: [(0, '4.232')] |
| [2024-08-26 11:36:37,848][15500] Updated weights for policy 0, policy_version 920 (0.0006) |
| [2024-08-26 11:36:40,013][15500] Updated weights for policy 0, policy_version 930 (0.0006) |
| [2024-08-26 11:36:41,231][15329] Fps is (10 sec: 19251.2, 60 sec: 19251.2, 300 sec: 19148.8). Total num frames: 3829760. Throughput: 0: 4831.3. Samples: 957712. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) |
| [2024-08-26 11:36:41,231][15329] Avg episode reward: [(0, '4.430')] |
| [2024-08-26 11:36:42,285][15500] Updated weights for policy 0, policy_version 940 (0.0006) |
| [2024-08-26 11:36:44,353][15500] Updated weights for policy 0, policy_version 950 (0.0006) |
| [2024-08-26 11:36:46,231][15329] Fps is (10 sec: 19251.0, 60 sec: 19319.4, 300 sec: 19161.3). Total num frames: 3928064. Throughput: 0: 4818.3. Samples: 971696. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) |
| [2024-08-26 11:36:46,231][15329] Avg episode reward: [(0, '4.300')] |
| [2024-08-26 11:36:46,389][15500] Updated weights for policy 0, policy_version 960 (0.0006) |
| [2024-08-26 11:36:48,528][15500] Updated weights for policy 0, policy_version 970 (0.0006) |
| [2024-08-26 11:36:50,226][15329] Component Batcher_0 stopped! |
| [2024-08-26 11:36:50,226][15453] Stopping Batcher_0... |
| [2024-08-26 11:36:50,226][15329] Component RolloutWorker_w1 process died already! Don't wait for it. |
| [2024-08-26 11:36:50,226][15329] Component RolloutWorker_w2 process died already! Don't wait for it. |
| [2024-08-26 11:36:50,226][15453] Loop batcher_evt_loop terminating... |
| [2024-08-26 11:36:50,226][15329] Component RolloutWorker_w4 process died already! Don't wait for it. |
| [2024-08-26 11:36:50,226][15329] Component RolloutWorker_w7 process died already! Don't wait for it. |
| [2024-08-26 11:36:50,226][15453] Saving /home/ai24/condaprojects/droid/d0/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... |
| [2024-08-26 11:36:50,234][15500] Weights refcount: 2 0 |
| [2024-08-26 11:36:50,235][15500] Stopping InferenceWorker_p0-w0... |
| [2024-08-26 11:36:50,235][15329] Component InferenceWorker_p0-w0 stopped! |
| [2024-08-26 11:36:50,236][15500] Loop inference_proc0-0_evt_loop terminating... |
| [2024-08-26 11:36:50,244][15507] Stopping RolloutWorker_w6... |
| [2024-08-26 11:36:50,245][15329] Component RolloutWorker_w6 stopped! |
| [2024-08-26 11:36:50,245][15502] Stopping RolloutWorker_w0... |
| [2024-08-26 11:36:50,245][15504] Stopping RolloutWorker_w5... |
| [2024-08-26 11:36:50,245][15507] Loop rollout_proc6_evt_loop terminating... |
| [2024-08-26 11:36:50,245][15329] Component RolloutWorker_w0 stopped! |
| [2024-08-26 11:36:50,245][15502] Loop rollout_proc0_evt_loop terminating... |
| [2024-08-26 11:36:50,245][15504] Loop rollout_proc5_evt_loop terminating... |
| [2024-08-26 11:36:50,245][15329] Component RolloutWorker_w5 stopped! |
| [2024-08-26 11:36:50,245][15329] Component RolloutWorker_w3 stopped! |
| [2024-08-26 11:36:50,245][15506] Stopping RolloutWorker_w3... |
| [2024-08-26 11:36:50,245][15506] Loop rollout_proc3_evt_loop terminating... |
| [2024-08-26 11:36:50,251][15453] Saving /home/ai24/condaprojects/droid/d0/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... |
| [2024-08-26 11:36:50,282][15453] Stopping LearnerWorker_p0... |
| [2024-08-26 11:36:50,282][15453] Loop learner_proc0_evt_loop terminating... |
| [2024-08-26 11:36:50,282][15329] Component LearnerWorker_p0 stopped! |
| [2024-08-26 11:36:50,282][15329] Waiting for process learner_proc0 to stop... |
| [2024-08-26 11:36:50,649][15329] Waiting for process inference_proc0-0 to join... |
| [2024-08-26 11:36:50,649][15329] Waiting for process rollout_proc0 to join... |
| [2024-08-26 11:36:50,649][15329] Waiting for process rollout_proc1 to join... |
| [2024-08-26 11:36:50,649][15329] Waiting for process rollout_proc2 to join... |
| [2024-08-26 11:36:50,649][15329] Waiting for process rollout_proc3 to join... |
| [2024-08-26 11:36:50,649][15329] Waiting for process rollout_proc4 to join... |
| [2024-08-26 11:36:50,649][15329] Waiting for process rollout_proc5 to join... |
| [2024-08-26 11:36:50,650][15329] Waiting for process rollout_proc6 to join... |
| [2024-08-26 11:36:50,650][15329] Waiting for process rollout_proc7 to join... |
| [2024-08-26 11:36:50,650][15329] Batcher 0 profile tree view: |
| batching: 9.4596, releasing_batches: 0.0135 |
| [2024-08-26 11:36:50,650][15329] InferenceWorker_p0-w0 profile tree view: |
| wait_policy: 0.0000 |
| wait_policy_total: 1.7129 |
| update_model: 2.0211 |
| weight_update: 0.0005 |
| one_step: 0.0019 |
| handle_policy_step: 199.5574 |
| deserialize: 3.7122, stack: 0.7094, obs_to_device_normalize: 42.1515, forward: 107.5218, send_messages: 10.5853 |
| prepare_outputs: 29.5555 |
| to_cpu: 23.1926 |
| [2024-08-26 11:36:50,650][15329] Learner 0 profile tree view: |
| misc: 0.0027, prepare_batch: 10.4142 |
| train: 21.0947 |
| epoch_init: 0.0036, minibatch_init: 0.0031, losses_postprocess: 0.3384, kl_divergence: 0.3259, after_optimizer: 8.8810 |
| calculate_losses: 8.0679 |
| losses_init: 0.0013, forward_head: 0.3760, bptt_initial: 5.1180, tail: 0.2861, advantages_returns: 0.0794, losses: 1.5331 |
| bptt: 0.5979 |
| bptt_forward_core: 0.5717 |
| update: 3.2889 |
| clip: 0.3501 |
| [2024-08-26 11:36:50,650][15329] RolloutWorker_w0 profile tree view: |
| wait_for_trajectories: 0.1006, enqueue_policy_requests: 5.5463, env_step: 97.7689, overhead: 5.3251, complete_rollouts: 0.2158 |
| save_policy_outputs: 6.3718 |
| split_output_tensors: 2.2561 |
| [2024-08-26 11:36:50,650][15329] Loop Runner_EvtLoop terminating... |
| [2024-08-26 11:36:50,650][15329] Runner profile tree view: |
| main_loop: 212.8401 |
| [2024-08-26 11:36:50,650][15329] Collected {0: 4005888}, FPS: 18821.1 |
|
|
