| == per-step latency |
| -- laya-browser-v10 |
| 3 questions / 65 options / 2804 tokens: agent.predict 57.4 ms | fast 55.5 ms (tokenize 3.6 + forward 51.0 + post 0.3) |
| 3 questions / 28 options / 1750 tokens: agent.predict 41.8 ms | fast 40.7 ms (tokenize 2.5 + forward 38.4 + post 0.2) |
| 3 questions / 31 options / 1910 tokens: agent.predict 46.2 ms | fast 45.2 ms (tokenize 3.2 + forward 41.2 + post 0.2) |
| 3 questions / 31 options / 1988 tokens: agent.predict 47.4 ms | fast 45.9 ms (tokenize 2.7 + forward 43.2 + post 0.1) |
| -- laya-browser-v10s |
| 3 questions / 65 options / 2750 tokens: agent.predict 23.9 ms | fast 20.7 ms (tokenize 2.6 + forward 17.9 + post 0.1) |
| 3 questions / 28 options / 1743 tokens: agent.predict 17.0 ms | fast 16.7 ms (tokenize 2.0 + forward 14.6 + post 0.1) |
| 3 questions / 31 options / 1910 tokens: agent.predict 19.1 ms | fast 18.6 ms (tokenize 2.1 + forward 16.7 + post 0.1) |
| 3 questions / 31 options / 1988 tokens: agent.predict 19.8 ms | fast 19.0 ms (tokenize 1.9 + forward 17.1 + post 0.1) |
| == suite laya-browser-v10 tau=0 x3 |
| == 28/48 passed (58%, 3 run(s) per task) | median wall 1.4s |
| per task: wiki-einstein=0/3 wiki-search=0/3 hn-new=3/3 hn-login-page=3/3 gh-issues=1/3 py-downloads=3/3 books-travel=3/3 books-open-book=3/3 internet-dropdown=0/3 internet-checkbox=3/3 books-page2=0/3 quotes-tag-love=3/3 hn-past=3/3 ddg-search=3/3 arxiv-search=0/3 flights=0/3 |
| server calls=679 escalated=0 (0%) |
| == suite laya-browser-v10 tau=0.7 x3 |
| == 20/48 passed (42%, 3 run(s) per task) | median wall 2.4s |
| per task: wiki-einstein=0/3 wiki-search=0/3 hn-new=0/3 hn-login-page=1/3 gh-issues=0/3 py-downloads=3/3 books-travel=3/3 books-open-book=3/3 internet-dropdown=0/3 internet-checkbox=3/3 books-page2=0/3 quotes-tag-love=3/3 hn-past=3/3 ddg-search=1/3 arxiv-search=0/3 flights=0/3 |
| server calls=531 escalated=416 (78%) |
| == suite laya-browser-v10s tau=0 x3 |
| == 24/48 passed (50%, 3 run(s) per task) | median wall 1.3s |
| per task: wiki-einstein=0/3 wiki-search=0/3 hn-new=3/3 hn-login-page=3/3 gh-issues=0/3 py-downloads=3/3 books-travel=3/3 books-open-book=3/3 internet-dropdown=0/3 internet-checkbox=3/3 books-page2=0/3 quotes-tag-love=3/3 hn-past=3/3 ddg-search=0/3 arxiv-search=0/3 flights=0/3 |
| server calls=731 escalated=0 (0%) |
| == suite laya-browser-v10s tau=0.7 x3 |
| == 25/48 passed (52%, 3 run(s) per task) | median wall 1.5s |
| per task: wiki-einstein=0/3 wiki-search=0/3 hn-new=3/3 hn-login-page=3/3 gh-issues=0/3 py-downloads=3/3 books-travel=3/3 books-open-book=3/3 internet-dropdown=0/3 internet-checkbox=3/3 books-page2=0/3 quotes-tag-love=3/3 hn-past=3/3 ddg-search=1/3 arxiv-search=0/3 flights=0/3 |
| server calls=487 escalated=412 (85%) |
| FINAL_DONE |
|
|
