AlphaZero Training Run: 62-Iteration Self-Play Metrics

Policy loss, value loss, game length, and MCTS agreement metrics across 62 iterations of AlphaZero-style reinforcement learning.
# iteration
loss_policy_train
loss_value_train
loss_policy_val
loss_value_val
loss_soft_policy_train
loss_soft_policy_val
loss_aux_value_train
loss_aux_value_val
loss_aux_value_0_train
loss_aux_value_0_val
loss_aux_value_1_train
loss_aux_value_1_val
loss_aux_value_2_train
loss_aux_value_2_val
loss_aux_value_3_train
loss_aux_value_3_val
gradient_steps
game_length_avg
game_length_stddev
game_length_min
game_length_max
game_wins
game_losses
game_draws
policy_entropy_avg
policy_max_prob_avg
policy_entropy_high_branch_avg
policy_max_prob_high_branch_avg
policy_agreement_avg
policy_agreement_high_branch_avg
policy_surprise_avg
value_z_avg
value_q_avg
value_z_stddev
value_q_stddev
value_correction_avg
value_correction_high_branch_avg
value_q_spread_avg
value_q_spread_high_branch_avg
value_error_early_avg
value_error_mid_avg
value_error_late_avg
value_network_stddev
lr
q_weight
mcts_sims
replay_samples
samples_iter
time_selfplay_secs
time_train_secs
+
1 1 3.864787 0.320826 3.18191 0.242259 3.646052 3.076387 0.170796 0.031286 0.061016 0.005071 0.041854 0.009963 0.716512 0.133743 0.0 0.0 35 354.264 67.721033 134 418 206 257 37 1.069399 0.552025 1.627311 0.355778 0.306516 0.152918 0.900903 0.117202 -0.043852 0.755335 0.117126 0.256406 0.220012 0.110637 0.128793 0.703323 0.707441 0.717725 0.043437 0.0005 0.028333 100 178444 178444 286.273461 77.393945
2 2 3.054179 0.214878 2.736785 0.18571 2.956313 2.728005 0.025733 0.01787 0.036853 0.028107 0.038285 0.029353 0.048863 0.028961 0.0 0.0 66 363.372 59.736836 132 420 235 235 30 1.107233 0.555301 1.653229 0.386097 0.212524 0.147646 1.226714 0.125226 0.066841 0.75366 0.302264 0.161567 0.139284 0.075105 0.073412 0.623991 0.541579 0.498636 0.311493 0.0005 0.056667 197 334478 156034 311.520347 122.90891
3 3 2.473027 0.319545 2.383122 0.304262 2.611523 2.547672 0.030155 0.017883 0.043951 0.021795 0.04879 0.029612 0.051183 0.033476 0.0 0.0 42 306.762 80.571219 125 456 246 242 12 0.914362 0.649239 1.223786 0.563321 0.343803 0.29213 1.169231 0.107064 0.078501 0.913341 0.397952 0.118311 0.109984 0.055504 0.064856 0.795905 0.72426 0.626296 0.413562 0.0005 0.085 293 214466 214466 565.548518 90.27873
4 4 2.306242 0.283898 2.178379 0.285746 2.373558 2.262347 0.023122 0.027903 0.033743 0.03843 0.035401 0.045739 0.040325 0.047388 0.0 0.0 92 290.198 101.386364 99 488 223 266 11 0.636061 0.749145 0.864953 0.690746 0.414222 0.362223 1.309917 0.052395 0.030916 0.88627 0.428252 0.157282 0.157085 0.05787 0.074392 0.807009 0.671657 0.560194 0.441084 0.0005 0.113333 390 468004 253538 776.247988 171.735797
5 5 2.120492 0.266878 2.084083 0.281335 2.201041 2.173777 0.022068 0.019783 0.029671 0.025418 0.034216 0.0314 0.040508 0.03751 0.0 0.0 125 245.736 82.902221 91 448 297 197 6 0.727392 0.718925 1.036619 0.622322 0.347107 0.255754 1.275226 0.092726 0.062839 0.952357 0.48553 0.178129 0.207145 0.073739 0.089232 0.855319 0.713809 0.555246 0.479771 0.0005 0.141667 487 635428 167424 624.092566 233.373341
6 6 2.056717 0.254081 2.004317 0.244936 2.128085 2.089925 0.025475 0.023007 0.035926 0.038931 0.040308 0.031487 0.044802 0.038463 0.0 0.0 159 224.934 85.427499 84 460 227 271 2 0.507362 0.801921 0.702605 0.743406 0.388656 0.307349 1.443071 0.064813 0.037782 0.960654 0.475798 0.198953 0.211323 0.075591 0.088419 0.876541 0.746356 0.578257 0.496498 0.0005 0.17 583 810439 175011 721.00008 297.277773
7 7 1.980565 0.240194 1.962871 0.247953 2.056309 2.038054 0.024335 0.023682 0.035475 0.028279 0.036753 0.039402 0.043138 0.045329 0.0 0.0 184 199.75 63.037604 71 438 253 243 4 0.709711 0.730818 1.050833 0.624401 0.331796 0.203324 1.272419 0.052391 0.052032 0.982562 0.453649 0.125269 0.11675 0.056044 0.065202 0.901564 0.776227 0.608777 0.460589 0.0005 0.198333 680 939431 128992 609.804331 345.188357
8 8 1.934722 0.224824 1.926557 0.237877 2.009667 1.996964 0.023222 0.030667 0.031103 0.045675 0.036661 0.050567 0.042819 0.049246 0.0 0.0 209 205.662 65.041493 92 434 297 201 2 0.669525 0.741777 1.023188 0.622877 0.319223 0.180888 1.501361 0.056121 0.060203 0.982131 0.472652 0.203874 0.193068 0.091324 0.104065 0.890773 0.743353 0.584211 0.457012 0.0005 0.226667 777 1065506 126075 655.579999 390.819171
9 9 1.905073 0.210875 1.928705 0.205652 1.952473 1.973959 0.022604 0.02032 0.028725 0.02477 0.03595 0.031809 0.042882 0.040527 0.0 0.0 198 191.228 71.703305 79 462 234 265 1 0.450463 0.823819 0.639126 0.766002 0.38914 0.288425 1.395133 0.044485 0.032008 0.976307 0.470015 0.16768 0.161381 0.074023 0.086891 0.903919 0.743153 0.570822 0.480495 0.0005 0.255 873 1011396 160356 953.552464 371.128067
10 10 1.885457 0.202162 1.930772 0.194228 1.936023 1.970428 0.023354 0.021064 0.028933 0.025361 0.037461 0.033421 0.044752 0.041666 0.0 0.0 174 183.046 61.119096 70 460 255 240 5 0.657052 0.752794 1.023239 0.635632 0.362461 0.230714 1.293476 0.062379 0.061959 0.981138 0.480829 0.122319 0.116329 0.05668 0.06547 0.893507 0.749269 0.558663 0.493849 0.0005 0.283333 970 888992 131134 815.300974 326.27391
11 11 1.845871 0.191741 1.825237 0.184761 1.896899 1.87675 0.02634 0.023318 0.03383 0.02774 0.042235 0.03806 0.049186 0.044817 0.0 0.0 172 190.236 63.771156 70 456 274 226 0 0.661495 0.752031 1.068277 0.625464 0.433967 0.306839 1.176193 0.103805 0.096743 0.980855 0.551427 0.096549 0.095214 0.046351 0.053343 0.853389 0.680149 0.469069 0.555274 0.0005 0.311667 1067 878536 156968 1076.774026 322.572067
12 12 1.769169 0.18186 1.772084 0.176119 1.83411 1.844026 0.028092 0.024488 0.036672 0.028815 0.045173 0.039682 0.051839 0.047161 0.0 0.0 166 152.794 46.067033 73 468 249 250 1 0.535325 0.799832 0.891145 0.685752 0.475168 0.291591 1.142953 0.106931 0.078072 0.989762 0.506 0.09566 0.093859 0.051289 0.056117 0.880376 0.72019 0.516044 0.519486 0.0005 0.34 1163 848198 144673 1061.730934 311.39659
13 13 1.743754 0.158582 1.726435 0.153854 1.81128 1.799296 0.025183 0.021926 0.030586 0.025736 0.040735 0.035313 0.04855 0.042916 0.0 0.0 168 153.432 51.590012 68 478 231 268 1 0.579754 0.784928 0.941172 0.672857 0.472446 0.284693 1.125724 0.159865 0.117188 0.97555 0.534392 0.094677 0.101375 0.043354 0.04914 0.861803 0.653665 0.446512 0.536952 0.0005 0.368333 1260 857260 138054 1159.378612 314.991295
14 14 1.685476 0.14208 1.668084 0.138423 1.76341 1.752266 0.023658 0.021398 0.028364 0.024813 0.038216 0.034359 0.045757 0.042141 0.0 0.0 170 160.676 50.682769 71 446 245 253 2 0.599818 0.777932 1.027391 0.641592 0.508001 0.298964 0.997862 0.153056 0.133498 0.979717 0.556548 0.083556 0.078853 0.04172 0.045313 0.82244 0.643421 0.445224 0.566471 0.0005 0.396667 1357 868419 137234 1142.575539 318.540086
15 15 1.633348 0.129286 1.62438 0.124074 1.726803 1.724655 0.026111 0.022418 0.031841 0.026108 0.042237 0.036109 0.049651 0.044523 0.0 0.0 162 152.096 44.158881 71 448 252 248 0 0.641061 0.763771 1.046578 0.633618 0.467355 0.287425 1.120743 0.144587 0.131918 0.986322 0.553629 0.093773 0.095643 0.045512 0.051104 0.842547 0.646832 0.445605 0.56377 0.0005 0.425 1453 829014 120951 1088.839052 304.436716
16 16 1.598881 0.116255 1.61431 0.111924 1.69243 1.705957 0.027271 0.022414 0.033341 0.026035 0.044324 0.036799 0.051996 0.043674 0.0 0.0 168 157.488 62.225829 70 470 205 292 3 0.515829 0.808925 0.916341 0.681383 0.546984 0.333553 0.994348 0.1293 0.102312 0.968616 0.561537 0.085636 0.084022 0.045192 0.049098 0.805688 0.618475 0.448373 0.573782 0.0005 0.453333 1550 859841 161961 1555.666082 315.398319
17 17 1.593311 0.102901 1.63572 0.100119 1.68449 1.719233 0.025457 0.022686 0.030927 0.026931 0.041195 0.036452 0.04865 0.044124 0.0 0.0 162 151.646 50.007686 73 458 239 259 2 0.606764 0.774544 0.994513 0.650092 0.487137 0.317486 1.052224 0.142098 0.139445 0.97955 0.564526 0.108476 0.105311 0.050597 0.058315 0.857885 0.631607 0.435654 0.566821 0.0005 0.481667 1647 827530 124657 1252.224998 303.262991
18 18 1.587619 0.092381 1.6032 0.106847 1.679098 1.697721 0.025076 0.032382 0.029541 0.04396 0.040489 0.052156 0.048468 0.057924 0.0 0.0 164 157.474 68.425911 66 498 187 312 1 0.55268 0.79346 0.906181 0.684672 0.51655 0.371133 1.050794 0.15175 0.136338 0.961803 0.563531 0.086824 0.084632 0.044678 0.05112 0.817798 0.636742 0.441428 0.581601 0.0005 0.51 1743 835065 152208 1654.426047 305.547479
19 19 1.610415 0.083619 1.599307 0.084605 1.688095 1.671362 0.025586 0.024777 0.030147 0.028787 0.041211 0.039683 0.04995 0.048696 0.0 0.0 169 152.524 65.137036 67 476 221 277 2 0.347574 0.866966 0.551663 0.804141 0.489161 0.33413 1.286698 0.09309 0.048168 0.975624 0.539497 0.12587 0.108489 0.071734 0.078992 0.848466 0.643014 0.460272 0.55665 0.0005 0.538333 1840 864236 167225 1836.418729 315.763165
20 20 1.595458 0.071229 1.577088 0.084741 1.680167 1.659707 0.0261 0.036645 0.030566 0.046531 0.041796 0.060673 0.050703 0.067048 0.0 0.0 168 149.222 52.376834 56 434 190 308 2 0.741654 0.72722 1.174968 0.599107 0.508026 0.350912 0.878517 0.258383 0.24159 0.955657 0.695365 0.069528 0.068033 0.033206 0.036553 0.60014 0.417148 0.261225 0.690961 0.0005 0.566667 1937 858578 131576 1515.447511 314.417207
21 21 1.59314 0.06514 1.614174 0.06185 1.676747 1.695504 0.026925 0.027885 0.032138 0.030517 0.043084 0.045214 0.051482 0.054957 0.0 0.0 169 181.222 65.463889 80 446 200 296 4 0.712622 0.734498 1.105734 0.611729 0.418859 0.298281 1.200802 0.17239 0.169155 0.965164 0.605477 0.149025 0.145808 0.058951 0.067606 0.751185 0.602074 0.418549 0.587895 0.0005 0.595 2033 864264 126637 1545.836148 316.323246
22 22 1.60039 0.053024 1.598467 0.052799 1.687232 1.684615 0.024862 0.025759 0.02848 0.027929 0.039581 0.040929 0.048451 0.051915 0.0 0.0 170 163.154 71.88651 72 484 106 392 2 0.579809 0.784635 0.956234 0.67096 0.554535 0.40083 0.968256 0.189099 0.163138 0.956453 0.680677 0.06939 0.06171 0.035676 0.039853 0.60233 0.459763 0.335824 0.691466 0.0005 0.623333 2130 868803 166500 2125.255328 318.410751
23 23 1.602958 0.044593 1.581573 0.046362 1.688489 1.67309 0.023233 0.023619 0.025804 0.027185 0.036903 0.037762 0.045971 0.04591 0.0 0.0 175 141.606 49.076621 67 464 224 276 0 0.474655 0.824219 0.818843 0.715103 0.566828 0.385691 1.027002 0.164254 0.140212 0.983393 0.598531 0.078073 0.073074 0.045051 0.04849 0.778576 0.589688 0.397085 0.614012 0.0005 0.651667 2227 893433 149287 1898.299594 327.279244
24 24 1.595729 0.039257 1.626009 0.037647 1.68055 1.700121 0.022925 0.023532 0.025702 0.024704 0.036395 0.037164 0.045237 0.0488 0.0 0.0 175 143.31 50.031409 56 480 225 273 2 0.452515 0.830092 0.743443 0.740151 0.541361 0.372322 1.079644 0.142532 0.125331 0.978512 0.57856 0.088087 0.083255 0.049506 0.056019 0.844911 0.607704 0.425367 0.597537 0.0005 0.68 2323 892272 151047 2091.257054 327.032849
25 25 1.546695 0.033551 1.562203 0.031339 1.657497 1.672241 0.024161 0.021427 0.027211 0.022543 0.038371 0.033928 0.047395 0.043157 0.0 0.0 168 151.93 50.27943 70 454 153 347 0 0.715677 0.735949 1.173823 0.596624 0.529346 0.351666 0.870213 0.302912 0.275798 0.947208 0.657627 0.059701 0.058459 0.02993 0.034055 0.629014 0.420513 0.286895 0.667637 0.0005 0.708333 2420 855542 130495 1881.03623 313.541955
26 26 1.538404 0.029395 1.547581 0.026681 1.642496 1.65031 0.023465 0.020417 0.025977 0.021728 0.03747 0.032568 0.046685 0.04189 0.0 0.0 166 160.646 56.818911 71 480 337 162 1 0.659241 0.753713 1.000794 0.649235 0.483701 0.364335 1.074254 0.196677 0.182989 0.969865 0.590161 0.080498 0.076719 0.03729 0.044168 0.80305 0.566641 0.367638 0.58894 0.0005 0.736667 2517 848363 124397 1859.219043 311.037866
27 27 1.631154 0.02767 1.647569 0.026203 1.730439 1.742618 0.025741 0.02487 0.028204 0.02789 0.040729 0.03893 0.050666 0.048654 0.0 0.0 27 145.534 55.786386 63 494 302 196 2 0.588341 0.781781 0.919811 0.681857 0.538247 0.39895 0.994642 0.200528 0.185835 0.960672 0.617199 0.074665 0.072845 0.037001 0.04466 0.758544 0.5237 0.367913 0.620541 0.001 0.765 2200 138128 138128 1917.634576 61.65351
28 28 1.600756 0.024276 1.582233 0.022054 1.712803 1.671691 0.027358 0.024486 0.030597 0.026372 0.043283 0.038719 0.054096 0.049308 0.0 0.0 54 144.894 46.512523 67 478 260 239 1 0.602088 0.777247 0.963369 0.664628 0.516011 0.373695 1.007537 0.231677 0.199585 0.963148 0.614058 0.08708 0.086147 0.042884 0.048546 0.758544 0.511941 0.331628 0.621181 0.001 0.793333 2200 274594 136466 1853.079969 100.700688
29 29 1.543052 0.024962 1.551533 0.018264 1.65123 1.652743 0.030234 0.02828 0.034632 0.029014 0.048585 0.044553 0.059042 0.059062 0.0 0.0 78 151.666 52.19121 72 462 225 274 1 0.613045 0.770348 0.962094 0.659778 0.478082 0.328206 1.067367 0.162183 0.16351 0.976478 0.571112 0.08935 0.083074 0.043636 0.049889 0.848223 0.610607 0.393213 0.569636 0.001 0.821667 2200 398010 123416 1621.624428 147.57809
30 30 1.527891 0.021392 1.539996 0.015603 1.641985 1.652806 0.030204 0.025168 0.034146 0.025874 0.048385 0.040407 0.059328 0.052234 0.0 0.0 105 143.46 46.034774 67 438 256 244 0 0.624463 0.768474 0.951779 0.671045 0.528727 0.391771 0.932016 0.220292 0.187015 0.969605 0.634996 0.072329 0.07237 0.036162 0.041647 0.739475 0.486038 0.300159 0.635651 0.001 0.85 2200 536658 138648 1837.105643 196.869067
31 31 1.510002 0.019791 1.534467 0.022667 1.617605 1.630937 0.027905 0.028226 0.03165 0.033988 0.044745 0.045121 0.055141 0.053103 0.0 0.0 136 142.962 50.19889 54 492 200 300 0 0.463335 0.826755 0.780327 0.727602 0.585815 0.43367 0.954705 0.149996 0.123276 0.982096 0.584539 0.071861 0.068297 0.040636 0.045073 0.824785 0.576924 0.375868 0.592216 0.001 0.85 2200 695336 158678 2095.163666 255.098031
32 32 1.504124 0.018803 1.492304 0.017774 1.610304 1.593604 0.026777 0.02448 0.029637 0.025709 0.042879 0.038837 0.053755 0.05042 0.0 0.0 165 167.818 61.528537 69 464 262 238 0 0.59749 0.776705 0.97375 0.660258 0.510189 0.340975 1.003694 0.204991 0.179096 0.965246 0.593617 0.11552 0.114116 0.04235 0.05088 0.774082 0.548422 0.391943 0.580342 0.001 0.85 2200 843630 148294 1984.36426 309.579498
33 33 1.502483 0.019177 1.520489 0.018532 1.604864 1.617476 0.026436 0.024772 0.029569 0.027223 0.042196 0.039014 0.052558 0.050305 0.0 0.0 197 154.378 56.565706 64 442 203 295 2 0.49421 0.814516 0.759661 0.73467 0.55437 0.426652 0.992558 0.148494 0.140722 0.977814 0.598636 0.087007 0.081614 0.045056 0.052803 0.809672 0.60133 0.383847 0.609081 0.001 0.85 2200 1006377 162747 2109.743674 369.513571
34 34 1.501502 0.019428 1.507263 0.019588 1.608812 1.612028 0.027127 0.024046 0.030404 0.024599 0.043295 0.037293 0.053953 0.050321 0.0 0.0 222 139.394 51.979138 48 466 160 338 2 0.629697 0.76975 0.963851 0.670826 0.552312 0.428338 0.912507 0.254125 0.226709 0.955053 0.67695 0.063272 0.061216 0.032487 0.03824 0.680525 0.419521 0.269889 0.686476 0.001 0.85 2200 1134855 128478 1765.623216 416.722142
35 35 1.494336 0.018457 1.472721 0.015414 1.604291 1.583125 0.026272 0.023132 0.028832 0.024855 0.041872 0.036589 0.052825 0.047244 0.0 0.0 248 155.92 63.776246 63 478 208 292 0 0.668505 0.751627 1.010599 0.649487 0.493075 0.375868 0.941319 0.2078 0.206914 0.956619 0.61813 0.093083 0.090281 0.038921 0.046342 0.76766 0.566063 0.358647 0.608787 0.001 0.85 2200 1269299 134444 1827.243433 466.175742
36 36 1.48397 0.019453 1.505326 0.016983 1.595019 1.619247 0.026652 0.023615 0.029877 0.026228 0.042452 0.036972 0.053017 0.047382 0.0 0.0 278 156.378 62.415376 62 472 144 356 0 0.559364 0.790212 0.882157 0.694187 0.561391 0.424078 0.892993 0.174826 0.161521 0.973995 0.634178 0.071024 0.068513 0.036611 0.043839 0.736014 0.516523 0.335731 0.638252 0.001 0.85 2200 1423026 153727 2037.006878 522.487341
37 37 1.475587 0.017219 1.478916 0.0149 1.585306 1.589145 0.024368 0.023677 0.026455 0.023768 0.0388 0.037476 0.049421 0.050104 0.0 0.0 308 168.054 73.029768 67 478 232 267 1 0.586444 0.778209 0.841015 0.703811 0.475624 0.394771 1.1106 0.13432 0.120005 0.964745 0.542202 0.108002 0.105444 0.043713 0.054562 0.851533 0.651803 0.478176 0.539126 0.001 0.85 2200 1576473 153447 2098.989394 578.610096
38 38 1.465006 0.017169 1.473249 0.014236 1.579261 1.588865 0.024315 0.022332 0.026446 0.023558 0.038669 0.035525 0.049223 0.046063 0.0 0.0 337 158.98 70.460199 55 478 207 292 1 0.637908 0.762002 0.992976 0.657282 0.547516 0.419244 0.855569 0.233136 0.21015 0.947589 0.626923 0.061087 0.059253 0.030223 0.035999 0.689646 0.450105 0.318761 0.636309 0.001 0.85 2200 1721015 144542 1909.775954 630.718938
39 39 1.461264 0.01743 1.475532 0.016604 1.573932 1.589149 0.024254 0.023681 0.026516 0.024779 0.038542 0.037377 0.049134 0.048563 0.0 0.0 369 152.204 61.626118 66 482 220 279 1 0.512251 0.807529 0.8033 0.71953 0.559175 0.423127 0.94167 0.12596 0.12609 0.977466 0.588403 0.0753 0.074171 0.038687 0.04585 0.834417 0.609512 0.39925 0.58956 0.001 0.85 2200 1886897 165882 2191.403032 690.374504
40 40 1.45703 0.016847 1.458281 0.014758 1.567668 1.569762 0.02314 0.021837 0.025111 0.022456 0.036696 0.034688 0.047142 0.046099 0.0 0.0 391 159.088 70.768837 65 488 250 248 2 0.473134 0.821397 0.74801 0.738572 0.56835 0.454738 0.952098 0.129152 0.113965 0.964031 0.558192 0.068169 0.064717 0.038168 0.045193 0.813643 0.589736 0.415874 0.565413 0.001 0.85 2200 2000000 178325 2343.412783 730.773342
41 41 1.447584 0.016466 1.474226 0.019915 1.556948 1.587155 0.022741 0.026505 0.02447 0.029285 0.03608 0.042885 0.046616 0.052717 0.0 0.0 391 151.478 62.343095 64 490 258 242 0 0.46031 0.825292 0.689681 0.758249 0.561468 0.447036 0.979701 0.129924 0.105911 0.977776 0.560312 0.07319 0.071048 0.043558 0.051654 0.84655 0.606777 0.409171 0.570298 0.001 0.85 2200 2000000 168145 2259.010202 733.335143
42 42 1.439119 0.015933 1.438965 0.017365 1.55053 1.549249 0.022521 0.022245 0.024018 0.024501 0.035577 0.03565 0.046464 0.045365 0.0 0.0 391 169.0 62.847371 69 470 358 142 0 0.683194 0.743043 1.039441 0.639553 0.513767 0.397493 0.894383 0.263804 0.240625 0.958827 0.644044 0.089614 0.087267 0.034734 0.040567 0.688416 0.464218 0.315169 0.632701 0.001 0.85 2200 2000000 148258 1990.163259 731.58971
43 43 1.438177 0.016148 1.440251 0.01419 1.544654 1.542344 0.021764 0.020372 0.023543 0.02029 0.034541 0.032049 0.044761 0.043494 0.0 0.0 391 161.352 79.63552 71 498 244 254 2 0.388365 0.848251 0.595903 0.784959 0.557086 0.445635 0.990283 0.101088 0.088031 0.955226 0.540583 0.089547 0.079906 0.051838 0.06248 0.845232 0.642681 0.452006 0.551334 0.001 0.85 2200 2000000 173844 2286.678973 730.369627
44 44 1.425366 0.016316 1.442016 0.018557 1.536779 1.554317 0.022089 0.024669 0.023908 0.028014 0.034871 0.039314 0.045332 0.048979 0.0 0.0 391 159.658 65.366452 62 464 308 191 1 0.656921 0.751374 0.967483 0.664112 0.534253 0.432223 0.825768 0.227825 0.201976 0.95692 0.626123 0.058065 0.057191 0.029876 0.036492 0.706818 0.476891 0.322827 0.624692 0.001 0.85 2200 2000000 147883 1961.597721 731.634271
45 45 1.435695 0.016486 1.438717 0.014131 1.544495 1.548456 0.02219 0.019105 0.024135 0.019825 0.035035 0.029984 0.04553 0.040119 0.0 0.0 391 148.996 66.979698 66 496 271 227 2 0.368999 0.857393 0.546611 0.80589 0.570004 0.467307 1.045475 0.108985 0.072766 0.967119 0.553043 0.088083 0.082921 0.055167 0.062585 0.823462 0.615631 0.40136 0.56292 0.001 0.85 2200 2000000 172304 2363.12772 731.133658
46 46 1.413448 0.015653 1.42459 0.014304 1.52541 1.53568 0.020678 0.020368 0.022153 0.021486 0.032702 0.032446 0.042742 0.042204 0.0 0.0 391 147.264 56.616732 59 446 263 236 1 0.551999 0.791152 0.825955 0.709695 0.531521 0.426446 0.914724 0.126189 0.117103 0.983349 0.545493 0.073291 0.073028 0.037489 0.045189 0.856586 0.652544 0.443002 0.5429 0.001 0.85 2200 2000000 148487 1930.738246 731.658938
47 47 1.396397 0.015194 1.408684 0.014436 1.51154 1.521602 0.020528 0.021489 0.021764 0.022518 0.032384 0.033757 0.042586 0.04489 0.0 0.0 391 146.446 46.780969 51 464 337 163 0 0.64404 0.758741 0.959276 0.669626 0.567808 0.451258 0.754924 0.237151 0.219979 0.968248 0.647667 0.058039 0.056217 0.029738 0.036318 0.715873 0.477352 0.313347 0.649182 0.001 0.85 2200 2000000 141981 1917.968596 731.21963
48 48 1.391687 0.015102 1.396846 0.013463 1.507451 1.511351 0.019972 0.019252 0.021254 0.019976 0.031473 0.030056 0.041608 0.040783 0.0 0.0 391 153.8 60.712635 64 478 243 256 1 0.604867 0.770967 0.886359 0.690049 0.541787 0.450494 0.872601 0.185175 0.162474 0.96531 0.59197 0.064294 0.060997 0.031908 0.039902 0.766272 0.552973 0.381144 0.593445 0.001 0.85 2200 2000000 148287 1946.923525 727.66864
49 49 1.379114 0.014961 1.378524 0.013703 1.499055 1.499705 0.020035 0.018847 0.021124 0.018727 0.031561 0.029678 0.041827 0.040936 0.0 0.0 391 139.886 44.507314 60 444 298 202 0 0.586252 0.779185 0.860614 0.702228 0.567265 0.462548 0.83353 0.194861 0.168031 0.977982 0.628236 0.060486 0.059147 0.033284 0.040446 0.750009 0.520905 0.351887 0.630133 0.001 0.85 2200 2000000 147023 1863.875768 727.928495
50 50 1.365788 0.014389 1.380076 0.013122 1.48942 1.501487 0.019557 0.018649 0.02072 0.019514 0.030911 0.029566 0.040864 0.039048 0.0 0.0 391 131.778 43.811514 53 484 285 215 0 0.490108 0.814873 0.742953 0.7395 0.589884 0.47054 0.843874 0.158295 0.124971 0.981082 0.584933 0.0596 0.05947 0.037217 0.043979 0.810012 0.564993 0.351262 0.591461 0.001 0.85 2200 2000000 144559 1819.94748 728.786112
51 51 1.359874 0.014513 1.367549 0.01348 1.481815 1.48814 0.019263 0.018336 0.020319 0.019621 0.03044 0.028842 0.04041 0.038005 0.0 0.0 391 135.466 43.404249 55 498 224 276 0 0.510903 0.807341 0.779897 0.725723 0.570765 0.457996 0.869952 0.136292 0.113218 0.986826 0.556629 0.07046 0.071239 0.036549 0.044577 0.869733 0.646225 0.424543 0.557536 0.001 0.85 2200 2000000 144117 1859.36987 727.803287
52 52 1.348599 0.014895 1.347117 0.013928 1.474516 1.471252 0.0199 0.018352 0.021303 0.019147 0.03151 0.028722 0.041427 0.038997 0.0 0.0 391 133.472 41.03712 55 448 286 214 0 0.605159 0.772589 0.883705 0.690765 0.559374 0.448502 0.800811 0.177221 0.158452 0.977937 0.583487 0.065018 0.063389 0.032533 0.039353 0.829824 0.57753 0.370176 0.580615 0.001 0.85 2200 2000000 126057 1597.16603 726.604658
53 53 1.33978 0.014636 1.336231 0.014018 1.467388 1.465913 0.020108 0.019793 0.021356 0.022157 0.031877 0.031527 0.042021 0.04033 0.0 0.0 391 142.672 49.40134 53 468 268 230 2 0.601627 0.771674 0.887454 0.687208 0.532979 0.423814 0.852638 0.17498 0.156644 0.976194 0.579721 0.072656 0.069903 0.035328 0.043077 0.821076 0.585022 0.383302 0.583011 0.001 0.85 2200 2000000 132101 1734.659378 727.170206
54 54 1.332465 0.014294 1.334156 0.01702 1.462583 1.469422 0.019853 0.02078 0.020795 0.02314 0.031494 0.033668 0.041707 0.0425 0.0 0.0 391 133.83 42.712259 60 446 239 260 1 0.570791 0.78411 0.837847 0.704884 0.553858 0.445095 0.855959 0.170748 0.150117 0.981602 0.577006 0.075516 0.072633 0.035519 0.044483 0.864856 0.630576 0.401815 0.572989 0.001 0.85 2200 2000000 126110 1622.574194 726.980076
55 55 1.320884 0.014177 1.337232 0.01486 1.454168 1.465355 0.019844 0.019468 0.020723 0.019913 0.031513 0.030079 0.041845 0.042086 0.0 0.0 391 150.456 47.820582 57 448 277 222 1 0.606047 0.77095 0.910132 0.677499 0.520069 0.407775 0.878787 0.203472 0.188995 0.969426 0.590068 0.097821 0.092619 0.041064 0.049063 0.81311 0.571887 0.359338 0.577359 0.001 0.85 2200 2000000 130802 1690.20232 727.513331
56 56 1.323014 0.014165 1.321301 0.015144 1.456065 1.45416 0.019677 0.021762 0.02056 0.023255 0.031237 0.034725 0.041356 0.045074 0.0 0.0 391 134.416 50.388639 56 488 242 257 1 0.429 0.835564 0.622198 0.780197 0.576644 0.484707 0.922425 0.118965 0.085759 0.984797 0.551297 0.066868 0.061492 0.042384 0.050513 0.867101 0.625497 0.410608 0.558505 0.001 0.85 2200 2000000 153436 2053.132314 726.763658
57 57 1.316037 0.014579 1.327117 0.012872 1.453426 1.465655 0.020099 0.018743 0.02102 0.019322 0.031902 0.02974 0.042233 0.039662 0.0 0.0 391 124.782 33.171531 60 364 237 263 0 0.435236 0.834157 0.645326 0.770775 0.590173 0.481207 0.919064 0.149979 0.115271 0.988689 0.565511 0.065432 0.061681 0.042449 0.049672 0.871754 0.622624 0.398794 0.575833 0.001 0.85 2200 2000000 137519 1747.019653 727.404668
58 58 1.308757 0.013503 1.325582 0.012469 1.446505 1.459705 0.019024 0.018331 0.019252 0.018372 0.030169 0.029493 0.040682 0.039465 0.0 0.0 391 127.236 40.01655 52 466 295 204 1 0.558701 0.789285 0.822686 0.711089 0.566232 0.454785 0.845153 0.158418 0.135902 0.979358 0.561462 0.067389 0.06709 0.033997 0.04172 0.847538 0.620627 0.39481 0.557516 0.001 0.85 2200 2000000 130367 1748.677676 727.136486
59 59 1.295545 0.013922 1.312865 0.016572 1.437396 1.449283 0.019374 0.023332 0.019834 0.025103 0.030797 0.037672 0.041235 0.047378 0.0 0.0 391 125.426 38.704322 61 427 281 219 0 0.493947 0.812451 0.728885 0.742184 0.580174 0.467649 0.867239 0.122345 0.103655 0.992488 0.563805 0.066892 0.06612 0.038299 0.046037 0.875773 0.617906 0.395052 0.565021 0.001 0.85 2200 2000000 134513 1722.172414 727.512374
60 60 1.302721 0.014069 1.319685 0.014328 1.44135 1.45519 0.019652 0.018409 0.019955 0.018403 0.031161 0.028894 0.042024 0.039913 0.0 0.0 391 131.76 35.734331 58 334 240 260 0 0.382318 0.852562 0.595851 0.785894 0.595447 0.464209 0.940825 0.116569 0.096554 0.993183 0.561035 0.081134 0.083369 0.051321 0.057639 0.880389 0.660247 0.422753 0.573411 0.001 0.85 2200 2000000 147595 1822.221514 727.842815
61 61 1.2974 0.013946 1.312984 0.012587 1.435012 1.445997 0.019455 0.017964 0.019837 0.01764 0.030888 0.028423 0.041506 0.039025 0.0 0.0 391 128.736 40.631149 66 434 225 274 1 0.524994 0.801443 0.796737 0.717361 0.559983 0.438831 0.857074 0.153566 0.12794 0.981491 0.547749 0.077931 0.075398 0.03716 0.045041 0.870226 0.62503 0.399675 0.542946 0.001 0.85 2200 2000000 130579 1671.410624 728.096484
62 62 1.287199 0.013724 1.312884 0.017074 1.424416 1.446529 0.019291 0.021245 0.019634 0.024171 0.030712 0.033763 0.041243 0.043145 0.0 0.0 391 116.54 29.893685 58 369 246 254 0 0.500447 0.811427 0.751687 0.733646 0.587302 0.468955 0.810102 0.142737 0.116387 0.989761 0.555325 0.062779 0.062844 0.039598 0.046514 0.880895 0.625373 0.394697 0.557588 0.001 0.85 2200 2000000 120999 1573.060033 728.66347

AlphaZero Training Run: 62-Iteration Self-Play Metrics — AI Analysis

Value loss plunged 96% (0.32 → 0.014) — the network learned to evaluate positions almost perfectly

Games shortened from 354 to 117 moves as the agent learned to win decisively instead of drifting

Training Summary

  • Policy loss dropped 67% (3.86 → 1.29) over 62 iterations, with most improvement in the first 15 iterations
  • Value loss fell 96% (0.32 → 0.014) and fully converged by iteration 30
  • Average game length shrank from 354 to 117 moves — a 67% reduction — as play became more decisive
  • Policy-MCTS agreement nearly doubled (31% → 59%), though the network still overrides tree search 41% of the time
  • Late-game value error improved from 0.72 to 0.39, but early-game error worsened (0.70 → 0.88) — the network struggles to evaluate opening positions

Training Summary

  • Policy loss fell 67% (3.86 → 1.29) over 62 iterations with train and validation tracking closely — no overfitting
  • Value loss dropped 96% (0.32 → 0.01), converging near zero by iteration 30
  • Average game length collapsed from 354 to 117 moves as the agent stopped drifting into long, inconclusive games
  • Policy agreement with MCTS nearly doubled (31% → 59%), but at 59% the network still frequently overrides the tree search
  • Early-position value error rose from 0.70 to 0.88 even as late-game error fell — the agent struggles to evaluate openings

Visualizations

Value Error by Game Phase
Policy Agreement with MCTS
Average Game Length
Value Loss Over Training

Early-Game Blind Spot

The value network's early-game error rose from 0.70 to 0.88 even as all other metrics improved. This suggests the network has learned strong tactical play but poor positional judgment in openings — a common pattern in self-play systems that lack opening books.

This dataset contains 62 records across 51 fields: iteration, loss_policy_train, loss_value_train, loss_policy_val, loss_value_val, loss_soft_policy_train, and 45 more.

62 rows · 51 columns · 2026-03-24

Embed this data story

Embedding launches soon. You'll be able to embed interactive charts and data tables on any website.