Currently I have only limited time for my project and I also want to take a bit of a break from programming. So after my release of iCE 3 I decided to go back to an area where I'm not so involved. I thought it is time to do a little evaluation parameter tuning again. Here the computer does the stuff on its own.
My idea was now to not tune the whole set but to keep the current values for material. Those are dominant terms (if the pawn value is badly wrong the eval is bad even if it has a good weight for the rook on a half open file). So if I keep the material values the other minor terms get more pressure and maybe they converge to a better set.
As the only tuning method that worked for me so far is the population based incremental learning (a GA type of optimization) I used this one. I used a population size of 256 and let the tuner run for almost 1500 generations.
After every 100 generations I took the current state of the tuning and let it run a small match (3000 games) against the base version.
After 100 generations the set was already only 50 ELO worse than the base version. This is pretty fast considering the fact that except from material all weights started from random values and converged to a set that gets about 3000 ELO on the CCRL scale. Over time the tuner was able to improve the set a bit further.
In the final iteration the final set was at -6 ELO (16000 games). After I optimized the build for the new set (the base version was also optimized) the new set was at +3 ELO.
In the tuning run the tuner played 750.000 games at very fast time controls.
So it confirms the method works but it is not able to improve the weights any further significantly. However it is better than the Texel method for iCE. After minimizing the evaluation error using this method the engine lost 24 ELO.
I think I will try some other methods in the future and hopefully can squeeze out a bit more than 3 ELO.
