But I already got some feedback from other testers that the new version is a significant improvement of more than 200 ELO.
This little tournament was run with the 32 bit version and a time control of 5 minutes a game. Lars Hallerström was so kind to give me permission to post them. Thanks again.
78 Ice 1.0 : 2703 840 (+367,=226,-247), 57.1 %
Alaric 707 : 40 (+ 16,= 10,- 14), 52.5 %
Amyan 1.7a : 40 (+ 18,= 9,- 13), 56.2 %
Chepla 0.63 : 40 (+ 28,= 8,- 4), 80.0 %
ET 8.01 : 40 (+ 23,= 8,- 9), 67.5 %
Tao 5.6 : 40 (+ 29,= 4,- 7), 77.5 %
Ufim 8.01 : 40 (+ 27,= 9,- 4), 78.8 %
Cyrano 0.6b17 : 40 (+ 15,= 12,- 13), 52.5 %
Fruit 2.3.1 : 40 (+ 10,= 10,- 20), 37.5 %
Ktulu 9 : 40 (+ 10,= 16,- 14), 45.0 %
Ruffian 1.0.0 : 40 (+ 15,= 12,- 13), 52.5 %
SOS 5.1 : 40 (+ 19,= 11,- 10), 61.3 %
WildCat 8 beta 5 : 40 (+ 18,= 8,- 14), 55.0 %
Cheng3 1.06b : 40 (+ 17,= 11,- 12), 56.2 %
Chronos 1.9.9 beta : 40 (+ 11,= 17,- 12), 48.8 %
Deuterium 12.01 : 40 (+ 15,= 13,- 12), 53.8 %
Dirty Sep 5 2012 : 40 (+ 16,= 12,- 12), 55.0 %
Pharaon 3.5.1 : 40 (+ 16,= 13,- 11), 56.2 %
Tornado 4.78 : 40 (+ 20,= 10,- 10), 62.5 %
Texel 1.01 : 40 (+ 19,= 10,- 11), 60.0 %
BobCat 3.25 : 40 (+ 11,= 11,- 18), 41.2 %
Bug2 1.9 : 40 (+ 14,= 12,- 14), 50.0 %
So here iCE gained 243 ELO points compared with iCE 0.3. This is a huge jump and I guess iCE 1.0 will now enter also the Top 100 in CCRL. So I probably need a new goal.
78 Ice 1.0 : 2703
183 Ice 0.3 : 2460
WOW!! +243 ELO Congratulations Thomas!
ReplyDeleteVery good improvement
I'm developing a chess engine from a tutorial from internet and it's pretty difficult to tune it, my goal it to reach at least 2500 ELO in CCRL as you iCE0.3 but each day I touch the code I feel that I'm wasting my time even reading http://chessprogramming.wikispaces.com because in that site there are many ways to improve the engine but many of those ways are obsolete or I dont understand how to apply it to the engine
Can you give me some light to found good resources to improve my engine?
Thanks and keep it up!
Hi, it depends on what you already have implemented. What features do you have already and does your engine already have an ELO rating ?
ReplyDeleteIn general I would recommend a PVS framework for the search. Get this stable. You can gain a lot of strength just by optimizing your move ordering. Then add a hash table. This will further help you with the move ordering. Hash table is easy to do wrong as it contains a lot of potential pitfalls and bugs are not so easy to find.
Then Null Move heuristics is very useful.
Most of the techniques in the wiki actually should work if implemented correctly but some overlap and you might find a feature not so useful because you already have a feature that performs something similar.
What features from the Wiki do you have doubts about ?
Im my latest release more tha 150 ELO I guess just come from choosing better weights or values for existing features.
And wasting time on something that does finally not work is part of the deal. The stronger your engine gets the more difficult it becomes to find something that makes it even stronger.
Thomas...
Hi, thanks for the reply
ReplyDeleteWell I've no tested vs other engine, because some < 2000 does not play 1min, but I think the engine it's like 1700-1800
The engine use bitboard, the evaluation it's pretty basic, it just eval PNBRK, now I'm implementing to eval the Isolated, Backwards and Passed pawns, I've found that the eval PNBRK from the Fruit 2.1 source it's good, I will test it
I've implemented the LMR (Late Move Reduction) from Glaurung also Fruit, according to my tests it's +100 ELO, even the LMR is not 100% complete using the common condition from here -> http://chessprogramming.wikispaces.com/Late+Move+Reductions
I tried to use PVS but the results were negative vs the LMR
I used this example -> http://chessprogramming.wikispaces.com/Principal+Variation+Search, in the Pseudo Code part and PVS + ZWS also I've tested it in a little 400games tournament in Arena 1 minute using top200.pgn as start position and without good results, but I will test more deeply
Right now the engine has no Hash implemented
The Null move also I dont know how to apply but according to some engine sources it just HashPosKey = 0; and/or move = 0; right?
I have many doubts, like the Fail-High, Fail-Low even Fails-Soft, upper bound, lower bound..... many technical things that I know what it the mening but I dont know how to apply to the engine.. it's complicated...
I have like 20 version of the engine to test between "improvements"
Thanks again for your time
Hi,
ReplyDeletethe most important basic feature in a chess engine is high quality move ordering. Some features like PVS or LMR rely on good move ordering and will not work if your ordering is bad.
To measure your move ordering simply count the number of fail highs (beta cutoffs) that happen during a search and also count the number when this cutoff was achieved with already the 1st move. Then build the percentage
pct = failhigh_1st_move / fail_high_total
You want this to be 90% or higher. Probably you need a transposition table for that and also have a look at MVV/LVA, Static Exchange Evaluation and Killer Moves.
Everything else can come later.
A null move is a move where you just switch the side to move on the board without moving a piece. So the opponent can make two moves in a row. If he can move twice and you find out that you are still better than your opponent, then your position is so strong the opponent will not select moves leading to it. You don't have to search this branch any longer.
The hashkey is not set to 0. The hashkey after the null move reflects that the side to move has changed.
Thomas...
Hi, thanks for the reply
ReplyDeleteHere a log from a perft test from the start position:
position startpos
go infinite
info score cp 30 depth 1 nodes 25 time 16 nps 1563 pv d2d4
Ordering:-1.#J, fhf:0, fh:0
info score cp 10 depth 2 nodes 277 time 16 nps 17313 pv d2d4 d7d5
Ordering:0.89, fhf:17, fh:19
info score cp 25 depth 3 nodes 1055 time 16 nps 65938 pv d2d4 d7d5 c1e3
Ordering:0.90, fhf:146, fh:163
info score cp 10 depth 4 nodes 2982 time 32 nps 93188 pv d2d4 d7d5 c1e3 c8e6
Ordering:0.90, fhf:758, fh:844
info score cp 20 depth 5 nodes 17981 time 32 nps 561906 pv d2d4 d7d5 c1e3 c8e6 b1c3
Ordering:0.87, fhf:2488, fh:2855
info score cp 10 depth 6 nodes 89142 time 110 nps 810382 pv d2d4 d7d5 c1e3 c8e6b1c3 g8f6
Ordering:0.85, fhf:22681, fh:26608
info score cp 25 depth 7 nodes 691989 time 578 nps 1197213 pv e2e4 e7e5 g1f3 g8f6 b1c3 f8d6 d2d4
Ordering:0.84, fhf:102192, fh:120992
info score cp 15 depth 8 nodes 3106308 time 2637 nps 1177000 pv e2e4 e7e5 g1f3 g8f6 b1c3 f8d6 d2d4 b8c6
Ordering:0.86, fhf:625142, fh:730977
info score cp 20 depth 9 nodes 14178599 time 10780 nps 1315000 pv d2d4 d7d5 b1c3 g8f6 c1f4 c8e6 e2e3 b8c6 f1d3
Ordering:0.84, fhf:1780916, fh:2125400
The lowest is 0.84, now I see where I have to start and it's improve first the move ordering
Now it has Null Move + PVS and still lost vs the LMR version but I think there are some things that I have to take a look more deeply
So much work... xD
Thanks Thomas