[ << ] [ >> ]           [Top] [Contents] [Index] [ ? ]

11. Tactical reading

The process of visualizing potential moves done by you and your opponent to learn the result of different moves is called "reading". GNU Go does three distinct types of reading: tactical reading which typically is concerned with the life and death of individual strings, Owl reading which is concerned with the life and death of dragons, and connection reading. In this Chapter, we document the tactical reading code, which is in ‘engine/reading.c’.

11.1 Reading Basics

What we call Tactical Reading is the analysis whether there is a direct capture of a single string, or whether there is a move to prevent such a direct capture.

If the reading module finds out that the string can get captured, this answer should (usually) be trusted. However, if it says it can be defended, this does not say as much. It is often the case that such a string has no chance to make a life, but that it cannot be captured within the horizon (and the cutoff heuristics) of the tactical reading.

The tactical reading is done by the functions in ‘engine/reading.c’. It is a minimax search that declares win for the attacker once he can physically take the string off board, whereas the defense is considered successful when the string has sufficiently many liberties. A string with five liberties is always considered alive. At higher depth within the search tree even fewer liberties cause GNU Go to give up the attack, See depthparams.

The reading code makes use of a stack onto which board positions can be pushed. The parameter stackp is zero if GNU Go is examining the true board position; if it is higher than zero, then GNU Go is examining a hypothetical position obtained by playing several moves.

The most important public reading functions are attack and find_defense. These are wrappers for functions do_attack and do_find_defense which are declared statically in ‘reading.c’. The functions do_attack and do_find_defense call each other recursively.

11.1.1 Organization of the reading code

The function do_attack and do_find_defense are wrappers themselves and call attack1, attack2, attack3 or attack4 resp. defend1, defend1, defend1 or defend1 depending on the number of liberties.

These are fine-tuned to generate and try out the moves in an efficient order. They generate a few moves themselves (mostly direct liberties of the string), and then call helper functions called ..._moves which suggest less obvious moves. Which of these functions get called depends on the number of liberties and of the current search depth.

11.1.2 Return Codes

The return codes of the reading (and owl) functions and owl can be 0, KO_B, KO_A or WIN. Each reading function determines whether a particular player (assumed to have the move) can solve a specific problem, typically attacking or defending a string.

A return code of WIN means success, 0 failure, while KO_A and KO_B are success conditioned on ko. A function returns KO_A if the position results in ko and that the player to move will get the first ko capture (so the opponent has to make the first ko threat). A return code of KO_B means that the player to move will have to make the first ko threat.

If GNU Go is compiled with the configure option ‘--enable-experimental-owl-ext’ then the owl functions also have possible return codes of GAIN and LOSS. A code of GAIN means that the attack (or defense) does not succeed, but that in the process of trying to attack or defend, an opponent's worm is captured. A code of LOSS means that the attack or defense succeeds, but that another friendly worm dies during the attack or defense.

11.1.3 Reading cutoff and depth parameters

Depth of reading is controlled by the parameters depth and branch_depth. The depth has a default value DEPTH (in ‘liberty.h’), which is set to 16 in the distribution, but it may also be set at the command line using the ‘-D’ or ‘--depth’ option. If depth is increased, GNU Go will be stronger and slower. GNU Go will read moves past depth, but in doing so it makes simplifying assumptions that can cause it to miss moves.

Specifically, when stackp > depth, GNU Go assumes that as soon as the string can get 3 liberties it is alive. This assumption is sufficient for reading ladders.

The branch_depth is typically set a little below depth. Between branch_depth and depth, attacks on strings with 3 liberties are considered, but branching is inhibited, so fewer variations are considered.

% %Currently the reading code does not try to defend a string by %attacking a boundary string with more than two liberties. Because %of this restriction, it can make oversights. A symptom of this is %two adjacent strings, each having three or four liberties, each %classified as DEAD. To resolve such situations, a function %small_semeai() (in ‘engine/semeai.c’) looks for such %pairs of strings and corrects their classification.

The backfill_depth is a similar variable with a default 12. Below this depth, GNU Go will try "backfilling" to capture stones. For example in this situation:

.OOOOOO.    on the edge of the board, O can capture X but
OOXXXXXO    in order to do so he has to first play at a in
.aObX.XO    preparation for making the atari at b. This is
--------    called backfilling.

Backfilling is only tried with stackp <= backfill_depth. The parameter backfill_depth may be set using the ‘-B’ option.

The fourlib_depth is a parameter with a default of only 7. Below this depth, GNU Go will try to attack strings with four liberties. The fourlib_depth may be set using the ‘-F’ option.

The parameter ko_depth is a similar cutoff. If stackp<ko_depth, the reading code will make experiments involving taking a ko even if it is not legal to do so (i.e., it is hypothesized that a remote ko threat is made and answered before continuation). This parameter may be set using the ‘-K’ option.

11.2 Hashing of Positions

To speed up the reading process, we note that a position can be reached in several different ways. In fact, it is a very common occurrence that a previously checked position is rechecked, often within the same search but from a different branch in the recursion tree.

This wastes a lot of computing resources, so in a number of places, we store away the current position, the function we are in, and which worm is under attack or to be defended. When the search for this position is finished, we also store away the result of the search and which move made the attack or defense succeed.

All this data is stored in a hash table, sometimes also called a transposition table, where Go positions are the key and results of the reading for certain functions and groups are the data. You can increase the size of the Hash table using the ‘-M’ or ‘--memory’ option see section Invoking GNU Go: Command line options.

The hash table is created once and for all at the beginning of the game by the function hashtable_new(). Although hash memory is thus allocated only once in the game, the table is reinitialized at the beginning of each move by a call to hashtable_clear() from genmove().

11.2.1 Calculation of the hash value

The hash algorithm is called Zobrist hashing, and is a standard technique for go and chess programming. The algorithm as used by us works as follows:

  1. First we define a go position. This positions consists of

    It is not necessary to specify the color to move (white or black) as part of the position. The reason for this is that read results are stored separately for the various reading functions such as attack3, and it is implicit in the calling function which player is to move.

  2. For each location on the board we generate random numbers:

    These random numbers are generated once at initialization time and then used throughout the life time of the hash table.

  3. The hash key for a position is the XOR of all the random numbers which are applicable for the position (white stones, black stones, and ko position).

11.2.2 Organization of the hash table

The hash table consists of 3 parts:

When the hash table is created, these 3 areas are allocated using malloc(). When the hash table is populated, all contents are taken from the Hash nodes and the Read results. No further allocation is done and when all nodes or results are used, the hash table is full. Nothing is deleted from the hash table except when it is totally emptied, at which point it can be used again as if newly initialized.

When a function wants to use the hash table, it looks up the current position using hashtable_search(). If the position doesn't already exist there, it can be entered using


Once the function has a pointer to the hash node containing a function, it can search for a result of a previous search using hashnode_search(). If a result is found, it can be used, and if not, a new result can be entered after a search using hashnode_new_result().

Hash nodes which hash to the same position in the hash table (collisions) form a simple linked list. Read results for the same position, created by different functions and different attacked or defended strings also form a linked list.

This is deemed sufficiently efficient for now, but the representation of collisions could be changed in the future. It is also not determined what the optimum sizes for the hash table, the number of positions and the number of results are.

11.2.3 Hash Structures

The basic hash structures are declared in ‘engine/hash.h’ and ‘engine/cache.c

typedef struct hashposition_t {
  Compacttype  board[COMPACT_BOARD_SIZE];
  int          ko_pos;
} Hashposition;

Represents the board and optionally the location of a ko, which is an illegal move. The player whose move is next is not recorded.

typedef struct {
  Hashvalue     hashval;
  Hashposition  hashpos;
} Hash_data;

Represents the return value of a function (hashval) and the board state (hashpos).

typedef struct read_result_t {
  unsigned int data1;	
  unsigned int data2;

  struct read_result_t *next;
} Read_result;

The data1 field packs into 32 bits the following fields:

komaster:  2 bits (EMPTY, BLACK, WHITE, or GRAY)
kom_pos : 10 bits (allows MAX_BOARD up to 31)
routine :  4 bits (currently 10 different choices)
str1    : 10 bits
stackp  :  5 bits

The data2 field packs into 32 bits the following fields:

status :   2 bits (0 free, 1 open, 2 closed)
result1:   4 bits
result2:   4 bits
move   :  10 bits
str2   :  10 bits

The komaster and (kom_pos) field are documented in See section Ko Handling.

When a new result node is created, 'status' is set to 1 'open'. This is then set to 2 'closed' when the result is entered. The main use for this is to identify open result nodes when the hashtable is partially cleared. Another potential use for this field is to identify repeated positions in the reading, in particular local double or triple kos.

typedef struct hashnode_t {
  Hash_data            key;
  Read_result        * results;
  struct hashnode_t  * next;
} Hashnode;

The hash table consists of hash nodes. Each hash node consists of The hash value for the position it holds, the position itself and the actual information which is purpose of the table from the start.

There is also a pointer to another hash node which is used when the nodes are sorted into hash buckets (see below).

typedef struct hashtable {
  size_t         hashtablesize;	/* Number of hash buckets */
  Hashnode    ** hashtable;	/* Pointer to array of hashnode lists */

  int            num_nodes;	/* Total number of hash nodes */
  Hashnode     * all_nodes;	/* Pointer to all allocated hash nodes. */
  int            free_node;	/* Index to next free node. */

  int            num_results;	/* Total number of results */
  Read_result  * all_results;	/* Pointer to all allocated results. */
  int            free_result;	/* Index to next free result. */
} Hashtable;

The hash table consists of three parts:

11.3 Persistent Reading Cache

Some calculations can be safely saved from move to move. If the opponent's move is not close to our worm or dragon, we do not have to reconsider the life or death of that group on the next move. So the result is saved in a persistent cache. Persistent caches are used for are used in the engine for several types of read results.

In this section we will discuss the persistent caching of tactical reading but the same principles apply to the other persistent caches.

Persistent caching is an important performance feature. However it can lead to mistakes and debugging problems—situations where GNU Go generates the right move during debugging but plays a wrong move during a game. If you suspect a persistent cache effect you may try loading the sgf file with the ‘--replay’ option and see if the mistake is repeated (see section Invoking GNU Go: Command line options).

The function store_persistent_cache() is called only by attack and find_defense, never from their static recursive counterparts do_attack and do_defend. The function store_persistent_reading_cache() attempts to cache the most expensive reading results. The function search_persistent_reading_cache attempts to retrieve a result from the cache.

If all cache entries are occupied, we try to replace the least useful one. This is indicated by the score field, which is initially the number of nodes expended by this particular reading, and later multiplied by the number of times it has been retrieved from the cache.

Once a (permanent) move is made, a number of cache entries immediately become invalid. These are cleaned away by the function purge_persistent_reading_cache(). To have a criterion for when a result may be purged, the function store_persistent_cache() computes the reading shadow and active area. If a permanent move is subsequently played in the active area, the cached result is invalidated. We now explain this algorithm in detail.

The reading shadow is the concatenation of all moves in all variations, as well as locations where an illegal move has been tried.

Once the read is finished, the reading shadow is expanded to the active area which may be cached. The intention is that as long as no stones are played in the active area, the cached value may safely be used.

Here is the algorithm used to compute the active area. This algorithm is in the function store_persistent_reading_cache(). The most expensive readings so far are stored in the persistent cache.

11.4 Ko Handling

The principles of ko handling are the same for tactical reading and owl reading.

We have already mentioned (see section Reading Basics) that GNU Go uses a return code of KO_A or KO_B if the result depends on ko. The return code of KO_B means that the position can be won provided the player whose move calls the function can come up with a sufficiently large ko threat. In order to verify this, the function must simulate making a ko threat and having it answered by taking the ko even if it is illegal. We call such an experimental taking of the ko a conditional ko capture.

Conditional ko captures are accomplished by the function tryko(). This function is like trymove() except that it does not require legality of the move in question.

The static reading functions, and the global functions do_attack and do_find_defense consult parameters komaster, kom_pos, which are declared static in ‘board.c’. These mediate ko captures to prevent the occurrence of infinite loops. During reading, the komaster values are pushed and popped from a stack.

Normally komaster is EMPTY but it can also be ‘BLACK’, ‘WHITE’, GRAY_BLACK, GRAY_WHITE or WEAK_KO. The komaster is set to color when color makes a conditional ko capture. In this case kom_pos is set to the location of the captured ko stone.

If the opponent is komaster, the reading functions will not try to take the ko at kom_pos. Also, the komaster is normally not allowed to take another ko. The exception is a nested ko, characterized by the condition that the captured ko stone is at distance 1 both vertically and horizontally from kom_pos, which is the location of the last stone taken by the komaster. Thus in this situation:


Here if ‘m’ is the location of kom_pos, then the move at ‘*’ is allowed.

The rationale behind this rule is that in the case where there are two kos on the board, the komaster cannot win both, and by becoming komaster he has already chosen which ko he wants to win. But in the case of a nested ko, taking one ko is a precondition to taking the other one, so we allow this.

If the komaster's opponent takes a ko, then both players have taken one ko. In this case komaster is set to GRAY_BLACK or GRAY_WHITE and after this further ko captures are even further restricted.

If the ko at kom_pos is filled, then the komaster reverts to EMPTY.

In detail, the komaster scheme is as follows. Color ‘O’ is to move. This scheme is known as scheme 5 since in versions of GNU Go through 3.4, several different schemes were included.

11.5 A Ko Example

To see the komaster scheme in action, consider this position from the file ‘regressions/games/life_and_death/tripod9.sgf’. We recommend studying this example by examining the variation file produced by the command:

  gnugo -l tripod9.sgf --decide-dragon C3 -o vars.sgf

In the lower left hand corner, there are kos at A2 and B4. Black is unconditionally dead because if W wins either ko there is nothing B can do.

 8 . . . . . . . .
 7 . . O . . . . .
 6 . . O . . . . .
 5 O O O . . . . .
 4 O . O O . . . .
 3 X O X O O O O .
 2 . X X X O . . .
 1 X O . . . . . .
   A B C D E F G H

This is how the komaster scheme sees this. B (i.e. X) starts by taking the ko at B4. W replies by taking the ko at A1. The board looks like this:

 8 . . . . . . . .
 7 . . O . . . . .
 6 . . O . . . . .
 5 O O O . . . . .
 4 O X O O . . . .
 3 X . X O O O O .
 2 O X X X O . . .
 1 . O . . . . . .
   A B C D E F G H

Now any move except the ko recapture (currently illegal) at A1 loses for B, so B retakes the ko and becomes komaster. The board looks like this:

 8 . . . . . . . .         komaster: BLACK
 7 . . O . . . . .         kom_pos: A2
 6 . . O . . . . .
 5 O O O . . . . .
 4 O X O O . . . .
 3 X . X O O O O .
 2 . X X X O . . .
 1 X O . . . . . .
   A B C D E F G H

W takes the ko at B3 after which the komaster is GRAY and ko recaptures are not allowed.

 8 . . . . . . . .         komaster: GRAY
 7 . . O . . . . .         kom_pos: B4
 6 . . O . . . . .
 5 O O O . . . . .
 4 O . O O . . . .
 3 X O X O O O O .
 2 . X X X O . . .
 1 X O . . . . . .
   A B C D E F G H

Since B is not allowed any ko recaptures, there is nothing he can do and he is found dead. Thus the komaster scheme produces the correct result.

11.6 Another Ko Example

We now consider an example to show why the komaster is reset to EMPTY if the ko is resolved in the komaster's favor. This means that the ko is filled, or else that is becomes no longer a ko and it is illegal for the komaster's opponent to play there.

The position resulting under consideration is in the file ‘regressions/games/ko5.sgf’. This is the position:

 . . . . . . O O 8
 X X X . . . O . 7
 X . X X . . O . 6
 . X . X X X O O 5
 X X . X . X O X 4
 . O X O O O X . 3
 O O X O . O X X 2
 . O . X O X X . 1
 F G H J K L M N

We recommend studying this example by examining the variation file produced by the command:

gnugo -l ko5.sgf --quiet --decide-string L1 -o vars.sgf

The correct resolution is that H1 attacks L1 unconditionally while K2 defends it with ko (code KO_A).

After Black (X) takes the ko at K3, white can do nothing but retake the ko conditionally, becoming komaster. B cannot do much, but in one variation he plays at K4 and W takes at H1. The following position results:

 . . . . . . O O 8
 X X X . . . O . 7
 X . X X . . O . 6
 . X . X X X O O 5
 X X . X X X O X 4
 . O X O O O X . 3
 O O X O . O X X 2
 . O O . O X X . 1
 F G H J K L M N

Now it is important the ‘O’ is no longer komaster. Were ‘O’ still komaster, he could capture the ko at N3 and there would be no way to finish off B.

11.7 Alternate Komaster Schemes

The following alternate schemes have been proposed. It is assumed that ‘O’ is the player about to move.

11.7.1 Essentially the 2.7.232 scheme.

11.7.2 Revised 2.7.232 version

11.8 Superstrings

A superstring is an extended string, where the extensions are through the following kinds of connections:

  1. Solid connections (just like ordinary string).
  2. Diagonal connection or one space jump through an intersection where an opponent move would be suicide or self-atari.
  3. Bamboo joint.
  4. Diagonal connection where both adjacent intersections are empty.
  5. Connection through adjacent or diagonal tactically captured stones. Connections of this type are omitted when the superstring code is called from ‘reading.c’, but included when the superstring code is called from ‘owl.c’.

Like a dragon, a superstring is an amalgamation of strings, but it is a much tighter organization of stones than a dragon, and its purpose is different. Superstrings are encountered already in the tactical reading because sometimes attacking or defending an element of the superstring is the best way to attack or defend a string. This is in contrast with dragons, which are ignored during tactical reading.

11.9 Debugging the reading code

The reading code searches for a path through the move tree to determine whether a string can be captured. We have a tool for investigating this with the ‘--decidestring’ option. This may be run with or without an output file.

Simply running

gnugo -t -l [input file name] -L [movenumber] --decidestring [location]

will run attack() to determine whether the string can be captured. If it can, it will also run find_defense() to determine whether or not it can be defended. It will give a count of the number of variations read. The ‘-t’ is necessary, or else GNU Go will not report its findings.

If we add ‘-o output file’ GNU Go will produce an output file with all variations considered. The variations are numbered in comments.

This file of variations is not very useful without a way of navigating the source code. This is provided with the GDB source file, listed at the end. You can source this from GDB, or just make it your GDB init file.

If you are using GDB to debug GNU Go you may find it less confusing to compile without optimization. The optimization sometimes changes the order in which program steps are executed. For example, to compile ‘reading.c’ without optimization, edit ‘engine/Makefile’ to remove the string -O2 from the file, touch ‘engine/reading.c’ and make. Note that the Makefile is automatically generated and may get overwritten later.

If in the course of reading you need to analyze a result where a function gets its value by returning a cached position from the hashing code, rerun the example with the hashing turned off by the command line option ‘--hash 0’. You should get the same result. (If you do not, please send us a bug report.) Don't run ‘--hash 0’ unless you have a good reason to, since it increases the number of variations.

With the source file given at the end of this document loaded, we can now navigate the variations. It is a good idea to use cgoban with a small ‘-fontHeight’, so that the variation window takes in a big picture. (You can resize the board.)

Suppose after perusing this file, we find that variation 17 is interesting and we would like to find out exactly what is going on here.

The macro 'jt n' will jump to the n-th variation.

(gdb) set args -l [filename] -L [move number] --decidestring [location]
(gdb) tbreak main
(gdb) run
(gdb) jt 17

will then jump to the location in question.

Actually the attack variations and defense variations are numbered separately. (But find_defense() is only run if attack() succeeds, so the defense variations may or may not exist.) It is redundant to have to tbreak main each time. So there are two macros avar and dvar.

(gdb) avar 17

restarts the program, and jumps to the 17-th attack variation.

(gdb) dvar 17

jumps to the 17-th defense variation. Both variation sets are found in the same sgf file, though they are numbered separately.

Other commands defined in this file:


dump will print the move stack.
nv moves to the next variation
ascii i j converts (i,j) to ascii

###############      .gdbinit file      ###############

# this command displays the stack

define dump
set dump_stack()

# display the name of the move in ascii

define ascii
set gprintf("%o%m\n",$arg0,$arg1)

# display the all information about a dragon

define dragon
set ascii_report_dragon("$arg0")

define worm
set ascii_report_worm("$arg0")

# move to the next variation

define nv
tbreak trymove

# move forward to a particular variation

define jt
while (count_variations < $arg0)

# restart, jump to a particular attack variation

define avar
tbreak sgffile_decidestring
tbreak attack
jt $arg0

# restart, jump to a particular defense variation

define dvar
tbreak sgffile_decidestring
tbreak attack
next 3
jt $arg0

11.10 Connection Reading

GNU Go does reading to determine if strings can be connected. The algorithms for this are in ‘readconnect.c’. As with the reading code, the connection code is not pattern based.

The connection code is invoked by the engine through the functions:

To see the connection code in action, you may try the following example.

gnugo --quiet -l connection3.sgf --decide-connection M3/N7 -o vars.sgf

(The file ‘connection3.sgf’ is in ‘regression/games’.) Examine the sgf file produced by this to see what kind of reading is done by the functions string_connect() and string_disconnect(), which are called by the function decide_connection.

One use of the connection code is used is through the autohelper macros oplay_connect, xplay_connect, oplay_disconnect and xplay_disconnect which are used in the connection databases.

[ << ] [ >> ]           [Top] [Contents] [Index] [ ? ]

This document was generated by Daniel Bump on February, 19 2009 using texi2html 1.78.