Library demo - multi-threaded operation (GNU Astronomy Utilities)

12.4.3 Library demo - multi-threaded operation

The following simple program shows how to use Gnuastro to simplify spinning off threads and distributing different jobs between the threads. The relevant thread-related functions are defined in Gnuastro’s thread related functions. For easy linking/compilation of this program, along with a first run, see Gnuastro’s BuildProgram. Before running, also change the filename and hdu variable values to specify an existing FITS file and/or extension/HDU.

This is a very simple program to open a FITS image, distribute its pixels between different threads and print the value of each pixel and the thread it was assigned to. The actual operation is very simple (and would not usually be done with threads in a real-life program). It is intentionally chosen to put more focus on the important steps in spinning of threads and how the worker function (which is called by each thread) can identify the job-IDs it should work on.

For example, instead of an array of pixels, you can define an array of tiles or any other context-specific structures as separate targets. The important thing is that each action should have its own unique ID (counting from zero, as is done in an array in C). You can then follow the process below and use each thread to work on all the targets that are assigned to it. Recall that spinning-off threads is itself an expensive process and we do not want to spin-off one thread for each target (see the description of gal_threads_dist_in_threads in Gnuastro’s thread related functions.

There are many (more complicated, real-world) examples of using gal_threads_spin_off in Gnuastro’s actual source code, you can see them by searching for the gal_threads_spin_off function from the top source (after unpacking the tarball) directory (for example, with this command):

$ grep -r gal_threads_spin_off ./

The code of this demonstration program is shown below. This program was also built and run when you ran make check during the building of Gnuastro (tests/lib/multithread.c), so it is already tested for your system and you can safely use it as a guide.

#include <stdio.h>
#include <stdlib.h>

#include <gnuastro/fits.h>
#include <gnuastro/threads.h>


/* This structure can keep all information you want to pass onto the
 * worker function on each thread. */
struct params
{
  gal_data_t *image;            /* Dataset to print values of. */
};




/* This is the main worker function which will be called by the
 * different threads. `gal_threads_params' is defined in
 * `gnuastro/threads.h' and contains the pointer to the parameter we
 * want. Note that the input argument and returned value of this
 * function always must have `void *' type. */
void *
worker_on_thread(void *in_prm)
{
  /* Low-level definitions to be done first. */
  struct gal_threads_params *tprm=(struct gal_threads_params *)in_prm;
  struct params *p=(struct params *)tprm->params;


  /* Subsequent definitions. */
  float *array=p->image->array;
  size_t i, index, *dsize=p->image->dsize;


  /* Go over all the actions (pixels in this case) that were assigned
   * to this thread. */
  for(i=0; tprm->indexs[i] != GAL_BLANK_SIZE_T; ++i)
    {
      /* For easy reading. */
      index = tprm->indexs[i];


      /* Print the information. */
      printf("(%zu, %zu) on thread %zu: %g\n", index%dsize[1]+1,
             index/dsize[1]+1, tprm->id, array[index]);
    }


  /* Wait for all the other threads to finish, then return. */
  if(tprm->b) pthread_barrier_wait(tprm->b);
  return NULL;
}




/* High-level function (called by the operating system). */
int
main(void)
{
  struct params p;
  char *filename="input.fits", *hdu="1";
  size_t numthreads=gal_threads_number();

  /* We are using * `-1' for `minmapsize' to ensure that the image is
     read into * memory and `1' for `quietmmap' (which can also be
     zero), see the "Memory management" section in the book. */
  int quietmmap=1;
  size_t minmapsize=-1;


  /* Read the image into memory as a float32 data type. */
  p.image=gal_fits_img_read_to_type(filename, hdu, GAL_TYPE_FLOAT32,
                                    minmapsize, quietmmap);


  /* Print some basic information before the actual contents: */
  printf("Pixel values of %s (HDU: %s) on %zu threads.\n", filename,
         hdu, numthreads);
  printf("Used to check the compiled library's capability in opening "
         "a FITS file, and also spinning-off threads.\n");


  /* A small sanity check: this is only intended for 2D arrays (to
   * print the coordinates of each pixel). */
  if(p.image->ndim!=2)
    {
      fprintf(stderr, "only 2D images are supported.");
      exit(EXIT_FAILURE);
    }


  /* Spin-off the threads and do the processing on each thread. */
  gal_threads_spin_off(worker_on_thread, &p, p.image->size, numthreads,
                       minmapsize, quietmmap);


  /* Clean up and return. */
  gal_data_free(p.image);
  return EXIT_SUCCESS;
}