Synchronizing Goroutines with Waitgroups

Disclaimer

Introduction

In this article we present, with several examples, the concepts of concurrent execution of goroutines in GO. We show how concurrent execution can fail if synchronization mechanisms are not used.

A Serial Function

  • Program function.go illustrates a serial function called myFunction.
/* function.go: a serial function*/

package main
import ("fmt"; "math/rand"; "os"; "strconv")

func myFunction(id int) {
   fmt.Printf("\nmyFunction %2d:",id)
   var sum float64
   for i := 0; i < rand.Intn(100000); i++ {
      sum+=rand.Float64()
   }
   fmt.Printf("\ndone %7.3e id %2d",sum,id)
}

func main() {
   if len(os.Args) < 2 {
      fmt.Printf("usage <filename> seed\n")
      return
   }
   seed, _  := strconv.Atoi(os.Args[1])
   rand.Seed(int64(seed))
   fmt.Printf("\n--start main program--")
   for id := 0; id < 20; id++ {
      myFunction(id)
   }
   fmt.Printf("\n--end main program--\n")
}
  • This function is passed an integer, which is the id of the function.
  • It computes the sum of 10000 random float64 numbers in the range [0, 1), using the function rand.Float64() and prints out the sum.
  • To use the rand function, we must import the package math/rand.
  • rand is a pseudo-random number generator. It will generate a different sequence depending on the seed (set using statement rand.Seed(int64(seed)), on line 21). The sequence will always be the same for a given seed.
  • The seed is entered on the command line. If we wanted the seed to be 3, we would execute: go run function.go 3
  • The seed is set using seed, _ := strconv.Atoi(os.Args[1]) [line 20]. This converts the string in position 1 of the command line into an int. The second value returned is an error code, which we presently ignore using the underscore symbol _
  • When seeding the random number generator we need to convert the seed from int to int64 thus: int64(seed) because rand.Seed expects an int64 argument.
  • In the fmt.Printf statement, the newline \n is placed in the beginning of the string. This is useful in experimenting with parallel and concurrent programs.
  • When we run this serial program we get the output:
$ go run function.go 3

--start main program--
myFunction  0:
done 5.027e+03 id  0
myFunction  1:
done 5.000e+03 id  1
myFunction  2:
done 5.033e+03 id  2
myFunction  3:
...
myFunction 17:
done 4.972e+03 id 17
myFunction 18:
done 4.948e+03 id 18
myFunction 19:
done 4.957e+03 id 19
--end main program--

Launching Multiple Goroutines

  • The only difference between goroutine1.go and the preceding function.go is the keyword go, before myFunction. The go creates a concurrent subroutine.
  • This causes the GO system to launch 20 myFunction()s concurrently.
/* goroutine1.go
   concurrent functions */
   
package main
import ("fmt"; "math/rand"; "os"; "strconv")

func myFunction(id int) {
   fmt.Printf("\nmyFunction %2d:",id)
   var sum float64 
   for i := 0; i < 10000; i++ {
      sum+=rand.Float64()
   }
   fmt.Printf("\ndone %7.3e id %2d",sum,id)
}

func main() {
   if len(os.Args) < 2 {
      fmt.Printf("usage <filename> seed\n")
      return
   }
   seed, _  := strconv.Atoi(os.Args[1])
   rand.Seed(int64(seed))
   fmt.Printf("\n--start main program--")
   for id := 0; id < 20; id++ {
      go myFunction(id)
   }
   fmt.Printf("\n--end main program--\n")
}
  • As shown in the following diagram, in function.go the main program executes myFunction(0. . . 19) serially and waits until myFunction(19) has completed before exiting (bold red line).
  • In goroutine1.go the main program fires off go myFunction(0. . . 19) concurrently. These goroutines take arbitrary times–the main program does not wait for them to complete (bold dashed red line).

Demonstrates the difference between a routine being executed serially multiple times vs multiple goroutines executing concurrently.
  • If you copy and paste goroutine1.go into your personal machine and run it repeatedly you will get odd results. Here are two examples:
go run goroutine1.go 3

--start main program--
myFunction  0:
myFunction  3:
myFunction  1:
myFunction  8:
myFunction  7:
myFunction 11:
myFunction 17:
myFunction 14:
myFunction 13:
myFunction  9:
myFunction 10:
--end main program--

In the above case we note that not all myFunctions() complete before the main program ends.

Here is another run:

$ go run goroutine1.go 3

--start main program--
--end main program--

Here the main program starts and ends without giving any of the myFunctions() a chance to complete.

Sleeping on the job (goroutine1a.go)

  • We can attempt to remedy the problems with goroutine1.go by making the main program sleep for a period of time after the goroutines have been launched, to give them all enough time to finish.
  • This can be done using the Sleep function from package time.
/* goroutine1a.go
   concurrent functions with sleep in main program */
   
package main
import ("fmt"; "math/rand"; "os"; "strconv"; "time")

func myFunction(id int) {
   fmt.Printf("\nmyFunction %2d:",id)
   var sum float64 
   for i := 0; i < 10000; i++ {
      sum+=rand.Float64()
   }
   fmt.Printf("\ndone %7.3e id %2d",sum,id)
}

func main() {
   if len(os.Args) < 3 {
      fmt.Printf("usage <filename> seed delay\n")
      return
   }
   seed, _  := strconv.Atoi(os.Args[1])
   delay, _ := strconv.Atoi(os.Args[2])
   rand.Seed(int64(seed))
   fmt.Printf("\n--start main program--")
   for id := 0; id < 20; id++ {
      go myFunction(id)
   }
   time.Sleep(time.Duration(delay)*time.Microsecond)
   fmt.Printf("\n--end main program--\n")
}
  • time.Sleep(. . . ) causes the main program to sleep for the specified time.
  • delay is entered as the second argument on the command line. It has to be converted from int to a variable of type time.Duration because (1) Sleep expects an argument of type time.Duration, (2) time.Microsecond is of type time duration, and (3) GO does not allow arithmetic operations on dissimilar types.
Using sleep in the main program to allow all goroutines to complete is not a good solution because the main program is idle and wasting resources while it is sleeping, and there is no guarantee that every goroutine will always take the same amount of time, from run to run.
  • Sleep is not a good solution because (1) we have to estimate the time for all goroutines to complete, (2) the main program is idle and wasting resources while it is sleeping, and (3) there is no guarantee that every goroutine will always take the same amount of time, from run to run.

Wait Groups for Synchronization

  • The correct solution (in goroutine2.go) is to use a synchronization variable of type WaitGroup from the sync package.
/* goroutine2.go
   concurrent functions with synchronization */
   
package main
import ("fmt"; "sync"; "math/rand"; "os"; "strconv")

func myFunction(id int, numFuncs *sync.WaitGroup) {
   fmt.Printf("\nmyFunction %2d:",id)
   var sum float64 
   for i := 0; i < 10000; i++ {
      sum+=rand.Float64()
   }
   fmt.Printf("\ndone %7.3e id %2d",sum,id)
   numFuncs.Done()
}

func main() {
   if len(os.Args) < 2 {
      fmt.Printf("usage <filename> seed\n")
      return
   }
   seed, _ := strconv.Atoi(os.Args[1])
   rand.Seed(int64(seed))
   var numFuncs sync.WaitGroup
   fmt.Printf("\n--start main program--")
   numFuncs.Add(20)
   for i := 0; i < 20; i++ {
      go myFunction(i, &numFuncs)
   }
   numFuncs.Wait()
   fmt.Printf("\n--end main program--\n")
}
  • We declare a variable numFuncs of type sync.WaitGroup, and set it to 20 (no. of goroutines).
  • Whenever a goroutine completes, it decrements this variable using numFuncs.Done().
  • The main program waits for numFuncs to become 0 using numFuncs.Wait(). When that happens it proceeds with the next statement, which happens to be the last statement in the program.
Using a waitGroup ensures that all goroutines terminate before the main program terminates.
  • This solution will work correctly regardless of the number of routines, the execution time of each routine, and the seed passed to the random number generator.
  • It is instructive to try this program out yourself.
  • Here is an example:
--start main program--
myFunction 19:
myFunction  9:
myFunction  3:
myFunction  7:
myFunction  4:
myFunction  8:
myFunction 11:
myFunction 15:
myFunction  5:
myFunction  2:
myFunction 13:
myFunction 18:
myFunction 12:
myFunction 10:
myFunction 14:
myFunction  1:
myFunction 17:
myFunction 16:
myFunction  6:
myFunction  0:
done 5.030e+03 id 12
done 5.011e+03 id 13
done 5.031e+03 id  7
done 5.030e+03 id  9
done 5.049e+03 id 19
done 4.999e+03 id  2
done 5.064e+03 id 15
done 4.967e+03 id  4
done 4.935e+03 id  6
done 4.973e+03 id  8
done 5.027e+03 id 10
done 4.987e+03 id 16
done 5.032e+03 id 17
done 5.050e+03 id  3
done 4.991e+03 id  0
done 5.007e+03 id  1
done 4.957e+03 id  5
done 5.006e+03 id 18
done 5.008e+03 id 14
done 4.999e+03 id 11
--end main program--
  • Note that each function starts after the start of the main program.
  • Each function ends before the end of the main program.
  • All 20 functions start
  • All 20 functions end
  • The functions do not start or end in numerical order: this is an important property of concurrent and parallel programs.
GO home
Next