Decryption go: goroutine state switching

Note: Non-members can read the full story in this link.

In Go, we use go to create a new goroutine. A goroutine is a lightweight thread managed by the Go runtime. We are all familiar with the states of operating system threads. In The time in computers: how long will it take to switch the context? we summarized the states of threads and the time it takes for thread scheduling.

Goroutines are similar; they have their states, and the runtime controls them.

The data structure of goroutines is defined in runtime2.go. The g.atomicstatus represents the state of a goroutine. Its range of values is also defined in the source code. Apart from several unused states and states related to GC, a goroutine may be in one of the following nine states:

StateDescription_GidleJust allocated and not yet initialized_GrunnableNo code execution, no ownership of stack, stored in the run queue_GrunningCan execute code, owns a stack, assigned to kernel thread M and processor P_GsyscallExecuting a system call owns a stack, not executing user code, assigned to kernel thread M but not in the run queue_GwaitingBlocked due to runtime, not executing user code, and not in the run queue, but may be in the waiting queue of a channel_GdeadNot in use, no code execution, may have allocated stack_GcopystackStack is being copied, no code execution, not in the run queue_GpreemptedBlocked due to preemption, not executing user code and not in the run queue, waiting to be woken up_GscanGC is scanning the stack space, with no code execution, and can coexist with other states

Among these states, the more common ones are _Grunnable, _Grunning, _Gsyscall, _Gwaiting, and _Gpreempted. We will focus on these states here. Goroutine state transitions are a complex process, and there are many methods to trigger goroutine state transitions. We cannot cover all transition routes here but will select some for discussion.

Although goroutine states defined during runtime are numerous and complex, we can aggregate these different states into three categories: Waiting, Runnable, and Running. During runtime, goroutines switch among these three states:

• Waiting: Goroutine is waiting for certain conditions to be met, such as the end of a system call. This includes states like _Gwaiting, _Gsyscall, and _Gpreempted.
• Runnable: Goroutine is ready to run and can be scheduled for execution. If there are many goroutines in the program, each goroutine may wait for more time. This corresponds to _Grunnable. At this point, the goroutine is in the local queue of P or the global queue.
• Running: Goroutine is running on a thread, corresponding to _Grunning.

Grunnable

A goroutine enters the Grunnable state under the following circumstances:

Goroutine Creation

In Go, including the main entry main in the user program, all goroutines are created through runtime.newproc -> runtime.newproc1. The former is a wrapper for the latter. The go keyword ultimately translates to a call to runtime.newproc. When runtime.newproc1 completes resource allocation and initialization, the new task's state is set to Grunnable and then added to the current P's local task queue.

func newproc1(fn *funcval, argp unsafe.Pointer, narg int32, callergp *g, callerpc uintptr) { 
   // --snip-- 
   // Get the current P from which to create a new G (newg). 
   _p_ := _g_.m.p.ptr() 
   newg := gfget(_p_) 
   // --snip-- 
   // Set the Goroutine state to Grunnable. 
   casgstatus(newg, _Gdead, _Grunnable) 
   // --snip-- 
   // Add the newly created G to the run queue. 
   runqput(_p_, newg, true) 
   // --snip-- 
 }

Wakeup of Blocked Tasks

When a blocked task (Gwaiting) is awakened due to certain conditions being met (such as writing data to a channel, which wakes up a task waiting to receive), the state of the waiting task (g1) is transitioned back to Grunnable and added to the task queue by calling runtime.ready. There is a more detailed explanation of goroutine blocking.

func ready(gp *g, traceskip int, next bool) { 
   // --snip-- 
   // Get the current g. 
   _g_ := getg() 
   // Transition the state from Gwaiting to Grunnable. 
   casgstatus(gp, _Gwaiting, _Grunnable) 
   // Add to the run queue. 
   runqput(_g_.m.p.ptr(), gp, next) 
   // --snip-- 
 }

Others

Another path is transitioning from Grunning and Gsyscall states to Grunnable, which will be discussed later. In short, a task in the Grunnable state must be in a task queue and ready to be scheduled for execution.

Grunning

All tasks in the Grunnable state may be retrieved by the scheduler (P&M) through the findrunnable function. Subsequently, their state is transitioned to Grunning, and finally, runtime.gogo is called to load the context and execute.

// One round of scheduler: find a runnable goroutine and execute it. 
 // Never returns. 
 func schedule() { 
   // --snip-- 
   // Pick a runnable g and execute. 
   if gp == nil { 
     gp, inheritTime = findrunnable() // blocks until work is available 
   } 
   // --snip-- 
   execute(gp, inheritTime) 
 }

// Schedules gp to 
 ​ 
  run on the current M. 
 func execute(gp *g, inheritTime bool) { 
   // Switch the current M to the new g. 
   _g_ := getg() 
   _g_.m.curg = gp 
   gp.m = _g_.m 
   // Transition the Grunnable state to Grunning. 
   casgstatus(gp, _Grunnable, _Grunning) 
   // --snip-- 
   // Execute the goroutine. 
   gogo(&gp.sched) 
 }

Go adopts a cooperative scheduling scheme. A running task needs to explicitly yield the processor.

After Go 1.2, the runtime also supports a certain degree of task preemption. When the system thread sysmon detects that a task is taking too long to execute or the runtime determines that garbage collection is necessary, the task is marked as "preemptible". Upon the next function call of the task, it yields the processor and switches back to the Grunnable state.

Gsyscall

To ensure high concurrency performance, the Go runtime first sets its state to Gsyscall before executing OS system calls using the runtime.entersyscall function (if the system call is blocking or takes too long to execute, the current M is detached from P). Upon return from the system call, the thread calls runtime.exitsyscall to attempt to regain P. If successful and the current task has not been preempted, its state transitions back to Grunning for continued execution. Otherwise, it is set to Grunnable and waits to be rescheduled for execution.

func reentersyscall(pc, sp uintptr) { 
   _g_ := getg() 
   // --snip-- 
   casgstatus(_g_, _Grunning, _Gsyscall) 
   // --snip-- 
   // Detach m and p. 
   pp := _g_.m.p.ptr() 
   pp.m = 0 
   _g_.m.oldp.set(pp) 
   _g_.m.p = 0 
   // --snip-- 
 }

func exitsyscall() { 
   _g_ := getg() 
   // --snip-- 
   // If P still exists, attempt to regain P. 
   if exitsyscallfast(oldp) { 
     // --snip-- 
     casgstatus(_g_, _Gsyscall, _Grunning) 
     // --snip-- 
     return 
   } 
   // --snip-- 
   // If P does not exist, Gsyscall -> Grunnable. 
   mcall(exitsyscall0) 
   // --snip-- 
 }

Gwaiting

When a task’s required resource or running condition cannot be met, it needs to call the runtime.park function to enter this state. Subsequently, unless the waiting condition is met, the task will remain in the waiting state and cannot be executed. Apart from the example of channels mentioned earlier, Go's timers, network IO operations, atomics, and semaphores can all cause tasks to be blocked.

// park continuation on g0. 
 func park_m(gp *g) { 
   // --snip-- 
   casgstatus(gp, _Grunning, _Gwaiting) 
   // --snip-- 
 }

func gopark(unlockf func(*g, unsafe.Pointer) bool, lock unsafe.Pointer, reason waitReason, traceEv byte, traceskip int) { 
   // --snip-- 
   mcall(park_m) 
 }

In runtime.park, lock is the lock that the goroutine needs to release when it is blocked (such as in channels), and reason is the reason for blocking, which facilitates debugging with gdb. When all tasks are in the Gwaiting state, it means that the current program has entered a deadlock. In this case, the runtime detects this situation and outputs the backtrace information of all Gwaiting tasks.

Gdead

When a task completes execution, it calls runtime.goexit to end. Its state is set to Gdead, and it enters the gFree list of the current P.

Conclusion

The state transitions of goroutines are similar to those of thread state transitions, but they appear more complex due to garbage collection reasons. However, if we remove the GC part, the state transitions of goroutines are similar to those of thread state transitions.

In the next article, I will summarize and analyze the state transitions of P.

If you like my articles, consider to :

1. Drop me a follow -> huizhou92
2. Leave a clap👏 (50 👏👏👏👏would be the best) and a comment if you want to interact with me.
3. Receive an email every time I post on Medium -> Click here!
4. If you find my article helpful to you, please buy me a cup of coffee