Detecting Goroutine Leaks Like Memory Leaks? goroutineleak Helps Us Pinpoint Goroutine Leaks Faster
Spotting Stuck Goroutines Before They Sink Your Service: Using Go’s Experimental goroutineleak + pprof to Treat Goroutine Leaks Like Memory…
Spotting Stuck Goroutines Before They Sink Your Service: Using Go’s Experimental goroutineleak + pprof to Treat Goroutine Leaks Like Memory Leaks
Introduction
In Go development, goroutine leaks are a common but subtle problem. When a goroutine is blocked on some synchronization primitive (such as a channel or mutex), and that primitive becomes permanently unreachable, the goroutine is effectively “leaked.”
Each leaked goroutine occupies at least 2 KB of stack memory. Over time, this can cause:
- Steadily increasing memory usage
- More pressure on the scheduler
- Eventually, OOM crashes or noticeable service performance degradation
Previously, developers typically relied on manual code review or third-party tools (such as goleak) to detect goroutine leaks—both low in efficiency and easy to miss issues.
But Go’s experimental feature goroutineleak changes that.
It integrates goroutine leak detection directly into the garbage collector (GC), so we can locate leaking goroutines via pprof, almost as easily as we detect memory leaks.
How GC Detects Goroutine Leaks
1) Lifecycle characteristics of leaked goroutines
A normal goroutine goes through a complete lifecycle: create → run → exit.
A leaked goroutine, on the other hand, gets stuck in a blocked state, and the synchronization primitive it’s blocked on has been deemed unreachable by GC.
2) sudog: the bridge between goroutines and synchronization primitives
The Go runtime tracks goroutines blocked on synchronization primitives via the sudog struct. Each sudog contains:
- A pointer to the blocked goroutine
- A pointer to the synchronization primitive (channel, mutex, etc.)
- Linked-list information for the wait queue
3) GC-based leak detection workflow
goroutineleak extends the GC to perform leak detection. The core workflow looks like this:
Key technical points:
- Special GC cycle: a GC cycle is triggered with additional leak detection logic
- Unreachability check: identify all goroutines blocked on unreachable synchronization primitives
- State marking: mark leaked goroutines with the
_Gleakedstate - Result exposure: expose results via the
/debug/pprof/goroutineleakendpoint inpprof
3. Usage: As Simple as pprof
1) Environment setup: install gotip
Since goroutineleak is currently under development and not available in a stable Go release, we can try it out in advance using gotip:
go install golang.org/dl/gotip@latest
gotip download2) Running a leak detection demo
Let’s write a small demo that intentionally creates a goroutine leak:
func main() {
// Start pprof server to expose goroutineleak profile
go func() {
log.Printf("pprof server started at http://localhost:6060")
if err := http.ListenAndServe(":6060", nil); err != nil {
log.Fatalf("failed to start pprof server: %v", err)
}
}()
// Create a goroutine leak
createLeakedGoroutine()
// Keep the program running to allow pprof inspection
fmt.Println("Demo program running...")
fmt.Println("Use this command to check for leaks:")
fmt.Println(" GOEXPERIMENT=goroutineleakprofile gotip tool pprof http://localhost:6060/debug/pprof/goroutineleak")
fmt.Println("Press Ctrl+C to exit")
// Wait for interrupt signal to gracefully exit
sigCh := make(chan os.Signal, 1)
signal.Notify(sigCh, syscall.SIGINT, syscall.SIGTERM)
<-sigCh
log.Println("Shutting down demo program...")
}
// createLeakedGoroutine intentionally creates a leaked goroutine
func createLeakedGoroutine() {
// Create a channel but never write to it or close it
ch := make(chan int)
// Start a goroutine that blocks forever on channel receive
go func() {
fmt.Println("Leaked goroutine started - waiting for channel data")
<-ch // This goroutine will never return
fmt.Println("Leaked goroutine should never reach this line")
}()
// The channel 'ch' is not accessible after this function returns,
// so both the channel and the goroutine are leaked
fmt.Println("Created a leaked goroutine - channel is now unreachable")
}Then run:
# Start the program with leak detection enabled
GOEXPERIMENT=goroutineleakprofile gotip run main.go3) Using pprof to locate leaks
In another terminal, run:
# Use the dedicated leak detection profile
GOEXPERIMENT=goroutineleakprofile gotip tool pprof http://localhost:6060/debug/pprof/goroutineleak4) Analyzing pprof results
# View leaked goroutines
(pprof) top
Showing nodes accounting for 1, 100% of 1 total
flat flat% sum% cum cum%
1 100% 100% 1 100% runtime.gopark
0 0% 100% 1 100% main.createLeakedGoroutine.func1
0 0% 100% 1 100% runtime.chanrecv
0 0% 100% 1 100% runtime.chanrecv1
# View the exact leaking code location
(pprof) list main.createLeakedGoroutine.func1
Total: 1
ROUTINE ======================== main.createLeakedGoroutine.func1 in /path/to/goroutineleak_example/main.go
1 1 (flat, cum) 100% of Total
. . 27:func createLeakedGoroutine() {
. . 28: // Create a channel but never write to it or close it
. . 29: ch := make(chan int)
. . 30:
. . 31: // Start a goroutine that blocks forever on channel receive
. . 32: go func() {
. . 33: fmt.Println("Leaked goroutine started - waiting for channel data")
1 1 34: <-ch // This goroutine will never return
. . 35: fmt.Println("Leaked goroutine should never reach this line")
. . 36: }()
. . 37:
. . 38: // The channel 'ch' is not accessible after this function returns,
. . 39: // so both the channel and the goroutine are leaked
. . 40: fmt.Println("Created a leaked goroutine - channel is now unreachable")
. . 41:}Pretty convenient?
4. What This Means for Developers
1) A unified debugging experience
Developers can reuse the pprof tooling they already know to detect goroutine leaks—no need to learn new tools or modify application code.
2) Higher debugging efficiency
- Automatically pinpoint the exact code location of leaks
- No need to manually track each goroutine’s lifecycle
- Combine with stack traces to quickly understand the root cause
3) From “reactive firefighting” to “proactive prevention”
- Integrate into CI/CD during development
- Periodically check in production to catch issues early
- Reduce performance degradation and outages caused by leaks
4) Deeper understanding of the Go runtime
By using goroutineleak, developers can better understand:
- How the Go runtime manages goroutines
- How synchronization primitives interact with goroutines
- The role of GC in resource management
Conclusion
The experimental goroutineleak feature gives Go developers an efficient, built-in way to detect goroutine leaks, turning a previously hidden problem into something as straightforward to inspect as memory leaks. Although it’s still experimental, it showcases the Go team’s ongoing effort to improve developer experience and service reliability.
For high-performance, highly reliable Go applications, goroutineleak is well worth trying—it helps you find problems faster and write more robust code.
Note: Right now, goroutineleak can only detect leaked goroutines; it cannot automatically free them. You still need to fix the underlying issues in your code (e.g., closing channels, ensuring synchronization primitives remain reachable, etc.) based on the detection results.
