Common Causes of Memory Leaks in Go: How to Avoid Them

Memory leaks are a common issue regardless of the programming language used. However, it’s relatively complex to write memory-leaking code in Go. This article will illustrate several scenarios where memory leaks may occur, allowing us to learn how to avoid them by studying these anti-patterns.

This article was first published in the Medium MPP plan. If you’re a Medium user, please follow me on Medium. Thank you!

Resource Leaks

Not Closing Opened Files

When you finish an open file, you should always call its Close method. Failing to do so may result in reaching the maximum number of file descriptors, preventing you from opening new files or connections. This can lead to a “too many open files” error, as shown below:

Code Example 1: Not closing files leads to file descriptor exhaustion.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13


func main() {  
    files := make([]*os.File, 0)  
    for i := 0; ; i++ {  
       file, err := os.OpenFile("test.log", os.O_CREATE|os.O_APPEND|os.O_WRONLY, 0644)  
       if err != nil {  
          fmt.Printf("Error at file %d: %v\n", i, err)  
          break  
       } else {  
          _, _ = file.Write([]byte("Hello, World!"))  
          files = append(files, file)  
       }  
    }  
}

Error Output:

1

Error at file 61437: open test.log: too many open files

On my Mac, the maximum number of file handles a process can open is 61,440. Go processes typically open three file descriptors (stderr, stdout, stdin), leaving a maximum of 61,437 open files. You can manually adjust this limit.

Not Closing `http.Response.Body`

Go has a well-known bug where forgetting to close the body of an HTTP request can lead to memory leaks. For example:

Code Example 2: Not closing the HTTP body causes a memory leak.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13


func makeRequest() {  
    client := &http.Client{}  
    req, err := http.NewRequest(http.MethodGet, "http://localhost:8081", nil)  
    res, err := client.Do(req)  
    if err != nil {  
       fmt.Println(err)  
    }  
    _, err = ioutil.ReadAll(res.Body)  
    // defer res.Body.Close()  
    if err != nil {  
       fmt.Println(err)  
    }  
}

For more details on this issue, you can refer to:

String and Slice Memory Leaks

The Go spec does not explicitly state whether substrings share memory with their original string. However, the compiler allows this behavior, which is generally good as it reduces memory and CPU usage. But this can sometimes lead to temporary memory leaks.

Code Example 3: Memory leak caused by strings.
https://gist.github.com/hxzhouh/e09587195e2d7aa2d5f6676777c6cb16

To prevent temporary memory leaks, we can use strings.Clone().

Code Example 4: Using strings.Clone() to avoid temporary memory leaks.

1
2
3
4
5
6


func Demo1() {  
    for i := 0; i < 10; i++ {  
       s := createStringWithLengthOnHeap(1 << 20) // 1MB  
       packageStr1 = append(packageStr1, strings.Clone(s[:50]))  
    }  
}

Goroutine Leaks

Goroutine Handler

Most memory leaks are due to goroutine leaks. For instance, the example below can quickly exhaust memory, resulting in an OOM (Out of Memory) error.

Code Example 5: goroutine handler leak.

1
2
3
4
5
6


for {  
       go func() {  
          time.Sleep(1 * time.Hour)  
       }()  
    }  
}

Misusing Channels

Incorrect usage of channels can also easily lead to goroutine leaks. For unbuffered channels, both the producer and consumer must be ready before writing to the channel, or it will block. In the example below, the function exits early, causing a goroutine leak.

Code Example 6: Unbuffered channel misuse causes a goroutine leak.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13


func Example() {
    a := 1
    c := make(chan error)
    go func() {
        c <- err
        return
    }()
    // Example exits here, causing a goroutine leak.
    if a > 0 {
        return
    }
    err := <-c
}

Simply changing it to a buffered channel can solve this issue: c := make(chan error, 1).

Misusing `range` with Channels

Channels can be iterated using range. However, if the channel is empty,the range will wait for new data, potentially blocking the goroutine.

Code Example 7: Misusing range causes a goroutine leak.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20


func main() {  
    wg := &sync.WaitGroup{}  
    c := make(chan any, 1)  
    items := []int{1, 2, 3, 4, 5}  
    for _, i := range items {  
       wg.Add(1)  
       go func() {  
          c <- i  
       }()  
    }  
    go func() {  
       for data := range c {  
          fmt.Println(data)  
          wg.Done()  
       }  
       fmt.Println("close")  
    }()  
    wg.Wait()  
    time.Sleep(1 * time.Second)  
}

To fix this, close the channel after calling wg.Wait().

Misusing `runtime.SetFinalizer`

If two objects are both set with runtime.SetFinalizer and they reference each other, they will not be garbage collected, even if they’re no longer in use. For more information, you can refer to my other article.

`time.Ticker`

This was an issue before Go 1.23. If ticker.Stop() is not called, it could cause a memory leak. As of Go 1.23, this issue has been fixed. More details here.

Misusing `defer`

Although using defer to release resources does not directly cause memory leaks, it can lead to temporary memory leaks in two ways:

Execution time: defer is always executed when the function ends. If your function runs for too long or never ends, the resources in defer may not be released in time.
Memory usage: Each defer adds a call point in memory. If used inside a loop, this may cause a temporary memory leak.

Code Example 8: Temporary memory leak caused by defer.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11


func ReadFile(files []string) {  
    for _, file := range files {  
	  f, err := os.Open(file)  
	  if err != nil {  
		 fmt.Println(err)  
		 return  
	  }  
	  // do something  
	  defer f.Close()  
    }  
}

This code causes a temporary memory leak and can lead to a “too many open files” error. Avoid using defer excessively unless necessary.

Conclusion

This article covered several behaviors in Go that can lead to memory leaks, with goroutine leaks being the most common. The improper use of channels, especially with select and range, can make detecting leaks more difficult. When faced with memory leaks, pprof can help quickly locate the problem, ensuring you write more robust code.

References

https://go101.org/article/memory-leaking.html