Go 1.23: new Iter package

Last week, Go 1 .23 entered the freeze period, meaning no new features will be added, and any already added features are unlikely to be removed. This is a great opportunity to preview the upcoming changes. In this article, Let’s learn about the new iter package.

This article is first published in the medium MPP plan. If you are a medium user, please follow me in medium. Thank you very much.

In Go 1.22, the range over func experimental feature was introduced, but it needed to be enabled by the parameter GOEXPERIMENT=rangefunc. In Go 1.23, this kind of iteration can be directly implemented with code.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14


func Backward(s []string) func(yield func(string) bool) {  
   return func(yield func(string) bool) {  
     for i := len(s) - 1; i >= 0; i-- {  
       yield(strings.ToUpper(s[i]))  
     }  
   }  
 }  
 ​  
 func ToUpperByIter() {  
   sl := []string{"hello", "world", "golang"}  
   for v := range Backward(sl) {  
     // do business   
   }  
 }

yield is a conventional name for callable functions passed into iterators.

Now, let’s consider how we would write the code to achieve the same functionality without using the iter package:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15


func Convert[S any, D any](src []S, mapFn func(s S) D) []D {  
     r := make([]D, 0, len(src))  
     for _, i := range src {  
        r = append(r, mapFn(i))  
     }  
     return r  
 }    
     
 func ToUpByString() {  
     sl := []string{"hello", "world", "golang"}  
     s0 := Convert(sl, func(v string) string { return strings.ToUpper(v) })  
     for _, v := range s0 {  
        // do business  
     }  
 }

Performance Comparison

Let’s compare the performance of the two methods:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13


➜  huizhou92 git:(master) ✗ go test -bench . -count=3  
 goos: darwin  
 goarch: arm64  
 pkg: huizhou92  
 cpu: Apple M1 Pro  
 BenchmarkToUpByString-10         8568332               128.7 ns/op  
 BenchmarkToUpByString-10         9310351               128.6 ns/op  
 BenchmarkToUpByString-10         9344986               128.5 ns/op  
 BenchmarkToUpByIter-10          12440120                96.22 ns/op  
 BenchmarkToUpByIter-10          12436645                96.25 ns/op  
 BenchmarkToUpByIter-10          12371175                96.64 ns/op  
 PASS  
 ok      huizhou92       8.162s

The result is clear: ToUpperByIter performs better because it doesn’t reallocate a new slice, making it more efficient than the previous method.

The goal of `iter`

The iter package aims to provide a unified and efficient iteration method. It offers a standard iteration interface for custom container classes (especially after the introduction of generics) and can replace some existing APIs that return slices. By using iterators and leveraging compiler optimization, performance can be improved. Additionally, it provides a standard iteration mechanism suitable for functional programming styles.

The Use of `iter`

iter supports two types of iterators:

1
2
3
4
5
6
7
8
9


// Seq is an iterator over sequences of individual values.  
 // When called as seq(yield), seq calls yield(v) for each value v in the sequence,  
 // stopping early if yield returns false.  
 type Seq[V any] func(yield func(V) bool)    
     
 // Seq2 is an iterator over sequences of pairs of values, most commonly key-value pairs.  
 // When called as seq(yield), seq calls yield(k, v) for each pair (k, v) in the sequence,  
 // stopping early if yield returns false.  
 type Seq2[K, V any] func(yield func(K, V) bool)

The map package has already been used iter to add methods such as All and Keys. Here is a reference to its implementation:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26


//https://go.googlesource.com/go/blob/c83b1a7013784098c2061ae7be832b2ab7241424/src/maps/iter.go#L12  
 // All returns an iterator over key-value pairs from m.  
 // The iteration order is not specified and is not guaranteed  
 // to be the same from one call to the next.  
 func All[Map ~map[K]V, K comparable, V any](m Map) iter.Seq2[K, V] {  
     return func(yield func(K, V) bool) {  
        for k, v := range m {  
           if !yield(k, v) {  
              return  
           }  
        }  
     }  
 }    
     
 // Keys returns an iterator over keys in m.  
 // The iteration order is not specified and is not guaranteed  
 // to be the same from one call to the next.  
 func Keys[Map ~map[K]V, K comparable](m Map) iter.Seq[K] {  
     return func(yield func(K) bool) {  
        for k := range m {  
           if !yield(k) {  
              return  
           }  
        }  
     }  
 }

Community Opinions

Pasted image 20240608215910
Photo by Ana Flávia on Unsplash

“In my opinion, yield is a complicated enough concept to cause a lot of bad, incomprehensible code to appear. This suggestion provides only a syntax sugar for writing something that is already more than possible in the language. I believe this goes against the rule of _One problem - one solution_. Please, let Go stay boring." Source

This is a common objection within the community. yield is not easy to understand, and we can implement iterators in many ways.

Conclusion

I support the addition of iter.
The iter package offers numerous possibilities for developers aiming to streamline their code and adopt more functional programming practices. However, its reception is mixed due to concerns about performance, complexity, and the learning curve.
As with any new tool, the key is to balance its use where it offers clear benefits while remaining mindful of the potential drawbacks. The Go community will undoubtedly continue to explore and debate the best ways to harness iter’s power without compromising the language’s foundational principles.

Performance Comparison

The goal of iter

The Use of iter

Community Opinions

Conclusion

Reference

The goal of `iter`

The Use of `iter`