Background

JSON is a lightweight data interchange format that has been favored by developers for its flexible features since the inception of Go’s encoding/json package ten years ago. Developers can customize the JSON representation of struct fields through struct tags, and also allow Go types to customize their own JSON representation. However, with the development of Go language and JSON standards, some functional deficiencies and performance limitations have gradually been exposed.

1. Missing functionality: For example, it is not possible to specify custom formatting for the time.Time type, and it is not possible to omit specific Go values during serialization, etc.
2. API deficiencies: For example, there is no simple way to correctly deserialize JSON from an io.Reader.
3. Performance limitations: The performance of the standard json package is not satisfactory, especially when dealing with large amounts of data.
4. Behavioral flaws: For example, the error handling for JSON syntax is not strict enough, and case-insensitive deserialization, etc.

Just like math/v2, Go official proposes encoding/json/v2 to solve the above problems. The main goal of this article is to analyze some issues about null values in encoding/json and how they are resolved in encoding/json/v2. This article does not involve other modifications in encoding/json/v2.

This article was first published in the Medium MPP plan. If you are a Medium user, please follow me on Medium. Thank you very much.

omitempty

In the encoding/json package, there is a description of omitempty as follows:

The “omitempty” option specifies that the field should be omitted from the encoding if the field has an empty value, defined as false, 0, a nil pointer, a nil interface value, and any empty array, slice, map, or string.

However, this predefined “empty” value judgment logic does not meet the needs of all actual scenarios. Let’s look at an example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

type Post struct {  
    Id         int64           `json:"id,omitempty"`  
    CreateTime time.Time       `json:"create_time,omitempty"`  
    TagList    []Tag           `json:"tag_list,omitempty"`  
    Name       string          `json:"name,omitempty"`  
    Score      float64         `json:"score,omitempty"`  
    Category   Category        `json:"category,omitempty"`  
    LikePost   map[string]Post `json:"like,omitempty"`  
}  
type Tag struct {  
    ID   string `json:"id"`  
    Name string `json:"name"`  
}  
type Category struct {  
    ID   float64 `json:"id"`  
    Name string  `json:"name"`  
}  
  
func main() {  
    b, _ := json.Marshal(new(Post))  
    fmt.Println(string(b))  
}

The output result is:

1

{"create_time":"0001-01-01T00:00:00Z","category":{"id":0,"name":""}}

Although omitempty is added to each field of Post, the result is not as expected.

1. omitempty cannot handle empty struct, such as Post.Category
2. The way omitempty handles time.Time is not what we understand as UTC =0, that is, 1970-01-01 00:00:00, but 0001-01-01T00:00:00Z.

omitzero Tag

In encoding/json/v2, a new tag omitzero will be added, which adds two functions to solve the above two problems. (This feature is still under development, but you can experience the new features in advance through go-json-experiment/json)

1. Better handling of time.Time.
2. Support for custom IsZero function.
   For example, the following code:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37

package main  
  
import (  
    "encoding/json"  
    "fmt"    v2_json "github.com/go-json-experiment/json"  
    "math"    "time")  
  
type Post struct {  
    Id         int64           `json:"id,omitempty,omitzero"`  
    CreateTime time.Time       `json:"create_time,omitempty,omitzero"`  
    TagList    []Tag           `json:"tag_list,omitempty"`  
    Name       string          `json:"name,omitempty"`  
    Score      ScoreType       `json:"score,omitempty,omitzero"`  
    Category   Category        `json:"category,omitempty,omitzero"`  
    LikePost   map[string]Post `json:"like,omitempty"`  
}  
type ScoreType float64  
  
func (s ScoreType) IsZero() bool {  
    return s < math.MinInt64  
}  
  
type Tag struct {  
    ID   string `json:"id"`  
    Name string `json:"name"`  
}  
type Category struct {  
    ID   float64 `json:"id"`  
    Name string  `json:"name"`  
}  
  
func main() {  
    v1String, _ := json.Marshal(new(Post))  
    fmt.Println(string(v1String))  
    v2String, _ := v2_json.Marshal(new(Post))  
    fmt.Println(string(v2String))  
}

Compared with encoding/json, encoding/json/v2 solves the above problems.

1
2

{"create_time":"0001-01-01T00:00:00Z","category":{"id":0,"name":""}}
{"score":0}

Conclusion

By introducing the omitzero tag, Go has taken an essential step in solving the pain points of “empty” value processing in JSON encoding. This solution not only meets developers’ needs for a more flexible definition of “empty” values but also maintains compatibility with existing systems. The landing time of omitzero is not determined yet, and it will not be available until the Go 1.24 version at the earliest. In addition, encoding/xml and other packages will also follow the json package and add the omitzero tag.
encoding/json/v2 also includes updates in other aspects, such as performance. Interested Gophers can learn about these changes in advance, and this blog will continue to pay attention to this proposal.

Go language is beginning to mature.