> ## Content Index > Fetch the complete content index at: https://huizhou92.com/llms.txt > Use this file to discover other available public pages before exploring further. # RPC Action 2: Using Protobuf and Custom Plugin - URL: https://huizhou92.com/rpc-action-2-using-protobuf-and-creating-a-custom-plugin/ - Published: 2024-08-18T16:28:14.000Z - Updated: 2026-09-08T02:30:40.000Z - Description: Discover how to define messages in Protobuf, generate Go code, and integrate it into your RPC services - Author: huizhou92 - Tags: #Migrated-1788833207488, #Import 2026-09-08 02:07 In the [previous article](https://levelup.gitconnected.com/implementing-a-simple-rpc-interface-in-go-a-step-by-step-guide-e4df4cbb9f91?source=your%5Fstories%5Fpage-------------------------------------), I implemented a simple RPC interface using the `net/rpc` package and tried out the `Gob` encoding that comes with `net/rpc` as well as `JSON` encoding to learn some basics of Golang RPC. In this post, I'll combine `net/rpc` with protobuf and create my own `protobuf` plugin to help us generate code, so let's get started. We must have used `gRPC` \+ `protobuf` during our work, they are not bound, `gRPC` can be encoded using JSON, and [protobuf](https://en.wikipedia.org/wiki/Protocol%5FBuffers?ref=huizhou92.com) can be implemented in other languages. It can also be implemented in other languages. > **Protocol Buffers* (*Protobuf*) is a* [*free and open-source*](https://en.wikipedia.org/wiki/Free%5Fand%5Fopen-source%5Fsoftware?ref=huizhou92.com)[*cross-platform*](https://en.wikipedia.org/wiki/Cross-platform%5Fsoftware?ref=huizhou92.com) *data format used to* [*serialize*](https://en.wikipedia.org/wiki/Serialization?ref=huizhou92.com) *structured data. It is useful in developing programs that communicate with each other over a network or for storing data. The method involves an* [*interface description language*](https://en.wikipedia.org/wiki/Interface%5Fdescription%5Flanguage?ref=huizhou92.com) *that describes the structure of some data and a program that generates source code from that description for generating or parsing a stream of bytes that represents the structured data.* ### Using Protobuf First, we write a `proto` file hello-service.proto that defines a message "String" ```go syntax = "proto3"; package api; option go_package="api"; ​ message String { string value = 1; } ``` Then use the `protoc` utility to generate the Go code for the message String ```bash protoc --go_out=. hello-service.proto ``` Then we modify the Hello function’s arguments to use the String generated by the `protobuf` file. ```go type HelloServiceInterface = interface { Hello(request api.String, reply *api.String) error } ``` Using it is similar to before, even though it is not as convenient as using string directly. So why should we use `protobuf`? As I said earlier, **using Protobuf to define language-independent RPC service interfaces and messages and then using the `protoc` tool to generate code in different languages is where its real value lies**. For example, use the official plugin `protoc-gen-go` to generate `gRPC` code. ```shell protoc --go_out=plugins=grpc. hello-service.proto ``` ### Plugin system for protoc To generate code from `protobuf` files, we must install the `protoc` , but the `protoc` We don't know our target language, so we need plugins to help us generate code. How does the`protoc` plugin system work? Take the above `grpc` as an example. There is a `--go_out` parameter here. Since the plugin we're calling is `protoc-gen-go`, the parameter is called go\_out; if the name were XXX, the parameter would be called XXX\_out. When `protoc` is running, it will first parse the `protobuf` file and generate a set of Protocol Buffers-encoded descriptive data. It will first determine whether or not the `go` plugin is included in the `protoc`, and then it will try to look for `protoc-gen-go` in `$PATH`, and if it can't find it, it will report an error, and then it will run `protoc-gen-go`. `protoc-gen-go` command and sends the description data to the plugin command via stdin. After the plugin generates the file contents, it then inputs Protocol Buffers encoded data to stdout to tell `protoc` to generate the specific file. `plugins=grpc` is a plugin that comes with `protoc-gen-go` in order to invoke it. If you don't use it, it will only generate a message in Go, but you can use this plugin to generate grpc-related code. ### Customize a protoc plugin If we add `Hello` interface timing to protobuf, can we customize a `protoc` plugin to generate code directly? ```typescript syntax = "proto3"; package api; option go_package="./api"; service HelloService { rpc Hello (String) returns (String) {} } message String { string value = 1; } ``` ### Objective For this article, my goal was to create a plugin that would then be used to generate RPC server-side and client-side code that would look something like this. ```go // HelloService_rpc.pb.go type HelloServiceInterface interface { Hello(String, *String) error } func RegisterHelloService( srv *rpc.Server, x HelloServiceInterface, ) error { if err := srv.RegisterName("HelloService", x); err != nil { return err } return nil } type HelloServiceClient struct { *rpc.Client } var _ HelloServiceInterface = (*HelloServiceClient)(nil) func DialHelloService(network, address string) ( *HelloServiceClient, error, ) { c, err := rpc.Dial(network, address) if err != nil { return nil, err } return &HelloServiceClient{Client: c}, nil } func (p *HelloServiceClient) Hello( in String, out *String, ) error { return p.Client.Call("HelloService.Hello", in, out) } ``` This would change our business code to look like the following ```go // service func main() { listener, err := net.Listen("tcp", ":1234") if err != nil { log.Fatal("ListenTCP error:", err) } _ = api.RegisterHelloService(rpc.DefaultServer, new(HelloService)) for { conn, err := listener.Accept() if err != nil { log.Fatal("Accept error:", err) } go rpc.ServeConn(conn) } } type HelloService struct{} func (p *HelloService) Hello(request api.String, reply *api.String) error { log.Println("HelloService.proto Hello") *reply = api.String{Value: "Hello:" + request.Value} return nil } // client.go func main() { client, err := api.DialHelloService("tcp", "localhost:1234") if err != nil { log.Fatal("net.Dial:", err) } reply := &api.String{} err = client.Hello(api.String{Value: "Hello"}, reply) if err != nil { log.Fatal(err) } log.Println(reply) } ``` Based on the generated code, our workload is already much smaller, and the chances of error are very small. A good start. Based on the api code above, we can pull out a template file: ```go const tmplService = ` package {{.PackageName}} import ( "net/rpc") {{$root := .}} type {{.ServiceName}}Interface interface { {{- range $_, $m := .MethodList}} {{$m.MethodName}}({{$m.InputTypeName}}, *{{$m.OutputTypeName}}) error {{- end}}} func Register{{.ServiceName}}( srv *rpc.Server, x {{.ServiceName}}Interface,) error { if err := srv.RegisterName("{{.ServiceName}}", x); err != nil { return err } return nil} type {{.ServiceName}}Client struct { *rpc.Client} var _ {{.ServiceName}}Interface = (*{{.ServiceName}}Client)(nil) func Dial{{.ServiceName}}(network, address string) ( *{{.ServiceName}}Client, error,) { c, err := rpc.Dial(network, address) if err != nil { return nil, err } return &{{.ServiceName}}Client{Client: c}, nil} {{range $_, $m := .MethodList}} func (p *{{$root.ServiceName}}Client) {{$m.MethodName}}( in {{$m.InputTypeName}}, out *{{$m.OutputTypeName}},) error { return p.Client.Call("{{$root.ServiceName}}.{{$m.MethodName}}", in, out)} {{end}} ` ``` The whole template is clear, and there are some placeholders in it, such as MethodName, ServiceName, etc., which we’ll cover later. ### How to develop a plug-in? Google released the Go language API [1](https://go.dev/blog/protobuf-apiv2?ref=huizhou92.com), which introduces a new package `google.golang.org/protobuf/compile R/protogen`, which greatly reduces the difficulty of plugins development: 1. First of all, we create a go language project, such as `protoc-gen-go-spprpc` 2. Then we need to define a `protogen.Options`, then call its `Run` method, and pass in a `func(*protogen.Plugin) error` callback. This is the end of the main process code. 3. We can also set the ParamFunc parameter of `protogen.Options`, so that `protogen` will automatically parse the parameters passed by the command line for us. Operations such as reading and decoding `protobuf` information from standard input, encoding input information into `protobuf` and writing stdout are all handled by `protogen`. What we need to do is to interact with `protogen.Plugin` to implement code generation logic. The most important thing for each service is the name of the service, and then each service has a set of methods. For the method defined by the service, the most important thing is the name of the method and the name of the input parameter and the output parameter type. Let’s first define a `ServiceData` to describe the meta information of the service: ```go // ServiceData type ServiceData struct { PackageName string ServiceName string MethodList []Method } // Method type Method struct { MethodName string InputTypeName string OutputTypeName string } ``` Then comes the main logic, and the code generation logic, and finally the call to `tmpl` to generate the code. ```go func main() { protogen.Options{}.Run(func(gen *protogen.Plugin) error { for _, file := range gen.Files { if !file.Generate { continue } generateFile(gen, file) } return nil }) } // generateFile function definition func generateFile(gen *protogen.Plugin, file *protogen.File) { filename := file.GeneratedFilenamePrefix + "_rpc.pb.go" g := gen.NewGeneratedFile(filename, file.GoImportPath) tmpl, err := template.New("service").Parse(tmplService) if err != nil { log.Fatalf("Error parsing template: %v", err) } packageName := string(file.GoPackageName) // Iterate over each service to generate code for _, service := range file.Services { serviceData := ServiceData{ ServiceName: service.GoName, PackageName: packageName, } for _, method := range service.Methods { inputType := method.Input.GoIdent.GoName outputType := method.Output.GoIdent.GoName serviceData.MethodList = append(serviceData.MethodList, Method{ MethodName: method.GoName, InputTypeName: inputType, OutputTypeName: outputType, }) } // Perform template rendering err = tmpl.Execute(g, serviceData) if err != nil { log.Fatalf("Error executing template: %v", err) } } } ``` ### Debug plugin Finally, we put the compiled binary execution file `protoc-gen-go-spprpc` in `$PATH`, and then run `protoc` to generate the code we want. ```go protoc --go_out=.. --go-spprpc_out=.. HelloService.proto ``` Because `protoc-gen-go-spprpc` has to depend on `protoc` to run, it's a bit tricky to debug. We can use ```lua fmt.Fprintf(os.Stderr, "Fprintln: %v\n", err) ``` to print the error log to debug. ### Summary That’s all there is to this article. We first implemented an `RPC` call using protobuf, and then created a `protobuf` plugin to help us generate the code. This opens the door for us to learn `protobuf` \+ `RPC`, and is our path to a thorough understanding of gRPC. I hope everyone can master this technology. ### Reference 1. [https://taoshu.in/go/create-protoc-plugin.html](https://taoshu.in/go/create-protoc-plugin.html?ref=huizhou92.com) 2. [https://chai2010.cn/advanced-go-programming-book/ch4-rpc/ch4-02-pb-intro.html](https://chai2010.cn/advanced-go-programming-book/ch4-rpc/ch4-02-pb-intro.html?ref=huizhou92.com) [文章索引](/article-index/)