The Go Protobuf resource is readable

There are a large number of protocols using Google Protocol Buffer. A brief introduction to Protobuf can be found in IBM's "Usage and Principles of Google Protocol Buffer". In simple terms, the advantages of Protobuf are (compared to XML) smaller, faster, simpler, and backward compatible. Disadvantages, the impact on my daily work is relatively poor readability, because Protobuf compression will be serialized, generate pb file, this file is binary, can not be human readable. However, in daily work, especially to troubleshoot problems, it is often necessary to look at the correctness of the resource file content, whether the upstream and downstream services send and receive package contents are correct, forge pb resources, etc. These contents are all pb and need to be converted to understand. This uses Go to write a two-stage program that uses JSON to forge pb resources and deserialize pb to print human-readable text.

JSON 转 pb

This is a very troublesome thing, but with the jsonpb library, things become very simple.

First define user.proto.

syntax = "proto3";

package user_info;

message UserInfo {
    message User {
        string username = 1;
        uint32 age      = 2;
        string graduate = 3;
    }
    
    repeated User user_list = 1;
}

Then convert to generate the user.pb.go file.

protoc --go_out=. user.proto

Write a JSON file, note that the name of the key needs to follow the name in user.pb.go, for example:

type UserInfo struct {
    UserList []*UserInfo_User `protobuf:"bytes,1,rep,name=user_list,json=userList" json:"user_list,omitempty"`
}

type UserInfo_User struct {
    Username string `protobuf:"bytes,1,opt,name=username" json:"username,omitempty"`
    Age      uint32 `protobuf:"varint,2,opt,name=age" json:"age,omitempty"`
    Graduate string `protobuf:"bytes,3,opt,name=graduate" json:"graduate,omitempty"`
}

user.pb.go has specified a field named in JSON, write JSON directly according to this The file is fine.

{
  "userList": [
    {
      "username": "lawrencelin",
      "age": 28,
      "graduate": "Tongji University"
    },
    {
      "username": "findingsea",
      "age": 28,
      "graduate": "Fudan University"
    }
  ]
}

Write the main code:

package main

import (
    "github.com/golang/protobuf/proto"
    "io/ioutil"
    "os"
    "fmt"
    "github.com/golang/protobuf/jsonpb"
    "user_proto"
)

func main()  {
    jsonFilePath := "/home/lawrence/GoglandProjects/JsonToPbIntro/json/user_info.json"
    pbFilePath := "/home/lawrence/GoglandProjects/JsonToPbIntro/pb/user_info.pb"

    buf, err := ioutil.ReadFile(jsonFilePath)
    if err != nil {
        fmt.Println("Read file err: ", err)
        os.Exit(0)
    }

    userInfo := &user_info.UserInfo{}

    if err = jsonpb.UnmarshalString(string(buf), userInfo); err != nil {
        fmt.Println("jsonpb UnmarshalString fail: ", err)
        os.Exit(0)
    }

    fmt.Println("user info pb: ", userInfo.String())

    data, err := proto.Marshal(userInfo)
    if err != nil {
        fmt.Println("proto Marshal fail: ", err)
        os.Exit(0)
    }

    if err = ioutil.WriteFile(pbFilePath, data, os.ModePerm); err != nil {
        fmt.Println("Write file err: ", err)
    }
}

The core function is UnmarshalString, the input is a JSON string, and the Protobuf object is output.

func UnmarshalString(str string, pb proto.Message) error

Running main.go, the user_info.pb file is generated and printed as follows:

user info pb:  user_list:<username:"lawrencelin" age:28 graduate:"Tongji University" > user_list:<username:"findingsea" age:28 graduate:"Fudan University" > 

Print Protobuf object

This should be very simple, because the Protobuf library provides string conversion Functions, like the C++ version of Protobuf, provide the DebugString() method directly, which can output readable print strings directly. But in Go, my intuition responded by calling the String() method, fmt.Println("user info pb: ", userInfo.String()) Print as a line.

user_list:<username:"lawrencelin" age:28 graduate:"Tongji University" > user_list:<username:"findingsea" age:28 graduate:"Fudan University" > 

Look at the String() method implementation, directly call CompactTextStringMethod:

func (m *UserInfo) String() string            { return proto.CompactTextString(m) }

// CompactText writes a given protocol buffer in compact text format (one line).
func CompactText(w io.Writer, pb Message) error { return compactTextMarshaler.Marshal(w, pb) }

// CompactTextString is the same as CompactText, but returns the string directly.
func CompactTextString(pb Message) string { return compactTextMarshaler.Text(pb) }

Note that this interface can only return compressed Text, this readability is very poor, how to output a readable Protobuf object?

After reading the documentation, I found that I should use the MarshalTextString interface to return a readable text format Protobuf object directly. The interface source code and comments are as follows:

// MarshalText writes a given protocol buffer in text format.
// The only errors returned are from w.
func MarshalText(w io.Writer, pb Message) error { return defaultTextMarshaler.Marshal(w, pb) }

// MarshalTextString is the same as MarshalText, but returns the string directly.
func MarshalTextString(pb Message) string { return defaultTextMarshaler.Text(pb) }

The method of calling is very simple, fmt.Println(proto.MarshalTextString(userInfo)), output:

user_list: <
  username: "lawrencelin"
  age: 28
  graduate: "Tongji University"
>
user_list: <
  username: "findingsea"
  age: 28
  graduate: "Fudan University"
>