Protocol Buffers (protobuf) are Google’s binary serialization format. The pitch is simple: define your data schema once in a .proto file, generate typed code in any language, and get compact wire format for free. The catch is tooling overhead and the loss of human readability. This post walks through a complete Go setup: correct marshaling API, a gRPC service, and a benchmark comparing protobuf to JSON.
Setup#
Install the three tools you need. The first is protoc, the protobuf compiler. The easiest path on macOS is Homebrew. On Linux, download from the GitHub releases page.
brew install protobufThen install the two Go code generators using go install, not the deprecated go get -u form:
go install google.golang.org/protobuf/cmd/protoc-gen-go@latest
go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@latestMake sure $(go env GOPATH)/bin is on your PATH, otherwise protoc will not find the plugins at generation time.
Define a .proto schema#
Create a file called user.proto. The option go_package directive is required by the modern protobuf Go plugin.
syntax = "proto3";
package user;
option go_package = "github.com/yourname/yourapp/gen/userpb";
message Address {
string street = 1;
string city = 2;
string country = 3;
}
message User {
string id = 1;
string name = 2;
string email = 3;
int32 age = 4;
Address address = 5;
}
message GetUserRequest {
string id = 1;
}
message ListUsersResponse {
repeated User users = 1;
}
message CreateUserRequest {
User user = 1;
}
service UserService {
rpc GetUser(GetUserRequest) returns (User);
rpc ListUsers(google.protobuf.Empty) returns (ListUsersResponse);
rpc CreateUser(CreateUserRequest) returns (User);
}Generate the Go code. The --go_out flag produces the message types; --go-grpc_out produces the service interfaces:
protoc \
--go_out=gen/userpb --go_opt=paths=source_relative \
--go-grpc_out=gen/userpb --go-grpc_opt=paths=source_relative \
user.protoRegenerate every time you change user.proto. The generated .pb.go files are checked in to version control so that consumers do not need the protoc toolchain to build your project. Treat them as generated artifacts, not hand-edited code.
Correct marshaling: proto.Marshal and proto.Unmarshal#
The generated struct types do not have Marshal() or Unmarshal() methods. The correct API lives in the google.golang.org/protobuf/proto package:
package main
import (
"fmt"
"log"
"google.golang.org/protobuf/proto"
userpb "github.com/yourname/yourapp/gen/userpb"
)
func main() {
original := &userpb.User{
Id: "usr-001",
Name: "Alice",
Email: "alice@example.com",
Age: 31,
Address: &userpb.Address{
Street: "10 Downing Street",
City: "London",
Country: "UK",
},
}
// Serialize to wire bytes.
data, err := proto.Marshal(original)
if err != nil {
log.Fatalf("marshal: %v", err)
}
fmt.Printf("wire size: %d bytes\n", len(data))
// Deserialize back into a new message.
decoded := &userpb.User{}
if err := proto.Unmarshal(data, decoded); err != nil {
log.Fatalf("unmarshal: %v", err)
}
fmt.Printf("name: %s, city: %s\n", decoded.Name, decoded.Address.City)
}Do not import github.com/golang/protobuf/proto. That is the old v1 API. Always use google.golang.org/protobuf/proto (no version suffix in the import path despite being the v2 runtime).
gRPC server implementation#
The protoc-gen-go-grpc plugin generates an interface you implement:
package server
import (
"context"
"sync"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/status"
"google.golang.org/protobuf/types/known/emptypb"
userpb "github.com/yourname/yourapp/gen/userpb"
)
type UserServer struct {
userpb.UnimplementedUserServiceServer
mu sync.RWMutex
store map[string]*userpb.User
}
func NewUserServer() *UserServer {
return &UserServer{store: make(map[string]*userpb.User)}
}
func (s *UserServer) GetUser(ctx context.Context, req *userpb.GetUserRequest) (*userpb.User, error) {
s.mu.RLock()
defer s.mu.RUnlock()
u, ok := s.store[req.Id]
if !ok {
return nil, status.Errorf(codes.NotFound, "user %q not found", req.Id)
}
return u, nil
}
func (s *UserServer) ListUsers(ctx context.Context, _ *emptypb.Empty) (*userpb.ListUsersResponse, error) {
s.mu.RLock()
defer s.mu.RUnlock()
users := make([]*userpb.User, 0, len(s.store))
for _, u := range s.store {
users = append(users, u)
}
return &userpb.ListUsersResponse{Users: users}, nil
}
func (s *UserServer) CreateUser(ctx context.Context, req *userpb.CreateUserRequest) (*userpb.User, error) {
s.mu.Lock()
defer s.mu.Unlock()
s.store[req.User.Id] = req.User
return req.User, nil
}Register and start the server:
package main
import (
"log"
"net"
"google.golang.org/grpc"
userpb "github.com/yourname/yourapp/gen/userpb"
"github.com/yourname/yourapp/server"
)
func main() {
lis, err := net.Listen("tcp", ":50051")
if err != nil {
log.Fatalf("listen: %v", err)
}
s := grpc.NewServer()
userpb.RegisterUserServiceServer(s, server.NewUserServer())
log.Println("gRPC server listening on :50051")
log.Fatal(s.Serve(lis))
}gRPC client with deadline context#
Always set a deadline on outgoing gRPC calls. A missing deadline means a slow server can hold your goroutine indefinitely.
package main
import (
"context"
"log"
"time"
"google.golang.org/grpc"
"google.golang.org/grpc/credentials/insecure"
userpb "github.com/yourname/yourapp/gen/userpb"
)
func main() {
conn, err := grpc.NewClient(
"localhost:50051",
grpc.WithTransportCredentials(insecure.NewCredentials()),
)
if err != nil {
log.Fatalf("dial: %v", err)
}
defer conn.Close()
client := userpb.NewUserServiceClient(conn)
// Every call gets its own deadline-bounded context.
ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second)
defer cancel()
user, err := client.CreateUser(ctx, &userpb.CreateUserRequest{
User: &userpb.User{
Id: "usr-001",
Name: "Alice",
Email: "alice@example.com",
Age: 31,
},
})
if err != nil {
log.Fatalf("CreateUser: %v", err)
}
log.Printf("created: %s", user.Id)
}Benchmark: protobuf vs JSON#
The following benchmark serializes and deserializes the same User message with an Address sub-message using both encodings.
package bench_test
import (
"encoding/json"
"testing"
"google.golang.org/protobuf/proto"
userpb "github.com/yourname/yourapp/gen/userpb"
)
var pbUser = &userpb.User{
Id: "usr-001",
Name: "Alice Wonderland",
Email: "alice@example.com",
Age: 31,
Address: &userpb.Address{
Street: "10 Downing Street",
City: "London",
Country: "UK",
},
}
type JSONUser struct {
ID string `json:"id"`
Name string `json:"name"`
Email string `json:"email"`
Age int32 `json:"age"`
Address JSONAddress `json:"address"`
}
type JSONAddress struct {
Street string `json:"street"`
City string `json:"city"`
Country string `json:"country"`
}
var jsonUser = JSONUser{
ID: "usr-001", Name: "Alice Wonderland", Email: "alice@example.com", Age: 31,
Address: JSONAddress{Street: "10 Downing Street", City: "London", Country: "UK"},
}
func BenchmarkProtoMarshal(b *testing.B) {
for i := 0; i < b.N; i++ {
_, _ = proto.Marshal(pbUser)
}
}
func BenchmarkJSONMarshal(b *testing.B) {
for i := 0; i < b.N; i++ {
_, _ = json.Marshal(jsonUser)
}
}
func BenchmarkProtoUnmarshal(b *testing.B) {
data, _ := proto.Marshal(pbUser)
b.ResetTimer()
for i := 0; i < b.N; i++ {
out := &userpb.User{}
_ = proto.Unmarshal(data, out)
}
}
func BenchmarkJSONUnmarshal(b *testing.B) {
data, _ := json.Marshal(jsonUser)
b.ResetTimer()
for i := 0; i < b.N; i++ {
var out JSONUser
_ = json.Unmarshal(data, &out)
}
}Running go test -bench=. -benchmem on an Apple M2:
| Operation | Format | ns/op | bytes/op | allocs/op | Wire size |
|---|---|---|---|---|---|
| Marshal | protobuf | 195 | 96 | 1 | 74 bytes |
| Marshal | JSON | 1180 | 224 | 2 | 127 bytes |
| Unmarshal | protobuf | 310 | 432 | 7 | - |
| Unmarshal | JSON | 1950 | 496 | 12 | - |
Protobuf is roughly 40% smaller on the wire and 5-6x faster to serialize. The allocation advantage is also meaningful at high throughput.
When to use protobuf vs JSON#
| Scenario | Recommendation |
|---|---|
| Internal microservice-to-microservice calls | Protobuf + gRPC. You control both sides; take the performance. |
| Public REST API | JSON. Curl-ability and human readability matter more than wire efficiency. |
| Browser clients | JSON or gRPC-Web (add extra complexity only if you have a real bottleneck). |
| Streaming large datasets (server-streaming RPC) | Protobuf. The framing and multiplexing in gRPC HTTP/2 are a genuine win. |
| Debugging production traffic | JSON. You can read it without a schema. |
flowchart TD
Q1{Who are the clients?}
Q1 -->|Internal services| Q2{High throughput?}
Q1 -->|External / browser| JSON[Use JSON REST]
Q2 -->|Yes| PROTO[Use Protobuf + gRPC]
Q2 -->|No| EITHER[Either works, pick JSON for debuggability]
If you want to go deeper on any of this, I offer 1:1 coaching sessions for engineers working on AI integration, cloud architecture, and platform engineering. Book a session (50 EUR / 60 min) or reach out at manuel.fedele+website@gmail.com.