Search Tech Journey

Find topics, journeys and posts

6-month learning plan58 / 130
back to blog
backend apisintermediate 55m read

S058 · gRPC & Protobuf — When RPC Wins

Binary framing + strict schemas + streaming = the internal-service default at Google, Netflix, Uber, Cloudflare. What Protobuf actually is, how gRPC uses HTTP/2, when RPC is the right shape, and a working polyglot client-server in Python.

🧠SoftwareM06 · Backend & APIs· Session 058 of 130 90 min

🎯 Define a Protobuf schema, generate client + server code, run unary and streaming RPCs, and articulate three concrete situations where gRPC beats REST for internal traffic.

Why this session exists

Inside every large tech company, the service-to-service protocol is almost never REST — it's gRPC (Google), Thrift (Meta), or another binary RPC. The reasons are unglamorous but decisive: strict schemas that break the build when contracts drift, ~5-10× smaller payloads over the wire, native streaming, and code generation across every language your microservices are written in. Learning gRPC is what separates ‘I write APIs’ from ‘I design the platform APIs everyone else consumes’.

You will be able to
  • Write a `.proto` file with messages, services, and streaming methods.
  • Generate Python client + server stubs and run a unary + a server-streaming RPC.
  • Explain Protobuf's wire format at a high level (tags, varints, wire types).
  • Contrast gRPC with REST and GraphQL on three axes: schema strictness, transport, streaming.
  • Pick the right RPC shape (unary, server-stream, client-stream, bidi) for a given problem.

Prerequisites

  • S055 · HTTP fundamentals — HTTP/2 in particular.
  • S056 · REST API design — the alternative you'll compare against.
  • S018 · Python packaging / venv — because we'll run codegen.


(a) Intuition · 5 min

REST is a friendly letter; gRPC is a shipping manifest
🌍 Real world

A letter is human-readable. Anyone can open it, read it, understand it, forward it. That flexibility is its strength — and its cost. Envelopes are bulky, addresses are inconsistent, and the postal system has to guess a lot.

A shipping manifest is a rigid, tabular document. Column 3 always means SKU, column 7 always means weight. Machines process thousands per second. Humans hate reading them raw, but scanners love them and mistakes are almost impossible.

💻 Code world

REST + JSON is the letter — human-readable, forgiving, but slow to parse and easy to drift. Great when you don't control the reader.

gRPC + Protobuf is the manifest — binary, strictly typed, generated code on both ends. Ten times smaller on the wire, orders of magnitude faster to parse, and impossible to send a request the server can't parse. Perfect when you own both ends of the pipe.

The four RPC shapes gRPC gives you

Not just request/response
  • Unary — client sends one request, server sends one response. The REST analogue.
  • Server-streaming — one request, many responses. Perfect for tailing logs, subscription feeds, download progress.
  • Client-streaming — many requests, one response. Ideal for large uploads, telemetry batches, sensor streams.
  • Bidirectional streaming — many both ways over one connection. Chat, live collaboration, gaming.

How gRPC got here

  1. 2001
    Google internal ‘Stubby’ RPC
    Google's original binary RPC. Fast, typed, but proprietary. Every service inside Google talks it.
  2. 2008
    Protobuf 2 open-sourced
    The serialisation layer without the RPC framework. Adopted at LinkedIn, Twitter, Netflix quickly.
  3. 2015
    gRPC 1.0 released
    Stubby, cleaned up and open-sourced. Sits on HTTP/2. Immediately adopted by Kubernetes, Envoy, CoreDNS.
  4. 2018
    Google ships gRPC-Web
    Bridges to browsers via a proxy. Web frontends can finally use gRPC without WebSockets.
  5. 2022
    Connect / gRPC over HTTP/1.1 + JSON
    Buf's Connect protocol adds REST-friendly transport for the same schemas. Best of both worlds for polyglot orgs.

(b) Visual walkthrough · 15 min

The build pipeline

The .proto is checked into a shared repo (often called a ‘schema registry’ or ‘proto repo’). All services regenerate on every build — the schema is the contract, and it's enforced by the compiler.

Anatomy of a message

What actually goes on the wire

Field tag (varint)
The unique number you assigned in .proto (e.g. tag 3). Encoded in 1 byte for tags 1-15.
tag
Wire type (3 bits)
0=varint (ints, bools), 1=64-bit, 2=length-delimited (strings, submessages), 5=32-bit. Packed with the tag.
type
Value (varint or bytes)
The actual data. Ints use varint encoding — small numbers take 1 byte. Strings prefix with a length.
value
Absent fields
Not sent at all. Protobuf has no null — just missing. Defaults kick in on the reader side.
missing

gRPC on HTTP/2

11
Client opens HTTP/2 connection

One TCP+TLS connection, kept warm. Every RPC is a stream on that connection.

22
Client sends HEADERS frame

`:path = /myservice.MyService/GetUser`, `content-type: application/grpc+proto`, custom metadata as headers.

33
Client sends DATA frame

The serialized Protobuf request body, prefixed with 1 byte (compression flag) + 4 bytes (length).

44
Server responds with HEADERS + DATA

Same framing in reverse. For streams, DATA frames continue until END_STREAM.

55
Trailers (Trailers-Only for errors)

gRPC-Status + gRPC-Message in HTTP trailers. Errors are structured, not free-form.

When to reach for gRPC vs REST vs GraphQL

gRPC · internal service-to-service

Strict schema, binary, streaming

  • 5-10× smaller payloads than JSON
  • Codegen for every mainstream language
  • Native streaming (server/client/bidi)
  • Poor browser story without gRPC-Web/Connect
  • Debugging needs specialised tools (grpcurl)
REST · public + simple

Human-readable, cacheable, universal

  • Every dev knows it, every language supports it
  • HTTP cache + CDN work out of the box
  • curl-friendly for debugging
  • Larger payloads (JSON overhead)
  • No streaming (SSE / WebSocket bolt-ons)
GraphQL · rich UI

Client-shaped queries over a graph

  • One request replaces N REST calls for UI screens
  • Fields chosen by client
  • Schema + tooling comparable to gRPC
  • Complex authz + cost analysis needed
  • Not a good fit for internal service-to-service

Common misconception
✗ What most people think

"gRPC is fast because Protobuf is a binary format. Binary beats JSON, that's the whole story."

✓ What is actually true

Encoding size is the smaller half. The larger wins are HTTP/2 multiplexing (many concurrent RPCs on one connection with no head-of-line blocking at the HTTP layer), no per-field key strings on the wire, no runtime type inference during parsing, and generated code that deserializes directly into structs without allocating a dictionary per message. On small payloads over a fast internal network, the difference is dominated by parsing and connection behaviour, not by bytes transmitted.

Why the myth is so sticky

The myth is sticky because "binary is smaller" is true, measurable, and easy to demo, so it explains the result you observed. It misleads because it predicts gRPC's advantage should vanish on small messages — and it doesn't. It also leads people to reach for gRPC purely for payload size when compression on JSON would have closed most of that particular gap.

Prove it to yourself

Look at what is actually on the wire and notice what is absent:

# JSON: field names shipped with every message
{"user_id": 12345, "is_active": true}   # ~36 bytes

# Protobuf: field NUMBER + wire type, no names
08 b9 60 10 01                            # 5 bytes
#  ^tag 1, varint   ^tag 2, varint

The field names exist only in the .proto, shared out of band at compile time. That is the real trick: the schema is not transmitted, so the message carries no self-description at all — which is also exactly why you cannot read a Protobuf message without its schema.

From first principles
Start with the question

Why must you never reuse a Protobuf field number, even for a field you deleted years ago? This looks like excessive caution — it is a hard correctness requirement.

  1. 1
    A Protobuf message on the wire is a sequence of (field number, wire type, value) triples. The field name is never transmitted.
    forced by · omitting names is precisely where the size and parse-speed advantage comes from
  2. 2
    Therefore the receiver identifies a field solely by its number, resolved against whatever .proto version it was compiled with.
    forced by · the message carries no self-description, so interpretation depends entirely on the reader's local schema
  3. 3
    In any real system, senders and receivers run different schema versions simultaneously — deploys are not atomic, and messages persist in queues, logs and databases.
    forced by · rolling deploys and durable storage guarantee old encodings meet new readers and vice versa
  4. 4
    So if field 7 was user_id (int64) and is later reused as created_at (int64), an old message decoded by new code yields a user ID silently interpreted as a timestamp.
    forced by · the wire types match, so nothing fails — the bytes parse perfectly into the wrong meaning
  5. 5
    No error is raised at any layer, so the corruption is silent and can persist in derived data indefinitely.
    forced by · type checking happens against the local schema, and the local schema says field 7 is a timestamp
⇒ Therefore

Therefore field numbers are permanent identity, not slots. The reserved keyword exists to make the compiler enforce this, and it should be used every single time a field is removed.

And note what this predicts: the same argument applies to enum values. Reusing an enum number silently changes the meaning of persisted data, which is why reserved works on enums too — and why every enum should have an explicit UNKNOWN = 0, since unset fields decode as zero and you need "unset" to be distinguishable from a real value rather than colliding with your first meaningful case.

Mental modelContract compiled into both ends

The .proto file is a shared contract compiled into client and server as native types. The network carries only numbered values; both ends already know what the numbers mean. Nothing is negotiated at runtime, nothing is inferred, nothing is self-describing.

Because the transport is HTTP/2, a single connection carries many concurrent streams, and a stream can flow in either direction for as long as it likes — which is why streaming RPCs exist at all rather than being bolted on.

  • Four call shapes, and choosing correctly is a design decision: unary (one request, one response), server streaming (subscriptions, large result sets), client streaming (uploads, telemetry ingestion), bidirectional (chat, live sync). Reaching for unary when the data is a stream forces polling.
  • Schema evolution rules are mechanical: adding an optional field is safe, removing a field requires reserved, changing a field's type or number is a break. Following these gives genuinely independent deploys; violating one produces silent corruption rather than a clean failure.
  • Unset and default are indistinguishable in proto3 scalars — a missing int is 0, a missing bool is false. If you need to know whether a value was set, use an explicit optional field or a wrapper type. Countless bugs live in this gap.
  • The binary wire format is unreadable without the schema. That is a real operational cost: curl, browser devtools, log inspection and proxies all stop working, so you need grpcurl, server reflection, and deliberate observability tooling.
🔔 Fires when you see

Fire this model when you see: a field that "sometimes comes back as zero" · a client and server disagreeing after a partial rollout · someone about to reuse a deleted field number · polling a service every second for updates · an internal API where JSON parsing shows up in a CPU profile.

The tradeoff

For a new internal service-to-service API: gRPC with Protobuf, or JSON over HTTP?

gRPC + Protobuf
+ you gain schema-enforced contracts caught at compile time in every language, generated clients so nobody hand-writes HTTP calls, first-class streaming, and materially lower CPU spent on serialization at high message rates.
− you pay you need a build pipeline for code generation and a story for distributing generated artefacts. Debugging loses curl and browser tools. Browsers cannot speak gRPC directly and need a gRPC-Web proxy. Load balancers must be HTTP/2-aware or all streams pin to one backend, because a single long-lived connection carries everything.
pick when internal service meshes with high call volume, multiple implementation languages, or genuine streaming needs
JSON over HTTP
+ you gain every tool works: curl, browsers, proxies, log aggregators, every HTTP load balancer. Zero build tooling, trivial onboarding, and a human can read a payload from a log line during an incident.
− you pay the contract lives in documentation and hope. Serialization CPU is real at high rates, there is no streaming primitive, and type mismatches surface at runtime in production rather than at build time.
pick when public APIs, low call volume, external consumers, or a small team where the schema fits in one head
gRPC internally, JSON gateway at the edge
+ you gain typed high-performance internals with a REST/JSON facade generated from the same .proto, so external consumers and browsers get plain HTTP while internal services keep the benefits.
− you pay one more hop and one more component to operate, plus a translation layer whose semantics you must own — error mapping between gRPC status codes and HTTP status codes is a genuine ongoing design problem.
pick when when you have both internal high-volume traffic and external consumers, which is the common end state for a platform of any size
What a senior engineer actually does

Use gRPC where the caller is a service you control and the call rate or type-safety pressure is real. Use JSON at any boundary crossing an organisational line, because the cost of a partner integrating with your .proto is far higher than your serialization savings.

The most commonly underestimated cost is load balancing. gRPC multiplexes over one long-lived HTTP/2 connection, so a naive L4 load balancer pins every request from that client to a single backend and your traffic distribution quietly becomes uneven. Confirm your infrastructure does HTTP/2-aware balancing before adopting gRPC broadly — this is discovered during an incident far more often than during design.


(c) Hands-on · 25 min

We'll build a small ‘echo’ service with unary + server-streaming RPCs, generate Python stubs, and see the wire messages fly.

1. Define the schema

Save as demo.proto:

syntax = "proto3";
package demo;
 
// Request/response messages
message SayHelloRequest {
  string name = 1;
  int32 shouts = 2;   // how many replies to stream back
}
 
message SayHelloResponse {
  string greeting = 1;
  int32 sequence = 2;
}
 
// Service definition
service Greeter {
  // Unary RPC: one request, one response
  rpc SayHello (SayHelloRequest) returns (SayHelloResponse);
 
  // Server-streaming: one request, many responses
  rpc SayHelloStream (SayHelloRequest) returns (stream SayHelloResponse);
}

2. Generate stubs

python -m pip install grpcio grpcio-tools
 
python -m grpc_tools.protoc \
  --proto_path=. \
  --python_out=. \
  --grpc_python_out=. \
  demo.proto

This produces demo_pb2.py (message classes) and demo_pb2_grpc.py (service stubs). You do not edit generated files.

3. Server

Save as server.py:

"""server.py — implement Greeter over the generated interface."""
import time
from concurrent import futures
 
import grpc
import demo_pb2
import demo_pb2_grpc
 
 
class GreeterServicer(demo_pb2_grpc.GreeterServicer):
    def SayHello(self, request, context):
        # unary: return one response
        return demo_pb2.SayHelloResponse(
            greeting=f"Hello, {request.name}!",
            sequence=0,
        )
 
    def SayHelloStream(self, request, context):
        # server-streaming: yield multiple responses on the same call
        for i in range(max(1, request.shouts)):
            yield demo_pb2.SayHelloResponse(
                greeting=f"Hello, {request.name}!",
                sequence=i,
            )
            time.sleep(0.2)
 
 
def serve() -> None:
    server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))
    demo_pb2_grpc.add_GreeterServicer_to_server(GreeterServicer(), server)
    server.add_insecure_port("127.0.0.1:50051")
    server.start()
    print("gRPC server listening on 127.0.0.1:50051")
    server.wait_for_termination()
 
 
if __name__ == "__main__":
    serve()

4. Client

Save as client.py:

"""client.py — call both RPCs, print responses + wire sizes."""
import grpc
import demo_pb2
import demo_pb2_grpc
 
 
def main() -> None:
    with grpc.insecure_channel("127.0.0.1:50051") as channel:
        stub = demo_pb2_grpc.GreeterStub(channel)
 
        # Unary
        req = demo_pb2.SayHelloRequest(name="Dinesh", shouts=1)
        print(f"[wire] request serialized bytes: {len(req.SerializeToString())}")
        resp = stub.SayHello(req)
        print(f"[unary]   {resp.greeting} (seq={resp.sequence})")
        print(f"[wire]    response serialized bytes: {len(resp.SerializeToString())}")
 
        # Server-streaming
        req = demo_pb2.SayHelloRequest(name="Dinesh", shouts=3)
        print("\n[stream]  server-streaming responses:")
        for msg in stub.SayHelloStream(req):
            print(f"  #{msg.sequence}: {msg.greeting}")
 
 
if __name__ == "__main__":
    main()

5. Run it

# terminal 1
python server.py
 
# terminal 2
python client.py

Expected output:

[wire] request serialized bytes: 10
[unary]   Hello, Dinesh! (seq=0)
[wire]    response serialized bytes: 17

[stream]  server-streaming responses:
  #0: Hello, Dinesh!
  #1: Hello, Dinesh!
  #2: Hello, Dinesh!

10 bytes for the request — compare to {"name":"Dinesh","shouts":1} in JSON (28 bytes). Nearly 3× smaller for a trivial payload. Real business messages compress even more.

Anatomy of the setup

What each artefact does

demo.proto
The single source of truth. Every service consuming or producing these messages regenerates from this file.
schema
demo_pb2.py
Generated message classes. Immutable-ish, typed, with SerializeToString / FromString methods. Never edit.
gen-msg
demo_pb2_grpc.py
Generated service classes: `GreeterStub` for clients, `GreeterServicer` base class for servers.
gen-svc
GreeterServicer
You override the methods declared in the .proto. Signature is enforced — you can't accidentally return the wrong type.
impl
server-streaming yield
The Python method becomes a generator. Every yield → one HTTP/2 DATA frame → one message to the client.
stream
grpc.insecure_channel
TLS is off for the demo. In production use `grpc.secure_channel` with credentials; you'll want mTLS for internal traffic.
transport
Try itBreak wire compatibility and watch it fail silently vs loudly

Edit demo.proto and change int32 shouts = 2; to int32 shouts = 5;. Rebuild the server only. Run the client (still on tag 2). The server will read shouts = 0 — silent data loss.

Then also change the type to string shouts = 5; on the server only, restart. The client sends an int; the server reads it as a mangled string. Parsing may fail or produce garbage.

The lesson: tags are the ABI. Reserve retired tags with reserved 2; and never repurpose them.

💡 Hint · Change a field's TAG number in .proto without re-generating on the client. The client will silently deserialise wrong fields. Now change a field's TYPE and re-generate one side only — you'll get parse errors. This is why Protobuf's backwards-compatibility rules matter.

(d) Production reality · 15 min

War story Google· 2015the entire internal service mesh
🔥 What broke

Google's internal RPC system (Stubby) was proprietary and enormous — every service, every language, every team ran on it. External adoption was zero because the code was private.

The rest of the industry had settled on REST + JSON, which Google engineers found painful compared to what they had internally: schema-first codegen, streaming, and 5-10× smaller wire payloads.

🧯 The fix
Google cleaned up Stubby, layered it on standard HTTP/2, and open-sourced it as gRPC. Within three years, Kubernetes, Envoy, etcd, CoreDNS, and Istio all standardised on gRPC. Today it's the de facto internal-service protocol across the cloud-native world.
🎓 Lesson to steal
Open-source the parts of your internal platform that solve universal problems. Google spent ~15 years on Stubby internally before gRPC captured the industry in ~3 years post-launch.
Post-mortem
War story Netflix· 2019internal microservice migration from Hystrix + REST
🔥 What broke
Netflix's original service mesh used REST + Hystrix for reliability. As the fleet grew past 1000 microservices, JSON parsing became a measurable CPU cost and REST's lack of streaming meant custom SSE/WebSocket bridges for real-time features (live playback events, personalisation updates).
🧯 The fix
Netflix adopted gRPC + Protobuf for internal service-to-service communication and standardised streaming for playback telemetry. CPU and bandwidth savings were substantial; new services default to gRPC unless there's a specific reason to expose REST.
🎓 Lesson to steal
At sufficient scale, JSON parsing alone can dominate CPU cost of a request. gRPC's efficiency pays back the codegen tax by 10× when your fleet is large.
Post-mortem
War story Common failure mode · everywherethe ‘.proto in the wrong repo’
🔥 What broke
Team A owns a service. Its .proto lives in the service repo. Team B, a client, forks it into their own repo and starts diverging. Six months later, A adds a field with tag=7. B, unaware, added tag=7 with a different type. Deploy day: silent misparsing across production.
🧯 The fix

The industry standard patterns:

  1. One central proto repository per org. Every service consumes generated packages from it (Buf, Bazel, or Nx pattern).
  2. Buf CLI (or similar) runs breaking-change detection on every PR. Removing a field, renumbering a tag, or narrowing a type fails CI.
  3. Versioned packages: demo.v1, demo.v2. Breaking changes mean a new package, not a mutated one.
🎓 Lesson to steal
Protos are contracts. Contracts need a single source of truth and automated breaking-change detection. Anything else is time-delayed suffering.

Where this shows up in the rest of the plan

gRPC is the backbone of cloud-native service meshes
S055 · HTTP fundamentals
HTTP/2 is the substrate. gRPC uses HEADERS + DATA + trailers.
S056 · REST design
The external counterpart. Most orgs run REST at the edge, gRPC internally.
S057 · GraphQL
The rich-UI alternative. Sometimes tunnelled through a gRPC backend.
S072 · Service mesh
Istio, Linkerd, Consul all speak gRPC natively — sidecars unwrap and re-wrap frames.
S117 · Streaming systems
gRPC bidi streaming is one of the delivery mechanisms for real-time features.
S128 · System design capstone
Every internal-services capstone assumes gRPC or an equivalent binary RPC.

(e) Recall + stretch · 10 min

Quick recall · click to reveal
★ = stretch question

Explain-out-loud test

Move on when you can teach these without notes:

  1. Why do big companies pick gRPC over REST internally?
  2. What is a Protobuf tag and why must you never renumber one?
  3. When would you specifically want a bidirectional stream instead of two unary RPCs?

What comes next

Hub: The 6-Month Learning Plan


Part of a 130-session evergreen learning series. Session structure: intuition → visual → hands-on → production war stories → recall. Duration: 90 minutes.