What are Protobufs?

Protobufs are a serialization format used in gRPC to send data, and one of the reasons why gRPC is faster than traditional HTTP APIs.

At a high level, Protobuf takes structured data and compresses it into a compact sequence of bytes.

These bytes aren’t random — they’re made up of small segments that together describe the data efficiently. Each field in the message is encoded using:

• Tags
• Wire types
• Encoding (like varints)
• Values

Let’s Start with Wire Types

Wire types define how the data is stored in binary.

Protobuf uses a few core ones:

• 0 → Varint (int32, int64, bool, enum) — variable length (1–10 bytes)

• 1 → 64-bit (fixed64, double) — always 8 bytes

• 2 → Length-delimited (string, bytes, nested messages) — length prefix + payload

• 5 → 32-bit (fixed32, float) — always 4 bytes

These types tell the decoder how to interpret the next set of bytes.

Next: Tags

Before writing any value, Protobuf writes a tag.

This tag tells the decoder:

• which field this is
• what type of data follows

The tag combines two things into one number:

• Field number
• Wire type

Using this formula:

tag = (field_number << 3) | wire_type

So instead of sending field names like "weight" or "name", Protobuf sends a small number that represents both identity and type.

Up Next: Varints

Varints are how Protobuf encodes integers efficiently.

Instead of fixed-size bytes, it uses variable-length encoding, where:

• Each byte stores 7 bits of data

• The most significant bit (MSB) is a continuation flag:

  • 1 → more bytes coming
  • 0 → last byte

Example: Encoding 300

1. Binary

300 → 100101100

2. Split into 7-bit chunks (from right)

0101100   0000010

3. Reverse (little-endian order)

0000010   0101100

4. Add continuation bits

1 0000010   (more bytes)
0 0101100   (last byte)

Final bytes:

10000010 00101100

Encoding Values (Putting It All Together)

Let’s take a simple message:

message Fruit {
  int32 weight = 1;
  string name = 2;
}

Field 1: weight = 150

• Field number = 1
• Wire type = 0 (varint)

tag = (1 << 3) | 0 = 8 → 08

Varint encoding of 150:

96 01

Final:

08 96 01

Field 2: name = "Apple"

• Field number = 2
• Wire type = 2 (length-delimited)

tag = (2 << 3) | 2 = 18 → 12

Now comes the important part

Length-Delimited Structure

For strings, Protobuf encodes:

1. Tag
2. Length of the string
3. Actual bytes (UTF-8)

UTF-8 Encoding

"Apple" → 41 70 70 6c 65

Length = 5 bytes

Final Encoding

12 05 41 70 70 6c 65

• 12 → tag
• 05 → length
• 41 70 70 6c 65 → "Apple"

Final Encoded Message

08 96 01 12 05 41 70 70 6c 65

Breakdown

• First 3 bytes → weight (150)
• Next 7 bytes → name ("Apple")

Why This Matters

This is why Protobuf is efficient:

• No field names → smaller payload
• Varints → fewer bytes for small numbers
• Length-delimited fields → structured but compact
• Binary format → faster parsing

Closing Thought

Efficiency isn’t just about algorithms — it’s also about how data is represented.

Protobufs don’t change your logic. They just make everything underneath… faster.

Read Part 2 here