Home > Notes > System-Design-Notes > grpc

What is RPC (Remote Procedure Call)?

RPC is a protocol that allows a program (client) to execute a subroutine or procedure on a remote server as if it were a local call, abstracting the underlying network communication. Introduced in the 1980s (e.g., Sun RPC), RPC enables distributed computing by treating remote functions as local, hiding details like marshalling (serializing arguments), network transport, and unmarshalling (deserializing results).

Core Principles of RPC

How RPC Works (Step-by-Step)

  1. Client Invocation: Calls stub function (e.g., add(2, 3)).
  2. Marshalling: Stub serializes arguments (2, 3) into packet with procedure ID.
  3. Transmission: Packet sent over network to server (via socket).
  4. Server Reception: Server stub receives, unmarshalls arguments, calls actual add() function.
  5. Execution: Server computes result (5).
  6. Marshalling Response: Stub serializes result, sends back.
  7. Client Reception: Client stub unmarshalls, returns 5 to caller.

Advantages of RPC

Disadvantages

RPC forms the basis for modern systems like ONC RPC, DCE RPC, and gRPC.

gRPC Protocol

gRPC is Google's modern evolution of RPC, built on HTTP/2 for transport and Protocol Buffers (Protobuf) for serialization. Launched in 2015, gRPC supports unary, streaming RPCs, bidirectional communication, and strong typing, making it ideal for microservices, mobile backends, and IoT. It emphasizes efficiency (binary format, multiplexing), reliability (TLS, deadlines), and extensibility (interceptors, metadata).

Key Features of gRPC

How gRPC Works (Step-by-Step)

  1. Define Contract: Write .proto file with messages (data types) and services (RPC methods).
  2. Generate Code: Use protoc compiler to create language-specific stubs (e.g., Python client/server classes).
  3. Server Implementation: Implement service methods; start gRPC server (binds to port).
  4. Client Invocation: Client stub makes call (e.g., stub.GetUser(request)); marshalls request, sends over HTTP/2.
  5. Transmission: HTTP/2 stream: Headers (method, metadata), body (marshalled request), trailers (status).
  6. Server Processing: Server stub unmarshalls, calls method, marshalls response.
  7. Response: Sent back via same stream; client unmarshalls and returns.

Protobuf Concepts (Integrated with gRPC)

Protobuf is gRPC's IDL and serialization layer—a schema-driven binary format for structured data.

Example .proto:

syntax = "proto3";
package user;

message User { int32 id = 1; string name = 2; }
message GetUserRequest { int32 id = 1; }

service UserService {
  rpc GetUser(GetUserRequest) returns (User);
}

Generated files explained below.

Files Generated After Writing .proto (Server Side)

After writing .proto and running protoc --python_out=. --grpc_python_out=. user.proto, two Python files are generated. These are language bindings for the schema—stubs for client/server.

1. user_pb2.py (Message Definitions)

class User(object): def init(self): self.id = 0 self.name = "" self._internal_metadata = ...

  def SerializeToString(self):
      # Binary serialization
      pass

  @classmethod
  def ParseFromString(cls, data):
      # Deserialize binary to object
      pass

`` - **Usage**: Create instances:user = user_pb2.User(id=1, name="Alice"); serialize:bytes = user.SerializeToString()`.

2. user_pb2_grpc.py (Service Stubs)

class UserServiceStub(object): def init(self, channel): self.GetUser = channel.unary_unary( '/user.UserService/GetUser', request_serializer=user_pb2.GetUserRequest.SerializeToString, response_deserializer=user_pb2.User.FromString, ) `` - **Usage**: Server: InheritUserServiceServicerand implementGetUser. Client: UseUserServiceStub(channel)to callstub.GetUser(request)`.

Nuances: Files are auto-generated; regenerate on .proto changes. Protobuf ensures type safety; gRPC handles transport.

How the Client Calls the Server (Request Flow)

gRPC calls are local-like but networked. The flow uses generated stubs for transparency.

Step-by-Step Client-Server Interaction

  1. Client Setup: Create channel (grpc.insecure_channel('localhost:50051') or secure with TLS).
  2. Stub Creation: stub = user_pb2_grpc.UserServiceStub(channel).
  3. Request Preparation: Create message: request = user_pb2.GetUserRequest(id=1).
  4. Marshalling: Stub serializes request (Protobuf to binary).
  5. Transmission: Sent over HTTP/2 stream:
  6. Headers: Method (/user.UserService/GetUser), metadata (e.g., auth).
  7. Body: Binary request.
  8. Trailers: Status (e.g., OK), details.
  9. Server Reception: Server stub receives binary, unmarshalls to GetUserRequest.
  10. Execution: Calls implemented method (e.g., query DB).
  11. Response: Marshalls User to binary, sends back via stream.
  12. Client Reception: Stub unmarshalls binary to User object; returns to caller.
  13. Error Handling: If failure (e.g., DEADLINE_EXCEEDED), raises grpc.RpcError with status.

Full Python Example (From Earlier, Expanded)

Server (server.py) – Implements service:

import grpc
from concurrent import futures
import user_pb2
import user_pb2_grpc
import time

class UserServiceServicer(user_pb2_grpc.UserServiceServicer):
    def GetUser(self, request, context):
        # Business logic (e.g., DB query)
        if request.id == 1:
            user = user_pb2.User(id=1, name="Alice", tags=["dev", "gRPC"])
        else:
            context.set_code(grpc.StatusCode.NOT_FOUND)
            context.set_details('User not found')
            return user_pb2.User()
        return user

    def ListUsers(self, request, context):
        users = [
            user_pb2.User(id=1, name="Alice", tags=["dev"]),
            user_pb2.User(id=2, name="Bob", tags=["ops"]),
        ]
        for user in users[:request.limit]:
            yield user
            time.sleep(0.5)  # Simulate streaming delay

def serve():
    server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))
    user_pb2_grpc.add_UserServiceServicer_to_server(UserServiceServicer(), server)
    server.add_insecure_port('[::]:50051')
    server.start()
    print("gRPC server listening on port 50051")
    server.wait_for_termination()

if __name__ == '__main__':
    serve()

Client (client.py) – Calls server:

import grpc
import user_pb2
import user_pb2_grpc

def run():
    with grpc.insecure_channel('localhost:50051') as channel:
        stub = user_pb2_grpc.UserServiceStub(channel)

        # Unary call
        try:
            response = stub.GetUser(user_pb2.GetUserRequest(id=1))
            print(f"Unary: User {response.id} = {response.name}")
        except grpc.RpcError as e:
            print(f"Unary Error: {e.code()} - {e.details()}")

        # Streaming call
        print("Streaming users:")
        try:
            for user in stub.ListUsers(user_pb2.ListUsersRequest(limit=2)):
                print(f"Streamed: {user.name} (ID: {user.id})")
        except grpc.RpcError as e:
            print(f"Streaming Error: {e.code()} - {e.details()}")

if __name__ == '__main__':
    run()

Running: 1. Generate files (as above). 2. Start server: python server.py. 3. Run client: python client.py. - Output: Unary user details; streamed users with delay.

Nuances in Client-Server Flow

gRPC and Protobuf enable efficient, typed RPCs. For bidirectional streaming or TLS setup, let me know!