Architecture Overview

The Virtufin WebSocket Manager is a distributed WebSocket connection management service built on .NET with gRPC API support and Dapr integration for state persistence and pub/sub messaging.

System Architecture

┌─────────────────────────────────────────────────────────────────────────────┐
│                           WebSocket Manager                                 │
│                                                                             │
│  ┌──────────────┐     ┌──────────────────┐     ┌────────────────────────┐  │
│  │ gRPC API     │────▶│ WebSocketService │────▶│ WebSocketClientWrapper │  │
│  │ (Protobuf)   │     │                  │     │                        │  │
│  └──────────────┘     └──────────────────┘     └───────────┬────────────┘  │
│                           │                                 │                │
│                           ▼                                 ▼                │
│                    ┌──────────────────┐           ┌─────────────────┐      │
│                    │DistributedStore  │           │  External       │      │
│                    │                  │           │  WebSocket      │      │
│                    └────────┬─────────┘           │  Servers        │      │
│                             │                     └─────────────────┘      │
│                             ▼                                            │
│                    ┌──────────────────┐                                  │
│                    │ DaprConnection   │                                  │
│                    │ Repository       │                                  │
│                    └────────┬─────────┘                                  │
│                             │                                            │
└─────────────────────────────┼────────────────────────────────────────────┘
                              │
              ┌───────────────┼───────────────┐
              ▼               ▼               ▼
      ┌──────────────┐ ┌──────────────┐ ┌──────────────┐
      │Dapr State    │ │Dapr Pub/Sub  │ │Dapr Runtime  │
      │Store        │ │              │ │              │
      └──────────────┘ └──────────────┘ └──────────────┘

Component Overview

gRPC API Layer

The entry point for all client operations. The WebSocketManagerService implements the protobuf-defined gRPC service, handling: - Connection lifecycle (connect, disconnect, list) - Message publishing control (start/stop publish) - Message sending (send, send raw)

Service Layer

Core business logic implemented by WebSocketService. Orchestrates: - Connection creation and management - Message routing to Dapr pub/sub - Ownership validation against instance IDs

Storage Layer

Distributed persistence via Dapr: - DistributedWebSocketConnectionStore - manages local cache + Dapr state - DaprConnectionRepository - handles raw Dapr state operations

Client Layer

Low-level WebSocket operations via WebSocketClientWrapper: - Connection establishment - Message sending/receiving - Auto-reconnect logic

Integration Layer

• DaprPublisher - publishes messages to Dapr pub/sub topics
• InstanceIdProvider - identifies which service instance owns each connection

Data Flow

Connecting to a WebSocket

1. Client sends Connect gRPC request with URL and autoReconnect flag
2. WebSocketManagerService delegates to WebSocketService
3. WebSocketService creates a WebSocketConnection via the store
4. WebSocketClientWrapper establishes the actual WebSocket connection
5. Receive loop is started to handle incoming messages
6. Connection is persisted to Dapr state store
7. Connection ID is returned to client

Publishing Messages

1. External WebSocket server sends a message
2. WebSocketClientWrapper receive loop captures the message
3. If connection has a topic configured, DaprPublisher is called
4. DaprPublisher publishes message to the configured Dapr pub/sub topic
5. Subscribers receive the message with connection metadata (ID, URL, timestamp)

Distributed State

• Each service instance maintains a local cache (ConcurrentDictionary)
• All state is persisted to Dapr state store for cross-instance sharing
• Instance ID ownership prevents cross-instance modifications
• On read, local cache is checked first, then Dapr state store

Key Design Decisions

Instance Ownership

Connections are owned by specific service instances (identified by InstanceIdProvider). Before any modification, the service verifies it owns the connection. This prevents race conditions in distributed deployments.

Two-Tier Storage

The DistributedWebSocketConnectionStore maintains both: - Local in-memory cache (fast access) - Dapr state store (distributed persistence)

This ensures minimal latency for local operations while supporting distributed scenarios.

Correlated Request-Response

SendAndWaitAsync wraps messages with correlation IDs, enabling correlation between requests and responses without modifying external WebSocket protocols.

Auto-Reconnect

The WebSocketClientWrapper implements exponential backoff reconnection, configurable via WebSocketOptions.ReconnectMaxAttempts and ReconnectBaseDelayMs.

Dependency Injection Setup

All services are registered as singletons in Program.cs:

builder.Services.AddSingleton<IWebSocketConnectionStore, DistributedWebSocketConnectionStore>();
builder.Services.AddSingleton<IWebSocketClientWrapper, WebSocketClientWrapper>();
builder.Services.AddSingleton<IWebSocketService, WebSocketService>();
builder.Services.AddSingleton<IDaprPublisher, DaprPublisher>();
builder.Services.AddSingleton<IDaprConnectionRepository, DaprConnectionRepository>();
builder.Services.AddSingleton<IInstanceIdProvider, DefaultInstanceIdProvider>();

The DaprClient is configured with custom HTTP and gRPC endpoints based on PortConstants.DaprHttpPortKey and PortConstants.DaprGrpcPortKey.