Skip to main content

Architecture Overview

SAGE3 is a distributed real-time collaborative platform. The backend is composed of three Node.js servers, a React web frontend, a Python AI service, and supporting infrastructure (Redis, Jupyter, ChromaDB). All servers are written in TypeScript except the AI service which is Python.

Architecture overview

System Components​

ComponentLanguagePurpose
HomebaseTypeScript / Node.jsMain API server — REST, WebSocket, auth, collections
Homebase-YJSTypeScript / Node.jsReal-time collaborative editing (Y.js CRDT + WebRTC)
Homebase-FilesTypeScript / Node.jsFile upload, processing, and static serving
WebappTypeScript / ReactBrowser frontend
SeerPython / FastAPIAI agent service (LLMs, image analysis, PDF, web)
Redis Stack—Primary database and pub/sub message bus
JupyterPythonKernel execution for SageCell
ChromaDB—Vector database for AI semantic search
LiveKit—Self-hosted video server (SFU) for screen sharing, optional

All external traffic enters through a Traefik reverse proxy that routes requests based on path:

  • /yjs, /rtc → Homebase-YJS
  • /api/assets/upload, /api/assets/static → Homebase-Files
  • /sfu → LiveKit (screen-sharing signaling; the media itself goes directly to LiveKit over UDP 7882 / TCP 7881)
  • Everything else → Homebase

1. Backend

1.A Homebase (Main Server)​

Homebase is the primary backend server. It manages authentication, the Redis database, real-time WebSocket subscriptions, and serves the React webapp.

webstack/apps/homebase/
webstack/libs/backend/

1.A.1 HTTP / WebSocket API​

Homebase exposes a REST API built with Express.js and a WebSocket channel for real-time subscriptions.

  • HTTP (REST): Standard CRUD operations on all collections (GET, POST, PUT, DELETE).
  • WebSocket: Clients subscribe to collections or individual documents and receive live CREATE, UPDATE, DELETE events whenever data changes.
  • File uploads: Handled via the Homebase-Files server (see 1.C), but coordinated through the main API.
webstack/apps/homebase/src/api/
webstack/libs/backend/src/libs/generics/

1.A.2 SAGEBase​

SAGEBase is SAGE3's Redis abstraction layer. It provides a document-oriented database API on top of Redis, with built-in pub/sub for real-time change notifications.

Key concepts:

  • Collections hold typed JSON Documents.
  • Documents support CREATE, READ, UPDATE, DELETE operations.
  • Subscriptions fire on CREATE, UPDATE, DELETE events and are used to push changes to all connected WebSocket clients.
  • Every document includes _id, _createdAt, _createdBy, _updatedAt, _updatedBy fields managed automatically.

Example usage:

const sbConfig: SAGEBaseConfig = {
projectName: 'SAGE3',
redisUrl: 'redis://localhost:6379',
};
await SAGEBase.init(sbConfig);

type Car = { make: string; model: string; year: number };

const cars = await SAGEBase.Database.collection<Car>('cars', { model: '', year: 0 });

cars.subscribe((message) => {
// Called on CREATE, UPDATE, DELETE
});

const ref = await cars.addDoc({ make: 'Subaru', model: 'WRX', year: 2019 }, '');
await ref.update({ year: 2020 });
await ref.delete();

SAGEBase is also published as a standalone npm package: @sage3/sagebase

webstack/libs/sagebase/

1.A.3 Collections​

SAGE3 maintains the following collections in Redis:

CollectionDescription
appsAll open application instances on all boards
boardsBoard metadata (name, room, layout, whiteboard state)
roomsRoom metadata (name, visibility, PIN)
usersRegistered user profiles
assetsFile asset metadata (path, type, size, derived data)
presenceLive user cursors, viewports, and activity
messageBoard-level chat messages
pluginsUploaded plugin applications
roommembersRoom membership records
annotationsWhiteboard drawing strokes
linksApp-to-app property links (Linker app)
insightAI analysis results

1.A.4 Authentication​

Authentication is handled via Passport.js. Supported strategies:

  • Google OAuth — sign in with Google account
  • Apple — Sign in with Apple
  • CILogon — federated institutional login (universities and research labs)
  • JWT — machine-to-machine access (used by pysage3 and automation scripts)
  • Guest — anonymous access with a display name
  • Spectator — read-only observer access

Sessions use RS256 JWT tokens. Configuration is managed in sage3-prod.hjson.

1.A.5 Asset Manager​

The Asset Manager handles file uploads, processing, and storage. Files are organized by room and accessible across all boards in that room.

On upload, files are processed depending on type:

  • Images → scaled to multiple resolutions, converted to WebP for efficient display. Large images are additionally tiled into DeepZoom format.
  • PDFs → each page rendered as images at multiple resolutions.
  • Video / Audio / CSV / Other → stored as-is with metadata extracted via EXIFTool.
webstack/apps/homebase/src/api/routers/custom/asset.ts
webstack/apps/homebase/src/processors/

1.B Homebase-YJS (Collaborative Editing Server)​

A dedicated server for Y.js CRDT-based collaborative editing and WebRTC signaling.

  • Y.js WebSocket (/yjs) — enables conflict-free real-time collaborative editing for apps like Notepad and CodeEditor. Multiple users can type simultaneously with automatic merge.
  • WebRTC signaling (/rtc) — room-based signaling for peer-to-peer connections used by the Drawing (TLDraw) app's collaborative canvas.
webstack/apps/homebase-yjs/

1.C Homebase-Files (File Server)​

A dedicated server for file upload and static asset delivery.

  • Handles POST /api/assets/upload for file uploads (via Multer).
  • Serves GET /api/assets/static/:file for direct file access.
  • Runs separately to avoid blocking the main Homebase server during large uploads.
webstack/apps/homebase-files/

1.D Seer (AI Agent Service)​

Seer is a Python FastAPI server that provides AI-powered features for SAGE3. It connects to external LLM providers and processes AI requests forwarded from Homebase.

LLM providers are not hard-coded. Seer reads the server's capability-driven model registry (services.models, see AI Configuration) at start-up and picks a model by matching a request's task to a model's declared capabilities. How each provider is spoken to is inferred from its config: a url plus a model api_version means Azure OpenAI, a url alone means any OpenAI-compatible endpoint (LiteLLM, vLLM, Ollama, …), and no url means openai.com. Adding a provider is a configuration change, not a code change.

AI agents:

AgentEndpointDescription
ChatPOST /askGeneral LLM Q&A with message history
SummaryPOST /summarySummarize the linked content
CodePOST /codeCode assistance and explanation
ImagePOST /imageImage understanding and bounding box detection
Image generationPOST /image-generationGeneric image generation; the prompt is used as written
WebPOST /web, POST /webshotWeb page scraping, summarization and screenshots
PDFPOST /pdfPDF document analysis and Q&A
MesonetPOST /mesonetHawaii sensor / weather data queries
IdeatorPOST /dimensions, /node, /abstract, /user-dimension, /summarize, /prose, /ideator/imageBrainstorming support for the SageIdeator app

GET /status reports service health.

Image generation vs. the Ideator. POST /image-generation is the generic path used by any app: it sends the caller's prompt unchanged. POST /ideator/image is SageIdeator's, and composes a brainstorming prompt from ideator concepts (dimensions, the brainstorming prompt) before generating. They share the client-construction code but not the prompt, so the ideator's framing does not leak into ordinary image requests.

Per-request user credentials. A request may carry a userllm block (apiKey, optional baseUrl, modelId) when the user has supplied their own key in the web app. When present it takes precedence over the configured providers for that one request. Such models are never cached — the model cache is keyed by provider name, which every user of a personal key shares, so caching would hand one user's client and key to the next request. The key is used and discarded; it is never persisted, and UserLLM carries a redacting representation so a log line that prints a request shows apiKey='***'. See Users bringing their own key.

Seer uses ChromaDB for semantic vector caching and communicates back to SAGE3 using the pysage3 Python client library.

seer/

1.E Redis​

SAGE3 uses Redis Stack which includes the RedisJSON and RediSearch modules.

  • RedisJSON — stores documents as native JSON (no serialization roundtrip).
  • RediSearch — enables field-level querying and indexing within collections.
  • Pub/Sub — used by SAGEBase to broadcast document change events to all subscribed WebSocket clients.

Redis key naming follows the pattern:

SAGE3:DB:<collection>:<document-uuid>

1.F JupyterLab / Kernel Server​

SAGE3 integrates with JupyterLab to provide SageCells — code cells placed directly on the board backed by Jupyter kernels.

  • The Kernel Server (kernelserver) is a Python FastAPI service that manages kernel lifecycle (create, list, interrupt, shutdown).
  • A JupyterLab instance provides the actual Python/R/Julia execution environment.
  • SageCell communicates with kernels via the Kernel Server API on Homebase at /api/kernels.

2. Frontend

SAGE3's frontend is a React 18 web application written in TypeScript. It is served by Homebase and runs in the browser or in the SAGE3 Electron desktop client.

2.A Electron Client​

The SAGE3 Electron client wraps the web app in a desktop application for macOS, Windows, and Ubuntu. It enables additional capabilities not available in a standard browser:

  • Webview streaming — stream the content of a Webview app to other users.
  • Multi-display support — span across tiled display walls.
  • Command line arguments for display configuration (--monitor, --fullscreen, --width, --height, etc.).

See the Electron Client page for development details.

2.B Webapp (React)​

The React frontend is organized as an Nx monorepo library structure.

webstack/apps/webapp/ # React entry point and routing
webstack/libs/frontend/ # Stores, hooks, and shared UI components
webstack/libs/applications/ # All SAGE3 app modules (22 supported apps + experimental)
webstack/libs/shared/ # Types, schemas, utilities (shared by frontend + backend)

Routes​

RoutePage
/Login
/homeRoom and board selection
/board/:roomId/:boardIdMain collaboration board
/accountUser settings
/adminAdmin dashboard (admin users only)

2.B.1 HTTP / WebSocket Communication​

webstack/libs/frontend/src/lib/api/http/api-https.ts # REST calls
webstack/libs/frontend/src/lib/api/ws/api-socket.ts # WebSocket subscriptions

2.B.2 Zustand Stores​

All frontend state is managed through Zustand stores. Components never call the API directly — they read and write through stores.

webstack/libs/frontend/src/lib/stores/
StoreDescription
AppStoreOpen applications on the current board (CRUD, batch ops)
BoardStoreBoard metadata and operations
RoomStoreRoom listing and management
UIStoreBoard UI state: scale, position, selected apps, panel visibility, lasso state
AssetStoreAsset library (files uploaded to the room)
UsersStoreRegistered users and their profiles
PresenceStoreLive user cursors and viewport positions
MessageStoreBoard chat messages
PluginStoreUploaded plugin apps
InsightStoreAI analysis results
AnnotationStoreWhiteboard drawing strokes
LinkStoreApp-to-app property links
KernelStoreJupyter kernel sessions
ScreenshareStoreScreen sharing: LiveKit room connection, remote tracks, local captures
TwilioStoreScreen sharing video state (legacy Twilio backend)

2.B.3 React Hooks​

Custom hooks for common patterns:

webstack/libs/frontend/src/lib/hooks/
  • useAuth — authentication state and login helpers
  • useUser — current user profile
  • useCursorBoardPosition — converts screen cursor position to board coordinates
  • useHotkeys — keyboard shortcut registration
  • useHexColor — Chakra UI color token resolution
  • useRouteNav — navigation helpers (go to board, go home)
  • useWindowResize — responsive layout triggers
  • usePeer — WebRTC peer connection management

2.B.4 Application Framework​

Each SAGE3 app is a self-contained module in webstack/libs/applications/src/lib/apps/<AppName>/:

  • index.ts — Zod schema defining the app's state shape, default values, and display name.
  • AppName.tsx — Two React components exported: AppComponent (the window) and ToolbarComponent (the toolbar).
  • styling.css — Optional app-specific CSS.

The app registry is auto-generated at webstack/libs/applications/src/lib/apps.ts and updated by running yarn regen.


3. Supporting Services

3.A Traefik (Reverse Proxy)​

Traefik handles TLS termination and routes incoming HTTPS traffic to the correct backend service based on URL path and priority rules. In production, all services are accessible through a single HTTPS port (443).

3.B OAuth Providers​

User authentication is delegated to third-party identity providers via Passport.js:

  • Google — OAuth 2.0
  • Apple — Sign in with Apple
  • CILogon — OpenID Connect for institutional (university) accounts
  • Keycloak — Self-hosted OpenID Connect identity provider
  • Guest / Spectator — no external provider required

Configuration lives in the auth section of sage3-prod.hjson. See Server Deployment for setup.

3.C Screen Sharing (LiveKit)​

Screen sharing runs on a self-hosted LiveKit server deployed alongside SAGE3. It is an SFU (selective forwarding unit): a user publishes their screen once and the server relays it to every viewer on the board, so bandwidth does not grow with the number of viewers as it would peer-to-peer. One LiveKit room exists per board, named after the board id. Homebase mints short-lived join tokens (/livekit/token, identity derived from the session), and LiveKit calls back into Homebase (/livekit/webhook, signature-verified) when a published track disappears, which is what removes a screenshare window the moment its publisher stops. The only configuration is a shared secret in the deployment's .env; without it the server offers no screen sharing. Which backend a server offers (livekit, twilio, or none) is reported to clients in the public configuration, and the UI only offers what exists.

Twilio Video remains supported as a legacy backend for servers that have not migrated. When both are configured, LiveKit is used. Twilio support will be removed in a future release.

3.D Fluentd (Logging)​

Fluentd aggregates server-side logs from all services. The logging verbosity is configurable (all, partial, none) in sage3-prod.hjson.