The problem
Large language models are stateless: every conversation starts from zero, and anything you taught the assistant yesterday is gone today. Existing memory tools tend to be bolted onto a single app for a single user — the knowledge can't follow you across assistants, and it can't be shared safely between people.
The approach
Recall is a standalone memory service that speaks the Model Context Protocol (MCP), so any MCP-compatible client — Claude, IDE agents, custom tools — can store and retrieve knowledge through the same server. Three design decisions shaped it:
- Hybrid retrieval, not just vectors. Memories are embedded with Voyage AI and stored in Postgres via pgvector, but semantic similarity alone misses exact identifiers and names. Recall runs vector search and keyword full-text search side by side, then fuses the ranked lists with Reciprocal Rank Fusion (RRF) so both kinds of relevance count.
- Multi-user from day one. OAuth sign-in with GitHub and Google, with sessions carried by EdDSA-signed JWTs, so every memory is scoped to its owner rather than to one global store.
- Real infrastructure, not a demo. The service is deployed the way a production API would be — containerized, load balanced, cached, and shipped by CI/CD with no long-lived cloud credentials.
Architecture
- API: FastAPI, containerized and running on AWS ECS Fargate (ARM64 for better price/performance), behind an Application Load Balancer with HTTPS.
- Storage: RDS Postgres with the pgvector extension — one database for both structured data and vector search.
- Cache: Valkey (Redis-compatible) for hot paths and session data.
- Frontend: Next.js dashboard deployed on Vercel.
- CI/CD: GitHub Actions authenticating to AWS with OIDC — deploys use short-lived federated credentials, so no AWS secrets live in the repo.
Highlights
- Hybrid semantic + keyword search fused with Reciprocal Rank Fusion.
- Multi-user OAuth (GitHub & Google) with EdDSA-signed JWT sessions.
- Keyless deployments: GitHub Actions → AWS via OIDC federation.
- ARM64 (Graviton) containers on Fargate for cost-efficient compute.
- Works with any MCP-compatible assistant, not one specific app.
Stack
- Python
- FastAPI
- PostgreSQL + pgvector
- Voyage AI embeddings
- Valkey (Redis)
- AWS ECS Fargate (ARM64)
- RDS
- ALB / HTTPS
- GitHub Actions + OIDC
- Next.js
- Vercel
- MCP