A private invite-only family archive with conversational AI search across decades of photos, scanned documents, and family history.
Échoes is a private, invite-only family history portal built for a single extended family. It brings a large vintage photo archive together with the documents a family gathers over the years, from letters and certificates to news clippings, all in one searchable, conversational space. Strong RLS and access controls keep deeply personal material safe, so the grandkids can ask the archive what they would once have asked an elder.
Échoes runs a production retrieval stack on Supabase with pgvector. Photos and documents are indexed for hybrid retrieval, fusing pgvector semantic search with PostgreSQL full-text search through Reciprocal Rank Fusion. Photo uploads move through a multi-stage AI pipeline: InsightFace (SCRFD, ONNX) detects faces server-side, MediaPipe maps a 478-point landmark mesh per face in the browser, and xAI Grok vision then describes the scene, estimates the era, and notes condition and identity cues. OpenAI embeddings are computed at the per-person-instance level, so a face ranks against every other appearance of that same person in the archive. That matters when a relative looks dramatically different from one decade to the next. Scanned documents take a separate path: vision-based OCR, SSN redaction enforcement, and chunk-level embeddings.
Its centerpiece is the Family Historian, a streaming RAG chat that answers natural-language questions across the entire archive and grounds every answer in citations. A query is first classified for intent and structured filters such as people, date range, location, and document type. It then runs through hybrid retrieval and is synthesized with multi-turn memory, with inline citations pointing back to the source photos and documents. A document-scoped variant narrows that same chat to a single open document, loading its chunks into context so you can interrogate just that page.
The Historian in action. Asking “Who served in the war?” returns a synthesized answer drawn from several records, a “From the archive” row of deduplicated reference tiles that link back to the exact photos, journals, and research docs behind it, and a set of suggested follow-up questions. Each query is classified for intent and structured filters, run through hybrid retrieval, and synthesized with multi-turn memory and inline citations.
Every photo in the archive can be filtered by decade and by family branch. On upload, each image runs through a multi-stage pipeline: InsightFace (SCRFD) detects faces, MediaPipe maps a 478-point landmark mesh per face, Grok vision adds a scene description along with era and identity cues, and OpenAI then embeds the result. Those steps make the same photo reachable three ways: by semantic search, by the people detected in it, or by the era and family filters you see here.
Identified family members, each shown with a portrait cropped from a photo they appear in. Every detected person in every photo gets its own per-instance OpenAI embedding, and face geometry is deliberately kept separate from identity so the embeddings never absorb unrelated scene description. That separation is what lets one person's appearances link up across decades, even when they age almost beyond recognition. A banner up top leads straight into the family tree.
A single person's page: a generated biography, every photo they appear in, and a relationship sidebar resolving parents, siblings, and children. Name resolution runs off aliases as the single source of truth (one preferred alias per person, enforced with a partial unique index), so every variant of a name collapses to one identity across the archive and in the Historian's answers.
Scanned letters, journals, certificates, and clippings, sorted into curated collections (here, “Bertha's WW2 Letters” and “Military Stories and Records”) alongside standalone documents. Each page flows through a vision-based OCR pipeline with chunk-level embeddings, behind an SSN-redaction guard that blocks any upload containing a detected social-security number before it is ever stored.
A stored document opened in the reader: a 1944 Western Union telegram reporting Pvt. Loren Schillie missing in action. A document-scoped Historian summary sits at the top, with a toggle between the scanned page image and its OCR'd transcript. In this mode the Historian loads the open document's chunks into context, so your questions are answered against exactly what you are reading.
The whole family rendered as a generational graph that runs from the great-grandparents down to the children's generation. Each person is a node, linked to the others by parent, sibling, spouse, and step-relationships. This same relationship data feeds the Historian, which is why a question like “who served in the war?” can reason about how people are related rather than only who happens to appear in a photo.
Streaming, citation-grounded conversational search across the entire archive. Query intent is classified, hybrid retrieval runs, results are synthesized with multi-turn memory and inline source citations. A document-scoped mode answers questions about an open document with its chunks loaded into context.
Each uploaded photo is analyzed for scene description, estimated era, suggested date, setting, condition, mood, visible text, color/BW classification, and per-person identity cues. Returns structured JSON that feeds metadata, search, and the people pipeline.
Scanned pages and PDFs are OCR'd via vision, chunked, and embedded for retrieval. Includes SSN redaction enforcement that blocks uploads containing detected social security numbers.
Semantic vector search fused with full-text search via Reciprocal Rank Fusion (RRF), scoped per asset type (photos, documents, people). Supports filter pushdown for date range, location, and document subtype.
Every detected person in every photo gets its own 1536-dim embedding. Similarity is ranked with Bayesian shrinkage and a gender filter so a face can be matched across decades despite appearance changes.
Natural-language queries are classified (asset_lookup / semantic / structured / combined) and parsed for entities: people with alias resolution, dates, and locations. The classification gates which retrieval strategy runs, keeping latency low for simple lookups while giving the Historian what it needs for synthesis questions.
Let's talk about what AI-powered web tools could do for your business.
Get in touch