Back to portfolio
ProjectLive

Échoes

A private family archive you can talk to. Ask it about decades of photos and letters, and let the Family Researcher follow the trail from one record to the next.

Tech Stack

Next.js 16TypeScriptSupabasepgvectorVercel AI SDKVercel AI GatewayxAI / GrokAnthropic / Claude OpusOpenAI embeddingsInsightFace (SCRFD)ONNX RuntimeMediaPipeTiptapTailwind CSS

Overview

Échoes is a private, invite-only archive built for one extended family. It gathers decades of vintage photos together with the letters, certificates, and clippings a family collects over the years, all in one place you can search and talk to. Only invited family members can get in, so the grandkids can ask the archive what they would once have asked an elder.

Some questions cannot be answered by one search. Ask “who does Bertha's obituary mention that we don't have on record?” and the evidence is spread across letters, an obituary, and the family tree, with no single document matching the question. The Family Researcher handles those. It searches, looks people up, checks how they are related, and searches again from what it just found, narrating each step on screen. When it spots people or relationships the tree is missing, it proposes them, and a family member confirms each one before anything is saved.

Underneath, every photo and document is read by AI when it is uploaded. Photos get a description and an estimated era, faces are matched across decades, and scanned pages become readable text. That is what lets the archive answer a question asked in plain English, even when the words do not match what is written on the page.

Inside the app

Screenshot gallery

The Family Historian

Asking “Who served in the war?” returns an answer drawn from several records, a row of reference tiles linking to the exact photos and documents behind it, and suggested follow-up questions.

The Family Researcher

The Historian's more capable sibling: given a question no single search can answer, it uses tools to search the archive, look people up, and check the family tree, then searches again from what it found, narrating every step. I have used it for real research on my own family history, and it works.

The photo archive

Every photo, filtered by family branch, decade, or theme, from gatherings and portraits to keepsakes and everyday moments. Each one is also searchable by what is in it and who is in it.

People

Identified family members, each with a portrait cropped from a photo they appear in. The same person is matched across decades, even when they age almost beyond recognition.

A person's profile

A single person's page: a generated biography, every photo and story they appear in, and their relationships grouped by generation, from grandparents through siblings and children. Every spelling and nickname resolves to one person.

A person's life story

One click turns everything the archive knows about a person, their photos, documents, key dates, and family connections, into a typeset, printable life story, with every fact footnoted to the letter, clipping, or record it came from. It is the keepsake families spend weeks assembling by hand, or pay someone to produce, ready to share in a couple of minutes.

Stories & documents

Scanned letters, journals, certificates, and clippings, sorted into collections like “Bertha's WW2 Letters” alongside standalone documents. Any upload containing a social security number is blocked before it is stored.

Document reader

A 1944 Western Union telegram reporting Pvt. Loren Schillie missing in action, with a summary up top and a toggle between the scanned page and its transcript. Questions asked here are answered from this document alone.

How it was built

Selected Design Decisions

Agent proposes, human approves, code writes. The Researcher never edits the family tree itself. After it answers, a separate pass compares the people the documents named against the tree and proposes what is missing, and a family member confirms, edits, or rejects each row before anything is written. An earlier version gave the model a tool to propose additions directly; it reliably skipped that step and sometimes claimed in prose to have done it. So proposing is no longer the model's to decide.

The Historian answers once. The Researcher follows the trail. Both run on the same search underneath. The Historian searches once and answers. The Researcher loops, and every step of that loop is visible on screen and saved afterward, with a fixed limit on how many steps it may take before it has to conclude from what it found. Relationships like “great-great-grandfather” come from the family tree, not from the model counting generations.

Think harder is a second model, not a longer prompt. After a standard run, a family member can send the exact same question to a stronger reasoning model. The button only appears when a different model is actually configured, so it never re-runs the same one and pretends to be deeper.

Under the Hood

AI Integrations

Family Researcher agent

xAI / Grok + Anthropic / Claude Opus

Multi-step search that plans, searches, checks the family tree, and searches again, showing each step live. Read-only: any proposed change to the tree goes to a person for review.

Family Historian chat

xAI / Grok

Conversational search across the whole archive, every answer linked to its source photos and documents. A document-scoped mode answers questions about only the page you are reading.

Photo understanding

xAI / Grok + OpenAI embeddings

Each photo gets a description, an estimated era, and a per-person signature so the same face can be found across decades of appearance changes.

Document reading

xAI / Grok (vision)

Scanned letters and records become searchable text. Uploads containing a social security number are blocked before they are stored.

Search that understands meaning

Supabase pgvector + full-text search

Meaning-based and keyword search combined, so a question in plain English finds the right records even when the words do not match.

Want to talk about this work?

I'm available for contract and full-time roles, and happy to walk through any of these systems in detail.

Get in touch