Engineering

41 postsslug · engineeringAll topics

September 22, 20265 min read

Flat image in, living character out

In short: give the pipeline a JPG or PNG of a character and it produces a layered rig plus a small web player that breathes, blinks, shows emotion and lip-syncs to a real voice. The steps are…

September 8, 20265 min read

Grounded answers or honest refusals: a rule for retrieval over personal files

In short: when a system answers questions over someone's own files, there is one rule I would not trade for anything: every answer quotes the person's own sources verbatim, or it is an honest refusal.…

August 25, 20265 min read

A second legal system: what breaks when a legal pipeline changes country

In short: taking a legal pipeline built for Canadian law to a second legal system is less about code that crashes than about assumptions: that a document is in English unless told otherwise, that a…

August 11, 20265 min read

What happens when the AI goes down mid-session

In short: when the model provider fails mid-sentence, the user sees whatever the companion had already said, followed by a calm, pre-written pause line. Their unanswered question is held, and the next…

July 28, 20265 min read

Emotion detection that must never slow the voice

In short: the emotion classifier in the supervised care companion runs beside the reply, never in front of it. It starts when the user's words arrive, nothing waits for it, and when it finishes it…

July 14, 20265 min read

Barge-in is the whole product

In short: a voice companion you cannot interrupt is a recording with extra steps. Barge-in, the user talking over the reply and the reply stopping, is what makes it feel like a conversation, and it…

June 16, 20265 min read

Warm intros from a graph: shortest path between two people

In short: put people and the organisations they belong to in one graph, and "who can introduce me" becomes a shortest-path query. I built a small demo of this on public Wikidata records for about six…

June 2, 20265 min read

An adversarial persona suite: how I attack my own chatbot

In short: I keep a file of attacks on my own chatbot, each one tied to a written rule, and run them through the real conversation graph against a live model. A broken hard rule fails the run with its…

May 19, 20265 min read

Fail closed: designing an AI for people who can't afford a bad answer

In short: when the user can't afford a bad answer, every part of the system that can fail has to fail towards the safe answer, not towards the model. In a supervised care product I'm building, a voice…

May 5, 20264 min read

Citations that point at the wrong paragraph

In short: a citation can be wrong in at least seven different ways, and only two of them involve the model. Most come from how long documents were cut into chunks. Name each failure, and give each one…

April 21, 20265 min read

When BM25 beat the embeddings

In short: on legal text, a keyword ranking function from the 1990s wins a whole class of queries that dense embeddings lose: citations, section numbers, defined terms. I run both and fuse the…

April 7, 20265 min read

How I evaluate retrieval before anyone asks a question

In short: before a single user types a question, I write the questions myself, mark which passage answers each one, and measure how often that passage comes back in the top k. That one number, recall…

March 24, 20265 min read

Laws have versions, and most RAG pretends they don't

In short: a section of a statute is not one text, it's a series of texts, each in force for a stretch of time. If your retrieval only knows the current one, it will answer a question about 2021 with…

March 10, 20265 min read

Ingesting a country's law when every province has its own format

In short: Canada's legislation lives on a different website for nearly every province and territory, as PDFs, Word files, an XML API and plain web pages. When a product depends on that text, the…

January 13, 20265 min read

RAG with a persona: when the knowledge base and the character disagree

In short: when a character and its knowledge base disagree, let the knowledge base win on content and the character win on voice, and let safety rules beat both. You enforce that in how you build the…

December 16, 20255 min read

Rules vs transformers for extracting metadata from legal documents

In short: on the same Spanish-language legal documents, the transformer path was more accurate and slower. The rules were fast, cheap and easy to explain. I kept both: rules by default, the…

November 18, 20255 min read

Rewriting a Next.js backend in Python without users noticing

In short: I moved an app's API out of Next.js routes and into a FastAPI service one endpoint at a time, keeping every request and response shape the same, and only pointed the frontend at the new…

November 4, 20255 min read

Streaming speech: where the latency actually goes

In short: in a voice pipeline that waits for each stage to finish, the first sound waits for the whole answer to be written and then spoken. Streaming doesn't make any stage faster. It lets the stages…

October 21, 20254 min read

A voice assistant in a weekend: LLM to speech over WebSockets

In short: a working voice assistant is one WebSocket, one language model call and one text-to-speech call, in that order. I built one in a weekend with FastAPI and OpenAI's APIs. It works, it's…

September 9, 20255 min read

Speech to facial animation: a week with Audio2Face

In short: NVIDIA's Audio2Face turns speech into a stream of ARKit blendshape weights, with emotion folded in. It gives a talking character believable lips with no animator, but it is one stage in a…

August 12, 20255 min read

Leakage: the bug that makes every metric lie

In short: leakage is when information from the test set, or from the answer itself, sneaks into training. The three kinds I check for every time are the same person on both sides of the split,…

July 29, 20255 min read

Classical ML or an LLM? The decision before touching code

In short: if the input is numbers or a signal and you have labelled examples, I start with a classical model. If the input is language and the job is to read or write it, I start with an LLM. Most of…

July 1, 20255 min read

Agents with opinions: a weekend multi-agent build

In short: I built a small stock analyzer where five agents, each with a fixed investing style, look at the same company and give their own reading. The per-agent outputs are not the interesting part.…

June 3, 20255 min read

Class imbalance in clinical data: what actually worked

In short: oversampling and class weights mostly move you along the same precision-recall curve. What actually worked was picking the decision threshold on purpose, on a validation set, against a…

May 6, 20255 min read

Data that doesn't match across sites

In short: when several sites collect "the same" data, the columns share names and almost nothing else. Before any model, write one data dictionary, map every site to it in code, and check the result…

April 22, 20254 min read

Explaining a model to a doctor

In short: doctors don't want to see how the model works. They want to know why it said this about this patient, in units they already use, and whether they should trust it. A simple bar chart with…

April 8, 20254 min read

Monitoring a model that listens to patients

In short: for a voice model in healthcare, the true labels arrive late or never, so you can't watch accuracy. You watch the recordings coming in, the scores going out, and the plumbing in between, and…

March 25, 20255 min read

Cloud Run or EC2: how I chose for a model that listens

In short: if the service takes a finished recording, scores it and answers, Cloud Run is usually the simpler and cheaper home. If it holds a live audio stream open, needs a GPU, or has to stay warm…

March 11, 20255 min read

Audio preprocessing that matters more than the model

In short: on medical audio, the steps before the model decide most of what the model can learn. Resampling, trimming, filtering, fixed-length windows and a split by patient each remove a shortcut the…

February 25, 20255 min read

The clinician said no, and was right

In short: a clinical model can score beautifully because it has learned something the clinic already decided. The fastest way to catch that is not a better metric. It is ten minutes with the person…

February 11, 20254 min read

What "in production" means in a hospital

In short: in a hospital, a model is in production when a clinician can see its output during real care, a named group has approved that, a named person owns it when it breaks, and the ward runs fine…

January 14, 20255 min read

Merging two sensor streams that tick at different rates

In short: to put a 100 Hz signal and irregular detections on one 10 Hz clock, summarise the dense stream per bin, count and carry forward the sparse one, and make sure no bin ever sees a sample from…

December 31, 20244 min read

The boring checklist that saved every project

In short: before I train anything I want three things: a record of where every piece of data came from, a dumb baseline to beat, and one number the people paying for the work actually care about. None…

December 17, 20244 min read

Respiratory sound classification, end to end

In short: I took a public set of lung sound recordings all the way from raw audio to a web demo anyone can upload a file to. The model was the smallest part. Most of the work was deciding what to…

December 3, 20244 min read

Running AI on your own machines

In short: when data is not allowed to leave the building, the smallest useful setup is two machines on the same network: one that asks, one with the GPU that answers. It takes about thirty lines of…

November 19, 20245 min read

Making a Sankey diagram people actually read

In short: a Sankey diagram works when a reader can follow one flow from left to right without reading a legend. Most unreadable ones fail on the same three things: too many nodes, an order nobody…

November 5, 20244 min read

Real-time transcription on a laptop: what Whisper gets wrong

In short: live on a laptop, Whisper invents text in silence, repeats itself, transcribes your speakers, cuts words at chunk edges and mangles names. Most of the fixes sit around the model, not in it:…

October 8, 20244 min read

What an earbud's motion sensor can tell you

In short: a six-axis motion sensor in an earbud sees enough to tell sitting from walking from nodding. Before any of that is learnable, though, you have to remove the differences between people and…

September 24, 20244 min read

The baseline before the fancy model: medical image segmentation

In short: before training a U-Net, write a threshold with a little cleanup and score it with the same metric. It's quick to write, it tells you how hard the problem really is, and on high-contrast…

September 10, 20244 min read

Experiment tracking I actually keep using

In short: I use DVC to version data, MLflow to record runs and models, and Weights and Biases when I want to watch a long training live or share charts. What makes them useful is one habit: every run…

July 30, 20245 min read

Federated learning: the model was the easy part

In short: when sites can't share their data, the model code is the smallest part of the work. Most of the effort goes into making the data mean the same thing everywhere, coping with sites that drop…