WorkSilicon
Oshkidar
Cross-modal AI: a vision AI taught a radar AI on a $2 chip to recognize the cat.
- Video → Radar
- one AI teaches another: a vision model labels every visit second by second, an automated ground-truth pipeline for the radar
- 62
- engineered radar features, each documented with the physics of why it separates a cat from a ghost
- 1,500
- gradient-boosted trees classify every window as cat, person, refill water, ghost or empty
- 1 Hz
- real-time inference on the chip, over a sliding 8-second window of 10 Hz radar frames
Oshkidar started as a household problem: a toilet that should flush after the cat has used it, and only for the cat. We took it on as a technical challenge: how much real-time machine-learning classification can run on a $2 microcontroller?
A 24 GHz mmWave radar on the cistern reports reflected energy in nine range gates, ten times a second. Once a second, the ESP32-C6 takes the last eight seconds of frames, subtracts the empty room’s own baseline and computes 62 features. Each feature carries a physical reason it should separate a cat from a person, the tank refilling, an empty room or a “ghost”: someone moving beyond the wall whose reflections flicker at the seat’s range. Gradient-boosted trees, 1,500 of them in 343 KB of flash, score every window across those five classes.
A window is evidence, not a decision. A Bayesian tracker holds a belief about who is in the room, with minutes of memory, and knows that its own flush is followed by a refill. A camera-fitted seat filter tells the seat from the door, and a state machine flushes once she leaves the seat, with a cooldown and a flush-rate limit.
The model is trained offline in Python on recorded radar days, validated one held-out day at a time and exported to C, and tests confirm the chip and the trainer compute the same answer. The device replaced a Raspberry Pi prototype and is in daily use, on firmware 3.2.0; since July, the learned model has had the final say on every flush.
Video teaches the radar
The most transferable part of the project is how the radar gets its ground truth. A camera, woken by the radar, runs a YOLO object detector on one frame a second, places the cat’s bounding box in a seat, door or floor zone, and writes a label for every second of the visit. Those labels are joined to the radar’s own windowed feature vectors, so each visit becomes labelled training and test data on its own: 2,707 seat windows from 46 visits in the first weeks alone. One AI system supervises another in a different sensing modality. Frames showing a person are never stored, and the camera only teaches: on the device, every decision is made from radar alone.
How it evolved
Four versions, each named for what the detector can see.
Version 1 · Snapshots · May to July 2026
The first detectors judged the room one radar frame at a time: a presence number against a threshold, then per-gate rules for the seat zone, for gentle versus violent movement, for distance and shape. Every release fixed the last failure and met a new one. The tank refill looked exactly like a cat and outlasted the cooldown, so one June day the toilet flushed itself every two minutes. Someone moving beyond the wall landed at the seat’s range with a cat’s values, and the toilet flushed for people in the next room.
Labelled 10 Hz captures held the lesson. In a single frame a cat and a ghost read the same; over a few seconds, a cat’s signal is steady and smooth while a ghost flickers and fades.
The shift: radar is a signal over time, never a snapshot. Value-based detection was retired.
Version 2 · Time · July to November 2026
Version 2 rebuilt detection around time. Every second, an 8-second window of nine-gate energy vectors, less the empty room’s baseline, becomes 62 engineered features, and 1,500 gradient-boosted trees score it as cat, person, refill, ghost or empty. A Bayesian occupancy tracker with minutes of memory turns those probabilities into a belief about the room, and expects a refill after its own flush. It ran in shadow first, suppressing 5 ghost flushes with no known misses, and on 20 July took the final say.
Ghost flushes stopped. But version 2 knew that a cat was there, not where. After using the toilet she often sits by the door, and the flush waited for her: 74 to 100 s after she left the seat, and minutes later after a door-sit.
The shift: from rules to probabilities, from frames to windows, from reacting to remembering.
Version 3 · Where she is · live since November 2026
Telling the seat from the door needed ground truth the radar could not produce itself, and the camera labeller supplied it. Against its labels the separation was clean: across 2,707 seat windows, a cat on the seat never fell below a seat-zone level that half of all door-sit windows did. The labels also overturned a summer assumption: the quiet stretches that radar alone had read as a still cat on the seat matched, on camera, a cat sitting by the door.
A camera-fitted seat filter now tells seat from door, and the flush follows her leaving the seat. On 19 November a visit was watched live: a door-sit, and a flush three minutes late. Her movement on the seat had mimicked a tank refill and split the visit in two. Firmware 3.2.0 fixed it that day, with a 20-second wait chosen by sweeping 35 camera-labelled days. Replayed over 103 visits: no flush while she was on the seat, and the flush 39 s after she leaves it at the median.
The shift: video became the radar’s teacher, and the flush, her reward, now comes on time.
Version 4 · Two timescales · built and tested offline
In November the data showed a new visitor: an exhaust fan that put steady energy on one range gate for up to three hours some mornings. The empty-room baseline was a single, deliberately slow estimate with a 12-hour half-life, so a three-hour nuisance never entered it. On 10 November a real visit scored too weakly against it to register, and no flush came. The windowed features had looked time-aware, but they sat on a reference that was, in practice, constant.
Several approaches to adaptive background modelling were weighed, and the answer was two timescales. A slow picture of the empty room sits beside a fast one for the current steady nuisance, and a Bayesian context filter says which fits, or that neither does: “this room is unfamiliar” is a real state. A nuisance earns the right to be subtracted on earlier occasions, never in the one that revealed it, and the cat model is never learned on the device; its labels still come from the camera. Offline, the fan qualifies on its 4th occasion and is subtracted from the 5th.
The shift: the room gets its own clock, separate from the cat’s. Built and tested offline; on the device next.

