Cheeto Cam
A self-hosted pet camera that knows which cat it's looking at
01 The story
A pet camera I built for my orange tabby, Cheeto. A Raspberry Pi 5 spots animals with YOLO, identifies the pet with CLIP and captions the visit with Florence-2, all on the Pi. A home server wakes the app with an empty push, so no detection detail passes through Google or Apple.
Cheeto Cam started in December 2024 as a Python script on a Raspberry Pi that watched a USB webcam for one orange cat and emailed a photo when it saw him. Over the next year it picked up YOLO detection, Gemini-written captions, weather, a hosted database, Pushover alerts and a web dashboard that made time-lapse GIFs. The detection history starts in April 2025.
In mid-2026 it became a real product, now called Critter Cam, in four repos: the Pi detector (public), a Flutter app for Android and iOS, a self-hosted backend on a refurbished Lenovo ThinkCentre (Supabase-compatible Postgres, auth, realtime and storage in Docker), and the web dashboard. The home network forwards no inbound ports. The server dials out through a Cloudflare Tunnel, and live video is token-gated.
All the AI runs on one Raspberry Pi 5, and no frame goes to a cloud model. YOLO11n finds an animal. A CLIP few-shot prototype, trained first on about 930 crops of Cheeto and later on confirm and correct taps in the app, decides who it is. Florence-2 writes a short activity caption. A small, unit-tested arbiter settles disagreements: when YOLO sees a 'bear' and the caption says 'squirrel', the only thing that can make a box that big at that distance is the cat.
Push was the hardest part. A self-hosted socket behind an Android foreground service silently stopped after six hours because of an Android 15 limit, and alerts went missing while the app looked healthy. The fix was to make FCM carry nothing: the payload is {"k":"det"}. The phone wakes, fetches the newest detection straight from the home server and builds the notification on the device. iOS gets a fixed generic message from APNs directly, and a notification service extension rewrites it with the real photo and caption.
The app is at 1.6.4 (build 21) and in private testing through Google Play internal testing and TestFlight. Households share access by QR code. It handles a USB webcam plus two wireless ESP32/AMB82 cameras, a 15-second rewind, battery saver and a Pi Zero desk frame. The newest detector work added audio from the outdoor camera and ignores birds.
02 What it does
- 01
Alerts that name the visitor, like 'Cheeto' or 'Unknown visitor', with a photo and a one-line activity caption
- 02
Visits, not noise: repeated frames group into one visit, and you choose first-of-visit, repeat-while-staying or every-detection alerts
- 03
Teach it your regulars: confirm or correct a sighting and the Pi rebuilds that pet's profile from your reviews
- 04
Live view with audio, and a rewind that plays the last 15 seconds as an MP4 made on demand
- 05
Multiple cameras (USB webcam, ESPHome XIAO ESP32-S3, AMB82-Mini with PIR), each with its own rotation, label and battery saver
- 06
Private households: join with a 10-character code or QR, owner, manager and member roles, and a recovery key for ownership
- 07
History calendar, server-side analytics, and visit, day or year-long GIFs to save or share
- 08
Android widget and shortcuts, an iOS notification extension with photo previews, and a Pi Zero desk frame showing the latest visitor
03 How it's built
The Pi is the only device that can write detections. The home server is the only one that talks to the internet, and only outbound through a Cloudflare Tunnel. Push messages carry no content: the phone wakes, fetches the detection from the home server and builds the notification itself. Pet embeddings never leave the Pi; the database stores only references to reviewed samples.
All 19 connections, as text
- Cameras: USB, XIAO ESP32-S3, AMB82-Mini → Pi 5 detector (one process per camera): USB, MJPEG, JPEG (motion-gated)
- Pi 5 detector (one process per camera) → YOLO11n: is an animal there?: every frame
- YOLO11n: is an animal there? → CLIP few-shot: which pet is it?: animal crop
- YOLO11n: is an animal there? → Florence-2: what is it doing?: context crop
- Pi 5 detector (one process per camera) → Supabase: auth, REST, realtime, storage: upload photos, then insert detection
- Supabase: auth, REST, realtime, storage → Postgres (RLS, household RPCs, stats): store
- Pi 5 detector (one process per camera) → Push dispatcher (FastAPI): notify
- Push dispatcher (FastAPI) → FCM (Android): data-only {"k":"det"}
- Push dispatcher (FastAPI) → APNs (iOS): generic alert
- Push dispatcher (FastAPI) → ntfy (self-hosted fallback): parallel fallback
- FCM (Android) → Critter Cam app (Flutter, Android/iOS): wake
- APNs (iOS) → Critter Cam app (Flutter, Android/iOS): wake + extension rewrite
- Critter Cam app (Flutter, Android/iOS) → Cloudflare Tunnel (outbound only): fetch newest detection, photos, stats
- Cloudflare Tunnel (outbound only) → Supabase: auth, REST, realtime, storage: path-allowlisted ingress
- Critter Cam app (Flutter, Android/iOS) → Live view + 15 s rewind server: token-gated live / rewind (via tunnel)
- Pi 5 detector (one process per camera) → Live view + 15 s rewind server: shares decoded frames
- Web dashboard PWA → Cloudflare Tunnel (outbound only): household dashboard
- Desk frame (Pi Zero W + LCD) → Supabase: auth, REST, realtime, storage: polls detections as a household member
- Critter Cam app (Flutter, Android/iOS) → Pi 5 detector (one process per camera): reviews become training samples (via DB)
04 Engineering notes
Three local models on one Raspberry Pi 5 (YOLO11n, CLIP ViT-B-32, Florence-2-base), shared by one detector process per camera through a cross-process flock semaphore, so captioning on one camera can't freeze the others
Few-shot identity from prototypes: a centroid plus up to six deliberately different exemplars per pet, scored 0.6/0.4, accepted only above 0.90 with a 0.015 margin over the runner-up, so two similar orange cats stay 'unknown' instead of swapped
Content-free push: an FCM data-only ping, then the phone fetches the detection over the tunnel. APNs is sent directly from the home server to a notification service extension, and dead tokens are pruned automatically
Postgres row-level security, with every client call going through a security-definer RPC that checks household membership. Aggregates run in Postgres, so opening the app costs about 17 KB of JSON
Rewind uses a ring buffer capped first by bytes, then by seconds, recording at 4 fps (about 1.3% of a core) and muxed with ffmpeg only on request. The mic is captured once and shared by rewind and live listeners
I built and tested iOS in GitHub Actions on macOS runners while developing on Windows. A Ruby script adds the notification-extension target, so there's no Xcode click-through
05 By the numbers
- 5,207
- Detections stored, April 2025 to July 2026
- 1,906
- Visits over 384 active days (counted at 5,192 detections)
- 10,415
- Photos in storage (880 MB)
- 930
- Reference crops of Cheeto in the identity training set
- 0
- Inbound router ports opened
- ~17 KB
- JSON pulled to open the app
- 15
- Containers on the home server, across 3 Docker stacks
- ~$80
- Home server: a refurbished ThinkCentre M700 Tiny
06 Stack
App
- Flutter (Android, iOS)
- Kotlin home-screen widget
- Swift notification service extension
- fl_chart
- video_player / just_audio
- Vanilla JS PWA dashboard
Backend
- Python detector on Raspberry Pi 5
- Self-hosted Supabase stack (Kong, PostgREST, GoTrue, Realtime, Storage)
- FastAPI push dispatcher
- ntfy
- Firebase Cloud Messaging
- APNs (HTTP/2, direct)
Data
- PostgreSQL with row-level security and household RPCs
- Supabase Storage (JPEG frames and thumbnails)
- On-Pi embeddings and prototypes (.npz)
Infrastructure
- Docker Compose on a Lenovo ThinkCentre M700 Tiny (Debian)
- Cloudflare Tunnel (no inbound ports)
- systemd units per camera
- Nightly pull-based backups
- Prometheus / Grafana / cAdvisor (LAN only)
- Firebase Hosting (dashboard)
- ESPHome and Arduino firmware for wireless cameras
- OpenSCAD outdoor enclosure
Tooling
- Ultralytics YOLO11n
- open_clip ViT-B-32-quickgelu
- Microsoft Florence-2-base (transformers)
- OpenCV
- ffmpeg / libx264
- pytest
- GitHub Actions (iOS build, TestFlight)
- OpenWeatherMap
Lines of code (approx.)
- Dart17,285
- Python6,895
- SQL4,841
- HTML3,109
- CSS1,135
- Arduino C++726
≈ 36,045 lines
Repositories
- cheetocam ↗public
- cheetocam-appprivate
- cheeto-infraprivate
- cat-detectorprivate
Four repos: cheetocam 109 commits (2024-12-02 to 2026-11-09), cat-detector 54 (2025-03-02 to 2026-08-01), cheetocam-app 68 (2026-07-31 to 2026-11-09) and cheeto-infra 36 (2026-07-31 to 2026-11-09). The 2026-11-09 commits on the app and infra repos are mostly snapshot and handoff commits; the last substantial app release commit is 2026-08-11 (1.6.4). The detector's 2026-11-09 commit is real code: outdoor-camera audio, lighter live frames and ignoring birds.