Running Local AI Without Heating the Entire House
What model size, quantization, and power measurements reveal about a genuinely useful local inference stack.
Field notes on local AI and inference, reliable DevOps pipelines and observability, Raspberry Pi builds, calm homelab networking, and the systems behind smooth gaming.
01 / FEATURED FIELD NOTE
LONG-FORM · TESTED IN THE LAB
What model size, quantization, and power measurements reveal about a genuinely useful local inference stack.
02 / RECENT TRANSMISSIONS
Serial debugging is still central in embedded work, and Rust is excellent for building reliable terminal tooling. I structure serial utilities as pipelines: read bytes, frame messages, parse the protocol, then route structured events out.
Writing `no_std` Rust firmware requires different habits than server Rust. Memory is constrained, timing is strict, and panic strategy must be deliberate.
Hardware abstraction layers are where embedded code often becomes difficult to test. I design traits around capabilities, then keep business logic independent from concrete drivers.
Hardware-connected software fails at boundaries: timing edges, flaky links, and rare protocol states. A strong test strategy must combine fast feedback with realistic integration checks.
03 / TOPICS
FIVE THREADS
04 / ACTIVE SERIES
ORDERED READING
Running useful AI on your own hardware without pretending the trade-offs do not exist.
Follow the seriesI'm Gábor Lepsényi, a technologist and incurable tinkerer based in Budapest. I write about running local AI and inference, keeping DevOps pipelines and observability boring, Raspberry Pi and embedded builds, calm homelab networking, and the systems behind smooth gaming.
No recycled release notes and no ten-minute miracle builds — just careful experiments, honest constraints, useful measurements, and enough context to reproduce the work. You can also find me writing in Hungarian at rust.hu and aws.hu.