Notes on medical-imaging AI, production ML systems, and what I've learned shipping research-grade work into clinical workflows. Three sections, three formats.
Narrative deep-dives on production medical-imaging AI.
Image-based RAG over ~1.6M medical images: MetaCLIP PE-Core-bigG-14-448 embeddings, FAISS for 1024-d ANN search, LLM synthesis across retrieved reports. The interesting decisions are the embedding model, the index choice, and the failure modes (visual-similarity ≠ clinical-similarity) nobody covers in tutorials.
Long-running orchestrated ML inference (15-30s) looks bad on a UI by default. Streaming partial results over SSE as each sub-model completes drops perceived latency 9× without changing the underlying compute. Architecture, FastAPI implementation, the five deployment gotchas nobody warns you about.
Three structurally different ways to combine models — feature-level fusion, prediction-level fusion, rules-based orchestration. The choice between them matters more than which models you pick. Most ML writing conflates the first two with the third.
OBB is a small but consistent win on rotated medical pathologies — +0.01 F1 on 2K images, F1 0.83 / recall 0.90 at 10K. The cost is real (annotation, training stability, runtime) but on the right side of the tradeoff for screening.
PaliGemma 3B with the last 5 SigLIP layers unfrozen, one prompt template per fine-tune, and WCE-or-Focal picked by operating point gets you most of the way to 10B's F1. Everything past that is uncharted in public writeups.
SQLite + an atomic-claim SQL pattern + a stuck-process watchdog will run your ML inference queue. Redis isn't free — make it earn its place.
The fusion math behind multi-model object detection ensembles — NMW vs NMS vs Soft-NMS vs WBF, explained with synthetic boxes and honest tradeoffs.
Three detection architectures fused into one ensemble — what each model contributes, what fusion actually buys, and the engineering numbers from the arXiv paper.
Almost ungoogleable combo — Electron + PyInstaller + AES-256 model decryption + admin-aware auto-updater — and the interactions between pieces are where everything goes wrong.
What actually dominated my first year as an AI Scientist — and it wasn't the model architectures.
Lab notes: setup → method → results → verdict.
Stanford AIMI's CheXagent-8b zero-shot on CheXpert pleural effusion: 70-75% precision and recall, neither traded off against the other. Strong rapid-prototyping baseline, insufficient as sole signal for triage. The interesting question is whether the gap to deployable closes with fine-tuning.
ImageNet's (mean, std) is the default in every PyTorch tutorial. For non-natural-image domains, computing custom stats is a one-time pass that lifts F1 by 1–3 points across binary chest-X-ray tasks — zero inference cost.
Strong embeddings, unshippable latency. 135 s/image on an A100, 27% batching gain, 32 GB peak memory — the foundation-model promise meets the production-inference budget.
Foundation-model VLMs and CNNs win on different axes — pick by task, not by hype. 3,566 chest X-rays, per-task numbers, cost, latency.
Educational primers, referenced from deeper posts.
Textbook predicts focal dominates at high imbalance. On Kermany subsampled to 58:1, plain BCE wins AUC (0.92), weighted BCE wins F1 at threshold 0.5, focal sits between — and the confidence histogram explains why the loss function can't fix calibration on its own.
Backprop produces one gradient per parameter per batch — but the resulting weight changes are wildly non-uniform across depth. Pretrained gradients are ~10× smaller than fresh-init, conv1 moves 0.34% while the fc head moves 38.67%, and the dynamic range across stages is what makes pretraining work.
YOLO collapses detection's two stages into one — 8,400 candidate boxes per 640×640 image across three scales, scored and pruned by NMS in a single forward pass. Visualized on a fine-tuned YOLOv8n detector on RSNA chest X-rays.
The DICOM standard is 5,000+ pages across thirty parts. For ML engineers reading from disk, the file format, a handful of tags, and the pixel processing rules cover almost every dataset you'll touch.
How the Feature Pyramid Network gives detectors both semantic depth AND spatial precision — visualized on a chest X-ray with all five pyramid levels.
Grad-CAM in thirty lines of PyTorch, demonstrated on a real chest X-ray classifier — plus the one failure mode that test-set accuracy hides.
Every CNN component explained against real activations from a chest X-ray classifier — no toy MNIST diagrams, no opaque equations.