supraj.dev
Home/Articles

The engineering notebook

Articles

Practical articles on AI, cloud and DevOps. Choose a story and get into the engineering.

Articles
50
Series
7
Topics
8

50 articles

AWS Data Perimeter Explained: Stopping Data Exfiltration with SCPs and RCPs

A practical guide to building a data perimeter on AWS: trusted identities, trusted resources and expected networks, what SCPs, RCPs, resource-based policies and VPC endpoint policies can and cannot stop, the condition keys that do the work, the confused deputy problem, and a staged rollout that won't break production.

Backups Are Not Disaster Recovery: Designing and Drilling Restores

A practical guide for engineers who "have backups" but have never restored a full production system: setting RTO and RPO with the business, choosing between backup and restore, pilot light, warm standby and active/active, keeping copies attackers cannot delete with AWS Backup Vault Lock, cross-account copies and S3 Object Lock, and proving recovery with restore testing and timed drills.

Prove What Runs in Your Cluster: Signing Images, SBOMs and Provenance

A practical guide to proving every production image was built by your pipeline from your source: digests vs tags, Sigstore keyless signing with Cosign, Fulcio and Rekor, SPDX vs CycloneDX SBOM attestations, SLSA provenance and Build levels, and enforcing the exact signer identity at admission with Kyverno ImageValidatingPolicy or the Sigstore policy-controller.

Delete Your Cloud Access Keys: Secretless Access with Workload Identity Federation

A practical guide to replacing long-lived AWS access keys and Google Cloud service account keys with short-lived federated tokens: how OIDC token exchange works, GitHub Actions to AWS and Google Cloud, EKS Pod Identity vs IRSA, GKE Workload Identity Federation, the trust conditions that stop other repos getting in, and how to retire old keys safely.

Your EKS Cluster Is Leaving Standard Support: A Zero-Downtime Upgrade Playbook

A practical playbook for a production Amazon EKS cluster several Kubernetes versions behind: what extended support costs, why the control plane moves one minor version at a time, how to find breaking changes, how to update VPC CNI, CoreDNS and kube-proxy, how to replace nodes without dropping traffic, why PodDisruptionBudgets stall drains, and what the 7-day rollback can and cannot undo.

Ingress NGINX Is Retired: Migrating to Gateway API Without an Outage

A practical guide for teams running dozens of services behind ingress-nginx: what retirement means, how Gateway API splits GatewayClass, Gateway and HTTPRoute, how to inventory hidden annotation behaviour, what ingress2gateway translates, and how to cut over with weighted DNS and a rollback plan.

Your Kubernetes Bill Doubled: Finding and Cutting the Waste

A practical guide for platform teams whose Kubernetes bill grew far faster than traffic: why the scheduler packs by requests, not usage, where the waste hides, how to measure it with Prometheus and OpenCost, and a step-by-step cut plan with VPA, Karpenter consolidation, spot capacity and quotas that does not cause outages.

Building Golden Paths: A Platform Team's First Internal Developer Platform

A practical guide for a 150-engineer company where every team builds its own CI, Terraform, Kubernetes manifests and dashboards: what platform engineering is and when a platform team is worth it, golden paths versus mandates, running the platform as a product, a first thin platform built from one service template, Backstage's software catalog and software templates, guardrails as policy as code, and how to measure success with DORA's software delivery metrics, adoption and developer satisfaction.

Stop Paging on CPU: SLOs and Burn-Rate Alerts, Explained Like a Mobile Data Plan

A practical guide for on-call teams drowning in CPU and disk pages: symptoms versus causes, availability and latency SLIs, choosing SLOs with the business, error budgets and policy, why simple error-rate alerts are noisy or slow, the Google SRE Workbook's multiwindow, multi-burn-rate parameters with worked arithmetic for 99.9%, and the Prometheus recording and alerting rules to build it.

Quantization Explained: How LLMs Fit in Less GPU Memory, and What It Costs

A plain-English guide to LLM quantization: what FP32, BF16, FP8 and INT4 cost in memory, why smaller weights speed up decoding, how scales and outliers decide accuracy, what GPTQ, AWQ, SmoothQuant and LLM.int8() actually do, how to quantize the KV cache, and how to pick a format in vLLM.

Inference Internals · Part 5

Speculative Decoding Explained: How a Small Model Makes a Big Model Answer Faster

A plain-English guide to speculative decoding: why each decode step reads all the weights for one token, how a small draft model guesses ahead while the big model checks every guess in one pass, the acceptance rule that keeps the output distribution identical, what drives the acceptance rate, how long a draft should be, and when it helps or hurts in vLLM.

Inference Internals · Part 4

Testing an Agent Harness: Evals That Catch Bad Agents Before Users Do

Part 3 of Agent Harness Engineering. What an eval is, the three kinds of grader, grading the outcome or the transcript, why you run many trials, capability and regression evals, turning real failures into tasks, and running evals as a CI gate. Explained like a test kitchen.

Agent Harness Engineering · Part 3

Continuous Batching Explained: How vLLM Serves Many Users at Once

A plain-English guide to continuous batching: why fixed batches leave the GPU idle while the longest answer finishes, how re-planning the batch at every step fixes it, why it needs PagedAttention, how prompts and answers share a step with chunked prefill, and what happens when KV memory runs out.

Inference Internals · Part 3

MLOps Part 2: From Notebook to a Training Pipeline

Why a notebook is not a pipeline, the stages of a training pipeline, the two validation gates, experiment tracking, a model registry and retraining triggers, explained with a bakery.

MLOps: From Model to Production · Part 2

Prompt Caching: The Cheapest Way to Cut Your AI Bill

A plain-English guide to prompt caching: what gets cached, why stable content goes first, how to read cache hits and misses in the usage fields, how long a cache lives, what writes and reads cost, how it relates to the KV cache, and when it doesn't help.

What Is NVIDIA NVKE? Explained Like a Wedding Hall

A simple guide to NVIDIA Kubernetes Engine (NVKE): what it is, who installs it, how it turns GPU machines into a Kubernetes cluster for each customer, and what the preview can't do yet.

PagedAttention Explained: How vLLM Stopped Wasting GPU Memory

A plain-English guide to PagedAttention: why older LLM servers wasted 60–80% of KV cache memory, how vLLM stores notes in small same-size blocks with a block table, how blocks are shared within and across requests, what happens when memory runs out, and how to tune it.

Inference Internals · Part 2

KV Cache Explained: Why AI Models Keep Notes

A plain-English guide to the KV cache: why a language model keeps notes for every word, what keys and values are, why the cache fills a GPU when many people chat at once, its drawbacks, and how GQA, MLA and FP8 shrink it. No equations.

Inference Internals · Part 1

What Is NVIDIA OpenShell? Explained Like a Gated Society

A plain-language guide to NVIDIA OpenShell: why AI agents need a sandbox, how the gateway, supervisor and sandbox fit together, how policies and hidden credentials work, and how to try it.

Agent Runtimes · Part 4

What Is Google AX? How It Works with Agent Substrate

Understand Google AX through a shared office: tasks, workspaces, runners, and how Agent Substrate provides the sandbox. Follow one simple README check.

Agent Runtimes · Part 3

Kubernetes Agent Sandbox, Explained Like a Hotel

What a sandbox is, why AI agents need one, why a normal container is not enough, and how kubernetes-sigs/agent-sandbox gives every agent its own locked room. Verified against the official v1beta1 API.

Agent Runtimes · Part 1

GPU Architecture, Layer by Layer

A plain-language deep dive into a modern GPU, using the NVIDIA H100: GPCs, TPCs and SMs, every unit inside an SM, the memory system, occupancy, Hopper's clusters and TMA, copy engines, NVLink and NVSwitch, MIG, the graphics units, and where TFLOPS numbers come from.

Masterclass · Part 3

How a GPU Works, and Why AI Can't Live Without It

A plain-language masterclass on GPUs. What a CPU is brilliant at and where it struggles, latency vs throughput, Streaming Multiprocessors, warps and divergence, latency hiding, memory bandwidth and the roofline, Tensor Cores and low precision. No hardware background needed.

Masterclass · Part 2

Inference Engineering Masterclass: How AI Actually Answers You (with vLLM)

A plain-language masterclass on inference engineering. Prefill and decode, the KV cache, batching, latency vs throughput, and how vLLM's PagedAttention, continuous batching, prefix caching and speculative decoding make AI fast and cheap. No AI background needed.

Masterclass · Part 1

What Is an Agent Harness? The Part of an AI Agent That Isn't the Model

Part 1 of Agent Harness Engineering. Why the same model can be a chatbot or a reliable coding agent, the seven parts every harness has, one bug fix traced through all of them, and the lessons Anthropic and OpenAI published about building harnesses.

Agent Harness Engineering · Part 1

Build an Agent Harness: A Working Blueprint in Python

Part 2 of Agent Harness Engineering. Build a real coding-agent harness in about 120 lines of Python, with a workspace jail, a policy that protects the tests, a done gate that runs the tests itself, checkpoints and traces. Then break it on purpose, and see how each piece grows up for production.

Agent Harness Engineering · Part 2

What Is an AI Agent? Explained From Zero, Step by Step

Part 1 of AI Agents from Zero. A spoon-fed explanation of what an AI agent really is, how it differs from a chatbot and a workflow, the five parts every agent has, and the think-act-observe loop, walked through one real example.

AI Agents from Zero · Part 1

How AI Agents Actually Work: Tool Calling, Memory and the Loop in Code

Part 2 of AI Agents from Zero. Open the hood: the exact messages an agent sends, how tool calling really works, three kinds of memory, how agents plan, and a complete working agent in about 80 lines of Python with no framework.

AI Agents from Zero · Part 2

AI Agent Design Patterns and Safety: Build Agents You Can Trust

Part 3 of AI Agents from Zero. The five building blocks behind most real agent systems with runnable code, when several agents beat one, a refund agent designed step by step, and how agents get attacked (prompt injection, the lethal trifecta) and defended.

AI Agents from Zero · Part 3

AI Agents in Production: The Complete Playbook

Part 4 of AI Agents from Zero. Everything between a working agent and one you can trust with real users: a simple reference architecture, reliability, identity and secrets, approvals, versioning, evals and red-teaming, safe rollouts, monitoring, cost, privacy, MCP security, an incident runbook, a launch plan and the checklist.

AI Agents from Zero · Part 4

AI Guardrails: The Model Is Not the Safety Layer

Guardrails are the runtime control plane around an LLM — what they are, why every production AI app needs them, and how to implement them with Bedrock Guardrails, NeMo Guardrails, and Llama Guard, with real configs.

The AWS MCP Server Is Now Generally Available

AWS announced the GA of their managed MCP Server, giving AI agents secure, authenticated access to 15,000+ AWS APIs, live documentation, and sandboxed script execution.

Building AI Agents with Google ADK: The Complete Production Guide

A deep, hands-on guide to Google's Agent Development Kit (ADK 2.0). Real Python code for tools, sequential/parallel/loop pipelines, sub-agent delegation, callbacks, evaluation, and Cloud Run deployment — plus an honest comparison with LangGraph, CrewAI, and AutoGen.

What is Vectorless RAG? (Explained Simply)

AI is getting smarter, but it doesn't always need complex vector databases. Here is a simple explanation of Vectorless RAG and why it's becoming popular.

Least-Privilege AWS IAM with an AI Agent

How iam-zero uses live CloudTrail logs and Anthropic Claude to audit over-permissive active policies, automatically opening secure pull requests.

Agentic Cloud Security · Part 1