Go
Think it through.

All four have a zero value of nil. A nil pointer dereference panics; a nil map read returns the zero value of the value type (safe) but a nil map write panics; a nil slice is safe to range over and append to; a nil channel blocks forever on send/receive. For SRE config structs, you must guard map writes (if cfg.Labels == nil { cfg.Labels = make(map[string]string) }) and always check pointers before dereferencing. Unmarshalling JSON/YAML into a struct with pointer fields is safe — the decoder allocates — but hand-constructed structs are not.

Context cancellation propagates from parent to all children — once a parent is cancelled, all derived contexts are cancelled too. Pattern for a clean-shutdown daemon: (1) ctx, cancel := signal.NotifyContext(context.Background(), syscall.SIGTERM, syscall.SIGINT); defer cancel(). (2) ticker := time.NewTicker(30 * time.Second); for { select { case <-ticker.C: poll(ctx); case <-ctx.Done(): return } }. (3) Inside poll, pass ctx to all k8s client calls — they abort in-flight requests immediately. (4) Use a sync.WaitGroup to wait for goroutines to drain before exit. Critical mistake: creating context.Background() inside the goroutine instead of passing the parent ctx — it orphans the goroutine and causes a 30-second hang on shutdown.

sync.Pool is a cache of temporary objects that the GC can evict at any collection cycle. Get() returns a pooled object or calls New() if empty; Put() returns it. Correctness bug: pooling bytes.Buffer and forgetting to call buf.Reset() before Put() — the next Get() returns a buffer with stale data, causing log lines or HTTP request bodies to contain fragments from a previous request. This is a real class of vulnerability in HTTP middleware. Memory bug: storing large objects (e.g., a 10 MB protobuf) in the pool — if request patterns are bursty, many large objects accumulate and inflate RSS. Correct pattern: always Reset() before Put(); pool only fixed-size or small objects; never pool objects that hold external resources. For SRE log parsers processing millions of lines/sec, pooling []byte scratch buffers via sync.Pool can reduce GC pressure by 40%+ if done correctly.

errors.New("msg") creates an opaque error with no unwrapping. fmt.Errorf("%w", err) wraps an existing error so callers can use errors.Is() / errors.As() to inspect the chain — essential for distinguishing transient vs fatal errors. Custom error types (implementing Error() string + optional Unwrap()) let you attach structured context (HTTP status code, retry-after, resource name). SRE CLI pattern: (1) At the bottom of the call stack, wrap with context: fmt.Errorf("fetching node %s metrics: %w", nodeName, err). (2) At main(), use errors.As() to detect known types and print operator-friendly messages. (3) For unknown errors, log the full chain to a debug log file but show only the outermost message to the terminal. Never fmt.Println(err.Error()) at every layer — it creates duplicate context in the chain.

When HeapInuse is stable but RSS grows, the leak is outside the Go heap. Causes: (1) CGo memory — C libraries (SQLite, librdkafka) allocate via malloc outside Go's GC. Diagnose: cat /proc/<pid>/smaps | grep -A15 "[heap]" to see anonymous pages; use Valgrind or jemalloc profiling for C heap. (2) Goroutine stack growth — 1M goroutines × 8 KB initial stack = 8 GB. Diagnose: runtime.NumGoroutine() metric over time; go tool pprof http://localhost:6060/debug/pprof/goroutine — look for goroutines blocked with no sender. (3) mmap'd files — os.OpenFile with large reads or syscall.Mmap accumulates mapped regions. Diagnose: cat /proc/<pid>/maps | wc -l. (4) OS-level fragmentation — Go's allocator returns memory to the OS lazily. Set GOMEMLIMIT=500MiB (Go 1.19+) to force more aggressive returns. Definitive diagnosis: go tool pprof http://localhost:6060/debug/pprof/heap?gc=1 captures a post-GC heap profile; compare with debug/pprof/allocs to find allocation sites that aren't being freed.

Go's memory model guarantees: a send on a channel happens-before the corresponding receive completes; closing a channel happens-before a receive of the zero value. Subtle race not caught by detector: using atomic.LoadInt64 / atomic.StoreInt64 for a flag but making a non-atomic decision based on two separate atomic reads. Example: if atomic.LoadInt64(&state) == StateLeader { atomic.AddInt64(&counter, 1) } — between the Load and the Add, another goroutine may change state. The race detector won't flag this because each individual atomic operation is data-race-free. This is a TOCTOU (Time of Check to Time of Use) race at the logical level. Fix: use a mutex or encode the state transition atomically with atomic.CompareAndSwapInt64. Another example: double-checked locking where the pointer load is atomic but the struct field access is not synchronized with the writer's store of the field content.

Go's scheduler uses a G-P-M model: G (goroutine, 2 KB initial stack), P (logical processor, holds a run queue; count = GOMAXPROCS), M (OS thread, executes Gs by acquiring a P). When GOMAXPROCS=8, at most 8 Gs run in parallel. Blocking syscalls: when an M blocks on a syscall (e.g., read()), the runtime detaches the P and hands it to another M so other Gs can continue — this is why Go handles 100K goroutines with only GOMAXPROCS threads active. CGo interaction: every CGo call crosses from Go stack to C stack on the same OS thread. The CGo call is treated as a blocking syscall — the P is handed off, temporarily exceeding GOMAXPROCS OS threads. Thread affinity with LockOSThread(): some C libraries (OpenGL, libasound) require all calls from the same OS thread. Pattern: func runCLibWorker() { runtime.LockOSThread(); defer runtime.UnlockOSThread(); for task := range taskCh { cLibCall(task) } }. Critical gotcha: if the goroutine exits without calling UnlockOSThread, the OS thread is retired and the C library's thread-local state is lost, causing crashes. Always defer UnlockOSThread and never let the goroutine exit while the C library holds thread-local resources.

A transparent gzip reader embeds a *gzip.Reader and delegates Read to it. The embedding promotes the Read method directly. However, io.ReadCloser also requires Close — so you must implement it: func (r *GzipReader) Close() error { return r.Reader.Close() }. This demonstrates interface composition: the struct satisfies io.ReadCloser simply by having the right methods, without explicit inheritance. The subtlety: gzip.Reader.Read returns io.EOF when the gzip stream ends, but if the underlying file has trailing data (e.g., concatenated gzip streams — common in .gz files produced by cat file1.gz file2.gz > combined.gz), Read returns io.EOF after the first stream, even though more compressed data exists. Proper behavior requires detecting the next gzip header and advancing the underlying io.Reader manually. Go's compress/gzip package has Reader.Multistream(false) to disable this (default true), but a correct implementation loops creating new gzip.Reader for each stream until the underlying reader is truly exhausted.

select with default is non-blocking: if no channel is ready, the default case executes immediately. If multiple cases are ready, Go's runtime uses a pseudo-random uniform selection to choose one — avoiding starvation from top-to-bottom order. Without default, select blocks until at least one channel is ready. The randomness is implemented via fastrand() in runtime.selectgo(). However, if one channel is perpetually ready (e.g., a ticker firing every millisecond), the default case never runs — the runtime does not prioritize default. To handle this, add a time.After case with a short timeout instead of default: case <-time.After(100 * time.Millisecond):. The default pattern is often misused in busy loops — for { select { case <-ch: handle(); default: } } burns CPU. Prefer blocking select with appropriate goroutine structure. The runtime's select implementation (in runtime/select.go) linearizes all cases, shuffles the poll order via a scrambled iteration order, ensuring fairness across goroutines.

PkgPath is an empty string for exported fields and the package path for unexported fields. This is the canonical way to check field visibility. Attempting to Set() an unexported field via reflection panics. However, you can mutate it using unsafe.Pointer: f := reflect.ValueOf(&s).Elem().FieldByName("unexported") then ptr := unsafe.Pointer(f.UnsafeAddr()). Since UnsafeAddr returns the field's address regardless of exported status, you can write to that memory — but this violates Go's encapsulation guarantees and future compiler changes may break it. The reflect package explicitly disallows Set on unexported fields because the reflect.Value's flag includes a flagStickyRO bit. To bypass, use reflect.NewAt(f.Type(), ptr).Elem().Set(...) — this creates a new reflect.Value pointing to the same memory without the read-only flag. This technique is used in packages like encoding/json (pre-Go 1.20 where reflect allowed it) and sql/driver. Safer alternatives: use go:linkname to access internal functions, or restructure the code to avoid needing this pattern.

//go:generate is a directive for go generate — it tells the tool to run a command (e.g., stringer, protoc, genny). It is placed above the package declaration in a .go file. //go:build (previously // +build) is a build constraint — it controls whether a file is included in a package during go build. To combine them: in the generator source: //go:generate stringer -type=Pill. Then in the generated file pill_string.go, add a build tag: //go:build amd64 || arm64 followed by a blank line then package main. The go generate step runs regardless of build tags (since it is a source-level operation), but the output file is only compiled on the target architectures. Other files without the tag provide fallback implementations. This pattern is common for platform-specific assembly or SIMD-optimized routines — generate stubs with //go:build and provide generic implementations elsewhere. go generate also respects GOOS/GOARCH if the generator needs to produce different output per platform. Build tags are evaluated at compile time, not generation time.

pprof CPU profiling works via SIGPROF — the runtime sets a timer (via setitimer on Linux) that sends SIGPROF to the process at 100 Hz (configurable via runtime.SetCPUProfileRate). The signal handler stops execution, captures the current PC and goroutine stack, increments a counter for that stack trace, then returns. This is statistical sampling, not tracing — it measures where the CPU is spending time, not wall-clock time. If a function is blocked on I/O, mutex contention, or channel operations, it does not consume CPU cycles, so SIGPROF never fires during that wait — the profile shows 0% even though wall-clock time is high. This is why CPU profiles are misleading for I/O-bound programs. Use net/http/pprof's ?debug=1 for CPU, and separate profile?name=heap for memory, profile?name=mutex for contention, profile?name=block for blocking. For wall-clock tracing, use runtime/trace which uses syscall.Getrusage and preemption signals (not SIGPROF) and records goroutine states (Runnable/Running/Waiting) to show where time is spent regardless of CPU usage.

map[int]struct{}{} uses zero bytes for the value — struct{}{} is a singleton zero-size type. map[int]bool uses 1 byte per entry (the bool). For large sets (millions of entries), the difference is ~1MB — negligible in most cases. The struct{} version reads more idiomatically for if set[key] { ... } — but both work since both zero-value bool and zero-value struct (which always returns false for ok check, and _, ok := m[k] works the same). The behavioral difference: m[key] = struct{}{} always overwrites, but m[key] = true is more explicit. map[int]bool allows the semantic triple: false meaning "exists but false", while struct{} cannot express this. However, this is rarely needed for sets. The struct{} version wins in debugging: a JSON marshal of struct{} produces null while bool produces false — neither is great. For public APIs, map[int]struct{} signals "set" intent more clearly to readers. Use map[int]bool only when the boolean value carries meaning beyond membership (e.g., "is active" vs "is inactive").

go test -fuzz implements coverage-guided fuzzing (like libFuzzer/AFL). It starts with a seed corpus (files in testdata/fuzz/<FuzzFuncName>/ and any f.Add() calls). For each input, the fuzzer instruments the binary to track which code edges are hit (via -cover mode with special counters). If a new input triggers a new code path, it is added to the corpus. Mutation strategies include: byte flip, arithmetic increment/decrement of integer values, interesting integer constants (0, 1, MAX_INT, etc.), splicing (combining two corpus entries), and walking through []byte strings. The fuzzer runs in a loop: pick a corpus entry, mutate it, run the test function with that input, check for panics or explicit failures via f.Fail() / t.Errorf(). Inputs that hit new coverage are added to the corpus, expanding the state space. The -fuzztime flag controls how long to fuzz. The runtime must be compiled with -race disabled (fuzzing + race detector is unsupported). The fuzz function signature must be func FuzzXxx(f *testing.F) with f.Fuzz(func(t *testing.T, data []byte) { ... }). Internally, testing/fstest uses the golang.org/x/tools/cmd/file2fuzz format for corpus files.

//go:linkname localname importpath.name allows a function or variable in one package to be linked to a symbol in another package — bypassing Go's visibility rules. The runtime uses it extensively: for example, time package links to runtime.nanotime to get monotonic time without exporting it. sync links to runtime.throw for fatal error handling. This avoids duplicating internal runtime helpers across the standard library. However, go:linkname is fragile: the target package's authors may change or remove the unexported symbol at any time — it is not covered by the Go 1 compatibility promise. When the linked function signature changes, your code silently compiles to incorrect behavior or crashes. Additionally, the go tool link does not enforce type checking on go:linkname targets — mismatched signatures produce runtime corruption, not compile errors. In application code: never use it. It is a last resort for deep runtime hacking, compiler-level tooling, or bridging standard library internals. Use exported APIs, or if missing, file a Go issue. The directive is not allowed in -race builds for some symbols.

wasm_exec.js provides a JavaScript runtime shim: it implements fs.write, fs.read (mapped to console/HTTP), nanotime via performance.now(), and a scheduler loop. Go's WASM target is GOOS=js — it runs a single goroutine on the main JS thread using cooperative scheduling: the Go runtime calls into a JS function go.run() which is a while(true) loop calling the generator-based goroutine scheduler. Goroutines are scheduled by the Go runtime's own M:P:G model implemented entirely in Go (no OS threads needed). GC runs synchronously — WASM has no gc instruction, so Go ships its own mark-sweep collector compiled to WASM, which pauses the single thread. The GOOS=wasip1 target (Go 1.21+) improves on this by targeting WASI preview 1 instead of the JS environment, dropping the wasm_exec.js dependency and supporting //go:wasmimport for direct WASM function imports. Both targets cannot use cgo or syscall packages. Memory is allocated via WebAssembly.Memory — Go's runtime manages its own heap within that linear memory. For production, use TinyGo for smaller WASM binaries (it does not include the full Go runtime).

Go's escape analysis runs during SSA compilation (in cmd/compile/internal/escape). A value escapes to the heap if: (1) its address is returned from the function, (2) it is assigned to an interface value (because the interface value is a {type, pointer} pair stored on heap), (3) it is captured by a closure that outlives the function's stack frame, (4) it is assigned to a global variable, (5) its size is too large for the stack (threshold: 10MB on 64-bit). Example of interface escape: var i interface{} = 42 — the integer escapes because the interface value dynamically allocates. This is a common performance issue. Closure escape: func() func() int { x := 42; return func() int { return x } } — x escapes because the closure references it after the outer function returns. To inspect: compile with go build -gcflags="-m -m" — this prints escape analysis decisions per variable. For example: ./main.go:5:6: 42 escapes to heap indicates the 42 literal moves to heap. Next-level optimization: allocate via sync.Pool for heap-allocated hot paths, or rewrite to pass by value. Use pprof heap profiles to confirm.

High mallocgc means frequent allocations. JSON marshaling allocates heavily because encoding/json uses reflection and allocates temporary buffers. Strategies: (1) Use encoding/json with json.Marshal for small structs, but pre-allocate a bytes.Buffer and use json.NewEncoder(buf) to reuse the buffer. (2) Use json.Marshaler implementation that writes directly to a buffer without reflection. (3) Replace encoding/json with jsoniter (github.com/json-iterator/go) — it generates per-struct codecs and avoids reflection for common types. (4) Use easyjson — generates marshal/unmarshal methods at compile time via go generate, reducing allocations by 80%+ for large structs. (5) Pool reusable buffers: var bufPool = sync.Pool{New: func() any { return new(bytes.Buffer) }} — get/populate/return buffers. (6) Use proto or flatbuffers instead of JSON for internal services. (7) Reduce pointer fields in structs — prefer value types over pointer types to reduce heap allocations. (8) Set GOMEMLIMIT=80% (Go 1.19+) to force the GC to run before RSS grows too large, but this increases CPU. Measure before/after with benchmem and pprof -diff_base.

Architecture: (1) Stage 1 — Reader: one goroutine reads the CSV line-by-line and sends each row to a buffered channel. Use encoding/csv.NewReader with Read() in a loop. (2) Stage 2 — Processor worker pool: N goroutines (N = runtime.GOMAXPROCS(0)) read from the input channel, process, and write to an output channel. (3) Stage 3 — Writer: one goroutine reads from output channel and writes to the output CSV. Bottlenecks: (a) CSV parsing — csv.Reader.Read() is CPU-bound. Use runtime.GOMAXPROCS() to match core count. (b) GC pressure — allocating per-row structs. Pool rows via sync.Pool. (c) Channel contention — if N > 100, mutex contention on channel operations. Use chan with buffer size 10,000. (d) Disk I/O — if reading/writing the same disk, I/O becomes the bottleneck (especially on HDD). Use separate disks or verify with iostat -x 1. Measure: pprof CPU profile to find hot functions; perf for cache misses; iostat and iotop for I/O; go tool trace to see goroutine blocking. The actual speedup is limited by Amdahl's law: if 20% of time is serial (CSV parsing if single-core constrained, or I/O), max speedup is 1/(0.2 + 0.8/16) ≈ 4x on 16 cores. Most production CSV tools hit disk I/O or JSON parsing as the bottleneck, not CPU.

The race detector (ThreadSanitizer, TSan) instruments every memory access by adding shadow-word checks. It detects races when two goroutines access the same memory location without synchronization AND at least one is a write. Why it might miss a concurrent map write: (1) TSan has a sampling rate — not every access is instrumented for performance. If the map write is rare enough (e.g., once per hour), the probability of detection is low. (2) TSan uses a happens-before analysis based on vector clocks — if the goroutines share no synchronization events (mutex, channel, atomic) before the race, TSan infers no ordering and detects it. But if they do share a prior synchronization (e.g., both previously acquired the same unrelated mutex), the vector clock may be large enough to create a false happens-before edge, hiding the race. (3) The actual sequence: goroutine A writes to map key "x" while goroutine B writes to map key "y" — Go's map implementation uses a single hash table structure, and even writing to different keys modifies shared internal state (bucket overflow pointers, tophash array). TSan sees these as distinct memory locations if the bucket is different — but Go's runtime has a global map lock check (hashGrow), and TSan may not instrument all internal map operations. (4) The crash (fatal error: concurrent map writes) is detected by Go's runtime itself via runtime.throw — not by TSan. The runtime explicitly checks for concurrent map writes at runtime using flag atomic in the map header. If the race occurs in un-instrumented code paths (assembly, cgo), TSan cannot see it. Fix: always use sync.Map or sync.RWMutex for concurrent access. Never rely solely on the race detector — it is a best-effort tool, not a proof of correctness.