queues

Production-grade distributed job queue from scratch: DLQ, retry policies, autoscaling workers, async concurrency control

ai-eval

Evaluation harness for LLM apps: LLM-as-judge, RAGAS for RAG quality, benchmark suites, A/B eval across model versions, regression detection dashboards

distributed-systems

Raft implementation with leader election, log replication, and linearizable reads across a multi-node cluster

rag

Retrieval-augmented generation pipeline with faithfulness scoring, retrieval recall, and quality gates via RAGAS

ml-foundations

Decoder-only transformer built from PyTorch primitives: self-attention, positional encoding, trained on a real dataset

load-testing

Coordinated load generation across container workers with autoscaling, real-time metrics, and multi-region test runs

ai-infra

LLM API gateway that routes requests across providers/models by cost, latency, and quality tier with automatic fallback, semantic caching, and per-tenant cost accounting

ml-inference

Serving layer with continuous batching, KV-cache reuse, and vLLM-style paged attention for high-throughput inference

vector-db

Purpose-built vector store with HNSW indexing, IVF partitioning, and a custom on-disk format for large embeddings

networking

L7 load balancer with consistent hashing, dynamic backend registration via etcd, and Prometheus metrics

k8s

Custom controller for a domain-specific resource with reconciliation loops, CRDs, and admission webhooks

data-pipeline

Ingestion pipeline: upload, chunking, transcode, async workers, object storage, and queue-driven fan-out

ai-agents

Real-time voice conversational agent with streaming ASR, LLM turns, and low-latency TTS via WebSockets