Tutorials
Build something that works: retrieval, agents, fine-tuning, evals, serving, with complete code.
Tutorials are the build-it pieces. Each one ends with a listing you can run, most of them on the Python standard library alone, and its numbers come from that listing rather than from a vendor page. There are seven so far, in three threads.
Retrieval: a minimal retrieval pipeline chunks, indexes and evaluates 216 arXiv abstracts with TF-IDF and reports Recall@k and MRR, and hybrid search then fuses BM25 with two embedding models on SciFact, where a published blend weight helped one model and hurt the other. Structured output and agents: structured output with JSON Schema builds a validate-and-retry loop, and the agent tool-use loop treats the loop as a state machine with six stop conditions and a token budget; both are measured over 2,000 simulated runs. Evaluation and cost: a golden test set scores two support bots on 48 items without an LLM judge, LLM-as-judge done carefully runs a pairwise judge harness in both orders over the same set, and counting tokens shows the same 20,000 characters coming to anywhere from 4,716 to 33,502 tokens.
If you are new here, the retrieval pipeline is the gentlest start, and the golden test set is the one most teams need first. The Evaluation section takes the measurement side further. Next on our list: a reranker over the hybrid-search results on the same SciFact split, and prompt caching once it can be measured against a hosted API.
-
Hybrid search in Python: BM25 plus embeddings, measured
BM25 and two embedding models fused by RRF and by a weighted blend on SciFact. A published blend weight helped one model and hurt the other.
-
Counting tokens before the bill: six tokenizers, six kinds of text
The same 20,000 characters came to 4,716 to 33,502 tokens depending on language and tokenizer. Measured counts, the divide-by-four rule tested, a forecast script.
-
LLM-as-judge done carefully: position bias, length, and agreement
A pairwise judge harness run in both orders over a 48-item golden set, with swap tests, a padding probe, and Cohen's kappa. Code and data included.
-
A golden test set for a chatbot, scored without an LLM judge
How to build a golden set for a support bot and score it with exact match, token F1, ROUGE-L, and per-item assertions. Two bots, 48 items, code and data included.
-
An agent tool-use loop from scratch: states and stop rules
A tool-use loop built as an explicit state machine with six stop conditions and a token budget, measured over 2,000 simulated episodes. Code included.
-
Structured output that never breaks: JSON Schema, validation, retries
A validate-and-retry loop for model JSON, with a small schema validator, a tolerant extractor, and measured retry rates from a 2,000-run simulation.
-
A minimal retrieval pipeline in Python: chunking, TF-IDF, evaluation
A complete retrieval pipeline in the standard library, run on 216 arXiv abstracts, with Recall@k and MRR measured for whole-document versus chunked indexing.