October 2026
3 articles-
Hybrid search in Python: BM25 plus embeddings, measured
BM25 and two embedding models fused by RRF and by a weighted blend on SciFact. A published blend weight helped one model and hurt the other.
-
Is your test set in the training data? Four overlap rules, measured
The GPT-3, PaLM, Llama 2 and GPT-4 contamination rules in one stdlib script. On GSM8K, 13-word overlap caught 69.1-95.6% of edited copies, 0.8% of rewordings.
-
TORQUE re-implemented: where keeping outliers around a rotation pays
A reading of arXiv 2609.36032v1, re-run on Qwen2.5-0.5B: keeping outliers cut activation error at every budget; the KV-cache gain faded once its bits were paid.
September 2026
13 articles-
Counting tokens before the bill: six tokenizers, six kinds of text
The same 20,000 characters came to 4,716 to 33,502 tokens depending on language and tokenizer. Measured counts, the divide-by-four rule tested, a forecast script.
-
What a quantisation format actually costs, derived from the source
Q4_K is not four bits per weight. Exact costs derived from ggml-common.h, joined to llama.cpp's measured size and perplexity tables, and the dominated formats named.
-
Where should a document live: context, KV cache, or weights?
A reading of arXiv 2609.17346v1, which compares in-context documents, KV-cache injection and fine-tuning on five benchmarks. Tables re-derived, code included.
-
LLM-as-judge done carefully: position bias, length, and agreement
A pairwise judge harness run in both orders over a 48-item golden set, with swap tests, a padding probe, and Cohen's kappa. Code and data included.
-
A golden test set for a chatbot, scored without an LLM judge
How to build a golden set for a support bot and score it with exact match, token F1, ROUGE-L, and per-item assertions. Two bots, 48 items, code and data included.
-
An agent tool-use loop from scratch: states and stop rules
A tool-use loop built as an explicit state machine with six stop conditions and a token budget, measured over 2,000 simulated episodes. Code included.
-
Structured output that never breaks: JSON Schema, validation, retries
A validate-and-retry loop for model JSON, with a small schema validator, a tolerant extractor, and measured retry rates from a 2,000-run simulation.
-
A minimal retrieval pipeline in Python: chunking, TF-IDF, evaluation
A complete retrieval pipeline in the standard library, run on 216 arXiv abstracts, with Recall@k and MRR measured for whole-document versus chunked indexing.
-
KV cache arithmetic: memory per token, batch size, and context cost
Bytes of KV cache per token from a config.json, worked for Qwen3-8B and Qwen3-32B at four context lengths on an 80 GB device, with GQA and int8 KV as the two levers.
-
Gemini 3.8 Flash released: what changed, from the primary sources
Google's 2 September 2026 release of Gemini 3.8 Flash, dated from the announcement, with the model page limits, the pricing expiry, and the eval table's footnotes.
-
Bootstrap CIs for eval deltas: is a 4-point gap on 500 items real?
A paired bootstrap for the accuracy gap between two systems on 500 items, with McNemar's test, coverage checked over 1,000 replays, and the code and per-item data.
-
EarlyEval explained: stopping agent benchmark runs early
A reading of arXiv:2609.02783, which trains classifiers to halt agent runs once the outcome is predictable, and what its Table III actually shows.
-
Covert Indirect Prompt Injection (ICoA) explained
A reading of arXiv:2608.30362, which splits attack success into covert and overt outcomes on AgentDojo and shows the covert share can be engineered.