23 results
Over the past two years RAG evolved from a 'linear pipeline' to 'loop-based reasoning'. It maps cleanly to five stages: Naive, Advanced, Modular, Graph, Agentic. The real inflection point is control moving from pipeline to agent — a System 1 → System 2 shift. Looking back at engineer-news's own RAG stack, it's stuck at the Naive edge — so this post also lays out what to fix next.
The previous post zoomed out for a five-stage panorama of RAG. This one zooms in on the five infra lessons any real RAG has to face: document ingestion, contextualized chunking, three-lane hybrid search, tensor reranker, and GraphRAG's semantic gap. Each lesson is checked against engineer-news's current stack, ending with a priority list for a personal site.
Anthropic proposes J-lens, an interpretability tool that captures the 'verbalizable' representations inside a Transformer, and uses it to show that Claude contains a privileged subspace analogous to the neuroscientific 'global workspace' — a small set of vectors that broadcast, drive reasoning, respond to external steering, and even leak signals during deception and evaluation awareness.
Three OpenAI engineers, five months, one million lines of AI-generated code, zero hand-written. The real value of this experiment isn't the numbers — it's the proof that Harness design can be engineered. Five concrete practices: making the app legible to agents, treating the repo as the source of truth, mechanizing architectural constraints, rewriting merge philosophy, and background entropy management.
Distilling Harness Engineering from concept and benchmark case into something you can start executing today: the four fixed failure modes of Agents, the 40% context sweet spot, the four-pillar framework the industry has converged on, and a three-phase roadmap from 'this afternoon' to 'fully automated in two weeks' — closing with six industry consensus points and three still-unsolved problems.
A breakdown of the defense-in-depth used by Claude Code, Codex, and others: rule-based keyword matching, classifiers, input/output scanning, execution sandboxes, cross-session behavioral monitoring, and the role of system prompts and skills.
Headroom compresses tool outputs, logs, and RAG chunks by 60–95% locally, before the request reaches your provider. The part worth stealing isn't the ratio — it's how it decides whether to compress at all using 'mask extraction + cache-mutation economics' — plus a reminder that its docs run ahead of its code.
Two 2026 products answering the same question: when a single model hits its ceiling, don't pick one model — orchestrate a pool of them behind one API. Fugu bets on learned coordination; Fusion bets on parallel deliberation plus a judge.
When an agent loads tools on demand, it's essentially applying RAG's 'retrieve-then-inject' pattern to tool schemas — the only difference is the retriever is the LLM's own reasoning instead of vector similarity.
Running an LLM locally with Ollama is simpler than you think: one line to install, one line to pull a model, one line to chat. This guide takes you from install to a working local RAG pipeline.
PageIndex swaps the vector DB for a hierarchical tree index plus an LLM Agent that reasons over it, and it shines on long structured documents (98.7% on FinanceBench). This site's Hybrid RAG instead runs vector search with a keyword fallback on the Cloudflare edge — a completely different set of tradeoffs.
Dexter breaks complex financial research questions into structured steps, automatically calls tools to fetch real-time financial data, then self-checks its results until the answer is complete enough—conceptually, it's 'Claude Code built specifically for financial research.'
Point your tools' endpoint at localhost:20128 and 9Router automatically does three-tier fallback across subscription services, cheap APIs, and free services — refreshing expired tokens on its own, translating between OpenAI and Claude formats, and using RTK to shave 20–40% off tool-output tokens.
Install one Rust binary, and the output of commands like git/grep/test/docker gets compressed before reaching the AI context — a 30-minute Claude Code session drops from roughly 118,000 tokens to about 23,900.
KV Cache reduces autoregressive Transformer generation from O(n²) — recomputing the full sequence for every new token — to O(n) per step, which is the core reason modern LLM inference is fast enough to be usable.
DeepSeek V3's 671B-parameter MoE architecture trained on just 2.78M H800 GPU-hours matches near-GPT-4 performance across multiple benchmarks, with API pricing at one-tenth of OpenAI's equivalent.
OpenAI released three models in spring 2025: GPT-4.1 for coding and instruction-following, o3 as the strongest reasoning model, and o4-mini hitting remarkable math and code performance at low cost — but the pricing strategy and API access limits left developers with mixed feelings.
LLM output quality is determined at three distinct layers: token-level decoding strategy, task-level workflow design, and model-level reasoning capability. Knowing which layer your problem lives in is the fastest path to fixing it.
The LLM Wiki isn't a query tool—it's an architecture that lets knowledge compound over time. The LLM actively builds and maintains a markdown knowledge base, instead of re-fetching from raw documents on every single query.
OpenAI's Michael Bolin breaks down the Codex CLI agent loop in depth: from prompt assembly, token inference, and tool execution, to context window management and auto compaction.
Small language models around 10B parameters can run on local hardware in real time, enabling dynamic NPC dialogue, procedural narrative generation, and adaptive game content. Research shows SLMs approach large model quality on short, well-constrained creative tasks — the key is curated training data and constrained inference design.
When an AI Agent performs poorly, it's not necessarily because the model is dumb. Starting from a small experiment where a Gemma 4 2B fixes a bug, this piece explains what a Harness is, how Harness Engineering differs from Prompt / Context Engineering, and how effective natural-language rules like agents.md really are.
ChatGPT's capabilities come from the first stage of massive self-supervised pre-training; but 'answering questions, understanding instructions, and not making things up' come from the latter two stages — instruction fine-tuning and RLHF. Capability and alignment are two separate things that are forged independently.