5 results
Titans introduces a neural memory module that updates itself via gradient descent at inference time, breaking the context-length ceiling of Transformers while staying near-linear in complexity.
AlphaFold's protein structure predictions earned the 2024 Nobel Prize in Chemistry. Here's what the MSA + Transformer architecture actually does and why it matters.
Transformer self-attention is inherently orderless — positional encoding is the fix. From sinusoidal absolute encoding, to learnable absolute encoding, to relative positional encoding, to RoPE (Rotary Position Embedding): modern LLMs almost universally use RoPE because it requires no parameters, naturally encodes relative distances, and can be extended to longer sequences.
AI video generation has been plagued by temporal drift and forgetting for years. In 2025, FramePack, Mixture of Contexts, and A2RD introduced systematic solutions that make long-form video generation genuinely viable.
When an AI Agent performs poorly, it's not necessarily because the model is dumb. Starting from a small experiment where a Gemma 4 2B fixes a bug, this piece explains what a Harness is, how Harness Engineering differs from Prompt / Context Engineering, and how effective natural-language rules like agents.md really are.