Table of Contents

Robot AI has always had a frustrating constraint: a model trained for Robot A doesn’t transfer to Robot B without starting over. NVIDIA’s Isaac GR00T N1, released at GTC 2025, is the first serious open attempt to break that constraint. Its architecture forced me to reconsider what a general-purpose robot AI should actually look like.

TL;DR

  • GR00T N1 is the world’s first open humanoid robot foundation model — open source, commercially licensed
  • Architecture: dual-system — a Vision-Language Model for high-level reasoning, a Diffusion Transformer for precise action generation
  • One model, multiple hardware platforms (Fourier GR-1, 1X Neo, and others) — cross-embodiment generalization is the core design goal
  • Training data: real captured motion + Isaac GR00T-Mimic synthetic data + internet video
  • GR00T N1.7 is in early commercial access; GR00T N2 (based on DreamZero research) is in development

Design Philosophy

Why “General” Is So Hard

Traditional robot AI models are task-specific and hardware-specific. Change the joint count of a robot arm or swap out a sensor configuration and you’re retraining from scratch. This makes robot AI development expensive and prevents the kind of knowledge accumulation that gives software its compounding advantages.

GR00T N1’s design goal: one model that, with appropriate fine-tuning, can perform manipulation tasks across different humanoid robot hardware platforms. This immediately means the architecture has to solve two fundamentally different problems simultaneously:

  1. Understanding the environment, language instructions, and task goals (high-level cognition)
  2. Precisely controlling tens of joints to produce continuous, dexterous motion (low-level action control)

The Dual-System Inspiration

GR00T N1’s architecture draws from the dual-process theory in cognitive science (Kahneman’s System 1 / System 2 framework):

  • System 2 (slow, deliberate): a Vision-Language Model that interprets the scene, understands language instructions, and plans action sequences
  • System 1 (fast, automatic): a Diffusion Transformer that generates continuous, precise motor control signals

This separation lets each subsystem use the architecture best suited to its problem class.

Core Architecture

System 2: The Vision-Language Model

The VLM receives multimodal input: camera images, language instructions, environment state. It answers high-level questions like “what’s the next step in this task?”:

  • Scene understanding: where is this object, how should I grasp it?
  • Instruction parsing: “move the red cup to the right side of the table”
  • Long-horizon planning: decomposing multi-step tasks into subtasks

The VLM’s output is not direct joint angles — it produces a high-level action representation or intent vector.

System 1: The Diffusion Transformer

The Diffusion Transformer takes the VLM’s high-level intent plus current sensor state (joint positions, force feedback, visual input) and generates continuous low-level action sequences.

Using a diffusion model for action generation captures something important: the same task can be accomplished in multiple valid ways. A diffusion model can represent this multimodal distribution of valid actions rather than collapsing to a single deterministic output. This is particularly valuable for dexterous manipulation where there are many valid grasping strategies.

graph TD
    A[Language instructions] --> VLM[System 2<br>Vision-Language Model]
    B[Camera images] --> VLM
    VLM --> C[High-level intent vector<br>Action plan]
    C --> DT[System 1<br>Diffusion Transformer]
    D[Joint state<br>Sensor feedback] --> DT
    DT --> E[Continuous action sequence<br>Joint control signals]
    E --> F[Robot execution]

Cross-Embodiment Generalization

GR00T N1’s ability to run on different hardware rests on abstracting the action representation. The model doesn’t output joint angles specific to one robot’s configuration — it produces action representations that can be mapped to different hardware configurations. For a new robot platform, you fine-tune rather than retrain from scratch.

Validated hardware includes: Fourier GR-1, 1X Neo, Agility Robotics Digit, and early testing on Boston Dynamics Atlas.

Training Data: Solving Robot Data Scarcity

Robot AI’s biggest bottleneck is the scarcity of high-quality training data. GR00T N1 uses three sources:

Real captured data: human demonstrations recorded via motion capture systems. High quality, but expensive to collect at scale.

Isaac GR00T-Mimic synthetic data: NVIDIA’s Isaac simulator generates synthetic training data at scale, including edge cases that are difficult to capture in real environments.

Internet video data: learning from internet video of humans performing manipulation tasks. Largest volume, but requires handling the absence of action labels and inconsistent viewpoints.

Comparison

DimensionGR00T N1Task-specific modelRT-X (Google)
Cross-hardware generalityHigh (design goal)Low (hardware-bound)Medium
Open accessOpen source + commercialUsually closedPartially open
Action generationDiffusion TransformerVariousSimilar
Data sourcesMixed (synthetic + real + video)Primarily realCross-robot real data
Fine-tuning difficultyMediumLow (task-specific)Medium

When to Use It (and When Not To)

Good fit:

  • Research groups or startups needing to deploy quickly across multiple robot platforms
  • General manipulation tasks (pick-and-place, assembly) as a research baseline
  • Starting from a pretrained model rather than training from scratch

Not a good fit:

  • Industrial scenarios requiring maximum precision on fixed hardware for specific tasks (a task-specific model will outperform)
  • Extremely low-latency real-time control (diffusion model inference latency needs evaluation)
  • Non-humanoid robots (designed for humanoid form factor; other configurations are not validated)

Overall Assessment

GR00T N1’s most significant contribution isn’t its current benchmark numbers — it’s establishing the robot foundation model paradigm: a general pretrained model, open to the industry for fine-tuning, accumulating cross-hardware knowledge the same way LLMs accumulated cross-domain language knowledge.

GR00T N2, based on DreamZero research and a new world-action model architecture, reportedly succeeds at new tasks in new environments more than twice as often as existing vision-language-action models. That iteration speed, combined with NVIDIA’s compute infrastructure advantages, suggests robot AI may advance faster than most people expect.

References

Ask this article

Answers come from this article only. Click any prompt below or open the chat at the bottom right.

🇺🇸 English

Robot AI has always had a frustrating constraint baked right into it. You train a model for Robot A, and it just... doesn't work on Robot B. Different joint count, different sensors, and you're starting over from scratch. NVIDIA's Isaac GR00T N1, which they unveiled at GTC 2025, is the first serious open attempt to break that wall — and honestly, digging into its architecture made me rethink what a general-purpose robot brain should even look like.

So let's start with the headline facts. GR00T N1 is the world's first open humanoid robot foundation model. Open source, commercially licensed — that combination matters. The architecture is what they call dual-system: one part is a Vision-Language Model that handles high-level reasoning, and the other is a Diffusion Transformer that handles precise action generation. And the whole point is that a single model can run across multiple hardware platforms — the Fourier GR-1, the 1X Neo, and others. That cross-hardware generalization isn't a side effect; it's the core design goal.

Now, why is "general" so hard in the first place? Traditional robot AI models are task-specific and hardware-specific. Change the number of joints in an arm, swap a sensor, and you retrain everything. That makes development expensive, and worse, it kills the compounding advantage that makes software so powerful — the ability to accumulate knowledge over time. Every robot project starts back at zero.

GR00T's goal is one model that, with some fine-tuning, can do manipulation tasks across different humanoid platforms. And the moment you say that out loud, you realize the architecture has to solve two completely different problems at the same time. First, understanding the world — the environment, the language instruction, the goal. That's high-level cognition. Second, precisely controlling dozens of joints to produce smooth, dexterous motion. That's low-level control. Those are genuinely different kinds of problems, and that's the key insight.

Here's where it gets clever. GR00T borrows from cognitive science — specifically Kahneman's idea of System 1 and System 2 thinking. System 2 is the slow, deliberate reasoning: that's the Vision-Language Model, interpreting the scene, understanding your instruction, planning out the sequence of actions. System 1 is the fast, automatic reflex: that's the Diffusion Transformer, generating the continuous motor signals. By splitting them apart, each subsystem gets to use the architecture that actually fits its job. You're not forcing one model to be good at two things it's fundamentally bad at combining.

Let's walk through each half. The Vision-Language Model takes in everything at once — camera images, your language instruction, the state of the environment. And it answers the big question: what's the next step here? It figures out where an object is and how to grasp it. It parses an instruction like "move the red cup to the right side of the table." And for longer tasks, it breaks the whole thing down into subtasks. Now, the crucial detail: the VLM does not output joint angles. It outputs a high-level intent — an action plan, an intent vector. It's thinking, not twitching.

Then that intent flows down to System 1, the Diffusion Transformer. This part takes the high-level intent plus the current physical state — joint positions, force feedback, live vision — and turns it into the actual continuous stream of motor commands. And here's why using a diffusion model is such a smart choice. The same task can be done in many equally valid ways. There are lots of good ways to grasp a cup. A diffusion model can represent that whole spread of valid options — that multimodal distribution — instead of collapsing everything down to one rigid, deterministic answer. For dexterous manipulation, where flexibility is everything, that's exactly what you want.

So picture the flow end to end: your language instruction and the camera images both feed into the Vision-Language Model. That produces a high-level intent and action plan. That intent, combined with the robot's current joint state and sensor feedback, feeds into the Diffusion Transformer. And that spits out a continuous sequence of joint control signals, which the robot executes. Reasoning on top, reflexes on the bottom.

Now, the part that makes cross-hardware actually work. The trick is abstracting the action representation. The model doesn't commit to joint angles that only make sense for one specific robot's body. It produces action representations that can then be mapped onto different hardware configurations. So when a new robot shows up, you fine-tune instead of retraining from zero. The validated hardware list is already interesting — the Fourier GR-1, the 1X Neo, Agility Robotics' Digit, and early testing on Boston Dynamics' Atlas.

Let's talk about data, because that's the real bottleneck in this whole field. High-quality robot training data is scarce, and GR00T pulls from three sources to fight that. First, real captured data — human demonstrations recorded through motion capture. Beautiful quality, but painfully expensive to gather at scale. Second, synthetic data from NVIDIA's Isaac GR00T-Mimic simulator, which can generate training data in huge volume, including weird edge cases you'd almost never catch in the real world. And third, internet video — learning from people doing manipulation tasks on video. That's the largest pool by far, but it's messy: no action labels, inconsistent camera angles. So it's a blend — real for quality, synthetic for scale and edge cases, internet video for sheer breadth.

How does this stack up against the alternatives? Compared to a task-specific model, GR00T trades a little raw performance for enormous flexibility. A task-specific model is easy to fine-tune and great on its one job, but it's locked to its hardware. GR00T is built for cross-hardware generality from the ground up, and it's genuinely open — open source plus commercial licensing — whereas task-specific models are usually closed. Against Google's RT-X, which sits in the middle on generality and is partially open, GR00T pushes harder on both openness and its mixed data strategy. RT-X leans on cross-robot real-world data; GR00T mixes synthetic, real, and video.

So when should you actually reach for this? It's a great fit if you're a research group or a startup that needs to deploy across several robot platforms quickly. It's a strong baseline for general manipulation — pick-and-place, assembly — and it lets you start from a pretrained model instead of building from nothing. But it's not for everyone. If you need maximum precision on fixed hardware for one specific industrial task, a dedicated task-specific model will beat it. If you need extremely low-latency real-time control, you have to seriously evaluate diffusion model inference latency first. And if you're not building a humanoid, this isn't your model — it's designed for the humanoid form factor, and other configurations just aren't validated.

Stepping back — the biggest contribution here isn't the benchmark numbers as they stand today. It's the paradigm. GR00T establishes the idea of a robot foundation model: a general pretrained base that the whole industry can fine-tune, that accumulates cross-hardware knowledge the same way large language models accumulated cross-domain language knowledge. And the trajectory is steep. GR00T N2, built on their DreamZero research and a new world-action model architecture, reportedly succeeds at brand-new tasks in brand-new environments more than twice as often as existing vision-language-action models. Pair that iteration speed with NVIDIA's compute advantage, and robot AI might move a lot faster than most people are bracing for.

So let me leave you with the three things worth holding onto. One: the dual-system split is the whole idea — a Vision-Language Model doing slow, deliberate reasoning, and a Diffusion Transformer doing fast, precise motion, each using the architecture that fits its job. Two: cross-embodiment generalization comes from abstracting the action representation, so a new robot means fine-tuning, not starting over — and that's what finally gives robotics the compounding, knowledge-accumulating advantage that software has always had. And three: this is really about a paradigm shift. The robot foundation model is now a thing that exists, it's open, and given how fast N2 is already moving, this field is about to get very interesting.

🇹🇼 中文

一支示範影片裡,有人假裝在割草、耙落葉,旁邊的人形機器人就跟著把草割了、把葉子耙了。看起來像雜耍,但真正的重點不在那台機器人身上,而在控制它的軟體——一個叫做 Sonic 的通用機器人控制器。這篇整理是以 Two Minute Papers 的介紹影片為事實依據。

先把定位講清楚:Sonic 的貢獻是控制軟體,不是硬體。在示範裡,一個人做各種動作,機器人能理解這些動作,把它翻譯成一連串三維空間裡的關節位置。換句話說,這是一套遙控控制器——人怎麼動,機器人就怎麼跟著動。而且它理解的是整個身體的動作,不是單純複製手臂軌跡。

正因為它掌握的是全身動作,你可以叫它爬進那些你不想進去的狹小空間。這件事本身就很有價值:探勘人類難以進入、或者危險的區域,像是協助救出被瓦礫壓住的人,或者未來在不讓人類冒險的前提下去探索其他星球。它甚至能打功夫——前提是,示範的那個人自己得會功夫。

Sonic 真正誇張的地方,是它的輸入幾乎可以是任何東西。你可以拍一段自己做動作的影片;可以用講的下指令;可以放音樂讓它跟著跳舞——影片因為版權沒放配樂,但確實做得到;比較簡單的任務,像是移動、模仿猴子,甚至直接打字就行。而且它的動作很有「表情」,你可以要它開心地走、偷偷摸摸地走、或者像個受傷的人一樣走。

這裡有個容易被低估的成就:它站得很穩,不會跌倒。影片特別點出,過去就算是模擬世界裡的簡單角色,光是教它走路不跌倒,都要成千上萬次嘗試。現在能直接做到這種穩定度,是一大跳躍。

那它是怎麼學會的?Sonic 先看了一億幀的人類動作,來理解人類怎麼動、動作之間怎麼銜接。關鍵在於,它不需要人工標註的動作標籤。你不用去解釋這個動作叫什麼、代表什麼意思,它就是看原始動作,自己學會如何在不同任務之間平順地切換,中間不會出現那種不自然的停頓。

整條處理流程大概是這樣:各種多模態輸入進來,先被轉成「人類動作」的表示,接著被編碼、量化成一組通用 token,最後由一個 decoder 翻成馬達指令,機器人就照著執行。這裡的通用 token 是整個設計的樞紐——不管你餵進去的是影片、聲音還是文字,都會先被壓成同一種抽象的 token 表示,後面才統一解碼成動作。

不過這裡有個大難點:機器人不是人。把人類動作對應到機器人動作其實非常難,因為機器人的身體構造跟人類根本不一樣。舉個影片裡的例子:你叫它轉身,它當然要轉身,但要轉多快?如果它想在瞬間轉一百八十度,機器人會把自己給甩壞。

為了解決這個,研究提出了一個叫 root trajectory spring model 的做法,對使用者那些突然、劇烈的指令做阻尼處理,避免機器人受傷——對,機器人也是會「受傷」的。裡面有一個隨時間變化的指數項,扮演物理煞車的角色:隨著時間增加,這一項會迅速趨近於零,逼著整個數學式平滑地衰減。它同時達成兩件事:第一,機器人不會因為指令太猛而弄傷自己;第二,動作會穩定地落在目標位置,不會在附近來回震盪個沒完。當然,阻尼也不能加過頭,加太重,機器人就變成一隻慢吞吞、什麼都做不了的鼻涕蟲。這個分寸很難拿捏。

接下來是這篇最反直覺的一點:訓練很貴,成品卻很輕。整套訓練用了一百二十八顆 GPU、跑了三天,成本不低。但重點來了——訓練完成後的成品極度輕量,只有大約四千兩百萬個參數。這個規模小到可以輕鬆在手機上跑,幾乎不吃資源,你不需要那種訓練等級的硬體才能用它。而且影片裡展示的所有模型,都會免費、永久地釋出給所有人,是公開研究,不是封閉的專有技術。

這個專案是由 professor Zhu 跟 Jim Fan 主導。Jim Fan 大約兩年前在 NVIDIA 創立了人形機器人實驗室,之後就持續產出突破性的研究。影片作者還點出一個有意思的類比:這個模型把一鍋雜亂、五花八門的輸入,壓縮成一種純粹、抽象的 token——就像你向很多人徵詢意見時,會聽到各種說法、甚至互相矛盾的建議,但把它們並排來看,往往能找到底層那個共通的道理。

最後提醒一下,這只是一個新興領域的早期成果,不是終點。

那今天收尾,記住三件事。第一,Sonic 是個控制軟體,把多模態輸入統一壓成通用 token,再解碼成全身動作,站得穩、還能有表情。第二,最核心的技術難點是人跟機器人身體構造不同,靠 spring model 的阻尼機制,讓劇烈指令不會把機器人甩壞。第三,也是最反直覺的:訓練要一百二十八顆 GPU 跑三天,成品卻只有四千兩百萬參數,小到手機能跑,而且完全開源。它證明了一件事——把龐雜的人類動作知識,壓進一個人人都能用的小 AI 控制器裡,是真的可行的。

Tags

Related Articles

84,395 Users, Effect Size 0.7: Why a Pretty Number Doesn't Mean a Self-Hypnosis App 'Works'

An npj Digital Medicine study analyzed 84,395 users and 282,893 sessions on the self-hypnosis app Reveri. Self-rated stress dropped with a Cohen's d of −0.71 to −0.78 (a 'large' effect). But it's a retrospective observational study with no control group and a single-item Likert measure — a great case study in why 'big N + big effect' still can't prove causation.