OPEN MODELS/IMITATION LEARNING/DIFFUSION POLICY
INTERMEDIATEIMITATION LEARNING

Diffusion Policy

Visuomotor policy learning via conditional diffusion

WHAT DOES THIS ACTUALLY DO?

Diffusion Policy borrows the same technology behind AI image generators (like Stable Diffusion) and applies it to robot movement. Instead of generating a picture, it generates the robot's next sequence of moves — and it handles uncertainty and multiple possible actions far better than older approaches.

PARAMETERS
~100M
RELEASED
2023-06
LICENSE
MIT
ARCHITECTURE
Vision...
CAN I ACTUALLY RUN THIS?
7
Intermediate
OUT OF 10

Well-documented with active community support. Mid-range GPU required. If you're comfortable with Python and basic ML, you can run demos in a few hours.

DOWNLOAD WEIGHTS
View on GitHub
3GB disk space required
OVERVIEW

Learns robot visuomotor policies by representing the action distribution as a conditional denoising diffusion process. State-of-the-art across simulation and real-robot benchmarks.

TRAINED ON

Various manipulation demonstration datasets

ARCHITECTURE
Vision Encoder + Diffusion UNet / Transformer
CAPABILITIES
Multi-modal action distributions
Visual observations
High precision control
Temporal consistency
REAL ROBOTICS USE CASES
MANUFACTURING

Precise part insertion with variable starting positions

RESEARCH

Gold standard baseline for manipulation papers

HOME ROBOTICS

Pouring and placing tasks with natural variation

HARDWARE REQUIREMENTS
MIN VRAM
8GB
REC VRAM
16GB
MIN RAM
16GB
DISK SPACE
3GB
JETSON
Compatible
CPU ONLY
GPU Required
TESTED GPUs
RTX 3080RTX 4080A100
DEPLOYMENT PATHS
Local PC
8GB VRAM minimumSUPPORTED
Docker
SUPPORTED
ROS2
Community adapters existSUPPORTED
Jetson AGX Orin
Tested and supportedSUPPORTED
Cloud
SUPPORTED
Isaac Sim
Sim training supportedSUPPORTED
RELATED PAPERS
Diffusion Policy: Visuomotor Policy Learning via Action Diffusion
READ ON ARXIV →
RELATED ORGANIZATIONS
Toyota Research InstitutePhysical Intelligence
ORGANIZATIONS
Columbia UniversityToyota Research InstituteMIT
DIFFICULTY
INTERMEDIATE

Requires some ML and Python experience

COMPUTE5/10
SETUP5/10
BEGINNER-FRIENDLY6/10
EDGE DEPLOYMENT
Jetson Compatible
CPU-only Not Available
Community →

Seeded and planned prompts — not live forum activity yet.