OPEN MODELS/VISION-LANGUAGE-ACTION/COGACT
ADVANCEDVISION-LANGUAGE-ACTION

CogACT

Componentized VLA with specialized diffusion action module

WHAT DOES THIS ACTUALLY DO?

CogACT separates thinking from doing — the language model figures out what to do, and a specialized diffusion model handles the precise physical movements. This split makes it especially good at tasks needing fine motor control.

PARAMETERS
~2B
RELEASED
2024-10
LICENSE
Apache 2.0
ARCHITECTURE
VLM...
CAN I ACTUALLY RUN THIS?
5
Advanced
OUT OF 10

Primarily a research model. Requires significant ML experience to set up and operate. Best suited for labs with existing robotics infrastructure.

DOWNLOAD WEIGHTS
Download from Hugging Face
10GB disk space required
OVERVIEW

A componentized Vision-Language-Action architecture using DiT-Large as its action module, conditioned on VLM outputs for precise manipulation.

TRAINED ON

Open X-Embodiment + proprietary manipulation data

ARCHITECTURE
VLM + DiT-Large Action Module
CAPABILITIES
High-precision manipulation
Modular architecture
Flexible action representations
Multi-task learning
REAL ROBOTICS USE CASES
MANUFACTURING

Precision screw insertion on assembly line

RESEARCH

Baseline for fine-grained manipulation benchmarks

WAREHOUSE

Careful handling of fragile items

HARDWARE REQUIREMENTS
MIN VRAM
12GB
REC VRAM
24GB
MIN RAM
32GB
DISK SPACE
10GB
JETSON
Not Supported
CPU ONLY
GPU Required
TESTED GPUs
RTX 3090RTX 4090A100
DEPLOYMENT PATHS
Local PC
24GB VRAM preferredSUPPORTED
Docker
SUPPORTED
Cloud
SUPPORTED
ROS2
No official integrationNOT SUPPORTED
Jetson
NOT SUPPORTED
Isaac Sim
NOT SUPPORTED
RELATED PAPERS
CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulation
READ ON ARXIV →
RELATED ORGANIZATIONS
Physical Intelligence
ORGANIZATIONS
CogACT Research Team
DIFFICULTY
ADVANCED

Serious GPU and ML engineering required

COMPUTE3/10
SETUP3/10
BEGINNER-FRIENDLY3/10
EDGE DEPLOYMENT
Jetson Not Supported
CPU-only Not Available
Community →

Seeded and planned prompts — not live forum activity yet.