Skip to content
Strands Harness
PYPython SDKTSTypeScript SDK
Strands
EVEvalsSHShell
Organizations
strands-agentsstrands-labs

Strands for Cosmos

Strands for Cosmos brings NVIDIA Cosmos to Strands Agents - giving an agent eyes that understand physics and hands that generate video, audio, and robot actions, all on local compute. It ships four model providers (Cosmos 3 omnimodal Reasoner and Generator, plus the Cosmos-Reason2 VLM) and 45 tools spanning the full pipeline: reasoning, generation, curation, post-training, quantization, edge deployment, and evaluation.

from strands import Agent
from strands_cosmos import Cosmos3ReasonerModel


agent = Agent(model=Cosmos3ReasonerModel(base_url="http://localhost:8000/v1"))
agent("Caption in detail: <video>scene.mp4</video>")

Getting started

Section titled “Getting started”
Terminal window
uv pip install strands-cosmos

The package is published to PyPI as strands-cosmos. Pick an extra for your task:

ExtraPulls inFor
(none)transformers, torch, torchvision, torchcodec, avCosmos-Reason2 VLM + tools
cosmos3openaiCosmos 3 reasoner client (vLLM server)
cosmos3-gendiffusers, cosmos_guardrail, soundfile, imageioCosmos 3 generator (in-process)
vllmvllm, openaiself-hosting vLLM
alleverything (heavy)kitchen sink

Two runtime surfaces

Section titled “Two runtime surfaces”

Cosmos 3 is a unified Mixture-of-Transformers that jointly understands and generates text, images, video, audio, and action. Strands for Cosmos exposes both:

  • Reasoner (Cosmos3ReasonerModel) - text + vision to text, served by vLLM.
  • Generator (Cosmos3GeneratorModel) - to image / video / audio / action, running in-process via Diffusers (no server).
from strands_cosmos import Cosmos3GeneratorModel


gen = Cosmos3GeneratorModel(model_id="nvidia/Cosmos3-Nano")
gen.generate(mode="text2video", prompt="A robot navigates a warehouse.", out_path="vid.mp4")
gen.generate(mode="text2video-with-sound", prompt="A robot pours water.", out_path="av.mp4", enable_sound=True)
gen.generate(mode="image2video", prompt="It moves forward.", image="frame.jpg", out_path="i2v.mp4")

Lightweight edge VLM

Section titled “Lightweight edge VLM”

For edge and Jetson deployments, the Cosmos-Reason2 VLM runs as a Strands model provider with a tiny footprint - verified on Jetson AGX Thor with Chain-of-Thought reasoning.

from strands import Agent
from strands_cosmos import CosmosVisionModel


agent = Agent(model=CosmosVisionModel(model_id="nvidia/Cosmos-Reason2-2B"))
agent("Caption in detail: <video>dashcam.mp4</video>")
agent("<image>robot_view.jpg</image> What should the robot do next?")
Section titled “Links”