LeRobot SO-101 Tabletop Environment in Isaac Sim 6.2
lerobot/so101-tabletop-isaac
LeRobot releases an Isaac Sim 6.2 environment for the SO-101 robotic arm, featuring a tabletop setup with wooden cubes and a green box. The environment supports the same policies as the MuJoCo version, with front and wrist cameras, joint position control, and a reward system for placing cubes in the box. It is available under the Apache-2.0 license.
The LeRobot SO-101 tabletop environment in Isaac Sim 6.2 provides a simulated workspace for robotic manipulation tasks, with detailed physics and rendering. It supports the same policies as the MuJoCo version, enabling cross-platform training and evaluation. The environment includes front and wrist cameras, joint position control, and a reward system for object placement.
Source: LeRobot — Model repository updates · Read original article ↗
Article text · Original source · English
On this page
SO-101 tabletop (Isaac Sim)
A LeRobot EnvHub environment built natively in NVIDIA Isaac Sim 6.2: an SO-101 arm in the turquoise of the LeRobot kits, four painted wooden 3 cm cubes and a green open box on a table in a simple room, simulated by PhysX and rendered with RTX.
The layout, cameras, observations, actions and LEROBOT_SIM match the MuJoCo version, lerobot/so101-tabletop-mujoco, so the same policies and commands run against both.
| Observation | pixels: front (scene camera) and wrist (gripper camera), 480×640 RGB; agent_pos: 6 joint positions |
| Action | 6 absolute joint position targets: shoulder_pan, shoulder_lift, elbow_flex, wrist_flex, wrist_roll in degrees, gripper 0-100 (the units of LeRobot's so101_follower with use_degrees=True) |
| Control rate | 30 fps (8 PhysX steps per control step) |
| Task | "Put all the cubes in the green box." |
| Reward | the fraction of cubes in the box at each step (0, 0.25, ... 1); info["cubes_in_box"] gives the count |
| Episode end | success (terminated) when all four cubes are in the box; truncated after 3600 steps (2 minutes), set with episode_length |
| Reset | seeded cube positions and yaws, joint noise |
Isaac Sim starts once per process, in make_env: about 20 s, plus a few minutes of shader compilation the first time on a machine or in a container. It holds one world, so make_env takes n_envs=1. A step takes about 80 ms on an RTX 3080 Laptop GPU, so lockstep evaluation is exact, while realtime runs slower than 30 fps. The arm asset is bundled, so the environment needs no network access at runtime.
With the LeRobot simulator server
From a LeRobot checkout, start the server. The image installs packages.txt (system libraries) and requirements.txt (Isaac Sim 6.2 from PyPI and NVIDIA's index, Python 3.12); it needs an NVIDIA RTX GPU and the NVIDIA Container Toolkit:
uv run scripts/run_sim.py --envhub lerobot/so101-tabletop-isaac
Then run a policy against it, for example:
lerobot-rollout --robot.type=remote --robot.endpoint=tcp/127.0.0.1:7448 \
--robot.deployment=envhub --robot.task=so101_tabletop \
--policy.path=<your SO-101 checkpoint> --fps=30
Observations reach policies as observation.images.front, observation.images.wrist and observation.state; use --rename_map if your checkpoint names its cameras differently.
In-process
Install Isaac Sim 6.2 next to LeRobot (uv pip install -r requirements.txt with UV_INDEX_STRATEGY=unsafe-best-match, or pip install -r requirements.txt), then:
from lerobot.envs.factory import make_env
envs = make_env("lerobot/so101-tabletop-isaac", n_envs=1, trust_remote_code=True)
How the scene is built
env.py is a compact example of the Isaac Sim 6 Python API:
| Part | Isaac Sim API |
|---|---|
| Arm | NVIDIA's RobotStudio/so101_new_calib asset (bundled in so101_new_calib/), referenced with stage_utils.add_reference_to_stage, driven through Articulation |
| Room, table, cubes, box, lights | GroundPlane, scaled Cube prims, DomeLight, DistantLight; GeomPrim colliders, RigidPrim bodies, RigidBodyMaterial |
| Looks | OmniPbrMaterial for the wooden cubes (textures/wood.png tinted per cube, with projected UVs since USD cubes have none) and for the arm's color (an override of the asset's own material), PreviewSurfaceMaterial for the table and the glossy box; colors are the MuJoCo scene's sRGB values, converted to Isaac's linear space |
| Cameras | RtxCamera + CameraSensor (RGB annotator); the wrist camera is parented to the arm's gripper link |
| Stepping | timeline paused, SimulationManager.step for physics, a Replicator capture step for the cameras |
Licenses
- This environment is Apache-2.0, including
textures/wood.png(generated procedurally for it). so101_new_calib/is NVIDIA's SO-101 asset from the Isaac Sim 6.2 asset library (Isaac/Robots/RobotStudio/so101_new_calib, thumbnails left out), unchanged (env.pyoverrides its color on the stage), under the Apache-2.0 license in that folder.- Isaac Sim is distributed under the NVIDIA Isaac Sim license.
env.pysetsOMNI_KIT_ACCEPT_EULA=YES; running this environment means you accept it.
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
- Environment
- SimulationOpen source S2
Reported numbers
observation pixels
480
View original evidence
ds run against both. Observation pixels : front (scene camera) and wrist (gripper camera), 480×640 RGB; agent_pos : 6 joint positions Action 6 absolute joint position targets: shoulder_pan , shoulder_lift , elbow_flex , wrist_flex , wrist_roll in degrees, gripper 0-100 (the units of LeRobot's so101_follower with use_degrees=True ) Control rate 30 fps (8 PhysX steps per control step) Task "Put all the cubes in the gre
Open source S3observation pixels
640
View original evidence
ds run against both. Observation pixels : front (scene camera) and wrist (gripper camera), 480×640 RGB; agent_pos : 6 joint positions Action 6 absolute joint position targets: shoulder_pan , shoulder_lift , elbow_flex , wrist_flex , wrist_roll in degrees, gripper 0-100 (the units of LeRobot's so101_follower with use_degrees=True ) Control rate 30 fps (8 PhysX steps per control step) Task "Put all the cubes in the gre
Open source S3control rate
30
View original evidence
ds run against both. Observation pixels : front (scene camera) and wrist (gripper camera), 480×640 RGB; agent_pos : 6 joint positions Action 6 absolute joint position targets: shoulder_pan , shoulder_lift , elbow_flex , wrist_flex , wrist_roll in degrees, gripper 0-100 (the units of LeRobot's so101_follower with use_degrees=True ) Control rate 30 fps (8 PhysX steps per control step) Task "Put all the cubes in the gre
Open source S3episode length
3,600
View original evidence
en box." Reward the fraction of cubes in the box at each step (0, 0.25, ... 1); info["cubes_in_box"] gives the count Episode end success (terminated) when all four cubes are in the box; truncated after 3600 steps (2 minutes), set with episode_length Reset seeded cube positions and yaws, joint noise Isaac Sim starts once per process, in make_env : about 20 s, plus a few minutes of shader compilation the first time on
Open source S4step time
≈80
View original evidence
a machine or in a container. It holds one world, so make_env takes n_envs=1 . A step takes about 80 ms on an RTX 3080 Laptop GPU, so lockstep evaluation is exact, while realtime runs slower than 30 fps. The arm asset is bundled, so the environment needs no network access at runtime. With the LeRobot simulator server From a LeRobot checkout, start the server. The image installs packages.txt (system libraries) and requ
Open source S5fps
30
View original evidence
irements.txt (Isaac Sim 6.2 from PyPI and NVIDIA's index, Python 3.12); it needs an NVIDIA RTX GPU and the NVIDIA Container Toolkit: uv run scripts/run_sim.py --envhub lerobot/so101-tabletop-isaac Then run a policy against it, for example: lerobot-rollout --robot.type=remote --robot.endpoint=tcp/127.0.0.1:7448 \ --robot.deployment=envhub --robot.task=so101_tabletop \ --policy.path=<your SO-101 checkpoint> --fps=30 Ob
Open source S6number of cubes
4
View original evidence
en box." Reward the fraction of cubes in the box at each step (0, 0.25, ... 1); info["cubes_in_box"] gives the count Episode end success (terminated) when all four cubes are in the box; truncated after 3600 steps (2 minutes), set with episode_length Reset seeded cube positions and yaws, joint noise Isaac Sim starts once per process, in make_env : about 20 s, plus a few minutes of shader compilation the first time on
Open source S4reward fraction
0
View original evidence
en box." Reward the fraction of cubes in the box at each step (0, 0.25, ... 1); info["cubes_in_box"] gives the count Episode end success (terminated) when all four cubes are in the box; truncated after 3600 steps (2 minutes), set with episode_length Reset seeded cube positions and yaws, joint noise Isaac Sim starts once per process, in make_env : about 20 s, plus a few minutes of shader compilation the first time on
Open source S4
- code: not_reported
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
SO-101 tabletop (Isaac Sim) A LeRobot EnvHub environment built natively in NVIDIA Isaac Sim 6.2: an SO-101 arm in the turquoise of the LeRobot kits, four painted wooden 3 cm cubes and a green open box on a table in a simple room, simulated by PhysX and rendered with RTX. The layout, cameras, observations, actions and LEROBOT_SIM match the MuJoCo version, lerobot/so101-tabletop-mujoco , so the same policies and comman
Open source S2
ds run against both. Observation pixels : front (scene camera) and wrist (gripper camera), 480×640 RGB; agent_pos : 6 joint positions Action 6 absolute joint position targets: shoulder_pan , shoulder_lift , elbow_flex , wrist_flex , wrist_roll in degrees, gripper 0-100 (the units of LeRobot's so101_follower with use_degrees=True ) Control rate 30 fps (8 PhysX steps per control step) Task "Put all the cubes in the gre
Open source S3
en box." Reward the fraction of cubes in the box at each step (0, 0.25, ... 1); info["cubes_in_box"] gives the count Episode end success (terminated) when all four cubes are in the box; truncated after 3600 steps (2 minutes), set with episode_length Reset seeded cube positions and yaws, joint noise Isaac Sim starts once per process, in make_env : about 20 s, plus a few minutes of shader compilation the first time on
Open source S4
a machine or in a container. It holds one world, so make_env takes n_envs=1 . A step takes about 80 ms on an RTX 3080 Laptop GPU, so lockstep evaluation is exact, while realtime runs slower than 30 fps. The arm asset is bundled, so the environment needs no network access at runtime. With the LeRobot simulator server From a LeRobot checkout, start the server. The image installs packages.txt (system libraries) and requ
Open source S5
irements.txt (Isaac Sim 6.2 from PyPI and NVIDIA's index, Python 3.12); it needs an NVIDIA RTX GPU and the NVIDIA Container Toolkit: uv run scripts/run_sim.py --envhub lerobot/so101-tabletop-isaac Then run a policy against it, for example: lerobot-rollout --robot.type=remote --robot.endpoint=tcp/127.0.0.1:7448 \ --robot.deployment=envhub --robot.task=so101_tabletop \ --policy.path=<your SO-101 checkpoint> --fps=30 Ob
Open source S6
Source：LeRobot — Model repository updates · huggingface.co