Setup & Tutorial

Get started with Action Atlas experiments and visualization

1

Clone the Repository

Clone with submodules to include LeRobot, LIBERO, and OpenVLA-OFT.

git clone --recursive https://github.com/CWRU-AISM/action-atlas.git cd action-atlas
2

Create Environment

Set up a conda environment with Python 3.12 and install PyTorch with CUDA support.

conda create -y -n actionatlas python=3.12 conda activate actionatlas pip install torch torchvision --index-url https://download.pytorch.org/whl/cu128
3

Install Dependencies

Install LeRobot (model policies for X-VLA, SmolVLA, GR00T, Pi0.5), LIBERO (environments), and project packages.

cd lerobot && pip install -e ".[pi]" && cd .. cd LIBERO && pip install -e . && cd .. pip install -e .
4

Verify Installation

Check that all model policies and LIBERO are available.

python -c "from lerobot.policies.xvla.modeling_xvla import XVLAPolicy; print('X-VLA OK')" python -c "from lerobot.policies.smolvla.modeling_smolvla import SmolVLAPolicy; print('SmolVLA OK')" python -c "from lerobot.policies.pi05.modeling_pi05 import PI05Policy; print('Pi0.5 OK')" python -c "import libero; print('LIBERO OK')" python experiments/grid_ablation.py --help

Running Experiments

All experiments use a unified CLI. Swap --model to run on any supported VLA.

1

Grid Ablation

Zero each layer one at a time and measure task success rate to find critical components.

python experiments/grid_ablation.py \ --model xvla --suite libero_object --n-episodes 3 --gpu 0
2

Vision Perturbation

Apply 24 image corruptions (noise, blur, color, spatial, extreme) and measure robustness.

python experiments/vision_perturbation.py \ --model smolvla --suite libero_goal --gpu 1
3

Counterfactual Prompting

Test how models respond to null, negated, or wrong language instructions.

python experiments/counterfactual.py \ --model pi05 --suite libero_spatial --gpu 2

SAE Interpretability Pipeline

The full pipeline: collect activations, train SAEs, identify concepts, then verify causality.

1

Collect Activations

Run baseline episodes and capture per-layer activations for SAE training.

python experiments/baseline.py \ --model xvla --suite libero_object \ --collect-activations --n-episodes 3
2

Train TopK SAEs

Train sparse autoencoders on collected activations (8x expansion, k=64).

python experiments/train_sae.py \ --activations-dir outputs/xvla_experiments/baseline_libero_object/activations \ --output-dir outputs/xvla_saes/libero_object
⚠️
Always use per-token SAEs. Mean-pooled SAEs cause severe task failure despite high reconstruction quality. Set TORCH_COMPILE_DISABLE=1 for all experiment scripts.
3

Concept Identification

Score features using Cohen's d to find task-selective SAE features.

python experiments/concept_id.py \ --sae-dir outputs/xvla_saes/libero_object \ --activations-dir outputs/xvla_experiments/baseline_libero_object/activations \ --suite libero_object
4

Concept Ablation & Steering

Remove or amplify specific concept features during live rollouts to verify causality.

# Ablation: remove features and measure impact python experiments/concept_ablation.py \ --model xvla --suite libero_object \ --sae-dir outputs/xvla_saes/libero_object \ --concept-results results/concept_id/libero_object/all_layers.json \ --layer transformer_L12 # Steering: amplify or suppress features python experiments/concept_steering.py \ --model xvla --layer transformer_L12 \ --strengths -2.0 -1.0 1.0 2.0

Running Action Atlas Locally

Action Atlas is deployed at action-atlas.com, but you can also run it locally to explore your own experiment data.

1

Install Visualization Dependencies

pip install flask flask-cors
2

Start the Backend

The Flask backend serves experiment data through a REST API.

cd action_atlas/backend python run.py --port 6006
3

Start the Frontend

The Next.js frontend provides the interactive visualization interface.

cd action_atlas/frontend npm install echo "NEXT_PUBLIC_API_URL=http://localhost:6006" > .env.local npm run dev

Open http://localhost:3002 to access Action Atlas locally.

Using Your Own Data

Point Action Atlas at your experiment outputs by setting environment variables:

export XVLA_ROLLOUTS_DIR=/path/to/your/xvla/data export SMOLVLA_ROLLOUTS_DIR=/path/to/your/smolvla/data export GROOT_ROLLOUTS_DIR=/path/to/your/groot/data

Platform Features

Feature Explorer

Browse UMAP scatter plots of SAE features, search by concept, and inspect individual feature activations across layers and suites.

Cross-Model Comparison

Compare layer criticality, concept representations, and robustness across X-VLA, SmolVLA, GR00T, Pi0.5, and OpenVLA-OFT.

Ablation Studies

Compare baseline vs. ablated behavior, view success rate deltas per concept, and explore layer-phase ablation matrices.

Perturbation Testing

Explore vision perturbation robustness results across 24 perturbation types and all supported models.