* Update README.md
* Update fake_inference.py
* Update fake_inference.py
* update paper link to arXiv version
* update paper link to arXiv version
* Update fake_inference.py
* add mot
* update libero example
* translate zh to en
* fix load model from hf
* lint
* lint
---------
Co-authored-by: yangping <yangping@x2robot.com>
* Update the details of the inference service startup
* add functions to compute and update robot-specific action/state normalization stats
* Update for pre-commit
* Union compute_action_statistics code into update_action_statistics
* Update for pre-commit
* Add default value for dof_config and agent_pos_config
---------
Co-authored-by: farmer <farmer@x2robot.com>
This release introduces significant performance optimizations, memory efficiency
improvements, and enhanced system robustness:
🚀 Performance Optimizations:
- Add three new fused CUDA kernels (rope_index, rot_pos_emb, get_window_index)
for accelerated multimodal preprocessing
- Implement FSDP2 support for distributed training with improved memory efficiency
- Add Torch.compile integration for additional performance gains
- Optimize memory usage: reduce peak allocation from 48GB to 24GB on 8-GPU setup
🔧 System Robustness:
- Fix missing token position inputs in prediction pipeline
- Add type-robust negation operations in RoPE CUDA kernels (half/bfloat16 support)
- Fix dataset root parameter initialization in LeRobot data loader
- Enhanced error handling and input validation across fusion operators
📚 Documentation & Usability:
- Add comprehensive memory usage benchmarks and hardware recommendations
- Update citation format with proper arXiv reference
- Improve training configuration documentation with quick start guide
- Add detailed API documentation for new fusion operators
🛠️ Technical Details:
- Version bump to 1.0.1
- New CUDA kernels: rope_index.cu, rot_pos.cu, window_index.cu
- FSDP2 state dict loading with distribute_tensor support
- Enhanced multimodal RoPE with 3D position encoding
- Window attention optimization for Vision Transformers
Breaking Changes: None - all changes are backward compatible
huangyanpei
Zihan Gao