103 lines
4.3 KiB
Markdown
103 lines
4.3 KiB
Markdown
|
三、仿真任务全流程（LIBERO 数据集微调+评估）
|
||
|
|
适用于单臂机器人仿真场景，基于 LeRobot 格式 LIBERO 数据集完成模型微调与仿真测试
|
||
|
|
3.1 下载 LIBERO 数据集
|
||
|
|
libero的huggingface路径：https://huggingface.co/datasets/lerobot/libero
|
||
|
|
huggingface-cli download lerobot/libero \
|
||
|
|
--repo-type dataset \
|
||
|
|
--local-dir /path/to/libero_all
|
||
|
|
3.2 配置训练 YAML 文件
|
||
|
|
复制官方示例配置，替换所有本地路径参数，适配本地环境
|
||
|
|
cp workspace/example/libero.yml /path/to/my_libero_config.yml
|
||
|
|
核心必填配置（需完整替换）：
|
||
|
|
参考：https://github.com/X-Square-Robot/wall-x/blob/main/workspace/example/libero.yml
|
||
|
|
model:
|
||
|
|
config_path: /path/to/wall-oss-0.5/config.json # 模型配置文件路径
|
||
|
|
processor_path: /path/to/Qwen2.5-VL-3B-Instruct # 处理器路径
|
||
|
|
pretrained_path: /path/to/Qwen2.5-VL-3B-Instruct # 预训练基座路径
|
||
|
|
|
||
|
|
data:
|
||
|
|
lerobot_config:
|
||
|
|
repo_id: /path/to/libero_all # 本地LIBERO数据集根目录
|
||
|
|
norm_stats_path: /path/to/libero_all_norm_stats.json # 归一化统计文件
|
||
|
|
key_mappings: # 数据集字段映射（固定适配LIBERO）
|
||
|
|
camera:
|
||
|
|
observation.images.faceImg: face_view
|
||
|
|
observation.images.rightImg: right_wrist_view
|
||
|
|
state: observation.state
|
||
|
|
action: action
|
||
|
|
|
||
|
|
checkpoint:
|
||
|
|
save_path: /path/to/libero_training_output # 训练结果保存路径
|
||
|
|
resume_from: /path/to/wall-oss-0.5/model.safetensors # 预训练权重路径
|
||
|
|
|
||
|
|
dof_config配置
|
||
|
|
# Libero delta action: pos3 + rot3 + gripper1 = 7, plus action_padding(19) = 26.
|
||
|
|
dof_config:
|
||
|
|
master_right_ee_cartesian_pos: 3 # delta position
|
||
|
|
master_right_ee_rotation: 3 # delta rotation (ZYX euler)
|
||
|
|
master_right_gripper: 1
|
||
|
|
action_padding: 19
|
||
|
|
ar_dof_config:
|
||
|
|
master_right_ee_cartesian_pos: 3
|
||
|
|
master_right_ee_rotation: 3
|
||
|
|
master_right_gripper: 1
|
||
|
|
action_padding: 19
|
||
|
|
# State: pos3 + rot3 + gripper2 = 8, plus action_padding(18) = 26.
|
||
|
|
agent_pos_config:
|
||
|
|
follow_right_ee_cartesian_pos: 3
|
||
|
|
follow_right_ee_rotation: 3
|
||
|
|
follow_right_gripper: 2
|
||
|
|
action_padding: 18
|
||
|
|
action_horizon: 10
|
||
|
|
action_horizon_flow: 10
|
||
|
|
|
||
|
|
|
||
|
|
关键适配说明：LIBERO 为7维单臂动作，需通过配置中 action_padding 补全至26维，匹配模型预训练维度（参考示例配置注释）
|
||
|
|
3.3 生成数据集归一化统计文件
|
||
|
|
训练前必须执行，生成数据均值、方差统计文件，保证训练稳定性
|
||
|
|
python scripts/compute_norm_stats.py \
|
||
|
|
--train_config /path/to/my_libero_config.yml \
|
||
|
|
--data_root /path/to/libero_all \
|
||
|
|
--output_path /path/to/libero_all_norm_stats.json
|
||
|
|
执行完成后，需确认配置文件中 norm_stats_path 与输出路径一致
|
||
|
|
3.4 启动模型微调训练
|
||
|
|
硬件要求：单卡训练最低需要 48G 显存，多卡训练推荐开启 FSDP 分布式训练
|
||
|
|
单GPU训练命令
|
||
|
|
CUDA_VISIBLE_DEVICES=0 \
|
||
|
|
python wall_x/trainer/fsdp_trainer/train_fsdp.py \
|
||
|
|
--config /path/to/my_libero_config.yml
|
||
|
|
多GPU训练（推荐）
|
||
|
|
CUDA_VISIBLE_DEVICES=0,1,2,3 \
|
||
|
|
torchrun --nproc_per_node=4 \
|
||
|
|
wall_x/trainer/fsdp_trainer/train_fsdp.py \
|
||
|
|
--config /path/to/my_libero_config.yml
|
||
|
|
断点合并说明
|
||
|
|
多卡FSDP训练会生成分片权重文件，推理前需合并为完整权重：
|
||
|
|
python scripts/merge_sharded_weights.py \
|
||
|
|
/path/to/sharded_checkpoint \
|
||
|
|
/path/to/merged_checkpoint
|
||
|
|
3.5 LIBERO 仿真推理评估
|
||
|
|
批量测试模型在仿真场景的任务完成效果，支持指定任务套件、测试次数
|
||
|
|
前置依赖校验（未安装需重新执行仿真依赖安装命令）：需提前安装 robosuite、MuJoCo、PyOpenGL 等仿真组件
|
||
|
|
常规批量评估
|
||
|
|
参考脚本：https://github.com/X-Square-Robot/wall-x/blob/main/scripts/run_libero.sh
|
||
|
|
CHECKPOINT_PATH=/path/to/checkpoint \
|
||
|
|
TRAIN_CONFIG_PATH=/path/to/my_libero_config.yml \
|
||
|
|
TASK_SUITE_NAME=libero_spatial \
|
||
|
|
NUM_TRIALS_PER_TASK=50 \
|
||
|
|
bash scripts/run_libero.sh
|
||
|
|
快速冒烟测试（调试用，单任务1次测试）
|
||
|
|
SMOKE=1 CHECKPOINT_PATH=/path/to/checkpoint bash scripts/run_libero.sh
|
||
|
|
核心环境变量说明
|
||
|
|
环境变量
|
||
|
|
参数说明
|
||
|
|
CHECKPOINT_PATH
|
||
|
|
训练完成的模型断点目录
|
||
|
|
TRAIN_CONFIG_PATH
|
||
|
|
训练使用的YAML配置文件路径
|
||
|
|
TASK_SUITE_NAME
|
||
|
|
仿真任务套件：libero_spatial / libero_object / libero_goal / libero_10
|
||
|
|
ALL_SUITES=1
|
||
|
|
开启后批量运行全部4类仿真任务套件
|
||
|
|
TASK_INDICES
|
||
|
|
指定测试任务序号，多任务用逗号分隔（如0,1,2）
|