"""DRAFT modality config for MLeggiero/g1-gr00t-inspire-red-ball (G1_INSPIRE). Registers the NEW_EMBODIMENT ("new_embodiment") modality config for fine-tuning GR00T-N1.7-3B on the Inspire-hand red-ball dataset. Imported by launch_finetune via --modality-config-path (importlib.import_module), which runs the assignment to MODALITY_CONFIGS below. === VERIFIED from the dataset files (meta/info.json, meta/modality.json, meta/stats.json) and the dataset README === - 85 episodes / 40,878 frames @ 60 fps; LeRobot v2.1 (same version as our apple dataset, which trains fine). - video: ego_view, observation.images.ego_view, [240, 424, 3] (RealSense D435). - state (baseline, observation.state[0:29]): left_arm(7) right_arm(7) left_hand(6) right_hand(6) waist(3). (state[29:63] is tactile -> not used here.) - action (30): left_arm(7) right_arm(7) left_hand(6) right_hand(6) base_height(1) navigate_command(3). - README explicitly marks left_arm/right_arm action as RELATIVE. - hand values are radians in [0, 1.7] (fingers; thumb_yaw [-0.1, 1.3]), order [pinky, ring, middle, index, thumb_pitch, thumb_yaw]. Per the author's replay_dataset.py (register_to_rad + load_from_dataset): Inspire register 1000 = OPEN / 0 = CLOSED, and the DATASET convention is the flip of the USD one -> dataset value ~1.7 = OPEN, 0 = CLOSED. See HAND-DIRECTION CAVEAT. === INFERRED / TO CONFIRM WITH THE AUTHOR (marked ASSUMPTION below) === A. hands/base_height/navigate_command rep = ABSOLUTE (README marks only the arms as relative; absolute is the default for the rest, but unconfirmed). B. action delta_indices = range(30) (chunk horizon). Model max is 40; the dataset ships no precomputed relative_stats, so the horizon is a free choice. The author's actual training horizon is unknown. C. state/video delta_indices = [0] (single current frame, matching every other G1 config in this repo). Whether the author used frame history is unconfirmed. Best fix for A-C: get the author's real g1_inspire_modality_config.py and use it verbatim instead of this reconstruction. === HAND-DIRECTION CAVEAT (per the dataset author) === The Inspire RH56DFTP 6-element hand command: the *manual* documents the finger direction BACKWARDS -- the real hand moves opposite. In THIS dataset the hand values are already radian flexion angles collected with the corrected driver, so TRAINING just learns these numbers (no action needed here). For DEPLOYMENT on a real Inspire hand, you MUST drive it with the same inverted-from-manual convention the author collected with, or the fingers go the wrong way. """ from gr00t.configs.data.embodiment_configs import MODALITY_CONFIGS from gr00t.data.types import ( ActionConfig, ActionFormat, ActionRepresentation, ActionType, ModalityConfig, ) _REL = ActionConfig( rep=ActionRepresentation.RELATIVE, type=ActionType.NON_EEF, format=ActionFormat.DEFAULT ) _ABS = ActionConfig( rep=ActionRepresentation.ABSOLUTE, type=ActionType.NON_EEF, format=ActionFormat.DEFAULT ) MODALITY_CONFIGS["new_embodiment"] = { "video": ModalityConfig( delta_indices=[0], # ASSUMPTION C: single current frame modality_keys=["ego_view"], ), "state": ModalityConfig( delta_indices=[0], # ASSUMPTION C modality_keys=["left_arm", "right_arm", "left_hand", "right_hand", "waist"], ), "action": ModalityConfig( delta_indices=list(range(30)), # ASSUMPTION B: chunk horizon 30 (<= model max 40) modality_keys=[ "left_arm", "right_arm", "left_hand", "right_hand", "base_height", "navigate_command", ], action_configs=[ _REL, # left_arm (VERIFIED relative, per README) _REL, # right_arm (VERIFIED relative, per README) _ABS, # left_hand (ASSUMPTION A: absolute) _ABS, # right_hand (ASSUMPTION A: absolute) _ABS, # base_height (ASSUMPTION A: absolute) _ABS, # navigate_command (ASSUMPTION A: absolute) ], ), "language": ModalityConfig( delta_indices=[0], modality_keys=["annotation.human.task_description"], ), }