Align2Act

Instruction-Tuned Models for Human-Aligned Autonomous Driving

Overview

Our approach transforms scenario data into textual descriptions and, by setting specific instructions, enables a fine-tuned LLM to generate InstructChain and trajectories that align with human driving behavior. The trajectory is subsequently applied in a simulated environment.

Results

Test14-random and Test14-hard benchmarks

Planners		Test14-random			Test14-hard
Type	Method	OLS	NR-CLS	R-CLS	OLS	NR-CLS	R-CLS
Expert	LogReplay	100.0	94.03	75.86	100.0	85.96	68.80
Rule-based	IDM	34.15	70.39	72.42	20.07	56.16	62.26
	PDM-Closed	46.32	90.05	91.64	26.43	65.07	75.18
Hybrid	GameFormer	79.35	80.80	79.31	75.27	66.59	68.83
	PDM-Hybrid	82.21	90.20	91.56	73.81	65.95	75.79
Learning-based	PlanCNN	62.93	69.66	67.54	52.4	49.47	52.16
	UrbanDriver	82.44	63.27	61.02	76.9	51.54	49.07
	GC-PGP	77.33	55.99	51.39	73.78	43.22	39.63
	PDM-Open	84.14	52.80	57.23	79.06	33.51	35.83
	PlanTF	87.07	86.48	80.59	83.32	72.68	61.7
LLM-based	Align2Act (LLaMA2 without Chain)	61.53	46.91	42.47	68.02	55.14	50.31
	Align2Act (InstructionChain)	85.17	70.31	66.96	81.12	57.37	52.95

Project Workflow

Getting Started

Dataset Setup

Follow the official documentation to set up the nuPlan dataset.

Setup Environment

• Create an environment using Python 3.10

conda create -n align2act python=3.10
source activate align2act

• Follow the official documentation to set up the LLaMA2-Accessory environment.
• Follow the official documentation to set up the nuplan-devkit environment. Make sure to set the following variables correctly:

- NUPLAN_DATA_ROOT="/teamspace/studios/this_studio/nuplan-devkit/nuplan/dataset"
- NUPLAN_MAPS="/teamspace/studios/this_studio/nuplan-devkit/nuplan/dataset/maps"
- NUPLAN_EXP_ROOT="/teamspace/studios/this_studio/nuplan-devkit/nuplan/exp" 

• Clone the Align2Act repository:

cd nuplan-devkit
git clone https://github.com/Jkanishkha0305/Align2Act.git

After setting up the environment, your directory structure should appear as follows:

├── LLaMA2-Accessory
├── nuplan-devkit
│  ├── Align2Act

Feature cache

This section preprocesses the dataset to enable faster subsequent data retrieval.

This refers to the implementation of planTF.

Execute the command below to generate 1M frames of training data in cache.cache_path.

You may need to:

• Modify cache.cache_path according to your setup.
• Adjust worker.threads_per_node based on your RAM and CPU capacity.

Please note that this step is time-intensive and may take dozens of hours to complete.

export PYTHONPATH=$(pwd)/nuplan-devkit:$PYTHONPATH
export PYTHONPATH=$(pwd)/nuplan-devkit/Align2Act:$PYTHONPATH

cd ./nuplan-devkit/Align2Act
python run_cache.py \
+caching=cache_llm \
scenario_builder=nuplan \
cache.cache_path=/teamspace/studios/this_studio/cache_1M \
cache.cleanup_cache=true \
scenario_filter=training_scenarios_1M \
worker.threads_per_node=40

Get training data from cache

This section of the code transforms cached data into a json file formatted for training LLM.

Use the command below to process limit_num cache entries (modifiable in the code).

It converts these entries into a json file and saves it at /teamspace/studios/this_studio/cache_1M/training_json/train.json:

cd ./nuplan-devkit/Align2Act
python cache2json.py \
+caching=cache_llm \
cache.cache_path=/teamspace/studios/this_studio/cache_1M

Training

Following the steps outlined above, you will obtain the train.json file suitable for training LLM.

For guidance on fine-tuning the model, please consult the official documentation.

Evaluation

After training, populate the following parameters in the llm_patches/llm_singleton.py file:

llama_config='/teamspace/studios/this_studio/LLaMA2-Accessory/accessory/configs/model/finetune/sg/llamaPeft_normBiasLora.json'
lora_config='/teamspace/studios/this_studio/LLaMA2-Accessory/accessory/configs/model/finetune/sg/llamaPeft_normBiasLora.json'
tokenizer_path='/teamspace/studios/this_studio/LLaMA2-Accessory/llama-2-7b/tokenizer.model'
pretrained_path='/teamspace/studios/this_studio/LLaMA2-Accessory/llama-2-7b/pretrained_path'

Then, execute the command below to initiate the simulation. You can run various types of simulations by modifying the simulation_type parameter:

export NUPLAN_DATA_ROOT="/teamspace/studios/this_studio/nuplan-devkit/nuplan/dataset"
export NUPLAN_MAPS="/teamspace/studios/this_studio/nuplan-devkit/nuplan/dataset/maps"
export NUPLAN_EXP_ROOT="/teamspace/studios/this_studio/nuplan-devkit/nuplan/exp"     

export PYTHONPATH=$(pwd)/nuplan-devkit:$PYTHONPATH
export PYTHONPATH=$(pwd)/nuplan-devkit/Align2Act:$PYTHONPATH
export PYTHONPATH=$(pwd)/LLaMA2-Accessory:$PYTHONPATH
cd ./nuplan-devkit/Align2Act
simulation_type=open_loop_boxes # closed_loop_nonreactive_agents closed_loop_reactive_agents
sh ./script/benchmarks_test14-hard.sh $simulation_type

Results after Benchmark Tests

test14-hard

open_loop_boxes

{   'aggregator_type': 'weighted_average',
    'log_name': None,
    'num_scenarios': 7.0,
    'planner_expert_average_heading_error_within_bound': 0.9490241969626653,
    'planner_expert_average_l2_error_within_bound': 0.3090423109016055,
    'planner_expert_final_heading_error_within_bound': 0.920009529746468,
    'planner_expert_final_l2_error_within_bound': 0.27812476656557045,
    'planner_miss_rate_within_bound': 0.2857142857142857,
    'planner_name': 'Align2Act',
    'scenario': 'final_score',
    'scenario_type': 'final_score',
    'score': 0.2833165956188029
},

closed_loop_nonreactive_agents

{   'aggregator_type': 'weighted_average',
    'corners_in_drivable_area': None,
    'drivable_area_compliance': 1.0,
    'driving_direction_compliance': 1.0,
    'ego_is_comfortable': 0.5714285714285714,
    'ego_is_making_progress': 0.2857142857142857,
    'ego_jerk': None,
    'ego_lane_change': None,
    'ego_lat_acceleration': None,
    'ego_lon_acceleration': None,
    'ego_lon_jerk': None,
    'ego_progress_along_expert_route': 0.2903926346289234,
    'ego_yaw_acceleration': None,
    'ego_yaw_rate': None,
    'log_name': None,
    'no_ego_at_fault_collisions': 1.0,
    'num_scenarios': 7.0,
    'planner_name': 'Align2Act',
    'scenario': 'final_score',
    'scenario_type': 'final_score',
    'score': 0.26785709132867425,
    'speed_limit_compliance': 0.9971754001104545,
    'time_to_collision_within_bound': 1.0
},

closed_loop_reactive_agents

{   'aggregator_type': 'weighted_average',
    'corners_in_drivable_area': None,
    'drivable_area_compliance': 1.0,
    'driving_direction_compliance': 1.0,
    'ego_is_comfortable': 0.5714285714285714,
    'ego_is_making_progress': 0.2857142857142857,
    'ego_jerk': None,
    'ego_lane_change': None,
    'ego_lat_acceleration': None,
    'ego_lon_acceleration': None,
    'ego_lon_jerk': None,
    'ego_progress_along_expert_route': 0.2903926346289234,
    'ego_yaw_acceleration': None,
    'ego_yaw_rate': None,
    'log_name': None,
    'no_ego_at_fault_collisions': 1.0,
    'num_scenarios': 7.0,
    'planner_name': 'Align2Act',
    'scenario': 'final_score',
    'scenario_type': 'final_score',
    'score': 0.26785709132867425,
    'speed_limit_compliance': 0.9971754001104545,
}

Planner Working

Visualization

Acknowledgement

nuplan-devkit | LLaMA2-Accessory | planTF