Issue: Inference Script's `--data` Argument Should Support Dynamic Input Lists

[curryqka]

Description

The current inference script (inference/llava1.5.py) uses a fixed JSON file path for the --data argument. However, this argument should ideally support dynamic input lists, such as a collection of paths that include both normal sensor inputs and special cases like rain, fog, or snow conditions.

Current Behavior

The script is hard-coded to use a single fixed JSON file path for --data:

--data data/DriveBench20K/drivebench-test-final3.json

This limits flexibility, especially when handling multiple input scenarios (e.g., normal conditions vs. adverse weather conditions).

Expected Behavior

The --data argument should accept a list of paths or a flexible input format that allows specifying multiple datasets, such as:

--data "data/normal.json,data/rain.json,data/snow.json"

Here is my possible improments but I can not ensure its correction:

Suggested Improvements

1. Modify the --data Argument: Allow it to accept a list of paths or a JSON file containing multiple dataset paths.
2. Update the Script Logic: Ensure the script can iterate over multiple datasets dynamically.
3. Support Special Cases: Explicitly handle different input scenarios (e.g., rain, fog, snow) by iterating over the provided paths.

Example Implementation

Here’s a potential modification to the script:

# Loop over the arrays and run the commands.
for i in "${!outputs[@]}"; do
    python inference/llava1.5.py \
        --model /high_perf_store/mlinfra-vepfs/wangjinghui/drive-bench/LLM_checkpoint/llava_1.5_7b_hf \
        --data "${data_paths[i]}" \  # Accepts a list of paths
        --output "res/llava-1.5-7b/${outputs[i]}" \
        --system_prompt prompt.txt \
        --num_processes "${GPU}" \
        --max_model_len 2048 \
        --corruption "${corruptions[i]}"
done

Additional Notes

This change would make the inference script more flexible and better suited for handling diverse input scenarios？？

[ldkong1205]

Hi @Daniel-xsy, please take a look at this issue, thanks!