feat: Reinstate disk-based shuffle

[andygrove]

This PR brings back the original disk-backed shuffle (effectively reverting #19).

There is a limitation that all workers must have the same local disk (so limited to single node) but we can follow up to add a mechanism for either:

• Using minio for shared persistence for shuffle files
• Reading shuffle files from other nodes via Arrow Flight

[andygrove]

I can now run TPC-H @ 100GB again:

Writing results to datafusion-ray-tpch-1731793169620.json
{
    "engine": "datafusion-python",
    "benchmark": "tpch",
    "data_path": "/mnt/bigdata/tpch/sf100",
    "query_path": "/home/andy/git/apache//datafusion-benchmarks/tpch/queries/",
    "1": [
        7.837432622909546
    ],
    "2": [
        4.042541742324829
    ],
    "3": [
        6.542836427688599
    ],
    "4": [
        2.6251494884490967
    ],
    "5": [
        11.944225072860718
    ],
    "6": [
        0.9280209541320801
    ],
    "7": [
        17.949591398239136
    ],
    "8": [
        10.611652612686157
    ],
    "9": [
        37.41403603553772
    ],
    "10": [
        9.629725456237793
    ],
    "11": [
        3.0758209228515625
    ],
    "12": [
        5.291697025299072
    ],
    "13": [
        6.631402015686035
    ],
    "14": [
        1.8772220611572266
    ],
    "15": [
        3.8080291748046875
    ],
    "16": [
        1.6926724910736084
    ],
    "17": [
        28.15924596786499
    ],
    "18": [
        28.37423038482666
    ],
    "19": [
        4.091530084609985
    ],
    "20": [
        5.51667594909668
    ],
    "21": [
        31.07892680168152
    ],
    "22": [
        1.856799602508545
    ]
}