Skip to content

Parallel partition exports fail when executed on the same node #1218

@Selfeer

Description

@Selfeer

How to run the tests:

s3/regression.py --clickhouse https://altinity-build-artifacts.s3.amazonaws.com/PRs/1177/76b3bd682e398bb711179650adc8708f13d280fa/build_amd_release/clickhouse-common-static_25.8.12.20667.altinityantalya_amd64.deb --clickhouse-version 25.8.12.20667 --storage minio --only "/s3/minio/part 3/export partition/parallel export partition/*" -l test.log --use-keeper

When running a set of tests where we check:

  1. Parallel ALTER TABLE ... EXPORT PARTITIONs on the same table, exporting different partitions
  2. Second test where we perform EXPORT PARTITIONs and in parallel we do other alters (like ADD COLUMN) on another random table.

We hit an issue where at some point export fails.

One of the fails provided the error in system.replicated_partition_exports

 7 │ FAILED    │ clickhouse1       │ 9            │ Cannot read all data in MergeTreeReaderCompact. Rows read: 9. Rows expected: 87: (while reading column extra): (while reading from part /external2/store/0e4/0e4bef3c-8620-42d0-b027-61fc422ac1e1/9_0_29_6/ in table default.source_a220fe61_d75f_11f0_ac3d_9cb1507b8e87 (0e4bef3c-8620-42d0-b027-61fc422ac1e1) located on disk external2 of type local, from mark 0 with max_rows_to_read = 87, offset = 0): While executing MergeTreeSequentialSource │
 8 │ COMPLETED │                   │ 4            │                  

After the error the table goes to READ ONLY mode and never recovers from that.

This happens both with and without export_merge_tree_partition_lock_inside_the_task enabled.

Here are the ClickHouse logs:

clickhouse_server.log

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions