vine: file type inconsistency

[JinZhou5042]

The cache-update message carries a type field for a newly created replica on a worker, but this type does not correspond to the actual type.

Here is the output by printing the cachename and the corresponding type for a replica inside the handle_cache_update function:

cachename: temp-rnd-bxhrzirbuxqwrnm, replica->type: 2
cachename: temp-rnd-kgjhquhucwtfqbj, replica->type: 2
cachename: temp-rnd-csfqdcgsxmygocc, replica->type: 0
cachename: temp-rnd-gwtoipaalahcems, replica->type: 2
cachename: temp-rnd-obatetkslhwcfua, replica->type: 2
cachename: temp-rnd-sfdwikyhjrflykp, replica->type: 2
cachename: temp-rnd-vajbdzycrlbcabe, replica->type: 2
cachename: temp-rnd-kfpwfsfbefddlid, replica->type: 0
cachename: temp-rnd-whwyvueptntmiqp, replica->type: 0
cachename: temp-rnd-ohvbftitvecdbzg, replica->type: 0
cachename: file-meta-85311becb4178d43633d12039060a688, replica->type: 0
cachename: temp-rnd-gnqkxecilnucdpz, replica->type: 2
cachename: file-meta-0e27e14c9c1e49ecf3072c8b6a67bd33, replica->type: 0
cachename: temp-rnd-wobacbfpdxskpmx, replica->type: 2
cachename: temp-rnd-nwprcqrnycqylxu, replica->type: 2
cachename: temp-rnd-tlxarqdoqpnxpgy, replica->type: 0
cachename: temp-rnd-ffgosrpohtcjtbc, replica->type: 2

There exists two problems:

1. The type value 0 doesn't correspond to any valid entry, since the manager's vine_file_type_t enumurator starts at 1;
2. TEMP files have inconsistent type values, some are 0 while others are 2 (If 0 were meant to represent VINE_FILE then 2 would correspond to VINE_TEMP but because the starting value is incorrect, both mappings are invalid).

This may cause problems when distinguishing between VINE_TEMP and VINE_FILE replicas. For example, we may need to identify and remove TEMP data if a worker accumulates too much and risks disk exhaustion.

[dthain]

Indeed, this problem is noted here in vine_cache:
https://github.com/cooperative-computing-lab/cctools/blob/master/taskvine/src/worker/vine_cache.c#L348

The underlying issue is that puturl is used to tell a worker to fetch a remote file at worker://addr/path. The file is then given a type of VINE_URL (2) because it was fetched by a url, which is not the right thing.

The puturl message should tell the worker the type of the original file. (And perhaps some other metadata as well?)

[btovar]

Given the design choice that the manager makes all the decisions, I am still not sure why the worker needs to know the type. And even so, the cache-update message message will be received by the manager which already knows the type. Changing the protocol here makes me think that some design invariant is being broken.

[dthain]

Good point, the manager will definitely know the type if it was the one that put it there.

But, if the worker is doing a scan of its contents after a reset, then the manager doesn't know anything, hence the extra detail in the cache-update messages.

Maybe it's worth asking whether the information is needed at all on the worker?

[btovar]

Then I think we need another type, like VINE_FILE_WORKER?

[dthain]

There may be some advantage in distinguishing between the operations fetch-from-another-worker and fetch-from-origin-url.

But the point is that the file fetched from a peer worker should be indistinguishable from the source copy when at rest.

If the original file was a FILE, then the peer-transferred copy should still be a FILE and not a URL, right?

[JinZhou5042]

Would it be worthwhile to distinguish between file types before and after materialization? For example, VINE_URL, VINE_BUFFER, VINE_MINI_TASK don't represent physcially existing files, but rather instructions that tell the worker how to obtain or generate the file. For the real file types, it looks only VINE_FILE and VINE_TEMP truly correspond to materialized and physical files that exist?

[btovar]

I think that's why the current implantation makes sense to me. The worker may want to know if it is a file or a url, but no more than that. After all, while the file structs at manager and worker refer to the same file information, they are used differently and we should not expect to match each other. The type at the manager tells us about the nature of the file, the type at the worker tells us about how to get it (or how it came to be). If we communicate the type from the worker in the cache-update message then this does not change the nature of the file from the manager perspective, it would only inform the manager how the file got there.

[dthain]

So then:

• Does it matter if a (fresh) manager receives an (incorrect) file type from the worker when it starts up and scans the cache?
• Does the worker use the file type in a meaningful way?

If the answer to either of those is "yes", then the manager needs to convey the correct file type when it uses puturl to perform a worker-to-worker transfer.

If the answer to both is "no", then does the worker need the file type at all?