fix: add configurable bar for name truncation for partial files #8641
Conversation
|
Rclone adds a hash of the file fingerprint to create a unique suffix. I guess if you have a lot of files with the same size/modtime/hash this could cause a problem. Perhaps rclone should include the filename in the hash too? That might be an easier fix. |
Yeah, our usecase is tens of terrabytes of data spread across tens to hundreds of millions of files where checksumming is a bottleneck, so we have that disabled. We also have very long and similar filenames I'd rather have the thing fail due to exceeding PATH_MAX instead of writing to the same partial file concurrently |
In the fingerprint, yes. I'll sub a PR for that |
|
closing in favor of #8643 |
What is the purpose of this change?
Currently, rclone truncates paths over a hundred characters to fit a partial suffix. Sometimes, this comes around and bites you when it turns out the ending, say 10 characters, are the only thing distinguishing one file being transferred from another. Thus, I have added this flag to make this limit not hardcoded.
Checklist