Apache Iceberg version

1.8.0

Query engine

Spark

Please describe the bug 🐞

Currently RewriteTablePaths procedures will only copy file names inside the staging folder. I believe this process is flawed, and one example is Dremio.
When having to deal with delete files, engines might save multiple deletion files with the same name, but a completely different path e.g. /hash1/delete_0_0_0.parquet and /hash2/delete_0_0_0.parquet.
RewriteTablePaths will fail in this case cause it will try to copy both files under the exact same path.

I think one approach for example could be to do something like filePath.replace(table.location(),stagingDir), but I am open to any other suggestions as well. Let me know your opinions!

Thanks!

Willingness to contribute

• I can contribute a fix for this bug independently
• I would be willing to contribute a fix for this bug with guidance from the Iceberg community
• I cannot contribute a fix for this bug at this time