Simple Metadata Backup

Overview

Each command that mutates the DuckLake metadata file should automatically create a backup before making changes. This provides a safety net for quick recovery from accidental data loss or corruption.

Implementation

Backup Creation:

• Before any metadata mutation operation, create a timestamped backup
• Backup filename format: tailpipe_ducklake.db.backup.YYYYMMDDHHMMSS
• Location: Same directory as the main metadata file (~/.tailpipe/data/<profile>/)

Backup Management:

• Keep only 1 previous backup (delete older backups)
• Backup is created before the mutation operation begins
• If the mutation fails, the backup remains available for restoration

Commands that trigger backup:

• collect (when creating new tables or inserting data)
• compact (when reorganizing data)
• partition delete
• Any other command that modifies the DuckLake database

Backup Restoration:

• Manual process: rename backup file to tailpipe_ducklake.db

Expected Outcome

• Always have a recent backup of metadata before any changes
• Minimal storage overhead (only 1 backup kept)
• Transparent to users (automatic backup creation)