XFS Quota Enforcement Issue: Only Applied at Volume Root, Not Inside _data for Second and Subsequent Podman Volumes

### Issue Description

When using the local volume driver with the `--opt o=size=10M` option on an XFS-backed storage (with project quotas enabled), Podman correctly creates the volume and assigns an XFS project quota. However, during container startup, Podman erroneously passes the “size” option as a mount parameter to the XFS mount command. Since XFS does not recognize a “size” mount option, the container fails to start with an error:

`mount: /mnt/data/containers/storage/volumes/testVolume/_data: fsconfig system call failed: xfs: Unknown parameter 'size'`


### Steps to reproduce the issue


1. **Prepare an XFS Partition with Project Quotas:**  
   - Format the device (e.g., `/dev/sdb1`) as XFS and mount it at `/mnt/data` with `prjquota` enabled.  
   -  `/etc/fstab` entry:  
       ```
      /dev/sdb1 /mnt/data/  xfs defaults,x-systemd.device-timeout=0,pquota 1 2
       ```
   - Mount the partition using:
     ```bash
     mount -a
     ```
2. **Configure Podman to Use the XFS Partition:**  
   - Edit `/etc/containers/storage.conf` to set graphroot:  
     ```toml
     [storage]
     driver = "overlay"
     graphroot = "/mnt/data/containers/storage"
     ```
   - Restart Podman if necessary.
3. **Create a Volume with a Size Option:**  
   - Create a volume with:
     ```bash
     podman volume create --driver local --opt o=size=10M testVolume
     ```
   - Verify via:
     ```bash
     xfs_quota -x -c "report -p" /mnt/data
     ```
     that a project quota is set (e.g., a hard limit of ~10MB).
4. **Attempt to Run a Container Using the Volume:**  
   - Start a container with:
     ```bash
     podman run --rm -v testVolume:/data busybox sh -c "dd if=/dev/zero of=/data/testfile bs=1M count=15"
     ```
   - The container fails with an error:
     ```
     mount: /mnt/data/containers/storage/volumes/testVolume/_data: fsconfig system call failed: xfs: Unknown parameter 'size'
     ```


### Describe the results you received

### **Actual Behavior:**
- When a volume is created with `--opt o=size=10M`, Podman sets the project quota as expected.
- However, at container startup, Podman issues a mount command that includes `-o size=10M`, which is rejected by the XFS mount system call, causing the container to fail to start.

---

### **Debug Logs and Analysis:**
- **Debug Log Snippet: podman volume create --opt device=/dev/sdb1 --opt type=xfs --opt o=size=10M testvol4**
```
[root@fedora abhi]# podman --log-level=debug volume create --opt device=/dev/sdb1 --opt type=xfs -o=o=size=10M testvol4
INFO[0000] podman filtering at log level debug
DEBU[0000] Called create.PersistentPreRunE(podman --log-level=debug volume create --opt device=/dev/sdb1 --opt type=xfs -o=o=size=10M testvol4)
DEBU[0000] Using conmon: "/usr/bin/conmon"
INFO[0000] Using sqlite as database backend
DEBU[0000] Using graph driver overlay
DEBU[0000] Using graph root /mnt/data/containers/storage
DEBU[0000] Using run root /run/containers/storage
DEBU[0000] Using static dir /mnt/data/containers/storage/libpod
DEBU[0000] Using tmp dir /run/libpod
DEBU[0000] Using volume path /mnt/data/containers/storage/volumes
DEBU[0000] Using transient store: false
DEBU[0000] [graphdriver] trying provided driver "overlay"
DEBU[0000] overlay: imagestore=/usr/lib/containers/storage
DEBU[0000] Cached value indicated that overlay is supported
DEBU[0000] Cached value indicated that overlay is supported
DEBU[0000] Cached value indicated that metacopy is being used
DEBU[0000] NewControl(/mnt/data/containers/storage/overlay): nextProjectID = 100001
DEBU[0000] Cached value indicated that native-diff is not being used
INFO[0000] Not using native diff for overlay, this may cause degraded performance for building images: kernel has CONFIG_OVERLAY_FS_REDIRECT_DIR enabled
DEBU[0000] backingFs=xfs, projectQuotaSupported=true, useNativeDiff=false, usingMetacopy=true
DEBU[0000] Initializing event backend journald
DEBU[0000] Configured OCI runtime runsc initialization failed: no valid executable found for OCI runtime runsc: invalid argument
DEBU[0000] Configured OCI runtime krun initialization failed: no valid executable found for OCI runtime krun: invalid argument
DEBU[0000] Configured OCI runtime ocijail initialization failed: no valid executable found for OCI runtime ocijail: invalid argument
DEBU[0000] Configured OCI runtime crun-vm initialization failed: no valid executable found for OCI runtime crun-vm: invalid argument
DEBU[0000] Configured OCI runtime crun-wasm initialization failed: no valid executable found for OCI runtime crun-wasm: invalid argument
DEBU[0000] Configured OCI runtime runc initialization failed: no valid executable found for OCI runtime runc: invalid argument
DEBU[0000] Configured OCI runtime runj initialization failed: no valid executable found for OCI runtime runj: invalid argument
DEBU[0000] Configured OCI runtime kata initialization failed: no valid executable found for OCI runtime kata: invalid argument
DEBU[0000] Configured OCI runtime youki initialization failed: no valid executable found for OCI runtime youki: invalid argument
DEBU[0000] Using OCI runtime "/usr/bin/crun"
INFO[0000] Setting parallel job count to 13
DEBU[0000] Validating options for local driver
DEBU[0000] NewControl(/mnt/data/containers/storage/volumes): nextProjectID = 200003
DEBU[0000] Setting quota project ID 200003 on /mnt/data/containers/storage/volumes/testvol4
DEBU[0000] SetQuota path=/mnt/data/containers/storage/volumes/testvol4, size=10000000, inodes=0, projectID=200003
testvol4
DEBU[0000] Called create.PersistentPostRunE(podman --log-level=debug volume create --opt device=/dev/sdb1 --opt type=xfs -o=o=size=10M testvol4)
DEBU[0000] Shutting down engines
INFO[0000] Received shutdown.Stop(), terminating!        PID=87400
```
- **Volume Inspect:**
```
[root@fedora abhi]# podman volume inspect testvol4
[
     {
          "Name": "testvol4",
          "Driver": "local",
          "Mountpoint": "/mnt/data/containers/storage/volumes/testvol4/_data",
          "CreatedAt": "2025-02-20T01:26:43.831051058+05:30",
          "Labels": {},
          "Scope": "local",
          "Options": {
               "SIZE": "10M",
               "device": "/dev/sdb1",
               "o": "size=10M",
               "type": "xfs"
          },
          "MountCount": 0,
          "NeedsCopyUp": true,
          "NeedsChown": true,
          "LockNumber": 3
     }
]

```
- **Debug Log Snippet: podman run**
```
  DEBU[0000] Running mount command: /usr/bin/mount -o size=10M -t xfs /dev/sdb1 /mnt/data/containers/storage/volumes/testvol4/_data
  DEBU[0000] Mount command failed with exit status 32
...
...
...
DEBU[0000] ExitCode msg: "mounting volume testvol4 for container a44a8a561b8148341e6db6aef76f1a25ffcbb620ba7d628138ad6efb1e155f58: mount: /mnt/data/containers/storage/volumes/testvol4/_data: fsconfig system call failed: xfs: unknown parameter 'size'.\n       dmesg(1) may have more information after failed mount system call.\n"
Error: mounting volume testvol4 for container a44a8a561b8148341e6db6aef76f1a25ffcbb620ba7d628138ad6efb1e155f58: mount: /mnt/data/containers/storage/volumes/testvol4/_data: fsconfig system call failed: xfs: Unknown parameter 'size'.
       dmesg(1) may have more information after failed mount system call.

```

- **Analysis:**
  - Although Podman’s volume creation process correctly sets up XFS project quotas (as confirmed by `xfs_quota`), it later passes the “size=10M” mount option when mounting the volume inside the container.
  - XFS does not recognize any mount option named “size”; project quotas are managed via the quota system, not as a mount parameter.
  - As a workaround, if the volume is created without the `--opt o=size=10M` option, Podman mounts the volume successfully, and quotas can be manually enforced using XFS tools.

### Describe the results you expected

### **Expected Behavior:**
- Podman should use the `--opt o=size=10M` parameter to set the XFS project quota on the volume (which it does) but **should not pass** a “size=10M” option to the mount system call when starting a container.
- The container should mount the volume normally, and the XFS quota should limit writes to ~10MB without causing a mount error.

---

### podman info output

```yaml
host:
  arch: amd64
  buildahVersion: 1.38.1
  cgroupControllers:
  - cpuset
  - cpu
  - io
  - memory
  - hugetlb
  - pids
  - rdma
  - misc
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon-2.1.12-3.fc41.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.1.12, commit: '
  cpuUtilization:
    idlePercent: 97.8
    systemPercent: 1.88
    userPercent: 0.32
  cpus: 4
  databaseBackend: sqlite
  distribution:
    distribution: fedora
    variant: workstation
    version: "41"
  eventLogger: journald
  freeLocks: 2044
  hostname: fedora
  idMappings:
    gidmap: null
    uidmap: null
  kernel: 6.12.11-200.fc41.x86_64
  linkmode: dynamic
  logDriver: journald
  memFree: 8684425216
  memTotal: 16758345728
  networkBackend: netavark
  networkBackendInfo:
    backend: netavark
    dns:
      package: aardvark-dns-1.13.1-1.fc41.x86_64
      path: /usr/libexec/podman/aardvark-dns
      version: aardvark-dns 1.13.1
    package: netavark-1.13.1-1.fc41.x86_64
    path: /usr/libexec/podman/netavark
    version: netavark 1.13.1
  ociRuntime:
    name: crun
    package: crun-1.19.1-1.fc41.x86_64
    path: /usr/bin/crun
    version: |-
      crun version 1.19.1
      commit: 3e32a70c93f5aa5fea69b50256cca7fd4aa23c80
      rundir: /run/crun
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +LIBKRUN +WASM:wasmedge +YAJL
  os: linux
  pasta:
    executable: /usr/bin/pasta
    package: passt-0^20250121.g4f2c8e7-2.fc41.x86_64
    version: |
      pasta 0^20250121.g4f2c8e7-2.fc41.x86_64
      Copyright Red Hat
      GNU General Public License, version 2 or later
        <https://www.gnu.org/licenses/old-licenses/gpl-2.0.html>
      This is free software: you are free to change and redistribute it.
      There is NO WARRANTY, to the extent permitted by law.
  remoteSocket:
    exists: true
    path: /run/podman/podman.sock
  rootlessNetworkCmd: pasta
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: false
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: true
  serviceIsRemote: false
  slirp4netns:
    executable: ""
    package: ""
    version: ""
  swapFree: 8589930496
  swapTotal: 8589930496
  uptime: 60h 45m 7.00s (Approximately 2.50 days)
  variant: ""
plugins:
  authorization: null
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  - ipvlan
  volume:
  - local
registries:
  search:
  - registry.fedoraproject.org
  - registry.access.redhat.com
  - docker.io
store:
  configFile: /usr/share/containers/storage.conf
  containerStore:
    number: 0
    paused: 0
    running: 0
    stopped: 0
  graphDriverName: overlay
  graphOptions:
    overlay.imagestore: /usr/lib/containers/storage
    overlay.mountopt: nodev,metacopy=on
  graphRoot: /mnt/data/containers/storage
  graphRootAllocated: 21406679040
  graphRootUsed: 464166912
  graphStatus:
    Backing Filesystem: xfs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Supports shifting: "true"
    Supports volatile: "true"
    Using metacopy: "true"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 1
  runRoot: /run/containers/storage
  transientStore: false
  volumePath: /mnt/data/containers/storage/volumes
version:
  APIVersion: 5.3.2
  Built: 1737504000
  BuiltTime: Wed Jan 22 05:30:00 2025
  GitCommit: ""
  GoVersion: go1.23.4
  Os: linux
  OsArch: linux/amd64
  Version: 5.3.2
```

### Podman in a container

No

### Privileged Or Rootless

Privileged

### Upstream Latest Release

Yes

### Additional environment details



**Environment:**
- **Podman Version:** 5.3.2
- **Operating System:** Fedora 41
- **Filesystem:** XFS with project quotas enabled (mounted with `prjquota`)
- **Podman Storage Configuration:**  
  - `graphroot = "/mnt/data/containers/storage"`



### Additional information


### **Impact:**
- This issue prevents the use of the `--opt o=size=10M feature for local volumes on XFS-backed storage in Podman, limiting the ability to enforce per-volume storage limits automatically.
- The only workaround is to create the volume without the size option and manually apply XFS quotas after volume creation, which is not ideal for automated or production environments.

---

### **Additional Information:**
- The issue appears to be isolated to the way Podman translates the `--opt o=size=10M` option into mount options for XFS.  
- Similar behavior is observed even though the XFS project quotas are properly applied at volume creation.
- Debug logs indicate that the unsupported mount option “size=10M” is passed during the container’s volume mount process.


Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

XFS Quota Enforcement Issue: Only Applied at Volume Root, Not Inside _data for Second and Subsequent Podman Volumes #25368

Issue Description

Steps to reproduce the issue

Describe the results you received

Actual Behavior:

Debug Logs and Analysis:

Describe the results you expected

Expected Behavior:

podman info output

Podman in a container

Privileged Or Rootless

Upstream Latest Release

Additional environment details

Additional information

Impact:

Additional Information:

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

XFS Quota Enforcement Issue: Only Applied at Volume Root, Not Inside _data for Second and Subsequent Podman Volumes #25368

Description

Issue Description

Steps to reproduce the issue

Describe the results you received

Actual Behavior:

Debug Logs and Analysis:

Describe the results you expected

Expected Behavior:

podman info output

Podman in a container

Privileged Or Rootless

Upstream Latest Release

Additional environment details

Additional information

Impact:

Additional Information:

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions