dokku · josegonzalez · Sep 30, 2021 · Sep 29, 2021
diff --git a/docs/advanced-usage/persistent-storage.md b/docs/advanced-usage/persistent-storage.md
@@ -5,10 +5,11 @@
 The preferred method to mount external containers to a Dokku managed container, is to use the Dokku storage plugin.
 
 ```
-storage:list <app>                             # List bind mounts for app's container(s) (host:container)
-storage:mount <app> <host-dir:container-dir>   # Create a new bind mount
-storage:report [<app>] [<flag>]                # Displays a checks report for one or more apps
-storage:unmount <app> <host-dir:container-dir> # Remove an existing bind mount
+storage:ensure-directory [--chown option] <directory>  # Creates a persistent storage directory in the recommended storage path
+storage:list <app>                                     # List bind mounts for app's container(s) (host:container)
+storage:mount <app> <host-dir:container-dir>           # Create a new bind mount
+storage:report [<app>] [<flag>]                        # Displays a checks report for one or more apps
+storage:unmount <app> <host-dir:container-dir>         # Remove an existing bind mount
 ```
 
 > The storage plugin is compatible with storage mounts created with the docker-options. The storage plugin will only list mounts from the deploy/run phase.
@@ -20,50 +21,77 @@ The storage plugin supports the following mount points:
 
 ## Usage
 
-This example demonstrates how to mount the recommended directory to `/storage` inside an application called `node-js-app`. For simplicity, the Dokku project recommends using the directory `/var/lib/dokku/data/storage` directory as the root host path for mounts. This directory is created on Dokku installation.
+### Creating storage directories
+
+> New as of 0.25.5
+
+A storage directory can be created with the `storage:ensure-directory` command. This command will create a subdirectory in the recommended `/var/lib/doku/data/storage` path - created during Dokku installation - and prepare it for use with an app.
 
 ```shell
-# we use a subdirectory inside of the host directory to scope it to just the app
-dokku storage:mount node-js-app /var/lib/dokku/data/storage/node-js-app:/storage
+dokku storage:ensure-directory lollipop
+```
+
+```
+-----> Ensuring /var/lib/dokku/data/storage/lollipop exists
+       Setting directory ownership to 32767:32767
+       Directory ready for mounting
 ```
 
-Dokku will then mount the shared contents of `/var/lib/dokku/data/storage/node-js-app` to `/storage` inside the container. Mounts are only available for containers crated via `run` and by the deploy process, and not during the build process. In addition, the host path is never auto-created by either Dokku or Docker, and should be an explicit path, not one relative to the current working directory.
+By default, permissions are set for usage with Herokuish buildpacks. These permissions can be changed via the `--chown` option according to the following table:
 
-> If the `/storage` path within the container had pre-existing content, the container files will be overrwritten. This may be an issue for users that create assets at build time but then mount a directory at the same place during runtime. Files are not merged.
+- `--chown herokuish` (default): Use `32767:32767` as the folder permissions.
+  - This is used for apps deployed with Buildpacks via Herokuish.
+- `--chown heroku`: Use `1000:1000` as the folder permissions.
+  - This is used for apps deployed with Cloud Native Buildpacks using the `heroku/buildpacks` builder.
+- `--chown packeto`: Use `2000:2000` as the folder permissions.
+  - This is used for apps deployed with Cloud Native Buildpacks using the `cloudfoundry/cnb` or `packeto` builders.
+- `--chown false`: Skips the `chown` call.
 
-Once you have mounted persistent storage, you will also need to restart the application. See the
-[process scaling documentation](/docs/processes/process-management.md) for more information.
+Users deploying via Dockerfile will want to specify `--chown false` and manually `chown` the created directory if the user and/or group id  of the runnning process in the deployed container do not correspond to any of the above options.
+
+> Warning: Failing to set the correct directory ownership may result in issues in persisting files written to the mounted storage directory.
+
+### Mounting storage into apps
+
+Dokku supports mounting both explicit host paths as well as docker volumes via the `storage:mount` command. This takes two arguments, an app name and a `host-path:container-path` or `docker-volume:container-path` combination.
 
 ```shell
-dokku ps:restart app-name
+# mount the directory into your container's /app/storage directory, relative to the container root (/)
+# explicit host paths _must_ exist prior to usage.
+dokku storage:mount node-js-app /var/lib/dokku/data/storage/node-js-app:/app/storage
+
+# mount the docker volume into your container's /app/storage directory, relative to the container root (/)
+# docker volumes _must_ exist prior to usage.
+dokku storage:mount node-js-app some-docker-volume:/app/storage
 ```
 
-A more complete workflow may require making a custom directory for your application and mounting it within your `/app/storage` directory instead. The mount point is *not* relative to your application's working directory, and is instead relative to the root (`/`) of the container.
+In the first example, Dokku will then mount the shared contents of `/var/lib/dokku/data/storage/node-js-app` to `/app/storage` inside the container.  The mount point is *not* relative to your app's working directory, and is instead relative to the root (`/`) of the container. Mounts are only available for containers created via `run` and by the deploy process, and not during the build process. In addition, the host path is never auto-created by either Dokku or Docker, and should be an explicit path, not one relative to the current working directory.
 
-```shell
-# creating storage for the app 'node-js-app'
-mkdir -p  /var/lib/dokku/data/storage/node-js-app
+> If the `/storage` path within the container had pre-existing content, the container files will be over-written. This may be an issue for users that create assets at build time but then mount a directory at the same place during runtime. Files are not merged.
 
-# set the directory ownership. Below is an example for herokuish
-# but see the `Directory Permissions` section for more details
-chown -R 32767:32767 /var/lib/dokku/data/storage/node-js-app
+Once persistent storage is mounted, the app requires a restart. See the [process scaling documentation](/docs/processes/process-management.md) for more information.
 
-# mount the directory into your container's /app/storage directory, relative to root
-dokku storage:mount app-name /var/lib/dokku/data/storage/node-js-app:/app/storage
+```shell
+dokku ps:restart app-name
 ```
 
-You can mount one or more directories as desired by following the above pattern.
+### Unmounting storage
+
+If an app no longer requires a mounted volume or directory, the `storage:unmount` command can be called. This takes the same arguments as the `storage:mount` command, an app name and a `host-path:container-path` or `docker-volume:container-path` combination.
+
+```shell
+# unmount the directory from your container's /app/storage directory, relative to the container root (/)
+dokku storage:unmount node-js-app /var/lib/dokku/data/storage/node-js-app:/app/storage
 
-### Directory Permissions
+# unmount the docker volume from your container's /app/storage directory, relative to the container root (/)
+dokku storage:unmount node-js-app some-docker-volume:/app/storage
+```
 
-The host directory should always be owned by the container user and group id. If this is not the case, files may not persist when written to mounted storage.
+Once persistent storage is unmounted, the app requires a restart. See the [process scaling documentation](/docs/processes/process-management.md) for more information.
 
-- Buildpacks via Herokuish: Use `32767:32767` as the file permissions
-- For Cloud Native Buildpacks: This will depend on the builder in question, but can be retrieved via the `CNB_USER_ID` and `CNB_GROUP_ID` environment variables on the builder image. Common builders are as follows:
-  - heroku/buildpacks: `1000:1000`
-  - cloudfoundry/cnb: `2000:2000`
-  - packeto: `2000:2000`
-- Dockerfile and Docker Image: Use the user and group id which corresponds to the one running the process within the container.
+```shell
+dokku ps:restart app-name
+```
 
 ### Displaying storage reports for an app
 
@@ -113,7 +141,7 @@ dokku storage:report node-js-app --storage-deploy-mounts
 
 ### Sharing storage across deploys
 
-Dokku is powered by Docker containers, which recommends in their [best practices](https://docs.docker.com/engine/userguide/eng-image/dockerfile_best-practices/#containers-should-be-ephemeral) that containers be treated as ephemeral. In order to manage persistent storage for web applications, like user uploads or large binary assets like images, a directory outside the container should be mounted.
+Dokku is powered by Docker containers, which recommends in their [best practices](https://docs.docker.com/engine/userguide/eng-image/dockerfile_best-practices/#containers-should-be-ephemeral) that containers be treated as ephemeral. In order to manage persistent storage for web apps, like user uploads or large binary assets like images, a directory outside the container should be mounted.
 
 ### Shared storage between containers
 
@@ -139,11 +167,11 @@ You cannot use mounted volumes during the build phase of a Dockerfile deploy. Th
 
 > Note: **This can cause data loss** if you bind a mount under `/app` in buildpack apps as herokuish will attempt to remove the original app path during the build phase.
 
-## Application User and Persistent Storage file ownership (buildpack apps only)
+## App User and Persistent Storage file ownership (buildpack apps only)
 
 > New as of 0.7.1
 
-By default, Dokku will execute your buildpack application processes as the `herokuishuser` user. You may override this by setting the `DOKKU_APP_USER` config variable.
+By default, Dokku will execute your buildpack app processes as the `herokuishuser` user. You may override this by setting the `DOKKU_APP_USER` config variable.
 
 > NOTE: this user must exist in your herokuish image.
 

diff --git a/plugins/storage/bin/chown-storage-dir b/plugins/storage/bin/chown-storage-dir
@@ -0,0 +1,27 @@
+#!/usr/bin/env bash
+set -eo pipefail
+[[ $DOKKU_TRACE ]] && set -x
+
+main() {
+  declare desc="chowns a storage directory"
+  declare DIRECTORY="$1" CHOWN_ID="$2"
+
+  if [[ -z "$DIRECTORY" ]]; then
+    echo " !     Please specify a directory to create" 1>&2
+    exit 1
+  fi
+
+  if [[ ! "$DIRECTORY" =~ ^[A-Za-z0-9\\_-]+$ ]]; then
+    echo " !     Directory can only contain the following set of characters: [A-Za-z0-9_-]" 1>&2
+    exit 1
+  fi
+
+  if [[ "$CHOWN_ID" != "32767" ]] && [[ "$CHOWN_ID" != "1000" ]] && [[ "$CHOWN_ID" != "2000" ]]; then
+    echo " !     Unsupported chown permissions. Supported values: 32767, 1000, 2000"
+    exit 1
+  fi
+
+  chown -R "$CHOWN_ID:$CHOWN_ID" "${DOKKU_LIB_ROOT}/data/storage/$DIRECTORY"
+}
+
+main "$@"
diff --git a/plugins/storage/help-functions b/plugins/storage/help-functions
@@ -27,6 +27,7 @@ help_desc
 fn-help-content() {
   declare desc="return help content"
   cat <<help_content
+    storage:ensure-directory [--chown option] <directory>, Creates a persistent storage directory in the recommended storage path
     storage:list <app>, List bind mounts for app's container(s) (host:container)
     storage:mount <app> <host-dir:container-dir>, Create a new bind mount
     storage:report [<app>] [<flag>], Displays a checks report for one or more apps

diff --git a/plugins/storage/install b/plugins/storage/install
@@ -5,9 +5,40 @@ set -eo pipefail
 trigger-storage-install() {
   declare desc="storage install trigger"
   declare trigger="install"
+  local storage_directory="${DOKKU_LIB_ROOT}/data/storage"
+  mkdir -p "${storage_directory}"
+  chown dokku:dokku "${storage_directory}"
 
-  mkdir -p "${DOKKU_LIB_ROOT}/data/storage"
-  chown dokku:dokku "${DOKKU_LIB_ROOT}/data/storage"
+  STORAGE_SUDOERS_FILE="/etc/sudoers.d/dokku-storage"
+
+  case "$DOKKU_DISTRO" in
+    debian)
+      echo "%dokku ALL=(ALL) NOPASSWD:$PLUGIN_AVAILABLE_PATH/storage/bin/chown-storage-dir *" >"$STORAGE_SUDOERS_FILE"
+      echo "Defaults env_keep += \"DOKKU_LIB_ROOT\"" >>"$STORAGE_SUDOERS_FILE"
+      ;;
+
+    ubuntu)
+      echo "%dokku ALL=(ALL) NOPASSWD:$PLUGIN_AVAILABLE_PATH/storage/bin/chown-storage-dir *" >"$STORAGE_SUDOERS_FILE"
+      echo "Defaults env_keep += \"DOKKU_LIB_ROOT\"" >>"$STORAGE_SUDOERS_FILE"
+      ;;
+
+    opensuse)
+      echo "%dokku ALL=(ALL) NOPASSWD:$PLUGIN_AVAILABLE_PATH/storage/bin/chown-storage-dir *" >"$STORAGE_SUDOERS_FILE"
+      echo "Defaults env_keep += \"DOKKU_LIB_ROOT\"" >>"$STORAGE_SUDOERS_FILE"
+      ;;
+
+    arch)
+      echo "%dokku ALL=(ALL) NOPASSWD:$PLUGIN_AVAILABLE_PATH/storage/bin/chown-storage-dir *" >"$STORAGE_SUDOERS_FILE"
+      echo "Defaults env_keep += \"DOKKU_LIB_ROOT\"" >>"$STORAGE_SUDOERS_FILE"
+      ;;
+
+    centos | fedora | rhel)
+      echo "%dokku ALL=(ALL) NOPASSWD:$PLUGIN_AVAILABLE_PATH/storage/bin/chown-storage-dir *" >"$STORAGE_SUDOERS_FILE"
+      echo "Defaults:dokku !requiretty" >>"$STORAGE_SUDOERS_FILE"
+      echo "Defaults env_keep += \"DOKKU_LIB_ROOT\"" >>"$STORAGE_SUDOERS_FILE"
+
+      ;;
+  esac
 }
 
 trigger-storage-install "$@"
diff --git a/plugins/storage/internal-functions b/plugins/storage/internal-functions
@@ -5,6 +5,62 @@ source "$PLUGIN_AVAILABLE_PATH/storage/functions"
 set -eo pipefail
 [[ $DOKKU_TRACE ]] && set -x
 
+cmd-storage-ensure-directory() {
+  declare desc="creates a persistent storage directory in the recommended storage path"
+  declare cmd="storage:ensure-directory"
+  [[ "$1" == "$cmd" ]] && shift 1
+  declare DIRECTORY CHOWN_FLAG
+
+  CHOWN_FLAG=herokuish
+  skip=false
+  for arg in "$@"; do
+    if [[ "$arg" == "--chown" ]]; then
+      skip=true
+      continue
+    fi
+
+    if [[ "$skip" == "true" ]]; then
+      CHOWN_FLAG="$arg"
+      skip=false
+      continue
+    fi
+
+    if [[ -z "$DIRECTORY" ]]; then
+      DIRECTORY="$arg"
+    fi
+  done
+
+  if [[ -z "$DIRECTORY" ]]; then
+    dokku_log_fail "Please specify a directory to create"
+  fi
+
+  if [[ ! "$DIRECTORY" =~ ^[A-Za-z0-9\\_-]+$ ]]; then
+    dokku_log_fail "Directory can only contain the following set of characters: [A-Za-z0-9_-]"
+  fi
+
+  if [[ "$CHOWN_FLAG" == "herokuish" ]]; then
+    CHOWN_FLAG="32767"
+  elif [[ "$CHOWN_FLAG" == "heroku" ]]; then
+    CHOWN_FLAG="1000"
+  elif [[ "$CHOWN_FLAG" == "packeto" ]]; then
+    CHOWN_FLAG="2000"
+  elif [[ "$CHOWN_FLAG" == "false" ]]; then
+    CHOWN_FLAG="false"
+  else
+    dokku_log_fail "Unsupported chown permissions"
+  fi
+
+  local storage_directory="${DOKKU_LIB_ROOT}/data/storage/$DIRECTORY"
+  dokku_log_info1 "Ensuring ${storage_directory} exists"
+  mkdir -p "${storage_directory}"
+  if [[ "$CHOWN_FLAG" != "false" ]]; then
+    dokku_log_verbose_quiet "Setting directory ownership to $CHOWN_FLAG:$CHOWN_FLAG"
+    sudo "$PLUGIN_AVAILABLE_PATH/storage/bin/chown-storage-dir" "$DIRECTORY" "$CHOWN_FLAG"
+  fi
+
+  dokku_log_verbose_quiet "Directory ready for mounting"
+}
+
 cmd-storage-report() {
   declare desc="displays a storage report for one or more apps"
   declare cmd="storage:report"

diff --git a/plugins/storage/subcommands/ensure-directory b/plugins/storage/subcommands/ensure-directory
@@ -0,0 +1,6 @@
+#!/usr/bin/env bash
+set -eo pipefail
+[[ $DOKKU_TRACE ]] && set -x
+source "$PLUGIN_AVAILABLE_PATH/storage/internal-functions"
+
+cmd-storage-ensure-directory "$@"
diff --git a/tests/unit/storage.bats b/tests/unit/storage.bats
@@ -5,6 +5,7 @@ load test_helper
 setup() {
   global_setup
   create_app
+  rm -rf "$DOKKU_LIB_ROOT/data/storage/rdmtestapp*"
 }
 
 teardown() {
@@ -26,6 +27,86 @@ teardown() {
   assert_output "$help_output"
 }
 
+@test "(storage) storage:ensure-directory" {
+  run /bin/bash -c "test -d $DOKKU_LIB_ROOT/data/storage/$TEST_APP"
+  echo "output: $output"
+  echo "status: $status"
+  assert_failure
+
+  run /bin/bash -c "dokku storage:ensure-directory @"
+  echo "output: $output"
+  echo "status: $status"
+  assert_failure
+
+  run /bin/bash -c "dokku storage:ensure-directory $TEST_APP/"
+  echo "output: $output"
+  echo "status: $status"
+  assert_failure
+
+  run /bin/bash -c "dokku storage:ensure-directory $TEST_APP/$TEST_APP"
+  echo "output: $output"
+  echo "status: $status"
+  assert_failure
+
+  run /bin/bash -c "dokku storage:ensure-directory $TEST_APP"
+  echo "output: $output"
+  echo "status: $status"
+  assert_success
+  assert_output_contains "Setting directory ownership to 1000:1000" 0
+  assert_output_contains "Setting directory ownership to 2000:2000" 0
+  assert_output_contains "Setting directory ownership to 32767:32767" 1
+
+  run /bin/bash -c "test -d $DOKKU_LIB_ROOT/data/storage/$TEST_APP"
+  echo "output: $output"
+  echo "status: $status"
+  assert_success
+
+  run /bin/bash -c "dokku storage:ensure-directory $TEST_APP"
+  echo "output: $output"
+  echo "status: $status"
+  assert_success
+
+  run /bin/bash -c "dokku storage:ensure-directory --chown false $TEST_APP"
+  echo "output: $output"
+  echo "status: $status"
+  assert_success
+  assert_output_contains "Setting directory ownership to 1000:1000" 0
+  assert_output_contains "Setting directory ownership to 2000:2000" 0
+  assert_output_contains "Setting directory ownership to 32767:32767" 0
+
+  run /bin/bash -c "dokku storage:ensure-directory $TEST_APP --chown false"
+  echo "output: $output"
+  echo "status: $status"
+  assert_success
+  assert_output_contains "Setting directory ownership to 1000:1000" 0
+  assert_output_contains "Setting directory ownership to 2000:2000" 0
+  assert_output_contains "Setting directory ownership to 32767:32767" 0
+
+  run /bin/bash -c "dokku storage:ensure-directory --chown heroku $TEST_APP"
+  echo "output: $output"
+  echo "status: $status"
+  assert_success
+  assert_output_contains "Setting directory ownership to 1000:1000" 1
+  assert_output_contains "Setting directory ownership to 2000:2000" 0
+  assert_output_contains "Setting directory ownership to 32767:32767" 0
+
+  run /bin/bash -c "dokku storage:ensure-directory --chown packeto $TEST_APP"
+  echo "output: $output"
+  echo "status: $status"
+  assert_success
+  assert_output_contains "Setting directory ownership to 1000:1000" 0
+  assert_output_contains "Setting directory ownership to 2000:2000" 1
+  assert_output_contains "Setting directory ownership to 32767:32767" 0
+
+  run /bin/bash -c "dokku storage:ensure-directory --chown herokuish $TEST_APP"
+  echo "output: $output"
+  echo "status: $status"
+  assert_success
+  assert_output_contains "Setting directory ownership to 1000:1000" 0
+  assert_output_contains "Setting directory ownership to 2000:2000" 0
+  assert_output_contains "Setting directory ownership to 32767:32767" 1
+}
+
 @test "(storage) storage:mount, storage:list, storage:umount" {
   run /bin/bash -c "dokku storage:mount $TEST_APP /tmp/mount:/mount"
   echo "output: $output"