这是indexloc提供的服务,不要输入任何密码
Skip to content

Running devpod on kubernetes provider in restricted Pod (non-root, limited capabilities) leads to "operation not permitted" error #1934

@Davidgo123

Description

@Davidgo123

What happened?
devpod crashes while run "devpod up ..." with the error "failed to probe user enviroment variables: probe user env: fork/exec /usr/bin/zsh: operation not permitted, probe user env: fork/exec /usr/bin/zsh: operation not permitted".

Provider is a restricted kubernetes (non-root constraint, limited capabilities)

What did you expect to happen instead?
Successful startup of devpod remote session with vscode

How can we reproduce the bug? (as minimally and precisely as possible)

My devcontainer.json:

{
  "name": "DevPod",
  "build": {
       "dockerfile": "Dockerfile"
   },
  "remoteUser": "vscode",
  "containerUser": "vscode",
  "updateRemoteUserUID": true,
  "workspaceFolder": "/home/vscode/workspaces",
  "workspaceMount": "source=${localEnv:PROJECT_PATH},target=/home/vscode/workspaces,type=bind,consistency=cached",
  "remoteEnv": {
    "SHELL": "/usr/bin/zsh",
   }
}

My Dockerfile (minimal):

FROM mcr.microsoft.com/devcontainers/python:3.12-bookworm
ARG USERNAME=vscode
ARG USERGROUP=vscode
USER root

# ---------------- install zsh ----------------
RUN apt-get update && apt-get install -y --no-install-recommends zsh

# ---------------- create devpod folders and files and change ownership ----------------
RUN mkdir -p /var/run/devpod /var/cache/devpod /var/devpod /tmp/devpod
RUN echo '{}' > /etc/envfile.json && chown $USERNAME:$USERGROUP /etc/envfile.json
RUN echo '{}' > /var/run/devpod/result.json && chown $USERNAME:$USERGROUP /var/run/devpod/result.json
RUN chown -R $USERNAME:$USERGROUP /run/devpod/ /var/devpod /var/run/devpod /var/cache/devpod /tmp/devpod
ENV SHELL="/usr/bin/zsh"
USER $USERNAME
WORKDIR /home/$USERNAME/workspaces

My provider.yaml:

name: kubernetes
version: v0.0.1
options:
  PROJECT_PATH:
    description: "Path to local repo content"
    required: true
  HOME_PATH:
    description: "Home path for kube config path"
    required: true
agent:
  containerInactivityTimeout: 10m
  path: ${DEVPOD}
  driver: kubernetes
  kubernetes:
    clusterRole: xxxx
    kubectlPath: "${HOME_PATH}\\Tools"
    kubernetesConfig: "${HOME_PATH}\\.kube\\config"
    kubernetesNamespace: devpod-ns
    diskSize: 10Gi
    podManifestTemplate: "${PROJECT_PATH}\\.devcontainer\\pod_manifest_template.yaml"
    strictSecurity: true
    storageClass: azureblob-nfs-premium
    inactivityTimeout: 10m
    podTimeout: 10m
exec:
  command: |-
    ${DEVPOD} helper sh -c "${COMMAND}"

My pod_manifest_template.yaml:

apiVersion: v1
kind: Pod
metadata:
  name: devpod
spec:
  containers:
    - name: devpod
      securityContext:
        runAsNonRoot: true
        runAsUser: 1000
        runAsGroup: 1000
        capabilities:
          drop:
            - ALL
          add:
            - CHOWN
            - SETUID
            - SETGID
            - FOWNER
            - DAC_OVERRIDE

Local Environment:

  • DevPod Version: v0.6.15
  • Operating System: windows
  • ARCH of the OS: AMD64

DevPod Provider:

  • Cloud Provider: Kubernetes
  • Kubernetes Provider: Client Version: v1.34.0, Kustomize Version: v5.7.1, Server Version: v1.33.2

Anything else we need to know?

Windows powershell commands to init / start devpod

devpod provider add ".devcontainer\provider.yaml" `
      -o HOME_PATH="$HOME" `
      -o PROJECT_PATH="$PWD" `
      --name "kubernetes" `
      --debug

devpod up "$PWD" `
   --id "devcontainer-$env:USERNAME" `
   --provider kubernetes `
   --workspace-env PROJECT_PATH="$PWD" `
   --ide vscode `
   --debug 

Logs

17:33:48 info Create Pod 'devpod-default-de-bc5bd'
17:33:48 info Waiting for DevContainer Pod 'devpod-default-de-bc5bd' to come up...
17:33:51 debug execute inject script
17:33:51 debug Received line after pong: done
17:33:51 debug done exec
17:33:51 debug done inject
17:33:51 debug done injecting
17:33:51 debug Done InjectAgentAndExecute
17:33:51 debug Injected into container
17:33:51 info Setup container...
17:33:51 debug Inject and run command: '/usr/local/bin/devpod' helper ssh-server --stdio --debug
17:33:51 debug Attempting to create SSH client
17:33:52 debug SSH client created
17:33:52 debug SSH session created
17:33:52 info Execute SSH server command: zsh -c '/usr/local/bin/devpod' agent container setup --setup-info '..........................'
17:33:53 debug Created logger
17:33:53 debug Received ping from agent
17:33:53 debug Start setting up container...
17:33:53 debug Syncing mounts...  [type=bind,src=C:\Users\XXXX\Repos\devpod ,dst=/home/vscode/workspaces,consistency=cached type=bind]
17:33:53 info Copy C:\Users\XXXX\Repos\devpod into DevContainer /home/vscode/workspaces
17:33:56 info Uploaded 0.26 MB
17:34:27 info Uploaded 0.92 MB
17:34:33 debug Credentials server started on port 14213...
17:34:34 debug Incoming client connection at /
17:34:35 debug Credentials server started...
17:34:35 error Error configuring git credentials: add git credential helper: exit status 255
17:34:35 info Chown workspace...
17:34:35 info Chown projects...
17:34:36 debug Patch etc environment & profile...
17:34:36 debug Run lifecycle hooks commands...
17:34:36 debug running user env probe with shell "zsh", probe "loginInteractiveShell", user "vscode" and command "cat /proc/self/environ"
17:34:36 debug running user env probe with shell "zsh", probe "loginInteractiveShell", user "vscode" and command "printenv"
17:34:36 warn failed to probe user environment variables: probe user env: fork/exec /usr/bin/zsh: operation not permitted, probe user env: fork/exec /usr/bin/zsh: operation not permitted
17:35:48 debug Done setting up environment
17:35:48 debug Setup VSCode...
17:35:48 info Install extensions '....'
17:35:48 debug Start DevPod Container Daemon with Inactivity Timeout 10m
17:35:48 debug Connection to SSH Server closed
17:35:48 debug Done executing ssh server helper command
17:35:48 debug Done setting up container
17:35:48 17:35:48 debug debug Connection to SSH Server closed
done inject
17:35:48 debug done injecting
17:35:48 debug Done InjectAgentAndExecute
17:35:481 7:35:48 debugdebug  Done executing ssh server helper command
done exec
17:35:48 debug Done creating devcontainer
17:35:48 info Run 'ssh devcontainer-xxxx.devpod' to ssh into the devcontainer
17:35:48 debug No dotfiles repo specified, skipping
17:35:48 info Starting VSCode...
17:35:49 debug Run VSCode command code --disable-extension ms-vscode-remote.remote-containers --new-window --folder-uri=vscode-remote://ssh-remote+devcontainer-xxxx.devpod//home/vscode/workspaces

-> After that the scripts runs to the end, start vscode, tries to connect to the kubernetes devpod pod and fails

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions