need docs and tests

Can you describe this deployment scenario from repo to deployment?

Now that the implementation is out of the way 😜 what is the justification for adding this new method?

Read the attached docs for more reasoning :)

Basically this "solves" the issue where Dockerfile deployments currently have no way of running multiple processes. We treat the docker-compose.yml as a method of defining separate process types.

Can you include some example repo artifacts for this method? Even better a test app would be great.

Heh I've managed to stay this long without having tests!

https://github.com/josegonzalez/dokku-compose-app

It's more or less the sample docker-compose application, but modified to understand environment variables for the redis url.

+1

Please do not +1 feature pull requests. We'll merge the features when they are ready :)

@michaelshobbs If I add my test app, would you mind adding actual tests for it?

please add a test app in this PR and i can add tests to the documented spec. (side note: make sure those docs clarify the expected behavior)

Why?

@josegonzalez I would like to run multiple instances (web, workers) using Dockerfile deployment method (no Procfile).

Currently if I have Dockerfile, dokku creates only web process and there is no way to run other type of processes (for example workers). Using docker-compose standard, dokku can automatically create web and workers processes which are defined in docker-compose.yml file.

How do you imagine linking works? Really what we want is a way to define multiple images or the same image with different start commands. Compose more or less replicates a lot of what dokku does and supporting it properly isn't feasible.

The above could work but would leave everyone upset with how things are implemented/not supported.

I understand your concerns about django-compose which overlaps dokku functionality. And I agree with you.

So what would you like us to do? Ideally we do not introduce a dokku.yml :)

@josegonzalez For me the removal of Zero-Downtime deploys would be a dealbreaker. What would be biggest benefit be for the Docker-compose?

Being able to run multiple commands for a Dockerfile-based deploy.

@josegonzalez I suggest to add a new option to specify path to Dockerfile (as environmental variable). It gives possibility to create separate Dockerfiles for web and workers.

For example, I can have the following repository structure:

/Dockerfile
/worker/Dockerfile
/src/
...

And then I can create a worker by defining path to /worker/, e.g. dokku config:set my_app DOCKERFILE_PATH=/worker/

@josegonzalez I suggest to add a new option to specify path to Dockerfile (as environmental variable). It gives possibility to create separate Dockerfiles for web and workers.

This is already possible via the DOCKER_BUILD_OPTIONS part of the docker-options plugin. You can specify dokku docker-options:add myapp build -b worker/Dockerfile or some such incantation.

You can't do that for a specific process though.

I've had success using a Procfile and the DYNO environment variable and a docker-entrypoint.sh script to dispatch/start the appropriate worker. I suspect this is the right approach currently. You use the Procfile to declare your worker names and then the DYNO var contains "worker:1", etc. so you know which one to start in your entrypoint script. The Dockerfile itself defines the dokku app. Separate Dockerfiles would not fit particularly well into dokku's model I don't think.

@mmerickel would you be willing to write an documentation for this?

Note: even in such a deploy, you don't get the ability to run different dockerfiles. I suppose that is okay.

Closing in favor of using the Procfile behavior that @pmclanahan is going to work on :)

@josegonzalez is there any feedback about the feature @pmclanahan is working on?

That ended up being our Procfile support as documented here.

There is another use-case for this which, I believe, is much more compelling -- dev-prod parity. Currently you can use dokku plugins to link to other services, which is great for production but if you app depends on those services you can no longer use it locally. Docker-compose allows you to have the same exact environment, complete with orchestrated services locally and and in production.

Currently, I get around this by using vagrant to orchestrate local docker containers.

It would be great to be able to deploy a docker-compose file as well. It seems to be supported by heroku as well, so I know i'm not crazy: https://devcenter.heroku.com/articles/local-development-with-docker-compose

This just doesn't work with zero-downtime deploys. Docker-compose doesn't handle that properly at all, and it's not clear to me how we'd handle conditionally starting/stopping certain services once they are running.

If you'd like to contribute to this, please see me on chat - my nick is savant - and we can discuss any potential issues before you start working on this.

@josegonzalez I thought I saw the new v3 compose config supports docker services. Perhaps dokku automatically turns it's zero downtime off in the case of compose and we leave it up to the docker-compose.yml author to do what they want. shrug

@michaelshobbs you may as well just shove sshfront + docker-compose if you're going to do that. I don't see the value of using docker-compose for infra if you are tossing out the best parts of Dokku.

I get your point. To be fair zero-downtime is just part of the build -> deploy -> proxy pipeline. The point about using docker-compose to represent prod environments in development is also valid. Just throwing it out there that this might be possible with the latest version of docker-compose.

dokku does offer some other goodies like automatically setting up nginx proxies for your containers, plugins (especially letsencrypt...)

I mean if someone wants to do the work of refactoring our pipeline to make it easy to make this an engine, then they may want to speak to me directly on chat.

I'm locking this thread as I don't think it's productive - the code here isn't mergeable anymore anyhow - and would rather we go back to the drawing board on how this might fit in.

I mentioned engines above, and have had this convo with @michaelshobbs via video, but if you are interested, please chat with me directly.