Resolution

• Pin the .deb requirements to nginx (>= 1.4.6)
• Remove the calls to apt-add-repository that set up the nginx PPA
• Ensure all included features still work
• Update the docs

As a bonus, once we stop using apt-add-repository, we can also

• drop software-properties-common and python-software-properties from package dependencies

Rationale

(See discussion below for more flavor)

• The version in the LTS repo works fully for dokku's core needs.
• The LTS package is less volatile (it only receives security fixes), so the underlying platform is less likely to break during routine, unattended upgrades. E.g., there's no risk of things breaking overnight when a new release of nginx gets uploaded on the PPA.
• By policy, the LTS team will backport a security fix from 1.10 to 1.4 if they have to. The PPA is weill within its rights to just tell you to upgrade to 1.10.

Discussion

(ported from #1629)

@callahad:

I'd also request: "Works with distro-provided nginx." Right now, the nginx-vhosts/dependencies file tries to add a PPA and install nginx from it, which seems a bit extreme. I want the host system to be as bland and stable as possible. Newer software is easy enough to run inside my deployed containers. :)

As it stands, that would mean supporting: nginx 1.4.6 on Ubuntu 14.04 and 1.6.2 on Debian 8. From reading the changelogs, I don't think we depend on anything introduced in 1.6 or later, and there are no open CVEs in the distro-provided packages.

@michaelshobbs:

@callahad the (current) primary install base is definitely Ubuntu and thus, we would need to be comfortable with nginx 1.4.6. IIRC, the move to PPA installation was to jump to 1.6.x from 1.4.x.

@callahad:

Are we actually depending on anything in 1.6, though? The PPA line was added back in 2013 with the intention of upgrading to 1.4 for websocket support. Now that 1.4 is in 14.04 LTS's repos, maybe we don't need the PPA?

@mmerickel:

The PPA is 1.8. LTS is pretty out of date.

@michaelshobbs:

I think we don't have enough information to determine the impacts of downgrading nginx due to the ability to customize with a config template.

We've already been shipping with a PPA (as was noted) since 2013. What do we as a project gain by removing this?

@callahad:

@michaelshobbs Stability in Dokku's foundation, the removal of an external dependency, and greater harmony with the host environment, since Dokku wouldn't need to run roughshod over /etc/apt/sources.list.d/.

I'm not suggesting downgrading, just setting the Depends field in the package control file to nginx (>= 1.4.6) and ensuring that all standard Dokku functionality works with that version of Nginx. Users who need features only available in newer versions of nginx for their custom templates could freely upgrade nginx from the PPA or backports repo. We just wouldn't mandate it for a standard Dokku installation.

@callahad:

The broader principle I'm advocating for is explicitly defining Dokku's minimum requirements and attempting to be as minimally invasive as possible. We'd gain more confidence in the underlying system, have fewer moving targets to debug, and would get to piggyback on the existing machinery that keeps LTS releases stable and secure. Maybe this discussion should go to a different RFC?

@josegonzalez:

Removing the PPA works for me. The big thing here is just ensuring users have a secure by default setup. If we can depend upon the LTS version for that, awesome.

@mmerickel:

Considering the oddities recently addressed with logrotate and permissions, we might be opening up more bugs if we support multiple nginx distributions. For example making sure those new search/replace scripts work with 2 distributions instead of 1. That can also be an issue for a user who wants a newer version of nginx because dokku is currently doing things to those files that may not be appropriate for all distributions.

So far I think most of the changes were done on plugin:install versus at deploy-time or anything of that sort, so if you drop the ppa and everything that touches those files then it'd be upt o a user to fix it when they install their ppa which is fine of course. Since we force a particular nginx right now though, certain parts of dokku can be a bit more predictable.

@michaelshobbs:

Stability can mean a few things here. The assumption I'm reading seems to imply that removing a change made 2+ years ago will move that needle somehow. IMO, this has worked fine for that long and seems to more likely open us up to varying issues due to folks expecting 1.8 (now) and/or due to some installations using 1.4.x, others using 1.6.x and still others on a PPA. I think it's important to weigh other possible impacts as well as the intended ones.

@callahad:

we might be opening up more bugs if we support multiple nginx distributions

That's effectively where we are right now, though. As soon as nginx 1.10 is released, the PPA will update, and all of a sudden new users are on a new, untested-by-us version of nginx. We wouldn't see that sort of churn if we stuck with the LTS packages from the distro.

As an individual self-hosting a dokku instance, the most important thing to me is that I can set the system up, turn on automatic security updates, and trust that things won't break in the middle of the night. And that's the explicit contract that LTS releases provide: the feature set of packages is frozen, but vulnerabilities still get patched.

By tracking the latest release of nginx, we're building atop a foundation where the underlying capabilities are in much greater flux. That means my server is more likely to break in the middle of the night, and ain't nobody got time for that. :)

@michaelshobbs:

I feel it must be said, especially in the flame scarred landscape that is the Internet, that I greatly appreciate an individual's desire and commitment to clearly convey their point of view.

With that out of the way, 😄, I'm fine with removing the PPA requirement but couldn't we also gain stability here by pinning to 1.8?

@callahad:

🍻

couldn't we also gain stability here by pinning to 1.8?

Yep! But I do think 1.4 makes sense given it's what's in 14.04 LTS, so pinning to 1.8 would still require folks to use the PPA, and the PPA doesn't make the same guarantees as the LTS repos. E.g., LTS will backport a security fix from 1.10 to 1.4 if it has to. The PPA will just tell you to upgrade to 1.10. ;)

@michaelshobbs:

Fair point. I'm still a bit concerned with downgrading our default and the potential ripple effect but I suppose we can see what happens there.

@josegonzalez as the keeper of all things .deb (:wink:) you want to put this bit in a PR?

ok now @callahad no more thread jacking 😉 and back on topic. anybody else have thoughts on this plugin?

@mmerickel:

That's effectively where we are right now, though. As soon as nginx 1.10 is released, the PPA will update, and all of a sudden new users are on a new, untested-by-us version of nginx. We wouldn't see that sort of churn if we stuck with the LTS packages from the distro.

Yes, but the distribution I'm talking about is the actual packager... So you wouldn't expect certain files that dokku is tweaking/stomping to change from version to version of nginx from the same packager (ppa) normally.