-
Notifications
You must be signed in to change notification settings - Fork 13
Support standalone builds #57
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
da2cfb5
to
3ae1469
Compare
e5ffc31
to
4601220
Compare
Thanks for the shout-out! Credit for the intial development of this mechanism for Octave.app goes to @schoeps, though: he wrote the whole thing, I just took over maintenance and did some refinements. This sounds like a promising approach. Let me know if there are tweaks I can do to Octave.app's octave-app-bundler or createdmg code to make it reusable for other applications like your OpenFOAM here. Note that I'm thinking of migrating Octave.app from Homebrew to MacPorts for this mechanism. Homebrew only supports the latest 3 versions of macOS, and I want to have Octave.app continue to support macOS versions back to at least 10.14 Mojave. Partly because I use 10.14 myself due to being an Aperture and iTunes user, and partly because Octave.app is targeted at students and academics, and they seem to tend to use older macOS versions. I don't know if OpenFOAM has a similar userbase. Also Homebrew does not officially support installing to alternate prefix locations (like you need to do for this app bundler approach), and many of its "bottle" binaries are not relocatable, so you end up doing a build from source for lots of big packages, which will affect your CI run time limit here. I don't know how relocatable MacPorts are compared to Homebrew, but their binary system seems more amenable to users or extenders setting up their own binary build farm, so I think it would be feasible to have Octave.app do its own MacPorts dependencies builds to avoid having to build from source each time you build Octave.app, whereas it seems pretty difficult to set up one's own custom Homebrew bottle distribution. This migration to MacPorts would be a big project, so it's not happening any time soon. |
Thank you for chiming in! And acknowledgements to @schoeps too in that case. These alternative standalone builds are still in an early stage (I'm dog-fooding them right now), so for now I'm happy with the current scheme. Although, if this works out, I can see the point of having a generic dependency bundler, that takes a Brewfile (assuming Homebrew—or a similar list of requirements for a different "backend") and a directory (custom prefix), and takes care of installing the required dependencies there leaving as small a footprint as possible. Of course it could do more than that: caching builds/making and managing bottles for a custom prefix could potentially save a lot of compilation time, as you mention; and it could even perform some kind of version management. It'd take a bunch of development work though, but I see the point of it and there have to be other projects that could benefit too from something like that. As for Homebrew vs. MacPorts and supporting older macOS releases, I'll have to stick with Homebrew at least for now, both for compatibility with the regular builds that are dependent Homebrew, plus the fact the app is currently built on GitHub Actions (Intel builds) and locally (Apple silicon), which already limits how far back I can go in terms of macOS versions. I do expect our apps to have similar userbases—although it has surprised me to see a lot of downloads for the Apple silicon variants recently (currently outnumbering Intel 2:1), but I don't really draw any conclusions from just that. Still, I'm curious: where do you run the Octave.app bundler script when making the releases? (I guess you can do it locally on your Mojave Mac). As it stands, supporting old versions of macOS with this app would imply more than just having compatible dependencies, so sourcing the dependencies from MacPorts might make sense but is not something I'd need for this project right now. I still think it'd be a good idea to try anything that might improve compatibility and/or reduce compile times for dependencies for use cases like ours. From my experience using MacPorts a few years back, I remember that it used to build a lot of stuff from source even on a default install (but that might have changed in the last few years). I did not attempt a relocated install with MacPorts. |
All totally reasonable.
Huh! That's a lot more than I would have expected. But I agree: it's a simple data point to draw much of a conclusion on.
This is my experience too. I'm running a default install of MacPorts on some of my Macs, and it builds a lot more stuff from source than Homebrew does. I'm just hoping that its greater customizability lets me put some work in to supporting my own binaries for custom builds and alternate install locations. |
Trying out a fully standalone (i.e., no system-wide Homebrew installation required) build, as suggested and discussed here.
In the standalone build, a full Homebrew install is packaged inside the virtual disk. This adds ~2 GB to the app size, which is a lot. OTOH, with the regular build, you'd have to install the same stuff anyway (although in that case it can be shared between OpenFOAM versions; and precompiled bottles can be used instead of building everything from source). EDIT: the internal Homebrew install can be cleaned up, so it's a little less than 2 GB, and it might be an acceptable compromise to many (most?) users (and especially non-Homebrew users). Standalone builds are also less likely to break due to upgraded dependencies, and the user should not even need a compiler (Xcode or CLT) to use them (unless they actually need to compile something). Build times for dependencies that do not have relocatable Homebrew bottles still add significant time (see below). Building the app no longer requires a system Homebrew install
This draft PR is 100% experimental, and likely to fail CI for exceeding the 6-hour job time limit... EDIT: I did have to split the build workflows into two separate jobs to stop us from hitting that limit. It still takes ~6 hours for a full build on macOS runners (~2 hours for dependencies + ~4 hours for OpenFOAM itself; note that macOS runners are not the fastest), which is a lot of time (but caching can somewhat help here). All in all I've started to like this idea a lot more than when I first tried it
EDIT: Extra details follow
I refined this PR by adopting a similar approach to https://github.com/octave-app/octave-app-bundler (I'd like acknowledge the maintainer of that project, @apjanke, who coincidentally also started a project attempting to build OpenFOAM for Mac with Homebrew). Dependencies are installed under a new
usr
directory in the volume using Homebrew. On standalone builds, the Homebrew installation itself is stripped from the volume, leaving only the dependencies. On non-standalone builds (which are still possible by setting the make variableDEPENDENCIES_KIND=homebrew
), theusr
directory is replaced with a symlink to the Homebrew prefix location (which is also backwards compatible with apps built prior to this PR).