这是indexloc提供的服务,不要输入任何密码
Skip to content

Support standalone builds #57

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jul 17, 2022
Merged

Support standalone builds #57

merged 1 commit into from
Jul 17, 2022

Conversation

gerlero
Copy link
Owner

@gerlero gerlero commented Jul 9, 2022

Trying out a fully standalone (i.e., no system-wide Homebrew installation required) build, as suggested and discussed here.

In the standalone build, a full Homebrew install is packaged inside the virtual disk. This adds ~2 GB to the app size, which is a lot. OTOH, with the regular build, you'd have to install the same stuff anyway (although in that case it can be shared between OpenFOAM versions; and precompiled bottles can be used instead of building everything from source). EDIT: the internal Homebrew install can be cleaned up, so it's a little less than 2 GB, and it might be an acceptable compromise to many (most?) users (and especially non-Homebrew users). Standalone builds are also less likely to break due to upgraded dependencies, and the user should not even need a compiler (Xcode or CLT) to use them (unless they actually need to compile something). Build times for dependencies that do not have relocatable Homebrew bottles still add significant time (see below). Building the app no longer requires a system Homebrew install

This draft PR is 100% experimental, and likely to fail CI for exceeding the 6-hour job time limit... EDIT: I did have to split the build workflows into two separate jobs to stop us from hitting that limit. It still takes ~6 hours for a full build on macOS runners (~2 hours for dependencies + ~4 hours for OpenFOAM itself; note that macOS runners are not the fastest), which is a lot of time (but caching can somewhat help here). All in all I've started to like this idea a lot more than when I first tried it

EDIT: Extra details follow
I refined this PR by adopting a similar approach to https://github.com/octave-app/octave-app-bundler (I'd like acknowledge the maintainer of that project, @apjanke, who coincidentally also started a project attempting to build OpenFOAM for Mac with Homebrew). Dependencies are installed under a new usr directory in the volume using Homebrew. On standalone builds, the Homebrew installation itself is stripped from the volume, leaving only the dependencies. On non-standalone builds (which are still possible by setting the make variable DEPENDENCIES_KIND=homebrew), the usr directory is replaced with a symlink to the Homebrew prefix location (which is also backwards compatible with apps built prior to this PR).

@gerlero gerlero force-pushed the standalone branch 30 times, most recently from da2cfb5 to 3ae1469 Compare July 13, 2022 16:49
@gerlero gerlero force-pushed the standalone branch 21 times, most recently from e5ffc31 to 4601220 Compare July 17, 2022 14:58
@gerlero gerlero marked this pull request as ready for review July 17, 2022 16:57
@gerlero gerlero merged commit 53903a1 into main Jul 17, 2022
@gerlero gerlero deleted the standalone branch July 17, 2022 17:03
@apjanke
Copy link

apjanke commented Jul 18, 2022

Thanks for the shout-out! Credit for the intial development of this mechanism for Octave.app goes to @schoeps, though: he wrote the whole thing, I just took over maintenance and did some refinements.

This sounds like a promising approach. Let me know if there are tweaks I can do to Octave.app's octave-app-bundler or createdmg code to make it reusable for other applications like your OpenFOAM here.

Note that I'm thinking of migrating Octave.app from Homebrew to MacPorts for this mechanism. Homebrew only supports the latest 3 versions of macOS, and I want to have Octave.app continue to support macOS versions back to at least 10.14 Mojave. Partly because I use 10.14 myself due to being an Aperture and iTunes user, and partly because Octave.app is targeted at students and academics, and they seem to tend to use older macOS versions. I don't know if OpenFOAM has a similar userbase. Also Homebrew does not officially support installing to alternate prefix locations (like you need to do for this app bundler approach), and many of its "bottle" binaries are not relocatable, so you end up doing a build from source for lots of big packages, which will affect your CI run time limit here. I don't know how relocatable MacPorts are compared to Homebrew, but their binary system seems more amenable to users or extenders setting up their own binary build farm, so I think it would be feasible to have Octave.app do its own MacPorts dependencies builds to avoid having to build from source each time you build Octave.app, whereas it seems pretty difficult to set up one's own custom Homebrew bottle distribution. This migration to MacPorts would be a big project, so it's not happening any time soon.

@gerlero
Copy link
Owner Author

gerlero commented Jul 21, 2022

Thank you for chiming in! And acknowledgements to @schoeps too in that case.

These alternative standalone builds are still in an early stage (I'm dog-fooding them right now), so for now I'm happy with the current scheme. Although, if this works out, I can see the point of having a generic dependency bundler, that takes a Brewfile (assuming Homebrew—or a similar list of requirements for a different "backend") and a directory (custom prefix), and takes care of installing the required dependencies there leaving as small a footprint as possible. Of course it could do more than that: caching builds/making and managing bottles for a custom prefix could potentially save a lot of compilation time, as you mention; and it could even perform some kind of version management. It'd take a bunch of development work though, but I see the point of it and there have to be other projects that could benefit too from something like that.

As for Homebrew vs. MacPorts and supporting older macOS releases, I'll have to stick with Homebrew at least for now, both for compatibility with the regular builds that are dependent Homebrew, plus the fact the app is currently built on GitHub Actions (Intel builds) and locally (Apple silicon), which already limits how far back I can go in terms of macOS versions. I do expect our apps to have similar userbases—although it has surprised me to see a lot of downloads for the Apple silicon variants recently (currently outnumbering Intel 2:1), but I don't really draw any conclusions from just that. Still, I'm curious: where do you run the Octave.app bundler script when making the releases? (I guess you can do it locally on your Mojave Mac). As it stands, supporting old versions of macOS with this app would imply more than just having compatible dependencies, so sourcing the dependencies from MacPorts might make sense but is not something I'd need for this project right now. I still think it'd be a good idea to try anything that might improve compatibility and/or reduce compile times for dependencies for use cases like ours. From my experience using MacPorts a few years back, I remember that it used to build a lot of stuff from source even on a default install (but that might have changed in the last few years). I did not attempt a relocated install with MacPorts.

@apjanke
Copy link

apjanke commented Jul 22, 2022

All totally reasonable.

a lot of downloads for the Apple silicon variants recently (currently outnumbering Intel 2:1)

Huh! That's a lot more than I would have expected. But I agree: it's a simple data point to draw much of a conclusion on.

I remember that it used to build a lot of stuff from source even on a default install

This is my experience too. I'm running a default install of MacPorts on some of my Macs, and it builds a lot more stuff from source than Homebrew does. I'm just hoping that its greater customizability lets me put some work in to supporting my own binaries for custom builds and alternate install locations.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants