Replies: 3 comments 2 replies
-
|
💬 Your Product Feedback Has Been Submitted 🎉 Thank you for taking the time to share your insights with us! Your feedback is invaluable as we build a better GitHub experience for all our users. Here's what you can expect moving forward ⏩
Where to look to see what's shipping 👀
What you can do in the meantime 💻
As a member of the GitHub community, your participation is essential. While we can't promise that every suggestion will be implemented, we want to emphasize that your feedback is instrumental in guiding our decisions and priorities. Thank you once again for your contribution to making GitHub even better! We're grateful for your ongoing support and collaboration in shaping the future of our platform. ⭐ |
Beta Was this translation helpful? Give feedback.
-
|
Issue opened in workflow repo: gautamkrishnar/blog-post-workflow#286 . As I noted there, web sockets with push notifications would be the best way to handle this, but it requires resources be online, e.g., a phone app, and listening for changes on a remote server. When working with items on GitHub.io pages though, workflows should be triggered on the GitHub.io repo and GitHub.com should pull the remote [static] resource from the GitHub.io page and render it using a simple IFRAME. There's no reason why a GitHub personal "repo" needs to be updated whenever a GitHub.io page is updated. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for the suggestion @ngie-eign I have updated the docs and the default template to match your suggestion. |
Beta Was this translation helpful? Give feedback.
-
|
I'll +1 this. Also, something is posting those dummy commits into my profile repo once every 2 months, and I can't find the workflow that's doing it, THAT is curious. The workload itself of pushing 1 commit once in two months is nothing. How about marking jobs as "not security/privacy crucial", also maybe marking them as "small", and keeping a pool of containers hot to run them, instead of spinning up/shutting down? That is assuming there is a noticeable cost to container creation/startup/shutdown, I don't know if it's actually the case. |
Beta Was this translation helpful? Give feedback.
-
|
@VioletGiraffe : I'd assume there's a fair bit that goes into spinning a container up and down because the container scheduler (I'm going to assume K8-based) needs to pick an area that meets the criteria specified in the manifest (arm? x64?), maybe needs to pick the least loaded area, maybe needs to deal with policy issues around exporting data (GDPR, etc), needs to spin up however many containers and replicas are required, run the tasks, gather the result(s), then tear down the containers. The logging/results gathering involved on the backend is also not cheap at scale. It goes into a mostly write once read many times ( WORM) key-value store or elastic file system like Cassandra, HDFS, etc, which is rebalanced/served/moved in a storage solution (if it's like AWS/Meta, there's a hot/warm storage solution that feeds into a cold storage/archival solution) until the data expires (I don't remember the exact timeout? Maybe a month?). And that's ignoring all of the caching layers involved and punting packets/datagrams around regions/datacenters as well (a good chunk of this stuff is probably replicated in multiple sites/regions). It's all quite complicated and very expensive at scale for simple tasks like taking a blog post and rendering it as markdown page. That's why I recommended that the workflow be applied directly to gautamkrishnar/blog-post-workflow#286 when possible, since it could run once on push and the rest of the time be handled as a static page. Big cloud providers like MS (Azure) are really good at hiding the costs/details from end-users :) (same with MS injecting CoPilot into all of their products). PS Sorry if it seems like I was for picking on your account. I was looking for VSCode + cppcheck integration and found cppcheck-vs-addin and I wanted to know what other work you'd done 💚. I stumbled on your workflows and stuff by accident ("blog posts? how does that dynamically update?", etc 🤔). |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
Why are you starting this discussion?
Product Feedback
What GitHub Actions topic or product is this about?
Schedule & Cron Jobs
Discussion Details
Problem
The cron scheduling syntax allows one to specify when cron jobs should be run using absolute times in UTC.
Not all workflows are created equal though: some workflows are low priority, some workflows are high priority, and some workflows are not critical to complete during a specific time period.
Request
Add a new means of scheduling cron jobs during off-peak hours, e.g., 12-5 AM, etc, in whatever local region the job is being run or localized in.
Case Study
Some workflows recommend being run once an hour, e.g., gautamkrishnar/blog-post-workflow. This is very wasteful resource-wise at scale: the workflow spins up at least one container, requests the XML/RSS feed over the network, and commits the rendered version of the XML/RSS feed once an hour.
In reality, the need for running every hour is no doubt overkill.
For example, in this repo, as of writing:
This is a lot of compute and network resources that are wasted when extrapolated out over the past 2 years.
More Discussion
Yes, I'm aware of the fact that the caching mechanisms likely mitigate/amortize some of this "pain" associated with this workflow as the dev-container picked is no doubt cached and the network is likely handled internally in MS's infrastructure. My statements are being made with a dev/SRE hat on from past jobs in large scale distributed systems, and as an engineer who is concerned with how many resources are being wasted spinning up datacenters for workflows like this.
CC: @gautamkrishnar, @VioletGiraffe
Beta Was this translation helpful? Give feedback.
All reactions