+
GitHub header

Degraded Performance for GitHub Actions MacOS Runners

Incident Report for GitHub

Resolved

On October 1, 2025 between 07:00 UTC and 17:20 UTC, Mac hosted runner capacity for Actions was degraded, leading to timed out jobs and long queue times. On average, the error rate was 46% and peaked at 96% of requests to the service. XL and Intel runners recovered by 10:10 UTC, with the other types taking longer to recover.

The degraded capacity was triggered by a scheduled event at 07:00 UTC that led to a permission failure on Mac runner hosts, blocking reimage operations. The permission issue was resolved by 9:41 UTC, but the recovery of available runners took longer than expected due to a combination of backoff logic slowing backend operations and some hosts needing state resets.

We deployed changes immediately following the incident to address the scheduled event and ensure that similar failures will not block critical operations in the future. We are also working to reduce the end-to-end time for self-healing of offline hosts for quicker full recovery of future capacity or host events.
Posted Oct 01, 2025 - 16:55 UTC

Update

We are seeing some recovery for image queueing and continuing to monitor.
Posted Oct 01, 2025 - 16:27 UTC

Update

We are continuing work to restore capacity for our MacOS ARM runners.
Posted Oct 01, 2025 - 14:41 UTC

Update

Our team continues to work hard on restoring capacity for the Mac runners.
Posted Oct 01, 2025 - 13:58 UTC

Update

Work continues on restoring capacity on the Mac runners.
Posted Oct 01, 2025 - 13:12 UTC

Update

MacOS ARM runners continue to be at reduced capacity, causing queuing of jobs. Investigation is ongoing.
Posted Oct 01, 2025 - 12:32 UTC

Update

Work continues to bring the full runner capacity back online. Resources are focused on improving the recovery of certain runner types.
Posted Oct 01, 2025 - 11:51 UTC

Update

We are continuing to see recovery of some runner capacity and investigating slow recovery of certain runner types.
Posted Oct 01, 2025 - 11:11 UTC

Update

We are seeing recovery of some runner capacity, while also investigating slow recovery of certain runner types.
Posted Oct 01, 2025 - 10:30 UTC

Update

MacOS runners are coming back online and starting to process queued work.
Posted Oct 01, 2025 - 09:44 UTC

Update

We are continuing to deploy the necessary changes to restore MacOS runner capacity.
Posted Oct 01, 2025 - 08:59 UTC

Update

We have identified the cause and are deploying a change to restore MacOS runner capacity.
Posted Oct 01, 2025 - 08:27 UTC

Update

Customers using GitHub Actions Mac OS runners are experiencing job start delays and failures. We are aware of this issue and actively investigating.
Posted Oct 01, 2025 - 08:17 UTC

Update

Actions is experiencing degraded performance. We are continuing to investigate.
Posted Oct 01, 2025 - 08:09 UTC

Investigating

We are currently investigating this issue.
Posted Oct 01, 2025 - 07:59 UTC
This incident affected: Actions.
点击 这是indexloc提供的php浏览器服务,不要输入任何密码和下载