getting an error with test-bot using github provided macos runners

[ipatch]

Output of brew config

na

Output of brew doctor

na

Description of issue

Run brew test-bot --only-setup
==> Using Homebrew/brew 4.6.20-50-g90ebad1417 (Merge pull request #20980 from Homebrew/stderr-puts-download-messages-if-not-tty)
==> Using Homebrew/homebrew-core 2fb23a752d5 (Merge pull request #252059 from Homebrew/bump-rust-1.91.0)
==> Testing FreeCAD/homebrew-freecad 8ebbefe (Merge 5682fd7645ed394c7649ead01571f8d81820bc4e into 1128a2307c51968be3932e2a92bfdd4f798935ca):
==> Running Setup#run!
==> brew install-bundler-gems --add-groups=ast,audit,bottle,formula_test,livecheck,style
==> brew config
HOMEBREW_VERSION: 4.6.20-50-g90ebad1
ORIGIN: https://github.com/Homebrew/brew
HEAD: 90ebad141725eda6a4cafb21522b96c2e73b09fc
Last commit: 3 hours ago
Branch: main
Core tap HEAD: 2fb23a752d53ebc8c4f176de7097ccc03e44c17d
Core tap last commit: 4 days ago
Core tap JSON: 06 Nov 23:59 UTC
Core cask tap HEAD: 2c38cc9054a159a5a7772343668c6f3d250124b3
Core cask tap last commit: 4 days ago
Core cask tap JSON: 06 Nov 23:59 UTC
HOMEBREW_PREFIX: /usr/local
HOMEBREW_CACHE: /Users/runner/Library/Caches/Homebrew
HOMEBREW_CASK_OPTS: ["--no-quarantine"]
HOMEBREW_COLOR: set
HOMEBREW_CURL_PATH: /usr/bin/curl
HOMEBREW_DEVELOPER: set
HOMEBREW_DOWNLOAD_CONCURRENCY: 8
HOMEBREW_FAIL_LOG_LINES: 150
HOMEBREW_GITHUB_API_TOKEN: set
HOMEBREW_GIT_EMAIL: [email protected]
HOMEBREW_GIT_NAME: BrewTestBot
HOMEBREW_GIT_PATH: /usr/bin/git
HOMEBREW_LOGS: /Users/runner/work/homebrew-freecad/homebrew-freecad/logs
HOMEBREW_MAKE_JOBS: 4
HOMEBREW_NO_ANALYTICS: set
HOMEBREW_NO_AUTO_UPDATE: set
HOMEBREW_NO_EMOJI: set
HOMEBREW_NO_ENV_HINTS: set
HOMEBREW_NO_INSTALLED_DEPENDENTS_CHECK: set
HOMEBREW_NO_INSTALL_CLEANUP: set
HOMEBREW_SORBET_RUNTIME: set
Homebrew Ruby: 3.4.5 => /usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/3.4.5/bin/ruby
CPU: quad-core 64-bit ivybridge
Clang: 17.0.0 build 1700
Git: 2.39.5 => /usr/bin/git
Curl: 8.7.1 => /usr/bin/curl
macOS: 15.7.1-x86_64
CLT: 16.4.0.0.1.1747106510
Xcode: 16.4 => /Applications/Xcode_16.4.app/Contents/Developer
==> brew doctor
==> FAILED
Full doctor output
  Please note that these warnings are just used to help the Homebrew maintainers
  with debugging if you file an issue. If everything you use Homebrew for is
  working fine: please don't worry or file an issue; just ignore this. Thanks!
  Warning: You have uncommitted modifications to Homebrew/homebrew-core.
  If this is a surprise to you, then you should stash these modifications.
  Stashing returns Homebrew to a pristine state but can be undone
  should you later need to do so for some reason.
    git -C "/usr/local/Homebrew/Library/Taps/homebrew/homebrew-core" stash -u && git -C "/usr/local/Homebrew/Library/Taps/homebrew/homebrew-core" clean -d -f
  
  Uncommitted files:
    ?? Formula/o/[email protected]
  
Error: 1 failed step!
brew doctor
Error: Process completed with exit code 1.

https://github.com/FreeCAD/homebrew-freecad/actions/runs/19153574155/job/54749268070

if i need to provide more information please let me know.

[ipatch]

pretty sure test-bot is broken with the current state using github actions. just tested github ci using test-bot provided by homebrew with a completely separate tap and am getting the same error,

https://github.com/ipatch/homebrew-us-05/actions/runs/19177074264/job/54824481059?pr=57

[ZhongRuoyu]

Can you try running brew test-bot --only-cleanup-before --tap=homebrew/core? You can put that next to the invocation without --tap=homebrew/core. Ultimately this is a problem with GitHub Actions' images which we don't control.

[heaths]

This alone did not work for me.

[ipatch]

i ended up adding the below snippet / step to my tests.yml file to resolve this issue.

    - name: sanatize homebrew-core tap
        run: |
          brew update-reset "$(brew --repository homebrew/core)" || true
          git -C "$(brew --repository homebrew/core)" clean -fd || true

[heaths]

Thanks. I ended up doing something similar. I also filed actions/runner-images#13284 against the hosted runner image since the repo appears to be dirty before any brew commands (besides getting the repo root, which I imagine would be read-only) are run.

[ZhongRuoyu]

@ipatch, @heaths: We have worked around this on Homebrew's side in Homebrew/brew#21025. You can now remove the workaround in the workflow.

[heaths]

Thanks. I don't need to update my workflow, right? I just always pick up the new version (I don't pin any versions currently for brew).

[heaths]

Also, in my workflow - created a while ago - it only passes the --only-cleanup-before without --cleanup, though the docs say that the former requires the latter. Is it actually implied? I added it anyway but just wanted to point out the confusion / discrepancy. (Of course, it didn't work yesterday, but sounds like it will going forward.)

[gromgit]

Also, in my workflow - created a while ago - it only passes the --only-cleanup-before without --cleanup, though the docs say that the former requires the latter. Is it actually implied?

It is, but only for GitHub Actions. A PR to fix the docs accordingly was merged just a few hours ago: Homebrew/brew#21035

[ZhongRuoyu]

Thanks. I don't need to update my workflow, right? I just always pick up the new version (I don't pin any versions currently for brew).

That's correct, you can drop the workaround and the original workflow should work now (as long as it contains brew test-bot --only-cleanup-before).