Linux: encoding issues prevent most brew commands

[davepgreene]

Additional Info

• I am using Windows Subsystem for Linux

Output of brew config

HOMEBREW_VERSION: 4.5.0
ORIGIN: https://github.com/Homebrew/brew
HEAD: 3332d3331b56e0aff675d3816d8ebfe564075299
Last commit: 15 hours ago
Branch: stable
Core tap JSON: 29 Apr 22:12 UTC
HOMEBREW_PREFIX: /home/linuxbrew/.linuxbrew
HOMEBREW_CASK_OPTS: []
HOMEBREW_DISPLAY: :0
HOMEBREW_EDITOR: nvim
HOMEBREW_MAKE_JOBS: 16
HOMEBREW_NO_ENV_HINTS: set
Homebrew Ruby: 3.4.3 => /home/dgreene/.local/share/mise/installs/ruby/3.4.3/bin/ruby
CPU: 16-core 64-bit zen5
Clang: N/A
Git: 2.49.0 => /home/linuxbrew/.linuxbrew/bin/git
Curl: 8.5.0 => /bin/curl
Kernel: Linux 5.15.167.4-microsoft-standard-WSL2 x86_64 GNU/Linux
OS: Ubuntu 24.04.2 LTS (noble)
WSL: 2 (Microsoft Store)
Host glibc: 2.39
/usr/bin/gcc: 14.2.0
/usr/bin/ruby: 3.2.3
Error: "\xE2" on US-ASCII
/home/dgreene/.local/share/mise/installs/ruby/3.4.3/lib/ruby/3.4.0/json/common.rb:221:in 'String#encode'
/home/dgreene/.local/share/mise/installs/ruby/3.4.3/lib/ruby/3.4.0/json/common.rb:221:in 'JSON::Ext::Parser.parse'
/home/dgreene/.local/share/mise/installs/ruby/3.4.3/lib/ruby/3.4.0/json/common.rb:221:in 'JSON.parse'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/api.rb:99:in 'Homebrew::API.fetch_json_api_file'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/api/formula.rb:50:in 'Homebrew::API::Formula.download_and_cache_data!'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/api/formula.rb:72:in 'Homebrew::API::Formula.all_formulae'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/formulary.rb:852:in 'Formulary::FromAPILoader.try_new'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/formulary.rb:1133:in 'block in Formulary.loader_for'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/formulary.rb:1132:in 'Array#each'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/formulary.rb:1132:in 'Formulary.loader_for'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/formulary.rb:926:in 'Formulary.factory'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/extend/os/linux/system_config.rb:31:in 'SystemConfig.formula_linked_version'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/extend/os/linux/system_config.rb:53:in 'block in SystemConfig.dump_verbose_config'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/extend/os/linux/system_config.rb:52:in 'Array#each'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/extend/os/linux/system_config.rb:52:in 'SystemConfig.dump_verbose_config'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/cmd/config.rb:21:in 'Homebrew::Cmd::Config#run'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/brew.rb:95:in '<main>'
Please report this issue:
  https://docs.brew.sh/Troubleshooting

Output of brew doctor

Error: "\xE2" on US-ASCII
/home/dgreene/.local/share/mise/installs/ruby/3.4.3/lib/ruby/3.4.0/json/common.rb:221:in 'String#encode'
/home/dgreene/.local/share/mise/installs/ruby/3.4.3/lib/ruby/3.4.0/json/common.rb:221:in 'JSON::Ext::Parser.parse'
/home/dgreene/.local/share/mise/installs/ruby/3.4.3/lib/ruby/3.4.0/json/common.rb:221:in 'JSON.parse'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/api.rb:99:in 'Homebrew::API.fetch_json_api_file'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/api/formula.rb:50:in 'Homebrew::API::Formula.download_and_cache_data!'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/api/formula.rb:72:in 'Homebrew::API::Formula.all_formulae'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/formulary.rb:852:in 'Formulary::FromAPILoader.try_new'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/diagnostic.rb:861:in 'block (2 levels) in Homebrew::Diagnostic::Checks#check_deleted_formula'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/diagnostic.rb:859:in 'Array#any?'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/diagnostic.rb:859:in 'block in Homebrew::Diagnostic::Checks#check_deleted_formula'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/diagnostic.rb:851:in 'Array#each'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/diagnostic.rb:851:in 'Enumerable#filter_map'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/diagnostic.rb:851:in 'Homebrew::Diagnostic::Checks#check_deleted_formula'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/cmd/doctor.rb:59:in 'block in Homebrew::Cmd::Doctor#run'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/cmd/doctor.rb:52:in 'Array#each'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/cmd/doctor.rb:52:in 'Homebrew::Cmd::Doctor#run'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/brew.rb:95:in '<main>'
Please report this issue:
  https://docs.brew.sh/Troubleshooting

Description of issue

As seen in the outputs, there's an encoding issue that's throwing an error when downloading the list of formulae. I've narrowed it down more specifically to the description field for the dbg-macro formula: https://github.com/Homebrew/homebrew-core/blob/master/Formula/d/dbg-macro.rb#L2. That said, I imagine if this wasn't a configuration issue on my part that there would be a ton of open issues here about it.

My only guess is that it has to do with my locale configuration but the output from locale seems ok...

LANG=en_US.UTF-8
LANGUAGE=
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=

Any suggestions? I'm kind of at a point where I don't know enough about what homebrew expects to be able to debug this further.

[Bo98]

What's the output of locale -a? Might not be finding UTF-8 in the locale database.

[davepgreene]

> locale -a
C
C.utf8
en_US.utf8
POSIX

[Bo98]

Does it work with HOMEBREW_FORCE_VENDOR_RUBY=1? Might be an issue with the Ruby being used:

Homebrew Ruby: 3.4.3 => /home/dgreene/.local/share/mise/installs/ruby/3.4.3/bin/ruby

[davepgreene]

Ah, yeah, found it. Aside from version, are there any particular features that the vendored ruby provides that a system one wouldn't?

[Bo98]

The system one might be compiled without locale charmap: https://github.com/ruby/ruby/blob/b42afa1dbcbb91e89852b3b3bc72484d7f0a5528/localeinit.c#L111. Though Ruby does try to compile with it by default so would be odd for a system one not to have it.

Not entirely sure of why it's different but I'll look and see if I can fix compatibility with system Rubies.

[davepgreene]

Out of curiosity, is there any reason not to just default to the vendored ruby? Is there ever a situation where Homebrew doesn't distribute the runtime?

[Bo98]

As long as you're using x86_64 or arm64, yeah there probably isn't much reason anymore to use system ruby and is a change I'm considering making for 4.5.1.

[huyz]

Note: brew doctor still gives the Error: "\xE2" on US-ASCII problem even with that envvar.