lmod issues since recent Lua update

[mathomp4]

I believe this morning, a brew upgrade brought in Lua 5.4. But in doing so, my lmod (also from brew) install seems to have broken:

❯ module
/Users/mathomp4/.homebrew/brew/opt/lua/bin/lua: version mismatch: app. needs 503.0, Lua core provides 504.0
stack traceback:
	[C]: in ?
	[C]: in function 'require'
	.../lmod/8.4.16/libexec/vendor/share/lua/5.3/posix/init.lua:23: in main chunk
	[C]: in function 'require'
	.../mathomp4/.homebrew/brew/Cellar/lmod/8.4.16/libexec/lmod:61: in main chunk
	[C]: in ?

From staring at it I guess it's from the luaposix rock? I do see in the lmod formula some hardcoded 5.3. Maybe?

I'm a pretty weak Lua person, so I'm not sure how to fix this. Is it possible to "move back" to the old version? I really only need Lua for lmod.

ETA: Note: I made a discussion because I have no way of knowing if this is a reproducible error, thus I can't file an issue.

[carlocab]

Sorry about that. It passed CI tests and there were about a dozen other formulae complaining much more loudly about the update, so I didn't look too closely.

I'll have a look. Can you give me a complete set of repro steps? The output of brew config and brew doctor would also be helpful.

[carlocab]

Despite my PR, please still answer my questions regarding repro steps and the output of brew config and brew doctor. I need this in order to devise a test to make sure lmod doesn't break with future updates. Thank you.

[mathomp4]

Here's the output of the brew commands. First config:

❯ brew config
HOMEBREW_VERSION: 2.6.0
ORIGIN: https://github.com/Homebrew/brew
HEAD: 1d5e354cc2ff048bd7161d95b3fa7f91dc9dd081
Last commit: 2 days ago
Core tap ORIGIN: https://github.com/Homebrew/homebrew-core
Core tap HEAD: e24f5681e7e2a9667361686f2f4896f0c480af06
Core tap last commit: 3 hours ago
Core tap branch: master
HOMEBREW_PREFIX: /Users/mathomp4/.homebrew/brew
HOMEBREW_REPOSITORY: /Users/mathomp4/.homebrew/brew
HOMEBREW_CELLAR: /Users/mathomp4/.homebrew/brew/Cellar
HOMEBREW_CASK_OPTS: ["--appdir=~/Applications"]
HOMEBREW_DISPLAY: /private/tmp/com.apple.launchd.PEl3ZEs8dp/org.macosforge.xquartz:0
HOMEBREW_EDITOR: /Users/mathomp4/.homebrew/brew/bin/vim
HOMEBREW_MAKE_JOBS: 16
Homebrew Ruby: 2.6.3 => /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/bin/ruby
CPU: 16-core 64-bit kabylake
Clang: 12.0 build 1200
Git: 2.29.2 => /Users/mathomp4/.homebrew/brew/bin/git
Curl: 7.64.1 => /usr/bin/curl
Java: 11.0.9.1
macOS: 10.15.7-x86_64
CLT: 12.0.32.21
Xcode: 12.0.1
XQuartz: 2.7.11 => /opt/X11

and doctor:

❯ brew doctor
Please note that these warnings are just used to help the Homebrew maintainers
with debugging if you file an issue. If everything you use Homebrew for is
working fine: please don't worry or file an issue; just ignore this. Thanks!

Warning: A newer Command Line Tools release is available.
Update them from Software Update in System Preferences or run:
  softwareupdate --all --install --force

If that doesn't show you an update run:
  sudo rm -rf /Library/Developer/CommandLineTools
  sudo xcode-select --install

Alternatively, manually download them from:
  https://developer.apple.com/download/more/.


Warning: Some installed formulae are deprecated or disabled.
You should find replacements for the following formulae:
  guile@2

Warning: You have unlinked kegs in your Cellar.
Leaving kegs unlinked can lead to build-trouble and cause brews that depend on
those kegs to fail to run properly once built. Run `brew link` on these:
  [email protected]
  md5sha1sum

Warning: Your Homebrew's prefix is not /usr/local.
Some of Homebrew's bottles (binary packages) can only be used with the default
prefix (/usr/local).
You will encounter build failures with some formulae.
Please create pull requests instead of asking for help on Homebrew's GitHub,
Twitter or any other official channels. You are responsible for resolving
any issues you experience while you are running this
unsupported configuration.


Warning: Your Xcode (12.0.1) is outdated.
Please update to Xcode 12.2 (or delete it).
Xcode can be updated from the App Store.

Wow. Long.

Note: I have to do things outside of /usr/local because this isn't a personal Mac, so I have no rights to install anywhere else. But I have never encountered an issue so far!

[mathomp4]

As for how to reproduce, well, I guess you just need lmod as it currently is and try to run, say, module or ml:

❯ module
/Users/mathomp4/.homebrew/brew/opt/lua/bin/lua: version mismatch: app. needs 503.0, Lua core provides 504.0
stack traceback:
	[C]: in ?
	[C]: in function 'require'
	.../lmod/8.4.16/libexec/vendor/share/lua/5.3/posix/init.lua:23: in main chunk
	[C]: in function 'require'
	.../mathomp4/.homebrew/brew/Cellar/lmod/8.4.16/libexec/lmod:61: in main chunk
	[C]: in ?
❯ ml
/Users/mathomp4/.homebrew/brew/opt/lua/bin/lua: version mismatch: app. needs 503.0, Lua core provides 504.0
stack traceback:
	[C]: in ?
	[C]: in function 'require'
	.../lmod/8.4.16/libexec/vendor/share/lua/5.3/posix/init.lua:23: in main chunk
	[C]: in function 'require'
	...athomp4/.homebrew/brew/Cellar/lmod/8.4.16/libexec/ml_cmd:62: in main chunk
	[C]: in ?

ml is a sort of "helper" function for module

[carlocab]

Does module list also give you an error?

[SMillerDev]

If I install it locally source /usr/local/opt/lmod/init/profile already gives an error. Could you install your fixed version locally and see if that works?

[carlocab]

I've proposed a fix here: Homebrew/homebrew-core#66159

Does this work for you?

[carlocab]

Yes. Do HOMEBREW_NO_AUTO_UPDATE=1 brew install -s lmod.

[mathomp4]

Well, I found out about HOMEBREW_NO_AUTO_UPDATE=1 but now:

❯ gh pr checkout 66159
From https://github.com/Homebrew/homebrew-core
 ! [rejected]              refs/pull/66159/head -> lmod-fix  (non-fast-forward)
exit status 1

[carlocab]

I think it's refusing to do that because you made changes to your local core repo before doing gh pr checkout. It might be better to just stay on master and do it the quick and dirty (copy and paste) way instead of trying to get gh to work...

[mathomp4]

@carlocab Okay. So, when I "hand-enter" all of your changes and run HOMEBREW_NO_AUTO_UPDATE=1 brew install -s lmod, I get lmod back and working! Huzzah!

[carlocab]

Great. The changes should be merged soon. Thanks for your patience.

[mathomp4]

Note, also, you might try consulting @rtmclay (creator and master magician of https://github.com/TACC/Lmod itself) if you are looking for a good way to detect an issue with lmod/luarocks/etc. He might know some lower-level thing that I am unaware of. (I've bothered him a few times when I've broken something in lmod.)

[carlocab]

AFAICT, the fix to your problem is easy: just rebuild lmod. (Unless there's some sort of subtle incompatibility I haven't picked up on.)

What bothers me is that CI didn't catch this breakage, when it really should've.

[mathomp4]

Hmm. So uninstall and reinstall lmod? Or is there is brew rebuild like command?

[carlocab]

Sorry, the fix on this end. On your end that won't quite work yet, as there needs to be an update to the lmod formula.

[mathomp4]

I did just try brew reinstall lmod but no joy. I'll try with -s...

[mathomp4]

And after your reply, I guess that's what did this:

checking for valid Lua version... 5.4
checking for lua modules: posix

Error: The follow lua module(s) are missing:  posix

You can not run Lmod without:  posix

Do not report this issue to Homebrew/brew or Homebrew/core!

These open issues may also help:
lmod: rebuild luarocks https://github.com/Homebrew/homebrew-core/pull/66159

Error: Your Xcode (12.0.1) is outdated.
Please update to Xcode 12.2 (or delete it).
Xcode can be updated from the App Store.


Error: A newer Command Line Tools release is available.
Update them from Software Update in System Preferences or run:
  softwareupdate --all --install --force

If that doesn't show you an update run:
  sudo rm -rf /Library/Developer/CommandLineTools
  sudo xcode-select --install

Alternatively, manually download them from:
  https://developer.apple.com/download/more/.

I'll wait for further instructions, and, I guess, update my Xcode as brew seems annoyed!