Investigate differences in clang format in CI and Windows

[ann0see]

Describe the bug
@pljones and @pgScorpio experienced unexpected clang format failures: see #2550 and #2551
To Reproduce
?

Expected behavior
Clang format on Windows should format the same as on the CI. I expect some unknown incompatibility.

Screenshots

Operating system
Windows and Linux

Version of Jamulus
N/A
Additional context
@hoffie

[pljones]

I'm running it on Linux, not Windows.

$ clang-format --version
clang-format version 10.0.0-4ubuntu1

I don't remember if we pinned CI to an earlier or later version...

[ann0see]

Ok. Thought you were developing on Windows too.

jamulus/.github/workflows/coding-style-check.yml

Line 20 in 5609296

clangFormatVersion: 10

It’s clang format 10 probably? So something else must be wrong.

[pljones]

I just tried running the clang-format check workflow on my repo:
https://github.com/pljones/jamulus/runs/5702476683?check_suite_focus=true
run against
pljones@b995489

So it's weird...

[pgScorpio]

I'm using clang-format Version 13.0.1 on Windows and macOS and Version 10.0.0-4ubuntu1 on Linux.
Both give the same result locally, but often auto-build does not have the same result.

1. auto-build clang-format breaks long lines too soon (config says ColomnLimit 150, but auto-build clang-format breaks sooner (just try a line of 149 chars long.).
2. auto-build clang-format does sometimes not align braces with if correctly. (See: here)
3. auto-build clang-format sometimes changes type* variable or type *variable to type * variable, but not consistently. (See here)

[ann0see]

Thanks for your comment. If I find time for it I’ll check what my clang-format does (but it should also be clang-format 10)

@passing knows clang format well, so he might know what goes wrong.

Else it might be worth opening an issue on the clang format action repo

[dcorson-ticino-com]

I had the same problem on my Qt6 work.
clang-format all OK locally on Win10, had errors when run on github.

[hoffie]

I didn't have time to investigate all hints yet, but I mainly suspect discrepancies in clang-format versions to be the cause.

I've checked https://github.com/jamulussoftware/jamulus/runs/5704004785?check_suite_focus=true (e36d3d6):

• This fails in CI
• When using the same clang-format 10 binary as in CI, this fails locally for me as well.
• clang-format 13.0.1 feels that it is clean and should not be changed.

I don't have a good idea what to do about this though. We've decided against using a newer clang-format version in CI due to it not being widely available on Linux distribution packages. Maybe we should revisit this decision.

What still needs investigation the case where Ubuntu's clang 10 disagrees as well.

[pljones]

What is the output of clang-format --version for the CI clang-format?

[pgScorpio]

@hoffie

What still needs investigation the case where Ubuntu's clang 10 disagrees as well.

Indeed. As said I use Version 13.0.1 on Windows and macOS and Version 10.0.0-4ubuntu1 on Linux.

And locally they all agree.
I cross-checked all of them by running make clang_format on the exact same repo on a USB drive on all three OS'es.
Both macOS and Linux give no change at all on the Windows clang-formated repo, so they all give the same result.
I even wondered if clang-format was actually working, but after manually changing some files they where the same again after clang-formating, so clang-format does work ;=))
But still some files from that same repo failed in the auto-build...
So there seems to be playing something else than just clang-format binary versions.
Perhaps somehow using a different clang-format configuration for auto-build ?
But still then I really wonder how, since only the long line breaking seems consistent, the other abnormalities I can't explain either.

[hoffie]

@pljones:

What is the output of clang-format --version for the CI clang-format?

clang-format version 10.0.0 (https://github.com/muttleyxd/clang-format-static-binaries 5b56bb49b977b80cd18f44df4444db93e2d30f46)
Ubuntu 20.04's clang-format package (clang-format version 10.0.0-4ubuntu1) is consistent with CI's clang-format for me.

@pgScorpio:

I'm using clang-format Version 13.0.1 on Windows and macOS and Version 10.0.0-4ubuntu1 on Linux. Both give the same result locally, but often auto-build does not have the same result.

Getting the same output from 13.0.1 on both Windows and macOS sounds reasonable to me. Getting the same output with 10.0.0-4ubuntu1 in the mentioned cases sounds surprising to me based on my tests. Can you confirm that the binary which is invoked is indeed extactly the version you've mentioned? Maybe some editors include their own (more-recent) clang-format?

1. auto-build clang-format breaks long lines too soon (config says ColomnLimit 150, but auto-build clang-format breaks sooner (just try a line of 149 chars long.).

I just tried it as part of a variable assignment/function call and could not reproduce anything unexpected with 149 columns, either with clang-format 10 or 13. Maybe it requires specific circumstances to trigger. As far as I understand, your local formatting (= clang-format 13) provides the better/correct result, right?

2. auto-build clang-format does sometimes not align braces with if correctly. (See: here)

Yep. v13 output seems more logical.

3. auto-build clang-format sometimes changes type* variable or type *variable to type * variable, but not consistently. (See here)

llvm/llvm-project#27727 mentions formatting problems for pointer receivers when using AlignConsecutiveDeclarations (which we do). It's fixed by clang-format 13.

@pljones:

I just tried running the clang-format check workflow on my repo: https://github.com/pljones/jamulus/runs/5702476683?check_suite_focus=true run against pljones@b995489

The #1432 PR check ran on 69a2411 and failed.
Your manual check ran on b995489.
There are differences between these two trees. I'm not saying that they should be relevant, but that at least explains why "clang behavior differs between repos" (it doesn't -- it differs when the files differ). The behavior is consistent with both clang-format 10 and 13. Adding an empty line before // clang-format on works around this.

All in all I can confirm that all the mentioned strangeness is reproducible with clang-format 10 (both with the binary from CI, but not with clang-format 13).
I cannot confirm any discrepancies regarding different output with the same clang-format base version or with the same version in different contexts/repos.

There are huge numbers of Issues for clang-format.

I see two ways foward:

• Disable clang-format for the problematic places. I don't like that as we have to remember reverting that at some point.
• Update CI's clang-format to v13 (this should be trivial). The only downside would be that people on Linux distributions with older versions (e.g. Ubuntu) may have to get clang-format from somewhere else.

In any case, we should specify which clang-format version we require (and mayb where to get it).