Test splitting fails with "invalid timing type filename" despite docs stating it's valid #1150
Description
Meta:
CircleCI CLI Version:
$ circleci version
Build Agent version: 1.0.282855-5d940a9a
Built: 2025-06-25T14:29:19+0000
CircleCI Tests Plugin CLI Version:
Installing circleci-tests-plugin-cli plugin.
circleci-tests-plugin-cli plugin Installed. Version: 1.0.8767-cce2816
Operating System:
$ uname -a
Linux db1d44f44033 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 GNU/Linux
CircleCI CLI Diagnostic:
$ circleci diagnostic
Error: unknown command "diagnostic" for "circleci"
Run 'circleci --help' for usage.
ERRO[0000] Exit with status 1 due to generic error error="unknown command \"diagnostic\" for \"circleci\""
Exited with code exit status 1
Current behavior:
Running the following job step in CircleCI:
- run:
name: "Run RSpec Parallel"
command: |
circleci tests glob "./spec/**/*_spec.rb" | circleci tests run --command="xargs bundle exec rspec --format progress -r rspec_junit_formatter --format RspecJunitFormatter -o /tmp/repo/test_results/rspec/junit.xml" --verbose --split-by=timings --timings-type=filename
no_output_timeout: 20m
...now results in this error:
Installing circleci-tests-plugin-cli plugin.
circleci-tests-plugin-cli plugin Installed. Version: 1.0.8767-cce2816
DEBU[2025-06-25T17:16:49Z] Attempting to read from stdin. This will hang if no input is provided.
DEBU[2025-06-25T17:16:49Z] There was an error when creating the sorter: invalid timing type filename
Error: invalid timing type filename
ERRO[2025-06-25T17:16:49Z] invalid timing type filename
Error: exit status 1
Exited with code exit status 1
Expected behavior:
The command should accept --timings-type=filename as a valid value.
This used to work reliably in our CI pipeline, and the CircleCI documentation still mentions filename as a supported option under test splitting.
When did this begin / Was this previously working?:
This began failing on or around June 25th, 2025.
Previously, this exact job and configuration had been running successfully for months.
We're using the latest auto-installed version of the circleci-tests-plugin-cli:
Version: 1.0.8767-cce2816
And CircleCI CLI version:
circleci version
0.1.27921+abcdef
Additional Information:
Plugin seems to now validate --timings-type strictly and rejects filename.
If filename has been deprecated, could you please:
Confirm the officially supported values (name? classname?).
Update the official documentation to reflect this change.
Consider providing a clearer changelog or migration path for those affected.
Thanks!