Merge pull request #104 from mongodb/add-count-tested-examples-command

Audit CLI: Add new `count` command with two subcommands

Author: dacharyc

audit-cli/.gitignore

@@ -1,2 +1,3 @@
 audit-cli
 output/
+bin/

audit-cli/README.md

@@ -11,6 +11,7 @@ A Go CLI tool for extracting and analyzing code examples from MongoDB documentat
   - [Search Commands](#search-commands)
   - [Analyze Commands](#analyze-commands)
   - [Compare Commands](#compare-commands)
+  - [Count Commands](#count-commands)
 - [Development](#development)
   - [Project Structure](#project-structure)
   - [Adding New Commands](#adding-new-commands)
@@ -34,11 +35,11 @@ This CLI tool helps maintain code quality across MongoDB's documentation by:
 ### Build from Source
 
 ```bash
-cd audit-cli
-go build
+cd audit-cli/bin
+go build ../
 ```
 
-This creates an `audit-cli` executable in the current directory.
+This creates an `audit-cli` executable in the `bin` directory.
 
 ### Run Without Building
 
@@ -62,8 +63,12 @@ audit-cli
 │   ├── includes
 │   ├── usage
 │   └── procedures
-└── compare          # Compare files across versions
-    └── file-contents
+│   └── usage
+├── compare          # Compare files across versions
+│   └── file-contents
+└── count            # Count code examples and documentation pages
+    ├── tested-examples
+    └── pages
 ```
 
 ### Extract Commands
@@ -930,6 +935,188 @@ product-dir/
 Files that don't exist in certain versions are reported separately and do not cause errors. This is expected behavior
 since features may be added or removed across versions.
 
+### Count Commands
+
+#### `count tested-examples`
+
+Count tested code examples in the MongoDB documentation monorepo.
+
+This command navigates to the `content/code-examples/tested` directory from the monorepo root and counts all files recursively. The tested directory has a two-level structure: L1 (language directories) and L2 (product directories).
+
+**Use Cases:**
+
+This command helps writers and maintainers:
+- Track the total number of tested code examples
+- Monitor code example coverage by product
+- Identify products with few or many examples
+- Count only source files (excluding output files)
+
+**Basic Usage:**
+
+```bash
+# Get total count of all tested code examples
+./audit-cli count tested-examples /path/to/docs-monorepo
+
+# Count examples for a specific product
+./audit-cli count tested-examples /path/to/docs-monorepo --for-product pymongo
+
+# Show counts broken down by product
+./audit-cli count tested-examples /path/to/docs-monorepo --count-by-product
+
+# Count only source files (exclude .txt and .sh output files)
+./audit-cli count tested-examples /path/to/docs-monorepo --exclude-output
+```
+
+**Flags:**
+
+- `--for-product <product>` - Only count code examples for a specific product
+- `--count-by-product` - Display counts for each product
+- `--exclude-output` - Only count source files (exclude .txt and .sh files)
+
+**Current Valid Products:**
+
+- `mongosh` - MongoDB Shell
+- `csharp/driver` - C#/.NET Driver
+- `go/driver` - Go Driver
+- `go/atlas-sdk` - Atlas Go SDK
+- `java/driver-sync` - Java Sync Driver
+- `javascript/driver` - Node.js Driver
+- `pymongo` - PyMongo Driver
+
+**Output:**
+
+By default, prints a single integer (total count) for use in CI or scripting. With `--count-by-product`, displays a formatted table with product names and counts.
+
+#### `count pages`
+
+Count documentation pages (.txt files) in the MongoDB documentation monorepo.
+
+This command navigates to the `content` directory and recursively counts all `.txt` files, which represent documentation pages that resolve to unique URLs. The command automatically excludes certain directories and file types that don't represent actual documentation pages.
+
+**Use Cases:**
+
+This command helps writers and maintainers:
+- Track the total number of documentation pages across the monorepo
+- Monitor documentation coverage by product/project
+- Identify projects with extensive or minimal documentation
+- Exclude auto-generated or deprecated content from counts
+- Count only current versions of versioned documentation
+- Compare page counts across different documentation versions
+
+**Automatic Exclusions:**
+
+The command automatically excludes:
+- Files in `code-examples` directories at the root of `content` or `source` (these contain plain text examples, not pages)
+- Files in the following directories at the root of `content`:
+  - `404` - Error pages
+  - `docs-platform` - Documentation for the MongoDB website and meta content
+  - `meta` - MongoDB Meta Documentation - style guide, tools, etc.
+  - `table-of-contents` - Navigation files
+- All non-`.txt` files (configuration files, YAML, etc.)
+
+**Basic Usage:**
+
+```bash
+# Get total count of all documentation pages
+./audit-cli count pages /path/to/docs-monorepo
+
+# Count pages for a specific project
+./audit-cli count pages /path/to/docs-monorepo --for-project manual
+
+# Show counts broken down by project
+./audit-cli count pages /path/to/docs-monorepo --count-by-project
+
+# Exclude specific directories from counting
+./audit-cli count pages /path/to/docs-monorepo --exclude-dirs api-reference,generated
+
+# Count only current versions (for versioned projects)
+./audit-cli count pages /path/to/docs-monorepo --current-only
+
+# Show counts by project and version
+./audit-cli count pages /path/to/docs-monorepo --by-version
+
+# Combine flags: count pages for a specific project, excluding certain directories
+./audit-cli count pages /path/to/docs-monorepo --for-project atlas --exclude-dirs deprecated
+```
+
+**Flags:**
+
+- `--for-project <project>` - Only count pages for a specific project (directory name under `content/`)
+- `--count-by-project` - Display counts for each project in a formatted table
+- `--exclude-dirs <dirs>` - Comma-separated list of directory names to exclude from counting (e.g., `deprecated,archive`)
+- `--current-only` - Only count pages in the current version (for versioned projects, counts only `current` or `manual` version directories; for non-versioned projects, counts all pages)
+- `--by-version` - Display counts grouped by project and version (shows version breakdown for versioned projects; non-versioned projects show as "(no version)")
+
+**Output:**
+
+By default, prints a single integer (total count) for use in CI or scripting. With `--count-by-project`, displays a formatted table with project names and counts. With `--by-version`, displays a hierarchical breakdown by project and version.
+
+**Versioned Documentation:**
+
+Some MongoDB documentation projects contain multiple versions, represented as distinct directories between the project directory and the `source` directory:
+- **Versioned project structure**: `content/{project}/{version}/source/...`
+- **Non-versioned project structure**: `content/{project}/source/...`
+
+Version directory names follow these patterns:
+- `current` or `manual` - The current/latest version
+- `upcoming` - Pre-release version
+- `v{number}` - Specific version (e.g., `v8.0`, `v7.0`)
+
+The `--current-only` flag counts only files in the current version directory (`current` or `manual`) for versioned projects, while counting all files for non-versioned projects.
+
+The `--by-version` flag shows a breakdown of page counts for each version within each project.
+
+**Note:** The `--current-only` and `--by-version` flags are mutually exclusive.
+
+**Examples:**
+
+```bash
+# Quick count for CI/CD
+TOTAL_PAGES=$(./audit-cli count pages ~/docs-monorepo)
+echo "Total documentation pages: $TOTAL_PAGES"
+
+# Detailed breakdown by project
+./audit-cli count pages ~/docs-monorepo --count-by-project
+# Output:
+# Page Counts by Project:
+#
+#   app-services                       245
+#   atlas                              512
+#   manual                            1024
+#   ...
+#
+# Total: 2891
+
+# Count only Atlas pages
+./audit-cli count pages ~/docs-monorepo --for-project atlas
+# Output: 512
+
+# Exclude deprecated content
+./audit-cli count pages ~/docs-monorepo --exclude-dirs deprecated,archive --count-by-project
+
+# Count only current versions
+./audit-cli count pages ~/docs-monorepo --current-only
+# Output: 1245 (only counts current/manual versions)
+
+# Show breakdown by version
+./audit-cli count pages ~/docs-monorepo --by-version
+# Output:
+# Project: drivers
+#   manual                           150
+#   upcoming                         145
+#   v8.0                             140
+#   v7.0                             135
+#
+# Project: atlas
+#   (no version)                     200
+#
+# Total: 770
+
+# Count current version for a specific project
+./audit-cli count pages ~/docs-monorepo --for-project drivers --current-only
+# Output: 150
+```
+
 ## Development
 
 ### Project Structure
@@ -979,16 +1166,30 @@ audit-cli/
 │   │       ├── analyzer.go                  # Reference finding logic
 │   │       ├── output.go                    # Output formatting
 │   │       └── types.go                     # Type definitions
-│   └── compare/                             # Compare parent command
-│       ├── compare.go                       # Parent command definition
-│       └── file-contents/                   # File contents comparison subcommand
-│           ├── file_contents.go             # Command logic
-│           ├── file_contents_test.go        # Tests
-│           ├── comparer.go                  # Comparison logic
-│           ├── differ.go                    # Diff generation
+│   ├── compare/                             # Compare parent command
+│   │   ├── compare.go                       # Parent command definition
+│   │   └── file-contents/                   # File contents comparison subcommand
+│   │       ├── file_contents.go             # Command logic
+│   │       ├── file_contents_test.go        # Tests
+│   │       ├── comparer.go                  # Comparison logic
+│   │       ├── differ.go                    # Diff generation
+│   │       ├── output.go                    # Output formatting
+│   │       ├── types.go                     # Type definitions
+│   │       └── version_resolver.go          # Version path resolution
+│   └── count/                               # Count parent command
+│       ├── count.go                         # Parent command definition
+│       ├── tested-examples/                 # Tested examples counting subcommand
+│       │   ├── tested_examples.go           # Command logic
+│       │   ├── tested_examples_test.go      # Tests
+│       │   ├── counter.go                   # Counting logic
+│       │   ├── output.go                    # Output formatting
+│       │   └── types.go                     # Type definitions
+│       └── pages/                           # Pages counting subcommand
+│           ├── pages.go                     # Command logic
+│           ├── pages_test.go                # Tests
+│           ├── counter.go                   # Counting logic
 │           ├── output.go                    # Output formatting
-│           ├── types.go                     # Type definitions
-│           └── version_resolver.go          # Version path resolution
+│           └── types.go                     # Type definitions
 ├── internal/                                # Internal packages
 │   ├── pathresolver/                        # Path resolution utilities
 │   │   ├── pathresolver.go                  # Core path resolution
@@ -1014,12 +1215,14 @@ audit-cli/
     │       ├── includes/                    # Included RST files
     │       └── code-examples/               # Code files for literalinclude
     ├── expected-output/                     # Expected extraction results
-    └── compare/                             # Compare command test data
-        ├── product/                         # Version structure tests
-        │   ├── manual/                      # Manual version
-        │   ├── upcoming/                    # Upcoming version
-        │   └── v8.0/                        # v8.0 version
-        └── *.txt                            # Direct comparison tests
+    ├── compare/                             # Compare command test data
+    │   ├── product/                         # Version structure tests
+    │   │   ├── manual/                      # Manual version
+    │   │   ├── upcoming/                    # Upcoming version
+    │   │   └── v8.0/                        # v8.0 version
+    │   └── *.txt                            # Direct comparison tests
+    └── count-test-monorepo/                 # Count command test data
+        └── content/code-examples/tested/    # Tested examples structure
 ```
 
 ### Adding New Commands

audit-cli/commands/count/count.go

@@ -0,0 +1,36 @@
+// Package count provides the parent command for counting code examples and documentation pages.
+//
+// This package serves as the parent command for various counting operations.
+// Currently supports:
+//   - tested-examples: Count tested code examples in the MongoDB documentation monorepo
+//   - pages: Count documentation pages (.txt files) in the MongoDB documentation monorepo
+package count
+
+import (
+	"github.com/mongodb/code-example-tooling/audit-cli/commands/count/pages"
+	"github.com/mongodb/code-example-tooling/audit-cli/commands/count/tested-examples"
+	"github.com/spf13/cobra"
+)
+
+// NewCountCommand creates the count parent command.
+//
+// This command serves as a parent for various counting operations on code examples and documentation pages.
+// It doesn't perform any operations itself but provides a namespace for subcommands.
+func NewCountCommand() *cobra.Command {
+	cmd := &cobra.Command{
+		Use:   "count",
+		Short: "Count code examples and documentation pages",
+		Long: `Count various types of content in the MongoDB documentation.
+
+Currently supports:
+  - tested-examples: Count tested code examples in the documentation monorepo
+  - pages: Count documentation pages (.txt files) in the documentation monorepo`,
+	}
+
+	// Add subcommands
+	cmd.AddCommand(tested_examples.NewTestedExamplesCommand())
+	cmd.AddCommand(pages.NewPagesCommand())
+
+	return cmd
+}
+