Very high memory consumption during "Discovering workflow directives" phase in large Next.js applications

[bholbein]

When using the Workflow SDK (withWorkflow()) in a large Next.js 15 application, the "Discovering workflow directives" phase consumes excessive memory, causing builds to fail with OOM errors on standard
build machines (16 GB RAM).

Environment

• Next.js: 15.5.6
• Workflow SDK: 4.0.1-beta.24 (also tested with 4.0.1-beta.21)
• Node.js: 20
• Build Environment: Vercel (16 GB RAM machines)
• Platform: macOS (local), Linux (Vercel)

Problem

The SDK scans the entire app/ directory (and other conventional directories like pages/, src/app/, src/pages/) to discover files containing 'use workflow' and 'use step' directives. In our application with
914 TypeScript/TSX files in the app/ directory but only ~11 files actually using workflow directives, this causes:

1. 16x memory increase compared to builds without the workflow plugin
2. Build failures due to OOM on 16 GB machines
3. "Discovering workflow directives" phase taking ~30 seconds

Memory Metrics

Scenario	RSS Memory	Heap Used
Build without VERCEL=1 (no workflow plugin)	267 MB	90 MB
Build with VERCEL=1 (workflow plugin active)	4,465 MB	2,716 MB
Vercel preview build	~15.7 GB (98% of 16 GB)	-

Root Cause Analysis

The issue might stems from @workflow/builders in base-builder.js:

  async discoverEntries(inputs, outdir) {
    await esbuild.build({
      entryPoints: inputs,  // ALL files from app/, pages/, etc.
      bundle: true,         // Full bundle for discovery
      // ...
    });
  }

And in @workflow/next:

dirs: ['pages', 'app', 'src/pages', 'src/app'], // HARDCODED, not configurable

The SDK uses esbuild to bundle every file in these directories as entry points just to discover which ones contain workflow directives. For large applications, this causes significant memory consumption.

[pranaygp]

@ijjk seems we need to lazy approach?