Flaky E2E tests - Cannot resolve import("#app/nuxt-vitest-app-entry")

[JessicaSachs]

Environment

• Operating System: Darwin
• Node Version: v22.17.1
• Nuxt Version: 4.0.3
• CLI Version: 3.28.0
• Nitro Version: 2.12.4
• Package Manager: pnpm@10.13.1
• Builder: -
• User Config: compatibilityDate, devtools, modules
• Runtime Modules: @nuxt/test-utils@3.19.2
• Build Modules: -

Reproduction

https://github.com/JessicaSachs/nuxt-vitest-flake-repro

I'm looking to use host in order to point my e2e tests at an already-built Nuxt application. To run the suite, first build the application:

pnpm build

And then run the tests via:

pnpm test:e2e

Describe the bug

I'm unable to run tests reliably. The E2E test run exits after first suite (Either Node or Nuxt) passes. I get flake 20% of the time. My coworker gets it about 80% of the time. Pretty sure it's related to the machine speed.

Additional context

I've tried doing things like forcing Vitest's parallelization to turn off and poolOptions/maxThreads to be 1. Can't find a workaround rn.

Logs

FAIL   nuxt  test/e2e/home.test.ts [ test/e2e/home.test.ts ]
Error: Failed to resolve import "#app/nuxt-vitest-app-entry" from "node_modules/.pnpm/@nuxt+test-utils@3.19.2_@playwright+test@1.54.1_happy-dom@18.0.1_magicast@0.3.5_playwri_9da3baa23bc5990d9fca0e90d773c266/node_modules/@nuxt/test-utils/dist/runtime/shared/nuxt.mjs". Does the file exist?
  Plugin: vite:import-analysis
  File: /Users/jess/projects/jess-ecosystem/nuxt-vitest-flake-repro/node_modules/.pnpm/@nuxt+test-utils@3.19.2_@playwright+test@1.54.1_happy-dom@18.0.1_magicast@0.3.5_playwri_9da3baa23bc5990d9fca0e90d773c266/node_modules/@nuxt/test-utils/dist/runtime/shared/nuxt.mjs:3:15
  1  |  export async function setupNuxt() {
  2  |    const { useRouter } = await import("#app/composables/router");
  3  |    await import("#app/nuxt-vitest-app-entry").then((r) => r.default());
     |                 ^
  4  |    const nuxtApp = useNuxtApp();
  5  |    await nuxtApp.callHook("page:finish");

[JessicaSachs]

I attempted to create an example within the Nuxt Test Utils repo under examples/. Pulling in the reproduction I posted above works fine within the Test Utils monorepo. I'm unable to cause a single failure within the monorepo.

I'm unsure what's going on.

I also ensured that all versions matched up between my broken and flaky reproduction and the Test Utils Monorepo resolved versions.

[JessicaSachs]

I downgraded. I think the workspaces changes ended up messing things up. I had quite a lot of flake - including this other error which I didn't report above.

The other thing I noticed in the downgrade is that the glob matcher I was using include: ['app/test/e2e/*.test.{ts,js}'], no longer spawned both a node and nuxt environment to start.

☠️ this was a tough one, fam. (Edit: Sorry - IDK why it marked as closed. This is very much a real issue.)

[danielroe]

you're running the test/e2e test in the node AND in the nuxt runtime environment.

that's why it's failing. it should only be run in the node environment.

I've updated the docs here - if you follow that project-based approach, that should resolve things.

[JessicaSachs]

This is because of the glob here:

The other thing I noticed in the downgrade is that the glob matcher I was using include: ['app/test/e2e/*.test.{ts,js}'], no longer spawned both a node and nuxt environment to start.

which causes an expansion of both environments for whatever reason starting in 3.19.x

[JessicaSachs]

From Discord, the resolution:

danielroe — 9:32 AM
are you using defineVitestConfig?

you should no longer use it - the example I provided in the setup now shows how to use defineConfig from vitest/config + then only use the nuxt defineProject within a projects array.

I think you'll find stuff just starts working

(having defineVitestConfig useable for the node or shared config caused a loooot of issues, even though this is what we originally documented)

Things did, indeed, start working. 🙏🏻 💯