test-server.ts should not be exported from `ai/test`

[mydea]

Description

If you import from ai/test, you get some nice test utils to test the ai package.

However, this seems to re-export from test-server.ts which seems to me as if it was an actual test, not a test util? This breaks somehow in our library now because it complains about the msw dependency and vitest not being setup etc 🤔 Is this exported here: https://github.com/vercel/ai/blob/37ae9e715502bcddaaecf46c06bc89614b8051ee/packages/provider-utils/src/test/index.ts on purpose?

AI SDK Version

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[mydea]

To be honest I am not 100% sure what is going on there, this does not seem to have changed recently, it just stuck out to me as weird/possibly unintended while I tried to debug our problem... we are seeing this error since a few hours, and are not quite sure what has changed to trigger this 😅

Error: Cannot find module 'msw'
Require stack:
- /sentry-javascript/dev-packages/node-integration-tests/suites/tracing/vercelai/v5/tmp_mf6url44_7fvj7n/node_modules/@ai-sdk/provider-utils/dist/test/index.js
- /sentry-javascript/dev-packages/node-integration-tests/suites/tracing/vercelai/v5/tmp_mf6url44_7fvj7n/node_modules/ai/dist/test/index.js
- /sentry-javascript/dev-packages/node-integration-tests/suites/tracing/vercelai/v5/tmp_mf6url44_7fvj7n/scenario.cjs
    at Module._resolveFilename (node:internal/modules/cjs/loader:1212:15)
    at Module._load (node:internal/modules/cjs/loader:1043:27)
    at Module.require (node:internal/modules/cjs/loader:1298:19)
    at Hook._require.Module.require (/sentry-javascript/node_modules/require-in-the-middle/index.js:167:34)
    at Hook._require.Module.require (/sentry-javascript/node_modules/require-in-the-middle/index.js:167:34)
    at require (node:internal/modules/helpers:182:18)
    at Object.<anonymous> (/sentry-javascript/dev-packages/node-integration-tests/suites/tracing/vercelai/v5/tmp_mf6url44_7fvj7n/node_modules/@ai-sdk/provider-utils/dist/test/index.js:105:18)

[gvkhna]

Can confirm got a few related errors, seemed like an import issue so rolled back a couple versions.

[gr2m]

That seems odd indeed.

We do import createTestServer in from @ai-sdk/provider-utils in many testfiles across several providers. Just removing the export causes problems

pnpm type-check:full

> type-check:full
> tsc --build tsconfig.with-examples.json

packages/gateway/src/gateway-embedding-model.test.ts:2:10 - error TS2305: Module '"@ai-sdk/provider-utils/test"' has no exported member 'createTestServer'.

2 import { createTestServer } from '@ai-sdk/provider-utils/test';
           ~~~~~~~~~~~~~~~~

packages/gateway/src/gateway-fetch-metadata.test.ts:1:10 - error TS2305: Module '"@ai-sdk/provider-utils/test"' has no exported member 'createTestServer'.

1 import { createTestServer } from '@ai-sdk/provider-utils/test';
           ~~~~~~~~~~~~~~~~

packages/gateway/src/gateway-language-model.test.ts:7:3 - error TS2305: Module '"@ai-sdk/provider-utils/test"' has no exported member 'createTestServer'.

7   createTestServer,
    ~~~~~~~~~~~~~~~~

packages/ai/src/tool/mcp/mcp-sse-transport.test.ts:2:3 - error TS2305: Module '"@ai-sdk/provider-utils/test"' has no exported member 'createTestServer'.

2   createTestServer,
    ~~~~~~~~~~~~~~~~

packages/ai/src/tool/mcp/mcp-sse-transport.test.ts:3:3 - error TS2305: Module '"@ai-sdk/provider-utils/test"' has no exported member 'TestResponseController'.

3   TestResponseController,
    ~~~~~~~~~~~~~~~~~~~~~~

packages/ai/src/ui/chat.test.ts:2:3 - error TS2305: Module '"@ai-sdk/provider-utils/test"' has no exported member 'createTestServer'.

2   createTestServer,
    ~~~~~~~~~~~~~~~~

packages/ai/src/ui/chat.test.ts:4:3 - error TS2305: Module '"@ai-sdk/provider-utils/test"' has no exported member 'TestResponseController'.

4   TestResponseController,
    ~~~~~~~~~~~~~~~~~~~~~~

packages/ai/src/ui/http-chat-transport.test.ts:1:10 - error TS2305: Module '"@ai-sdk/provider-utils/test"' has no exported member 'createTestServer'.

1 import { createTestServer } from '@ai-sdk/provider-utils/test';
           ~~~~~~~~~~~~~~~~

packages/ai/src/util/download/download.test.ts:1:10 - error TS2305: Module '"@ai-sdk/provider-utils/test"' has no exported member 'createTestServer'.

1 import { createTestServer } from '@ai-sdk/provider-utils/test';
           ~~~~~~~~~~~~~~~~

packages/anthropic/src/anthropic-messages-language-model.test.ts:8:3 - error TS2305: Module '"@ai-sdk/provider-utils/test"' has no exported member 'createTestServer'.

8   createTestServer,
    ~~~~~~~~~~~~~~~~

packages/amazon-bedrock/src/bedrock-chat-language-model.test.ts:3:3 - error TS2305: Module '"@ai-sdk/provider-utils/test"' has no exported member 'createTestServer'.

3   createTestServer,
    ~~~~~~~~~~~~~~~~

packages/amazon-bedrock/src/bedrock-embedding-model.test.ts:1:10 - error TS2305: Module '"@ai-sdk/provider-utils/test"' has no exported member 'createTestServer'.

1 import { createTestServer } from '@ai-sdk/provider-utils/test';
           ~~~~~~~~~~~~~~~~

packages/amazon-bedrock/src/bedrock-image-model.test.ts:1:10 - error TS2305: Module '"@ai-sdk/provider-utils/test"' has no exported member 'createTestServer'.

1 import { createTestServer } from '@ai-sdk/provider-utils/test';
           ~~~~~~~~~~~~~~~~

packages/angular/src/lib/chat.ng.test.ts:2:3 - error TS2305: Module '"@ai-sdk/provider-utils/test"' has no exported member 'createTestServer'.

2   createTestServer,
    ~~~~~~~~~~~~~~~~

packages/angular/src/lib/chat.ng.test.ts:4:3 - error TS2305: Module '"@ai-sdk/provider-utils/test"' has no exported member 'TestResponseController'.

4   TestResponseController,
    ~~~~~~~~~~~~~~~~~~~~~~

packages/angular/src/lib/completion.ng.test.ts:2:3 - error TS2305: Module '"@ai-sdk/provider-utils/test"' has no exported member 'createTestServer'.

2   createTestServer,
    ~~~~~~~~~~~~~~~~

packages/angular/src/lib/completion.ng.test.ts:3:3 - error TS2305: Module '"@ai-sdk/provider-utils/test"' has no exported member 'TestResponseController'.

3   TestResponseController,
    ~~~~~~~~~~~~~~~~~~~~~~

packages/angular/src/lib/structured-object.ng.test.ts:2:3 - error TS2305: Module '"@ai-sdk/provider-utils/test"' has no exported member 'createTestServer'.

2   createTestServer,
    ~~~~~~~~~~~~~~~~

packages/angular/src/lib/structured-object.ng.test.ts:3:3 - error TS2305: Module '"@ai-sdk/provider-utils/test"' has no exported member 'TestResponseController'.

3   TestResponseController,
    ~~~~~~~~~~~~~~~~~~~~~~

packages/assemblyai/src/assemblyai-transcription-model.test.ts:1:10 - error TS2305: Module '"@ai-sdk/provider-utils/test"' has no exported member 'createTestServer'.

1 import { createTestServer } from '@ai-sdk/provider-utils/test';
           ~~~~~~~~~~~~~~~~

packages/openai/src/chat/openai-chat-language-model.test.ts:4:3 - error TS2305: Module '"@ai-sdk/provider-utils/test"' has no exported member 'createTestServer'.

4   createTestServer,
    ~~~~~~~~~~~~~~~~

packages/openai/src/completion/openai-completion-language-model.test.ts:4:3 - error TS2305: Module '"@ai-sdk/provider-utils/test"' has no exported member 'createTestServer'.

4   createTestServer,
    ~~~~~~~~~~~~~~~~

packages/openai/src/embedding/openai-embedding-model.test.ts:2:10 - error TS2305: Module '"@ai-sdk/provider-utils/test"' has no exported member 'createTestServer'.

2 import { createTestServer } from '@ai-sdk/provider-utils/test';
           ~~~~~~~~~~~~~~~~

packages/openai/src/image/openai-image-model.test.ts:1:10 - error TS2305: Module '"@ai-sdk/provider-utils/test"' has no exported member 'createTestServer'.

1 import { createTestServer } from '@ai-sdk/provider-utils/test';
           ~~~~~~~~~~~~~~~~

packages/openai/src/responses/openai-responses-language-model.test.ts:7:3 - error TS2305: Module '"@ai-sdk/provider-utils/test"' has no exported member 'createTestServer'.

7   createTestServer,
    ~~~~~~~~~~~~~~~~

packages/openai/src/speech/openai-speech-model.test.ts:1:10 - error TS2305: Module '"@ai-sdk/provider-utils/test"' has no exported member 'createTestServer'.

1 import { createTestServer } from '@ai-sdk/provider-utils/test';
           ~~~~~~~~~~~~~~~~

packages/openai/src/transcription/openai-transcription-model.test.ts:1:10 - error TS2305: Module '"@ai-sdk/provider-utils/test"' has no exported member 'createTestServer'.

1 import { createTestServer } from '@ai-sdk/provider-utils/test';
           ~~~~~~~~~~~~~~~~

packages/azure/src/azure-openai-provider.test.ts:5:10 - error TS2305: Module '"@ai-sdk/provider-utils/test"' has no exported member 'createTestServer'.

5 import { createTestServer } from '@ai-sdk/provider-utils/test';
           ~~~~~~~~~~~~~~~~

packages/openai-compatible/src/chat/openai-compatible-chat-language-model.test.ts:5:3 - error TS2305: Module '"@ai-sdk/provider-utils/test"' has no exported member 'createTestServer'.

5   createTestServer,
    ~~~~~~~~~~~~~~~~

packages/openai-compatible/src/completion/openai-compatible-completion-language-model.test.ts:5:3 - error TS2305: Module '"@ai-sdk/provider-utils/test"' has no exported member 'createTestServer'.

5   createTestServer,
    ~~~~~~~~~~~~~~~~

packages/openai-compatible/src/embedding/openai-compatible-embedding-model.test.ts:3:10 - error TS2305: Module '"@ai-sdk/provider-utils/test"' has no exported member 'createTestServer'.

3 import { createTestServer } from '@ai-sdk/provider-utils/test';
           ~~~~~~~~~~~~~~~~

packages/openai-compatible/src/image/openai-compatible-image-model.test.ts:3:10 - error TS2305: Module '"@ai-sdk/provider-utils/test"' has no exported member 'createTestServer'.

3 import { createTestServer } from '@ai-sdk/provider-utils/test';

What we added more recently was the explicit import of the vitest methods, that might explain why you suddenly see the problems: #8277

I think we need to refactor how packages/provider-utils/src/test/test-server.ts works without depending on vitest methods

[alvarosevilla95]

Yeah this seems to break my tanstack start app, import MockLanguageModelV2 from ai-test, and when I run the app I run into this error:

Error: Vitest failed to access its internal state.

One of the following is possible:
- "vitest" is imported directly without running "vitest" command
- "vitest" is imported inside "globalSetup" (to fix this, use "setupFiles" instead, because "globalSetup" 
runs in a different context)
- "vitest" is imported inside Vite / Vitest config file
- Otherwise, it might be a Vitest bug. Please report it to https://github.com/vitest-dev/vitest/issues

[aayush-kapoor]

looking into this!

no reproduction of error needed anymore