WARNING: THIS SITE IS A MIRROR OF GITHUB.COM / IT CANNOT LOGIN OR REGISTER ACCOUNTS / THE CONTENTS ARE PROVIDED AS-IS / THIS SITE ASSUMES NO RESPONSIBILITY FOR ANY DISPLAYED CONTENT OR LINKS / IF YOU FOUND SOMETHING MAY NOT GOOD FOR EVERYONE, CONTACT ADMIN AT ilovescratch@foxmail.com
Skip to content

Conversation

@AryanBagade
Copy link

@AryanBagade AryanBagade commented Dec 4, 2025

Adds language model support to Replicate provider with support for popular models including Meta Llama 3.x, Mistral, DeepSeek, and Qwen.

  • Implements ReplicateLanguageModel class
  • Supports standard AI SDK parameters
  • Maintains backward compatibility with image models
  • Includes comprehensive documentation

Fixes #10257

Background

Replicate has been adding many language models (Meta Llama 3.x, Mistral, DeepSeek, etc.) that weren't supported in the AI SDK. The Replicate provider only supported image generation, nd the languageModel() method threw a NoSuchModelError. This prevented users/developers from using Replicate's extensive catalog of language models through the AI SDK.

Summary

This PR adds comprehensive language model support to the Replicate provider by:

  • Creating ReplicateLanguageModel class that implements LanguageModelV3
  • Adding ReplicateLanguageModelId type with 20+ popular models pre-configured ( open for reviewers to change later )
  • Implementing doGenerate() for non-streaming text generation via Replicate's predictions API
  • Implementing doStream() with basic streaming support (marked as experimental)
  • Supporting all standard AI SDK parameters (temperature, topP, topK, maxOutputTokens, seed, stopSequences)
  • Supporting provider-specific options via providerOptions.replicate
  • Updating documentation with language model usage examples
  • Maintaining full backward compatibility with existing image model functionality

Files Changed:

  • packages/replicate/src/replicate-language-model.ts (new)
  • Core implementation
  • packages/replicate/src/replicate-language-settings.ts (new) - Model type definitions
  • packages/replicate/src/replicate-provider.ts (modified) - Added languageModel() method
  • packages/replicate/src/index.ts (modified) - Exported new types
  • packages/replicate/README.md (modified) - Added language model documentation

Verification Steps:

  1. Built the package successfully:

    pnpm --filter @ai-sdk/replicate build
    # Build completed with no errors
    
  2. ✅ Verified module exports and types:

    • Created test script to verify all exports work correctly
    • Confirmed replicate.languageModel() method exists and
      returns proper model instance
    • Verified model implements all required LanguageModelV3
      methods: doGenerate, doStream, supportedUrls
  3. ✅ Tested with multiple model IDs:

    • meta/meta-llama-3.1-405b-instruct ✅
    • meta/meta-llama-3.1-70b-instruct ✅
    • mistralai/mixtral-8x7b-instruct-v0.1 ✅
    • deepseek-ai/deepseek-r1 ✅
    • Custom versioned model: custom-user/model:version123 ✅
  4. ✅ Verified backward compatibility:

    • Image models still work: replicate.image('black-forest-labs/flux-schnell')
    • No breaking changes to existing API
  5. ✅ Verified error handling:

    • Unsupported models (embeddingModel) correctly throw NoSuchModelError
  6. ✅ Ran quality checks:

    • TypeScript type-check: pnpm type-check:full ✅
    • ESLint on source files: All pass ✅
    • Prettier formatting: All files formatted correctly ✅

Checklist

  • Tests have been added / updated (for bug fixes / features)
  • Documentation has been added / updated (for bug fixes / features)
  • A patch changeset for relevant packages has been added (for bug fixes / features - run pnpm changeset in the project root)
  • I have reviewed this pull request (self-review)

Note on testing: Automated unit tests have not been added yet. The implementation has been manually tested. I'm happy to add unit tests following the patterns in replicate-image-model.test.ts if the maintainers would like and my approach and is correct!!!

Future Work

I believe following enhancements could be added in future PRs if desired:

  • Full streaming implementation: as of now streaming returns a placeholder message. A complete implementation would connect to Replicate's Server-Sent Events (SSE) stream URL and parse events in real-time.
  • Token usage reporting: Replicate API doesn't currently return token counts in responses. Could be added if/when Replicate provides this data.
  • Additional model presets: Can add more model IDs to ReplicateLanguageModelId as new models are released (also reviewers can change existing models as well

Adds language model support to Replicate provider with support for popular models including Meta Llama 3.x, Mistral, DeepSeek, and Qwen.

- Implements ReplicateLanguageModel class
- Supports standard AI SDK parameters
- Maintains backward compatibility with image models
- Includes comprehensive documentation

Fixes vercel#10257
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add Replicate Language Models Support

1 participant