✨ feat: support anthropic as model provider #1409

danielglh · 2024-02-28T04:33:13Z

💻 变更类型 | Change Type

🔀 变更说明 | Description of Change

Added a new model provider, Anthropic, along with the 3 models it currently provides.

close #1401

📝 补充信息 | Additional Information

Anthropic API Geographic Restrictions

Here is a list of locations that Anthropic API supports: https://docs.anthropic.com/claude/reference/supported-regions.

If a lobe-chat service happens to be deployed to a location that it doesn't support, it will not work and it will return 403 errors. For example, you may notice that it doesn't always work on Vercel.ai because Vercel deploys code to its edge server around the world and usually uses the edge server that is closest to your location to serve your requests, and some of its edge servers may not be allowed to make requests to Anthropic API.

Different API Parameters

As far as I know, Anthropic API doesn't seem to support these parameters: presence_penalty, frequency_penalty. Therefore these settings do not affect Anthropic models.

vercel · 2024-02-28T04:33:16Z

@danielglh is attempting to deploy a commit to the LobeHub Team on Vercel.

A member of the Team first needs to authorize it.

lobehubbot · 2024-02-28T04:33:24Z

👍 @danielglh

Thank you for raising your pull request and contributing to our Community
Please make sure you have followed our contributing guidelines. We will review it as soon as possible.
If you encounter any problems, please feel free to connect with us.
非常感谢您提出拉取请求并为我们的社区做出贡献，请确保您已经遵循了我们的贡献指南，我们会尽快审查它。
如果您遇到任何问题，请随时与我们联系。

codecov · 2024-02-28T17:47:54Z

Codecov Report

Attention: Patch coverage is 86.92308% with 34 lines in your changes are missing coverage. Please review.

Project coverage is 92.37%. Comparing base (aaedfa7) to head (035e6dd).
Report is 2 commits behind head on main.

Files	Patch %	Lines
src/libs/agent-runtime/anthropic/index.ts	82.35%	21 Missing ⚠️
src/app/api/chat/[provider]/agentRuntime.ts	38.46%	8 Missing ⚠️
src/app/api/errorResponse.ts	33.33%	2 Missing ⚠️
src/services/_auth.ts	50.00%	2 Missing ⚠️
src/services/file.ts	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1409      +/-   ##
==========================================
- Coverage   92.48%   92.37%   -0.12%     
==========================================
  Files         227      230       +3     
  Lines       12022    12278     +256     
  Branches     1457     1483      +26     
==========================================
+ Hits        11119    11342     +223     
- Misses        903      936      +33

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

arvinxx

Thanks for your PR! A really nice work. Most of codes are LGTM. Please rebase the main branch and add a test for this provider~ (you can refer to MoonshotAI 's test)

src/features/Conversation/Error/APIKeyForm/Anthropic.tsx

danielglh · 2024-02-29T08:49:41Z

Thanks for your PR! A really nice work. Most of codes are LGTM. Please rebase the main branch and add a test for this provider~ (you can refer to MoonshotAI 's test)

Cool. I've rebased main branch and made necessary changes. Regarding test, Anthropic API is not compatible with OpenAI API so it probably takes more time to work on it. I noticed that other providers that are not compatible with OpenAI API don't have tests either. Not sure where to find a better reference.

arvinxx · 2024-02-29T09:33:20Z

@danielglh I will add the google provider test soon. And you can use the simmliar method to add tests:

danielglh · 2024-02-29T09:51:32Z

@arvinxx sure will take a look

danielglh · 2024-02-29T15:10:48Z

Hmm probably need to take some time to check how the streaming should be implemented. For some reason it's quite unstable.

arvinxx · 2024-02-29T15:39:41Z

@danielglh what's the mean of the streaming is unstable?

danielglh · 2024-03-01T01:46:34Z

@danielglh what's the mean of the streaming is unstable?

@arvinxx for some reason the response stream was cut randomly when it was deployed to Vercel for preview

danielglh · 2024-03-01T03:41:41Z

@arvinxx I think it might be limitations of the edge servers Vercel uses for preview. The streaming works perfectly fine when I run lobe-chat locally and on a remote dev server I have.

arvinxx · 2024-03-01T04:33:10Z

@danielglh I know there is a timeout of stream on vercel, but I don't know the accurate value.

shigomany · 2024-03-01T19:35:47Z

locales/ru-RU/setting.json

@@ -21,6 +21,14 @@
    "sessionWithName": "Настройки сеанса · {{name}}"
  },
  "llm": {
+    "Anthropic": {
+      "title": "Антропный",


I think it's not worth translating, but leaving the Anthropic. So that it doesn't seem like it's a machine translation

@shigomany I agree but this seems to be a general issue to solve in the i18n project. Other providers' titles were mostly translated already

I consider to fix it later

danielglh · 2024-03-03T03:53:07Z

Thanks for your PR! A really nice work. Most of codes are LGTM. Please rebase the main branch and add a test for this provider~ (you can refer to MoonshotAI 's test)

@arvinxx tests added for Anthropic, plz check again

ckt1031 · 2024-03-04T15:14:34Z

Now claude-3.0 is released

arvinxx · 2024-03-04T16:33:46Z

@ckt1031 yes, @danielglh can we support Anthropic's new model in the PR?

src/libs/agent-runtime/anthropic/index.ts

ckt1031 · 2024-03-04T16:38:18Z

@ckt1031 yes, @danielglh can we support Anthropic's new model in the PR?

I believe not only adding the new model, but also support vision feature by Claude 3.0 😅.

arvinxx · 2024-03-05T00:51:03Z

@ckt1031 yes, @danielglh can we support Anthropic's new model in the PR?

I believe not only adding the new model, but also support vision feature by Claude 3.0 😅.

I have supported the vision

arvinxx · 2024-03-05T02:54:14Z

@danielglh can you fix the test of anthropic test? I think the rest is well done enough to ship.

vercel · 2024-03-05T03:00:56Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
lobe-chat	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Mar 5, 2024 2:39pm

danielglh · 2024-03-05T08:03:13Z

@danielglh can you fix the test of anthropic test? I think the rest is well done enough to ship.

@arvinxx tests fixed

danielglh · 2024-03-05T08:44:50Z

@arvinxx I'm testing this on Vercel and still experiencing weird streaming problems:

Any suggestions? Cannot reproduce the issue in the local dev environment. It runs perfectly fine locally.

danielglh · 2024-03-05T08:53:51Z

Found it ... not sure if it is a Vercel issue or an Anthropic SDK issue yet.

anthropics/anthropic-sdk-typescript#292

I would suggest that we go ahead and ship this first. Not everyone uses Vercel. We just need to make it clear that there's a known issue with Vercel (maybe include it in the document?).

arvinxx · 2024-03-05T12:12:16Z

@danielglh I agree.

danielglh · 2024-03-05T13:07:40Z

@arvinxx documents updated regarding the known issue with Anthropic SDK on Vercel/Cloudflare edge runtime.

lobehubbot · 2024-03-05T15:24:11Z

❤️ Great PR @danielglh ❤️

The growth of project is inseparable from user feedback and contribution, thanks for your contribution! If you are interesting with the lobehub developer community, please join our discord and then dm @arvinxx or @canisminor1990. They will invite you to our private developer channel. We are talking about the lobe-chat development or sharing ai newsletter around the world.
项目的成长离不开用户反馈和贡献，感谢您的贡献! 如果您对 LobeHub 开发者社区感兴趣，请加入我们的 discord，然后私信 @arvinxx 或 @canisminor1990。他们会邀请您加入我们的私密开发者频道。我们将会讨论关于 Lobe Chat 的开发，分享和讨论全球范围内的 AI 消息。

## [Version 0.132.0](v0.131.0...v0.132.0) <sup>Released on **2024-03-05**</sup> #### ✨ Features - **misc**: Support anthropic as model provider. <br/> <details> <summary><kbd>Improvements and Fixes</kbd></summary> #### What's improved * **misc**: Support anthropic as model provider, closes [#1409](#1409) ([a42cf8c](a42cf8c)) </details> <div align="right"> [![](https://img.shields.io/badge/-BACK_TO_TOP-151515?style=flat-square)](#readme-top) </div>

lobehubbot · 2024-03-05T15:27:34Z

🎉 This PR is included in version 0.132.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

* ✨ feat: support anthropic as model provider * ✨ feat: support anthropic with vision * ✅ test: fix some with test * 🐛 fix: anthropic error handling * ✅ chore: fix broken tests * 🐛 fix: remove unused imports * 📝 doc: add note for anthropic with vercel/cloudflare * 📝 docs: update docs * 📝 docs: update docs * 💄 style: update claude icons * 💄 style: update tags --------- Co-authored-by: arvinxx <[email protected]>

## [Version 0.132.0](lobehub/lobe-chat@v0.131.0...v0.132.0) <sup>Released on **2024-03-05**</sup> #### ✨ Features - **misc**: Support anthropic as model provider. <br/> <details> <summary><kbd>Improvements and Fixes</kbd></summary> #### What's improved * **misc**: Support anthropic as model provider, closes [lobehub#1409](lobehub#1409) ([a42cf8c](lobehub@a42cf8c)) </details> <div align="right"> [![](https://img.shields.io/badge/-BACK_TO_TOP-151515?style=flat-square)](#readme-top) </div>

## [Version 1.14.0](v1.13.0...v1.14.0) <sup>Released on **2024-03-05**</sup> #### ✨ Features - **misc**: Support anthropic as model provider. <br/> <details> <summary><kbd>Improvements and Fixes</kbd></summary> #### What's improved * **misc**: Support anthropic as model provider, closes [lobehub#1409](https://github.com/bentwnghk/lobe-chat/issues/1409) ([a42cf8c](a42cf8c)) </details> <div align="right"> [![](https://img.shields.io/badge/-BACK_TO_TOP-151515?style=flat-square)](#readme-top) </div>

## [Version 0.132.0](lobehub/lobe-chat@v0.131.0...v0.132.0) <sup>Released on **2024-03-05**</sup> #### ✨ Features - **misc**: Support anthropic as model provider. <br/> <details> <summary><kbd>Improvements and Fixes</kbd></summary> #### What's improved * **misc**: Support anthropic as model provider, closes [#1409](lobehub/lobe-chat#1409) ([a42cf8c](lobehub/lobe-chat@a42cf8c)) </details> <div align="right"> [![](https://img.shields.io/badge/-BACK_TO_TOP-151515?style=flat-square)](#readme-top) </div>

arvinxx requested changes Feb 29, 2024

View reviewed changes

src/features/Conversation/Error/APIKeyForm/Anthropic.tsx Outdated Show resolved Hide resolved

danielglh force-pushed the feat-anthropic-models branch 3 times, most recently from 9c22cfd to 5b108d7 Compare February 29, 2024 08:46

danielglh force-pushed the feat-anthropic-models branch from 3f6461b to e75b380 Compare February 29, 2024 14:58

shigomany reviewed Mar 1, 2024

View reviewed changes

danielglh force-pushed the feat-anthropic-models branch 3 times, most recently from 4ad0a01 to 4bce423 Compare March 3, 2024 03:52

danielglh force-pushed the feat-anthropic-models branch from 4bce423 to ab8249b Compare March 3, 2024 07:54

arvinxx reviewed Mar 4, 2024

View reviewed changes

src/libs/agent-runtime/anthropic/index.ts Outdated Show resolved Hide resolved

arvinxx reviewed Mar 4, 2024

View reviewed changes

src/libs/agent-runtime/anthropic/index.ts Outdated Show resolved Hide resolved

arvinxx reviewed Mar 4, 2024

View reviewed changes

src/libs/agent-runtime/anthropic/index.ts Outdated Show resolved Hide resolved

arvinxx force-pushed the feat-anthropic-models branch from ab8249b to 5af03f6 Compare March 4, 2024 18:23

vercel bot deployed to Preview March 5, 2024 03:06 View deployment

danielglh and others added 5 commits March 5, 2024 15:57

✨ feat: support anthropic as model provider

45783c6

✨ feat: support anthropic with vision

5a3877b

✅ test: fix some with test

165b71c

🐛 fix: anthropic error handling

36e4760

✅ chore: fix broken tests

206fb76

danielglh force-pushed the feat-anthropic-models branch from da692a1 to 206fb76 Compare March 5, 2024 07:59

🐛 fix: remove unused imports

554c1f4

📝 doc: add note for anthropic with vercel/cloudflare

4a490ab

arvinxx added 3 commits March 5, 2024 22:13

📝 docs: update docs

95bc285

📝 docs: update docs

9385aea

💄 style: update claude icons

8e30806

vercel bot deployed to Preview March 5, 2024 14:39 View deployment

💄 style: update tags

035e6dd

arvinxx merged commit a42cf8c into lobehub:main Mar 5, 2024
2 of 5 checks passed

lobehubbot added the released label Mar 5, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

✨ feat: support anthropic as model provider #1409

✨ feat: support anthropic as model provider #1409

danielglh commented Feb 28, 2024 •

edited by arvinxx

Loading

vercel bot commented Feb 28, 2024

lobehubbot commented Feb 28, 2024

codecov bot commented Feb 28, 2024 •

edited

Loading

arvinxx left a comment •

edited

Loading

danielglh commented Feb 29, 2024

arvinxx commented Feb 29, 2024 •

edited

Loading

danielglh commented Feb 29, 2024

danielglh commented Feb 29, 2024

arvinxx commented Feb 29, 2024

danielglh commented Mar 1, 2024

danielglh commented Mar 1, 2024 •

edited

Loading

arvinxx commented Mar 1, 2024

shigomany Mar 1, 2024

arvinxx Mar 3, 2024

danielglh Mar 3, 2024

arvinxx Mar 5, 2024

danielglh commented Mar 3, 2024

ckt1031 commented Mar 4, 2024

arvinxx commented Mar 4, 2024

ckt1031 commented Mar 4, 2024

arvinxx commented Mar 5, 2024

arvinxx commented Mar 5, 2024

vercel bot commented Mar 5, 2024 •

edited

Loading

danielglh commented Mar 5, 2024

danielglh commented Mar 5, 2024

danielglh commented Mar 5, 2024 •

edited

Loading

arvinxx commented Mar 5, 2024

danielglh commented Mar 5, 2024

lobehubbot commented Mar 5, 2024

lobehubbot commented Mar 5, 2024

✨ feat: support anthropic as model provider #1409

✨ feat: support anthropic as model provider #1409

Conversation

danielglh commented Feb 28, 2024 • edited by arvinxx Loading

💻 变更类型 | Change Type

🔀 变更说明 | Description of Change

📝 补充信息 | Additional Information

vercel bot commented Feb 28, 2024

lobehubbot commented Feb 28, 2024

codecov bot commented Feb 28, 2024 • edited Loading

Codecov Report

arvinxx left a comment • edited Loading

Choose a reason for hiding this comment

danielglh commented Feb 29, 2024

arvinxx commented Feb 29, 2024 • edited Loading

danielglh commented Feb 29, 2024

danielglh commented Feb 29, 2024

arvinxx commented Feb 29, 2024

danielglh commented Mar 1, 2024

danielglh commented Mar 1, 2024 • edited Loading

arvinxx commented Mar 1, 2024

shigomany Mar 1, 2024

Choose a reason for hiding this comment

arvinxx Mar 3, 2024

Choose a reason for hiding this comment

danielglh Mar 3, 2024

Choose a reason for hiding this comment

arvinxx Mar 5, 2024

Choose a reason for hiding this comment

danielglh commented Mar 3, 2024

ckt1031 commented Mar 4, 2024

arvinxx commented Mar 4, 2024

ckt1031 commented Mar 4, 2024

arvinxx commented Mar 5, 2024

arvinxx commented Mar 5, 2024

vercel bot commented Mar 5, 2024 • edited Loading

danielglh commented Mar 5, 2024

danielglh commented Mar 5, 2024

danielglh commented Mar 5, 2024 • edited Loading

arvinxx commented Mar 5, 2024

danielglh commented Mar 5, 2024

lobehubbot commented Mar 5, 2024

lobehubbot commented Mar 5, 2024

danielglh commented Feb 28, 2024 •

edited by arvinxx

Loading

codecov bot commented Feb 28, 2024 •

edited

Loading

arvinxx left a comment •

edited

Loading

arvinxx commented Feb 29, 2024 •

edited

Loading

danielglh commented Mar 1, 2024 •

edited

Loading

vercel bot commented Mar 5, 2024 •

edited

Loading

danielglh commented Mar 5, 2024 •

edited

Loading