当前位置：首页 > news >正文

OneAPI接入本地大模型+FastGPT调用本地大模型

news 来源：原创 2024/9/22 7:13:21

将Ollama下载的本地大模型配置到OneAPI中，并通过FastGPT调用本地大模型完成对话。

OneAPI配置

新建令牌

在这里插入图片描述

新建渠道

在这里插入图片描述

FastGPT配置

配置docker-compose

配置令牌和OneAPI部署地址
在这里插入图片描述

配置config.json

配置调用的渠道名称和大模型名称

{"systemEnv": {"pluginBaseUrl": "","vectorMaxProcess": 15,"qaMaxProcess": 15,"pgHNSWEfSearch": 100},"chatModels": [{"model": "qwen:1.8b", "name": "lingmouAIOllama", "maxContext": 8000, "maxResponse": 4000, "quoteMaxToken": 2000, "maxTemperature": 1, "vision": false, "defaultSystemChatPrompt": "" }],"qaModels": [{"model": "qwen:1.8b", "name": "lingmouAIOllama", "maxContext": 8000, "maxResponse": 4000, "quoteMaxToken": 2000, "maxTemperature": 1, "vision": false, "defaultSystemChatPrompt": "" }],"cqModels": [{"model": "qwen:1.8b", "name": "lingmouAIOllama", "maxContext": 8000, "maxResponse": 4000, "quoteMaxToken": 2000, "maxTemperature": 1, "vision": false, "defaultSystemChatPrompt": "" }],"extractModels": [{"model": "qwen:1.8b", "name": "lingmouAIOllama", "maxContext": 8000, "maxResponse": 4000, "quoteMaxToken": 2000, "maxTemperature": 1, "vision": false, "defaultSystemChatPrompt": "" }],"qgModels": [{"model": "gpt-3.5-turbo-1106","name": "GPT35-1106","maxContext": 1600,"maxResponse": 4000,"inputPrice": 0,"outputPrice": 0}],"vectorModels": [{"model": "text-embedding-v1","name": "lingmouAI","inputPrice": 0,"outputPrice": 0,"defaultToken": 700,"maxToken": 3000,"weight": 100},{"model": "text-embedding-ada-002","name": "lingmouAI","inputPrice": 0,"outputPrice": 0,"defaultToken": 700,"maxToken": 3000,"weight": 100}],"reRankModels": [],"audioSpeechModels": [{"model": "tts-1","name": "OpenAI TTS1","inputPrice": 0,"outputPrice": 0,"voices": [{ "label": "Alloy", "value": "alloy", "bufferId": "openai-Alloy" },{ "label": "Echo", "value": "echo", "bufferId": "openai-Echo" },{ "label": "Fable", "value": "fable", "bufferId": "openai-Fable" },{ "label": "Onyx", "value": "onyx", "bufferId": "openai-Onyx" },{ "label": "Nova", "value": "nova", "bufferId": "openai-Nova" },{ "label": "Shimmer", "value": "shimmer", "bufferId": "openai-Shimmer" }]}],"whisperModel": {"model": "whisper-1","name": "Whisper1","inputPrice": 0,"outputPrice": 0}
}