Chat Completion

OpenAI的ChatCompletion接口

支持OpenAI所有的Chat模型，包括最新版本的模型

关于该接口的详细参数您可以查阅OpenAI的官方文档：https://platform.openai.com/docs/api-reference/chat/create

您还可以通过此接口，以OpenAI ChatCompletion API的格式，调用许多非OpenAI模型，例如Anthropic的Claude系列模型，以及TA、ChatGLM、Cohere等等，具体支持的模型列表请以网站设置页面上的模型列表为准，文档中的内容可能不会及时维护

Gemini系列模型JSON模式支持[24-1109更新]

参考文档：
https://ai.google.dev/api/generate-content#v1beta.GenerationConfig
https://platform.openai.com/docs/api-reference/chat/create

具体实现细节

简单地转换一下数据格式而已：

if (body.response_format.type === 'text'):
{
req.generation_config.responseMimeType='text/plain'
}
elif (body.response_format.type === 'json_object'):
{
req.generation_config.responseMimeType='application/json'
}
elif (body.response_format.type === 'json_schema'):
{
req.generation_config.responseMimeType='application/json',
req.generation_config.responseSchema=body.json_schema
}

示例请求：

{
    "model": "gemini-1.5-pro-002",
    "messages": [
        {
            "role": "user",
            "content": "What is the weather in SF CA?"
        }
    ],
    "stream": false,
    "response_format": {
        "type": "json_object"
    }
}

响应内容：

{
    "id": "chatcmpl-8dImtLWPeWwKaJUPX4iazoKba4FVS",
    "object": "chat.completion",
    "created": 1731147163,
    "model": "gemini-1.5-pro-002",
    "choices": [
        {
            "index": 0,
            "message": {
                "role": "assistant",
                "content": "{\"location\": \"San Francisco, CA\", \"weather\": \"I do not have access to real-time information, such as live weather updates.  For the latest weather information, please check a reliable weather app or your local news.\"}\n"
            },
            "finish_reason": "stop"
        }
    ],
    "usage": {
        "prompt_tokens": 9,
        "completion_tokens": 50,
        "total_tokens": 59
    }
}

示例请求：

{
    "model": "gemini-1.5-pro-002",
    "messages": [
        {
            "role": "user",
            "content": "What is the weather in SF CA?"
        }
    ],
    "stream": false,
    "response_format": {
        "type": "json_schema",
        "json_schema": {
            "name": "get_current_weather",
            "description": "Get the current weather in a given location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description": "The city and state, e.g. San Francisco, CA"
                    },
                    "unit": {
                        "type": "string",
                        "enum": [
                            "celsius",
                            "fahrenheit"
                        ]
                    }
                },
                "required": [
                    "location"
                ]
            }
        }
    }
}

响应内容：

{
    "id": "chatcmpl-7nh9pv8TFs9vCwUcThfCNpDakonsL",
    "object": "chat.completion",
    "created": 1731147392,
    "model": "gemini-1.5-pro-002",
    "choices": [
        {
            "index": 0,
            "message": {
                "role": "assistant",
                "content": "{\"location\": \"San Francisco, CA\", \"weather\": {\"forecast\": [{\"date\": \"2024/01/20\", \"temperature\": \"12°C\", \"condition\": \"Cloudy\"}, {\"date\": \"2024/01/21\", \"temperature\": \"13°C\", \"condition\": \"Partly Cloudy\"}, {\"date\": \"2024/01/22\", \"temperature\": \"14°C\", \"condition\": \"Sunny\"}]}, \"current_condition\": {\"temperature\": \"12°C\", \"condition\": \"Cloudy\", \"wind\": \"10 mph\", \"humidity\": \"70%\"}}\n"
            },
            "finish_reason": "stop"
        }
    ],
    "usage": {
        "prompt_tokens": 9,
        "completion_tokens": 146,
        "total_tokens": 155
    }
}

Claude系列模型Tools函数调用支持[24-1107更新]

最近有用户反馈关于Claude系列模型在OpenAI兼容接口中的函数调用参数支持问题，因此于24-1107完成了相关的兼容性适配，现在您可以通过OpenAI兼容接口的Tools格式调用Claude系列模型，同时返回OpenAI格式的Tools响应数据。

具体实现细节：

Claude与OpenAI原生接口的ToolsCall输入输出都有一些区别，实现这个兼容花了一点心思，具体细节如下：

输入部分处理：

对于OpenAI到Claude请求体的tool_choice参数的转换

OpenAI body.parallel_tool_calls ==如果存在 则 取反==> Claude body.tool_choice.disable_parallel_tool_use
OpenAI body.tool_choice.type ==> Claude body.tool_choice.type
OpenAI body.tool_choice.function.name ==> Claude body.tool_choice.name

其中：

1. 当 `tool_choice` 为字符串类型时：
   - 如果值为 `"none"`，则 Claude 的 `tool_choices` 设置为 `undefined`
   - 如果值为 `"auto"`，则 Claude 的 `tool_choices` 设置为 `{ type: 'auto' }`
   - 如果值为 `"required"`，则 Claude 的 `tool_choices` 设置为 `{ type: 'any' }`

2. 当 `tool_choice` 为对象类型时：
   - Claude 的 `tool_choices` 将被设置为包含以下内容的对象：
     - `type` 固定设置为 `'tool'`
     - `name` 设置为 OpenAI 请求中 `tool_choice.function.name` 的值

3. 对于 `tools` 参数的转换：
   - 如果 OpenAI 请求中存在 `tools` 数组，则对数组中的每个工具对象进行转换
   - 对于每个类型为 `'function'` 的工具对象，转换规则如下：
     - `tool.function.name` ==> Claude `tools[].name`
     - `tool.function.description` ==> Claude `tools[].description`
     - `tool.function.parameters` ==> Claude `tools[].input_schema`
   - 如果工具对象不符合上述结构，则该参数不会被发送到Claude
   - 如果 OpenAI 请求中不存在 `tools` 数组，Claude 的 `tools` 参数不会被发送到Claude

输出部分处理：

Claude的流式输出与OpenAI的格式有较大差异，需要进行适配转换。主要包括以下几种情况：

1. 纯文本输出转换：
   - 当Claude返回 `content_block_delta` 且类型为 `text_delta` 时，需转换为OpenAI的格式：
     - 将 text_delta.text 转换到 OpenAI 的 choices[0].delta.content 中

2. 工具调用输出转换：
   - 当Claude返回工具调用相关事件时，需要进行以下转换：
     - 当收到 `content_block_start` 且类型为 `tool_use` 时：
       - 生成工具调用的初始结构，包含 tool_calls 数组
       - 将 Claude 的 tool_use.name 转换为 OpenAI 的 function.name
       - 生成一个唯一的 tool call id
       - 工具调用的 index 需要从 Claude 的基于1的索引转换为基于0的索引

     - 当收到 `content_block_delta` 且类型为 `input_json_delta` 时：
       - 将 partial_json 内容追加到对应工具调用的 arguments 字段中

3. 结束标识转换：
   - 当Claude返回 `message_delta` 且 stop_reason 为 "tool_use" 时：
     - 设置 OpenAI 格式的 finish_reason 为 "tool_calls"
   - 当Claude返回 `message_stop` 时：
     - 输出 OpenAI 格式的 usage 信息
     - 最后输出 "[DONE]" 标识

4. 通用字段转换：
   - 为所有输出添加 OpenAI 格式所需的通用字段：
     - id: `chatcmpl-${随机ID}`
     - object: "chat.completion.chunk"
     - created: 请求开始时间戳
     - model: 模型名称
     - system_fingerprint: `fp_${8位随机ID}`
     - choices[0].index: 0
     - choices[0].logprobs: null

示例请求：

{
  "model": "claude-3-5-haiku",
  "messages": [
    {
      "role": "user",
      "content": "What's the weather like in Boston today?"
    }
  ],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_current_weather",
        "description": "Get the current weather in a given location",
        "parameters": {
          "type": "object",
          "properties": {
            "location": {
              "type": "string",
              "description": "The city and state, e.g. San Francisco, CA"
            },
            "unit": {
              "type": "string",
              "enum": ["celsius", "fahrenheit"]
            }
          },
          "required": ["location"]
        }
      }
    }
  ],
  "tool_choice": "auto",
  "stream": false
}

非流返回(stream=false)

{
    "id": "chatcmpl-H84t7g0CY5f4Gqg7KviVCXzJ4EIJ0",
    "object": "chat.completion",
    "created": 1730973989,
    "model": "claude-3-5-haiku",
    "choices": [
        {
            "index": 0,
            "message": {
                "role": "assistant",
                "content": null,
                "tool_calls": [
                    {
                        "id": "toolu_01HB4BABmfcNDCJKG5eiVmQv",
                        "type": "function",
                        "function": {
                            "name": "get_current_weather",
                            "arguments": "{\"location\":\"Boston, MA\",\"unit\":\"fahrenheit\"}"
                        }
                    }
                ],
                "refusal": null
            },
            "logprobs": null,
            "finish_reason": "tool_calls"
        },
        {
            "index": 0,
            "message": {
                "role": "assistant",
                "content": "I'll help you check the current weather in Boston. I'll retrieve the weather information using the get_current_weather function.",
                "refusal": null
            },
            "logprobs": null,
            "finish_reason": "stop"
        }
    ],
    "usage": {
        "prompt_tokens": 376,
        "completion_tokens": 104,
        "total_tokens": 480
    }
}

流返回(stream=true)：

data: {"id":"chatcmpl-t5VoJU6Et68AzGIMAaDYjGydoQ8HQ","object":"chat.completion.chunk","created":1730992326,"model":"claude-3-5-haiku","system_fingerprint":"fp_t5VoJU6E","choices":[{"index":0,"delta":{"role":"assistant","content":"","refusal":null},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-t5VoJU6Et68AzGIMAaDYjGydoQ8HQ","object":"chat.completion.chunk","created":1730992326,"model":"claude-3-5-haiku","system_fingerprint":"fp_t5VoJU6E","choices":[{"index":0,"delta":{"role":"assistant","content":"I'll"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-t5VoJU6Et68AzGIMAaDYjGydoQ8HQ","object":"chat.completion.chunk","created":1730992326,"model":"claude-3-5-haiku","system_fingerprint":"fp_t5VoJU6E","choices":[{"index":0,"delta":{"role":"assistant","content":" help"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-t5VoJU6Et68AzGIMAaDYjGydoQ8HQ","object":"chat.completion.chunk","created":1730992326,"model":"claude-3-5-haiku","system_fingerprint":"fp_t5VoJU6E","choices":[{"index":0,"delta":{"role":"assistant","content":" you find out"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-t5VoJU6Et68AzGIMAaDYjGydoQ8HQ","object":"chat.completion.chunk","created":1730992326,"model":"claude-3-5-haiku","system_fingerprint":"fp_t5VoJU6E","choices":[{"index":0,"delta":{"role":"assistant","content":" the current weather in Boston"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-t5VoJU6Et68AzGIMAaDYjGydoQ8HQ","object":"chat.completion.chunk","created":1730992326,"model":"claude-3-5-haiku","system_fingerprint":"fp_t5VoJU6E","choices":[{"index":0,"delta":{"role":"assistant","content":"."},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-t5VoJU6Et68AzGIMAaDYjGydoQ8HQ","object":"chat.completion.chunk","created":1730992326,"model":"claude-3-5-haiku","system_fingerprint":"fp_t5VoJU6E","choices":[{"index":0,"delta":{"role":"assistant","content":" I"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-t5VoJU6Et68AzGIMAaDYjGydoQ8HQ","object":"chat.completion.chunk","created":1730992326,"model":"claude-3-5-haiku","system_fingerprint":"fp_t5VoJU6E","choices":[{"index":0,"delta":{"role":"assistant","content":"'ll"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-t5VoJU6Et68AzGIMAaDYjGydoQ8HQ","object":"chat.completion.chunk","created":1730992326,"model":"claude-3-5-haiku","system_fingerprint":"fp_t5VoJU6E","choices":[{"index":0,"delta":{"role":"assistant","content":" retrieve"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-t5VoJU6Et68AzGIMAaDYjGydoQ8HQ","object":"chat.completion.chunk","created":1730992326,"model":"claude-3-5-haiku","system_fingerprint":"fp_t5VoJU6E","choices":[{"index":0,"delta":{"role":"assistant","content":" the current"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-t5VoJU6Et68AzGIMAaDYjGydoQ8HQ","object":"chat.completion.chunk","created":1730992326,"model":"claude-3-5-haiku","system_fingerprint":"fp_t5VoJU6E","choices":[{"index":0,"delta":{"role":"assistant","content":" weather information"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-t5VoJU6Et68AzGIMAaDYjGydoQ8HQ","object":"chat.completion.chunk","created":1730992326,"model":"claude-3-5-haiku","system_fingerprint":"fp_t5VoJU6E","choices":[{"index":0,"delta":{"role":"assistant","content":" for"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-t5VoJU6Et68AzGIMAaDYjGydoQ8HQ","object":"chat.completion.chunk","created":1730992326,"model":"claude-3-5-haiku","system_fingerprint":"fp_t5VoJU6E","choices":[{"index":0,"delta":{"role":"assistant","content":" you"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-t5VoJU6Et68AzGIMAaDYjGydoQ8HQ","object":"chat.completion.chunk","created":1730992326,"model":"claude-3-5-haiku","system_fingerprint":"fp_t5VoJU6E","choices":[{"index":0,"delta":{"role":"assistant","content":"."},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-t5VoJU6Et68AzGIMAaDYjGydoQ8HQ","object":"chat.completion.chunk","created":1730992326,"model":"claude-3-5-haiku","system_fingerprint":"fp_t5VoJU6E","choices":[{"index":0,"delta":{"role":"assistant","content":null,"tool_calls":[{"index":0,"id":"toolu_01RdBwK8GsN7sm6dyDteDc3e","type":"function","function":{"name":"get_current_weather","arguments":""}}],"refusal":null},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-t5VoJU6Et68AzGIMAaDYjGydoQ8HQ","object":"chat.completion.chunk","created":1730992326,"model":"claude-3-5-haiku","system_fingerprint":"fp_t5VoJU6E","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":"{\"locati"}}]},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-t5VoJU6Et68AzGIMAaDYjGydoQ8HQ","object":"chat.completion.chunk","created":1730992326,"model":"claude-3-5-haiku","system_fingerprint":"fp_t5VoJU6E","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":"on\": \"Bosto"}}]},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-t5VoJU6Et68AzGIMAaDYjGydoQ8HQ","object":"chat.completion.chunk","created":1730992326,"model":"claude-3-5-haiku","system_fingerprint":"fp_t5VoJU6E","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":"n, M"}}]},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-t5VoJU6Et68AzGIMAaDYjGydoQ8HQ","object":"chat.completion.chunk","created":1730992326,"model":"claude-3-5-haiku","system_fingerprint":"fp_t5VoJU6E","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":"A\""}}]},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-t5VoJU6Et68AzGIMAaDYjGydoQ8HQ","object":"chat.completion.chunk","created":1730992326,"model":"claude-3-5-haiku","system_fingerprint":"fp_t5VoJU6E","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":", \"unit\": \"f"}}]},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-t5VoJU6Et68AzGIMAaDYjGydoQ8HQ","object":"chat.completion.chunk","created":1730992326,"model":"claude-3-5-haiku","system_fingerprint":"fp_t5VoJU6E","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":"ahrenh"}}]},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-t5VoJU6Et68AzGIMAaDYjGydoQ8HQ","object":"chat.completion.chunk","created":1730992326,"model":"claude-3-5-haiku","system_fingerprint":"fp_t5VoJU6E","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":"eit\"}"}}]},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-t5VoJU6Et68AzGIMAaDYjGydoQ8HQ","object":"chat.completion.chunk","created":1730992326,"model":"claude-3-5-haiku","system_fingerprint":"fp_t5VoJU6E","choices":[{"index":0,"delta":{},"logprobs":null,"finish_reason":"tool_calls"}]}

data: {"id":"chatcmpl-t5VoJU6Et68AzGIMAaDYjGydoQ8HQ","object":"chat.completion.chunk","created":1730992326,"model":"claude-3-5-haiku","choices":[],"usage":{"prompt_tokens":376,"completion_tokens":100,"total_tokens":476,"prompt_tokens_details":{"cached_tokens":0},"completion_tokens_details":{"reasoning_tokens":0}}}

data: [DONE]

Gemini Pro Vision使用示例

可以完全参照OpenAI的Vision说明：https://platform.openai.com/docs/guides/vision

示例输入：

{
    "model": "gemini-pro-vision",
    "messages": [
        {
            "role": "user",
            "content": [
                {
                    "type": "text", "text": "描述一下这张图片"
                },
                {
                    "type": "image_url",
                    "image_url": "https://pbs.twimg.com/media/GBEB1CbbIAAC28o?format=jpg&name=small"
                }
            ]
        }
    ],
    "stream": false
}

注：同样也支持Base64作为图像输入的，而且Base64可靠性更高，推荐Base64，这里放链接是为了防止字数太多。

返回结果：

{
  "id": "chatcmpl-gPyHaMj77C8Uca3UudUL5zxvAvI3N",
  "object": "chat.completion",
  "created": 1702560711,
  "model": "gemini-pro-vision",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": " 这是五个不同角色的表情包。他们分别是：\n- 胡桃\n- 宵宫\n- 早柚\n- 刻晴\n- 珊瑚宫心海"
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 8,
    "completion_tokens": 48,
    "total_tokens": 56
  }
}

流式请求（"stream": true）返回结果：

data: {"id":"chatcmpl-6tzfeCctklU06xs8BmDo3b1YcCold","object":"chat.completion.chunk","created":1702562069,"model":"gemini-pro-vision","choices":[{"index":0,"delta":{"role":"assistant","content":" 这是五个不同角色的表情包。他们分别是：\n- 胡桃\n- 宵宫\n- 早柚\n- 刻晴\n- 珊瑚"},"finish_reason":null}]}

data: {"id":"chatcmpl-6tzfeCctklU06xs8BmDo3b1YcCold","object":"chat.completion.chunk","created":1702562069,"model":"gemini-pro-vision","choices":[{"index":0,"delta":{"role":"assistant","content":"宫心海"},"finish_reason":null}]}

data: [DONE]

application/json

Body

string

必需

object

string

必需

created

integer

必需

model

string

必需

choices

array [object {3}]

必需

index

integer

可选

message

object

可选

finish_reason

string

可选

usage

object

必需

prompt_tokens

integer

必需

completion_tokens

integer

必需

total_tokens

integer

必需

示例

成功示例(NonStream)成功示例(Stream)

{
  "id": "chatcmpl-8IXolsgH8YDFIjiRZScLjNPylMxyr",
  "object": "chat.completion",
  "created": 1699430591,
  "model": "gpt-4-0613",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Test"
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 19,
    "completion_tokens": 1,
    "total_tokens": 20
  }
}

Gemini系列模型JSON模式支持[24-1109更新]

具体实现细节

Claude系列模型Tools函数调用支持[24-1107更新]

具体实现细节：

示例请求：

Gemini Pro Vision使用示例

请求参数

示例代码

返回响应

Chat Completion

Gemini系列模型JSON模式支持[24-1109更新]#

具体实现细节#

Claude系列模型Tools函数调用支持[24-1107更新]#

具体实现细节：#

示例请求：#

Gemini Pro Vision使用示例#

请求参数

示例代码

返回响应

Gemini系列模型JSON模式支持[24-1109更新]

具体实现细节

Claude系列模型Tools函数调用支持[24-1107更新]

具体实现细节：

示例请求：

Gemini Pro Vision使用示例