# `LangChain.ChatModels.ChatOpenAI`
[🔗](https://github.com/brainlid/langchain/blob/v0.14.2/lib/chat_models/chat_open_ai.ex#L1)

Represents the [OpenAI
ChatModel](https://platform.openai.com/docs/api-reference/chat/create).

Parses and validates inputs for making a requests from the OpenAI Chat API.

Converts responses into more specialized `LangChain` data structures.

- https://github.com/openai/openai-cookbook/blob/main/examples/How_to_call_functions_with_chat_models.ipynb

## ContentPart Types

OpenAI supports several types of content parts that can be combined in a single message:

### Text Content
Basic text content is the default and most common type:

    Message.new_user!("Hello, how are you?")

### Image Content
OpenAI supports both base64-encoded images and image URLs:

    # Using a base64 encoded image
    Message.new_user!([
      ContentPart.text!("What's in this image?"),
      ContentPart.image!("base64_encoded_image_data", media: :jpg)
    ])

    # Using an image URL
    Message.new_user!([
      ContentPart.text!("Describe this image:"),
      ContentPart.image_url!("https://example.com/image.jpg")
    ])

For images, you can specify the detail level which affects token usage:
- `detail: "low"` - Lower resolution, fewer tokens
- `detail: "high"` - Higher resolution, more tokens
- `detail: "auto"` - Let the model decide

### File Content
OpenAI supports both base64-encoded files and file IDs:

    # Using a base64 encoded file
    Message.new_user!([
      ContentPart.text!("Process this file:"),
      ContentPart.file!("base64_encoded_file_data",
        type: :base64,
        filename: "document.pdf"
      )
    ])

    # Using a file ID (after uploading to OpenAI)
    Message.new_user!([
      ContentPart.text!("Process this file:"),
      ContentPart.file!("file-1234", type: :file_id)
    ])

## Callbacks

See the set of available callbacks: `LangChain.Chains.ChainCallbacks`

### Rate Limit API Response Headers

OpenAI returns rate limit information in the response headers. Those can be
accessed using the LLM callback `on_llm_ratelimit_info` like this:

    handlers = %{
      on_llm_ratelimit_info: fn headers ->
        IO.inspect(headers)
      end
    }

    {:ok, chat} = ChatOpenAI.new(%{callbacks: [handlers]})

Handlers assigned to the model are fired by the model itself and receive only
the event's argument. Handlers assigned to an `LangChain.Chains.LLMChain`
receive the chain as an additional first argument:

    handler = %{
      on_llm_ratelimit_info: fn _chain, headers ->
        IO.inspect(headers)
      end
    }

    %{llm: ChatOpenAI.new!(%{})}
    |> LLMChain.new!()
    |> LLMChain.add_callback(handler)

When a request is received, something similar to the following will be output
to the console.

    %{
      "x-ratelimit-limit-requests" => ["5000"],
      "x-ratelimit-limit-tokens" => ["160000"],
      "x-ratelimit-remaining-requests" => ["4999"],
      "x-ratelimit-remaining-tokens" => ["159973"],
      "x-ratelimit-reset-requests" => ["12ms"],
      "x-ratelimit-reset-tokens" => ["10ms"],
      "x-request-id" => ["req_1234"]
    }

### Token Usage

OpenAI returns token usage information as part of the response body. The
`LangChain.TokenUsage` is added to the `metadata` of the `LangChain.Message`
and `LangChain.MessageDelta` structs that are processed under the `:usage`
key.

The OpenAI documentation instructs to provide the `stream_options` with the
`include_usage: true` for the information to be provided.

```elixir
chat = ChatOpenAI.new!(%{stream: true, stream_options: %{include_usage: true}})
```

The `TokenUsage` data is accumulated for `MessageDelta` structs and the final usage information will be on the `LangChain.Message`.

NOTE: Of special note is that the `TokenUsage` information is returned once
for all "choices" in the response. The `LangChain.TokenUsage` data is added to
each message, but if your usage requests multiple choices, you will see the
same usage information for each choice but it is duplicated and only one
response is meaningful.

## Tool Choice

OpenAI's ChatGPT API supports forcing a tool to be used.
- https://platform.openai.com/docs/api-reference/chat/create#chat-create-tool_choice

This is supported through the `tool_choice` options. It takes a plain Elixir
map to provide the configuration.

By default, the LLM will choose a tool call if a tool is available and it
determines it is needed. That's the "auto" mode.

## Parallel Tool Calls

By default, OpenAI models may decide to make multiple tool calls at once,
including calling the same tool multiple times. You can limit this behavior by
setting the `parallel_tool_calls`
[option](https://platform.openai.com/docs/api-reference/chat/create#chat_create-parallel_tool_calls)
to false.

### Example
For the LLM's response to make a tool call of the "get_weather" function.

    ChatOpenAI.new(%{
      model: "...",
      tool_choice: %{"type" => "function", "function" => %{"name" => "get_weather"}}
    })

## Log Probabilities

OpenAI can return log probability information for output tokens, which is
useful for evaluating model confidence, building classifiers, or debugging
token selection.

Enable with the `logprobs` option. Optionally set `top_logprobs` to receive
the N most likely tokens (0-20) at each position:

    chat = ChatOpenAI.new!(%{
      model: "gpt-4o",
      logprobs: true,
      top_logprobs: 3
    })

When enabled, the logprobs data from the API response is placed in the
`metadata` field under the `"logprobs"` key. This works for both
non-streaming (`LangChain.Message`) and streaming (`LangChain.MessageDelta`)
responses:

    # Non-streaming
    {:ok, [%Message{metadata: %{"logprobs" => logprobs}}]} =
      ChatOpenAI.call(chat, [message], [])

    # Streaming - logprobs appear on each delta chunk
    chat = ChatOpenAI.new!(%{model: "gpt-4o", stream: true, logprobs: true})
    # Each MessageDelta will have metadata: %{"logprobs" => ...}

    # logprobs contains the raw OpenAI response structure:
    # %{
    #   "content" => [
    #     %{
    #       "token" => "Hello",
    #       "logprob" => -0.0002,
    #       "bytes" => [72, 101, 108, 108, 111],
    #       "top_logprobs" => [
    #         %{"token" => "Hello", "logprob" => -0.0002, ...},
    #         %{"token" => "Hi", "logprob" => -8.53, ...},
    #         ...
    #       ]
    #     },
    #     ...
    #   ]
    # }

When `logprobs` is not enabled or the response contains no logprobs data,
`metadata` will be `nil`.

See the [OpenAI documentation](https://developers.openai.com/api/reference/resources/completions/methods/create)
for full details on the response structure.

## Azure OpenAI Support

To use `ChatOpenAI` with Microsoft's Azure hosted OpenAI models, the
`endpoint` must be overridden and the API key needs to be provided in some
way. The [MS Quickstart guide for REST
access](https://learn.microsoft.com/en-us/azure/ai-services/openai/chatgpt-quickstart?tabs=command-line%2Cjavascript-keyless%2Ctypescript-keyless%2Cpython-new&pivots=rest-api)
may be helpful.

In order to use it, you must have an Azure account and from the console, a
model must be deployed for your account. Use the Azure AI Foundry and Azure
OpenAI Service to deploy the model you want to use. The entire URL is used as
the `endpoint` and the provided `key` is used as the `api_key`.

The following is an example of setting up `ChatOpenAI` for use with an Azure
hosted model.

    endpoint = System.fetch_env!("AZURE_OPENAI_ENDPOINT")
    api_key = System.fetch_env!("AZURE_OPENAI_KEY")

    llm =
      ChatOpenAI.new!(%{
        endpoint: endpoint,
        api_key: api_key,
        seed: 0,
        temperature: 1,
        stream: false
      })

The URL itself specifies the model to use and the `model` attribute is
disregarded.

A fake example URL for the endpoint value:

`https://some-subdomain.cognitiveservices.azure.com/openai/deployments/gpt-4o-mini/chat/completions?api-version=2024-08-01-preview"`

## Service Tier

The `service_tier` option asks the provider to serve the request with a
particular processing tier, trading cost against latency:

    ChatOpenAI.new!(%{model: "gpt-5", service_tier: "priority"})

OpenAI accepts values such as `"auto"`, `"default"`, `"flex"`, `"scale"`,
`"priority"` and `"fast"`. OpenAI-compatible providers accept their own
subset, so the value is passed through without validation.

The tier that actually served the request can differ from the one requested.
When the provider reports it, it is kept in the token usage's `raw` map:

    %Message{metadata: %{usage: %TokenUsage{raw: %{"service_tier" => tier}}}} = message

When streaming, token usage (and the tier with it) is only reported when
`stream_options: %{include_usage: true}` is set. Some providers don't report a
tier at all.

## Token Limits

`max_tokens` sets the upper bound on generated tokens. It is sent as
`max_completion_tokens`, OpenAI's current name for the limit. OpenAI's
reasoning models require that name, and the limit includes reasoning tokens.

    ChatOpenAI.new!(%{model: "gpt-5", max_tokens: 4000})

Some OpenAI-compatible servers, such as Ollama's `/v1` endpoint, only accept
the older `max_tokens` key. They ignore `max_completion_tokens`, which leaves
the request with no limit at all. For those servers, swap the key with
`extra_body`:

    ChatOpenAI.new!(%{
      endpoint: "http://localhost:11434/v1/chat/completions",
      model: "llama3.2",
      max_tokens: 4000,
      extra_body: %{"max_completion_tokens" => nil, "max_tokens" => 4000}
    })

Keep the `max_tokens` field set as well, since telemetry reports the limit
from it.

## Provider-Specific Parameters

OpenAI-compatible providers accept parameters of their own that `ChatOpenAI`
has no field for. `extra_body` is a map of values merged into the request
body last, so they override anything the model computed:

    ChatOpenAI.new!(%{
      endpoint: "http://localhost:8000/v1/chat/completions",
      model: "Qwen/Qwen3-8B",
      extra_body: %{"top_k" => 20, "repetition_penalty" => 1.05}
    })

- Keys may be strings or atoms. A key naming one the model already sends
  replaces that value, so `%{"n" => 2}` overrides the `n` field.
- A `nil` value removes the key from the body. `%{"n" => nil}` stops the
  always-sent `n` from going out, for providers that reject it.
- Nested maps are merged key by key. Any other value replaces the existing
  one.

Overriding `stream`, `messages`, `tools` or `model` can break response
handling. Headers and transport options belong in `req_config`, not
`extra_body`. See `LangChain.Utils.merge_extra_body/2` for the full rules.

## Reasoning Model Support

OpenAI made some significant API changes with the introduction of their
"reasoning" models. This includes the `o1` and `o1-mini` models.

To enable this mode, set `:reasoning_mode` to `true`:

    model = ChatOpenAI.new!(%{reasoning_mode: true})

Setting `reasoning_mode` to `true` does at least the two following things:

- Set `:developer` as the `role` for system messages. The OpenAI documentation
  says API calls to `o1` and newer models must use the `role: :developer`
  instead of `role: :system` and errors if not set correctly.
- The `:reasoning_effort` option included in LLM requests. This setting is
  only permitted on a reasoning model. The `:reasoning_effort` values support
  the "low", "medium" (default), and "high" options specified in the OpenAI
  documentation. This instructs the LLM on how much time, and tokens, should
  be spent on thinking through and reasoning about the request and the
  response.

### Returned Thinking

OpenAI-compatible services other than OpenAI itself return a reasoning
model's thinking in a `reasoning_content` field beside `content`. It arrives
on the message for a single response and on each chunk when streaming.

That thinking becomes a `LangChain.Message.ContentPart` of type `:thinking`,
held separately from the answer text and ordered ahead of it:

    [
      %ContentPart{type: :thinking, content: "Comparing the tenths place..."},
      %ContentPart{type: :text, content: "9.9 is larger."}
    ]

Reaching such a service takes an `endpoint` for it, and often an extra header
supplied through `req_config`:

    ChatOpenAI.new!(%{
      endpoint: "https://api.cloudflare.com/client/v4/accounts/#{account_id}/ai/v1/chat/completions",
      api_key: api_key,
      model: "@cf/zai-org/glm-5.3-flash",
      reasoning_mode: true,
      reasoning_effort: "medium",
      service_tier: "priority",
      req_config: %{headers: [{"cf-aig-gateway-id", gateway_id}]}
    })

## Connection Retry Behavior

The `retry_count` option controls how many times a request is retried when
a pooled HTTP connection turns out to be stale (server closed it between
requests). This is a transport-level issue where retrying with a fresh
connection is the correct response.

**Only closed-connection errors are retried.** Timeouts, rate limits (429),
overloaded (529), authentication errors, and invalid requests all return
immediately -- they are not problems that a simple retry will fix.

| `retry_count` | Total HTTP requests |
|---|---|
| `0` | 1 (no retries) |
| `1` | 2 (1 initial + 1 retry) |
| `2` (default) | 3 (1 initial + 2 retries) |

Req's built-in HTTP retry is disabled to prevent the two retry layers from
compounding. See [GitHub issue #503](https://github.com/brainlid/langchain/issues/503).

When running LLM calls from a background job queue (e.g., Oban) that has its
own retry logic, set `retry_count: 0` so there are no hidden retries:

    ChatOpenAI.new!(%{model: "...", retry_count: 0})

# `t`
[🔗](https://github.com/brainlid/langchain/blob/v0.14.2/lib/chat_models/chat_open_ai.ex#L519)

```elixir
@type t() :: %LangChain.ChatModels.ChatOpenAI{
  api_key: term(),
  callbacks: term(),
  endpoint: term(),
  extra_body: term(),
  frequency_penalty: term(),
  json_response: term(),
  json_schema: term(),
  logprobs: term(),
  max_tokens: term(),
  model: term(),
  n: term(),
  org_id: term(),
  parallel_tool_calls: term(),
  reasoning_effort: term(),
  reasoning_mode: term(),
  receive_timeout: term(),
  req_config: term(),
  retry_count: term(),
  seed: term(),
  service_tier: term(),
  stream: term(),
  stream_options: term(),
  temperature: term(),
  tool_choice: term(),
  top_logprobs: term(),
  user: term(),
  verbose_api: term(),
  verbosity: term()
}
```

# `call`
[🔗](https://github.com/brainlid/langchain/blob/v0.14.2/lib/chat_models/chat_open_ai.ex#L948)

Calls the OpenAI API passing the ChatOpenAI struct with configuration, plus
either a simple message or the list of messages to act as the prompt.

Optionally pass in a list of tools available to the LLM for requesting
execution in response.

Optionally pass in a callback function that can be executed as data is
received from the API.

**NOTE:** This function *can* be used directly, but the primary interface
should be through `LangChain.Chains.LLMChain`. The `ChatOpenAI` module is more
focused on translating the `LangChain` data structures to and from the OpenAI
API.

Another benefit of using `LangChain.Chains.LLMChain` is that it combines the
storage of messages, adding tools, adding custom context that should be
passed to tools, and automatically applying `LangChain.MessageDelta`
structs as they are are received, then converting those to the full
`LangChain.Message` once fully complete.

# `content_part_for_api`
[🔗](https://github.com/brainlid/langchain/blob/v0.14.2/lib/chat_models/chat_open_ai.ex#L841)

Convert a ContentPart to the expected map of data for the OpenAI API.

# `content_parts_for_api`
[🔗](https://github.com/brainlid/langchain/blob/v0.14.2/lib/chat_models/chat_open_ai.ex#L832)

Convert a list of ContentParts to the expected map of data for the OpenAI API.

Thinking and unsupported parts are omitted. Both are response-side artifacts
that this API surface has no request representation for, and both reach it by
round-tripping a message the provider itself produced.

There is no agreed request representation for thinking across the services
that speak this API. A provider returning it in `reasoning_content` may
accept that field back, ignore it, or accept it only in a particular mode,
and the field is absent from the OpenAI request schema the rest of these
services are modeled on. Unsupported parts, such as the `redacted_thinking`
block Anthropic returns, hold opaque provider data with no meaning here at
all.

Nothing depends on returning either. Reasoning on this surface carries no
signature to validate and no continuity requirement, so a conversation sends
the answer text and any tool calls, and the model reasons afresh on the next
turn.

The omission is unconditional, which is what keeps prompt caching working.
A prefix cache is built from what the client sends rather than from what the
model produced, so a conversation that always omits thinking presents a
prefix that matches itself turn after turn. Omitting it on some turns and
including it on others would break the prefix at the first message that
differs and cost a cache miss for everything after it.

# `decode_stream`
[🔗](https://github.com/brainlid/langchain/blob/v0.14.2/lib/chat_models/chat_open_ai.ex#L1187)

```elixir
@spec decode_stream(
  {String.t(), String.t()},
  list()
) :: {[%{required(String.t()) => any()}], String.t()}
```

Decode a streamed response from an OpenAI-compatible server. Parses a string
of received content into an Elixir map data structure using string keys.

If a partial response was received, meaning the JSON text is split across
multiple data frames, then the incomplete portion is returned as-is in the
buffer. The function will be successively called, receiving the incomplete
buffer data from a previous call, and assembling it to parse.

# `for_api`
[🔗](https://github.com/brainlid/langchain/blob/v0.14.2/lib/chat_models/chat_open_ai.ex#L730)

```elixir
@spec for_api(
  struct(),
  LangChain.Message.t()
  | LangChain.PromptTemplate.t()
  | LangChain.Message.ToolCall.t()
  | LangChain.Message.ToolResult.t()
  | LangChain.Message.ContentPart.t()
  | LangChain.Function.t()
) :: %{required(String.t()) => any()} | [%{required(String.t()) => any()}]
```

Convert a LangChain Message-based structure to the expected map of data for
the OpenAI API. This happens within the context of the model configuration as
well. The additional context is needed to correctly convert a role to either
`:system` or `:developer`.

NOTE: The `ChatOpenAI` model's functions are reused in other modules. For this
reason, model is more generally defined as a struct.

# `for_api`
[🔗](https://github.com/brainlid/langchain/blob/v0.14.2/lib/chat_models/chat_open_ai.ex#L621)

```elixir
@spec for_api(
  t() | LangChain.Message.t() | LangChain.Function.t(),
  message :: [map()],
  LangChain.ChatModels.ChatModel.tools()
) :: %{required(atom()) => any()}
```

Return the params formatted for an API request.

# `new`
[🔗](https://github.com/brainlid/langchain/blob/v0.14.2/lib/chat_models/chat_open_ai.ex#L572)

```elixir
@spec new(attrs :: map()) :: {:ok, t()} | {:error, Ecto.Changeset.t()}
```

Setup a ChatOpenAI client configuration.

# `new!`
[🔗](https://github.com/brainlid/langchain/blob/v0.14.2/lib/chat_models/chat_open_ai.ex#L583)

```elixir
@spec new!(attrs :: map()) :: t() | no_return()
```

Setup a ChatOpenAI client configuration and return it or raise an error if invalid.

# `restore_from_map`
[🔗](https://github.com/brainlid/langchain/blob/v0.14.2/lib/chat_models/chat_open_ai.ex#L1659)

Restores the model from the config.

# `retry_on_fallback?`
[🔗](https://github.com/brainlid/langchain/blob/v0.14.2/lib/chat_models/chat_open_ai.ex#L1619)

```elixir
@spec retry_on_fallback?(LangChain.LangChainError.t()) :: boolean()
```

Determine if an error should be retried. If `true`, a fallback LLM may be
used. If `false`, the error is understood to be more fundamental with the
request rather than a service issue and it should not be retried or fallback
to another service.

# `serialize_config`
[🔗](https://github.com/brainlid/langchain/blob/v0.14.2/lib/chat_models/chat_open_ai.ex#L1630)

```elixir
@spec serialize_config(t()) :: %{required(String.t()) => any()}
```

Generate a config map that can later restore the model's configuration.

---

*Consult [api-reference.md](api-reference.md) for complete listing*