# `LangChain.ChatModels.ChatOpenAI`
[🔗](https://github.com/brainlid/langchain/blob/v0.15.3/lib/chat_models/chat_open_ai.ex#L1)

Represents the [OpenAI
ChatModel](https://platform.openai.com/docs/api-reference/chat/create).

Parses and validates inputs for making a requests from the OpenAI Chat API.

Converts responses into more specialized `LangChain` data structures.

- https://github.com/openai/openai-cookbook/blob/main/examples/How_to_call_functions_with_chat_models.ipynb

## OpenAI-compatible services

This module follows OpenAI's own API, including Azure OpenAI, and adopts
OpenAI's changes as they ship. Services that offer an OpenAI-compatible
endpoint (Cloudflare Workers AI, Groq, OpenRouter, vLLM, SGLang, Ollama, LM
Studio and others) often do not support those changes, and many ignore or
silently drop what they do not understand. For example, a system message
sent under the `developer` role, as `reasoning_mode` does, never reaches a
model served by SGLang or vLLM when the model's chat template has no
`developer` branch.

Use `LangChain.ChatModels.ChatOpenAICompatible` for those services. It sends
only the fields you set, always sends system messages as `system`, and never
falls back to the global OpenAI API key.

## ContentPart Types

OpenAI supports several types of content parts that can be combined in a single message:

### Text Content
Basic text content is the default and most common type:

    Message.new_user!("Hello, how are you?")

### Image Content
OpenAI supports both base64-encoded images and image URLs:

    # Using a base64 encoded image
    Message.new_user!([
      ContentPart.text!("What's in this image?"),
      ContentPart.image!("base64_encoded_image_data", media: :jpg)
    ])

    # Using an image URL
    Message.new_user!([
      ContentPart.text!("Describe this image:"),
      ContentPart.image_url!("https://example.com/image.jpg")
    ])

For images, you can specify the detail level which affects token usage:
- `detail: "low"` - Lower resolution, fewer tokens
- `detail: "high"` - Higher resolution, more tokens
- `detail: "auto"` - Let the model decide

### File Content
OpenAI supports both base64-encoded files and file IDs:

    # Using a base64 encoded file
    Message.new_user!([
      ContentPart.text!("Process this file:"),
      ContentPart.file!("base64_encoded_file_data",
        type: :base64,
        filename: "document.pdf"
      )
    ])

    # Using a file ID (after uploading to OpenAI)
    Message.new_user!([
      ContentPart.text!("Process this file:"),
      ContentPart.file!("file-1234", type: :file_id)
    ])

## Callbacks

See the set of available callbacks: `LangChain.Chains.ChainCallbacks`

### Rate Limit API Response Headers

OpenAI returns rate limit information in the response headers. Those can be
accessed using the LLM callback `on_llm_ratelimit_info` like this:

    handlers = %{
      on_llm_ratelimit_info: fn headers ->
        IO.inspect(headers)
      end
    }

    {:ok, chat} = ChatOpenAI.new(%{callbacks: [handlers]})

Handlers assigned to the model are fired by the model itself and receive only
the event's argument. Handlers assigned to an `LangChain.Chains.LLMChain`
receive the chain as an additional first argument:

    handler = %{
      on_llm_ratelimit_info: fn _chain, headers ->
        IO.inspect(headers)
      end
    }

    %{llm: ChatOpenAI.new!(%{})}
    |> LLMChain.new!()
    |> LLMChain.add_callback(handler)

When a request is received, something similar to the following will be output
to the console.

    %{
      "x-ratelimit-limit-requests" => ["5000"],
      "x-ratelimit-limit-tokens" => ["160000"],
      "x-ratelimit-remaining-requests" => ["4999"],
      "x-ratelimit-remaining-tokens" => ["159973"],
      "x-ratelimit-reset-requests" => ["12ms"],
      "x-ratelimit-reset-tokens" => ["10ms"],
      "x-request-id" => ["req_1234"]
    }

### Token Usage

OpenAI returns token usage information as part of the response body. The
`LangChain.TokenUsage` is added to the `metadata` of the `LangChain.Message`
and `LangChain.MessageDelta` structs that are processed under the `:usage`
key.

The OpenAI documentation instructs to provide the `stream_options` with the
`include_usage: true` for the information to be provided.

```elixir
chat = ChatOpenAI.new!(%{stream: true, stream_options: %{include_usage: true}})
```

The `TokenUsage` data is accumulated for `MessageDelta` structs and the final usage information will be on the `LangChain.Message`.

NOTE: Of special note is that the `TokenUsage` information is returned once
for all "choices" in the response. The `LangChain.TokenUsage` data is added to
each message, but if your usage requests multiple choices, you will see the
same usage information for each choice but it is duplicated and only one
response is meaningful.

## Tool Choice

OpenAI's ChatGPT API supports forcing a tool to be used.
- https://platform.openai.com/docs/api-reference/chat/create#chat-create-tool_choice

This is supported through the `tool_choice` options. It takes a plain Elixir
map to provide the configuration.

By default, the LLM will choose a tool call if a tool is available and it
determines it is needed. That's the "auto" mode.

## Parallel Tool Calls

By default, OpenAI models may decide to make multiple tool calls at once,
including calling the same tool multiple times. You can limit this behavior by
setting the `parallel_tool_calls`
[option](https://platform.openai.com/docs/api-reference/chat/create#chat_create-parallel_tool_calls)
to false.

### Example
For the LLM's response to make a tool call of the "get_weather" function.

    ChatOpenAI.new(%{
      model: "...",
      tool_choice: %{"type" => "function", "function" => %{"name" => "get_weather"}}
    })

## Log Probabilities

OpenAI can return log probability information for output tokens, which is
useful for evaluating model confidence, building classifiers, or debugging
token selection.

Enable with the `logprobs` option. Optionally set `top_logprobs` to receive
the N most likely tokens (0-20) at each position:

    chat = ChatOpenAI.new!(%{
      model: "gpt-4o",
      logprobs: true,
      top_logprobs: 3
    })

When enabled, the logprobs data from the API response is placed in the
`metadata` field under the `"logprobs"` key. This works for both
non-streaming (`LangChain.Message`) and streaming (`LangChain.MessageDelta`)
responses:

    # Non-streaming
    {:ok, [%Message{metadata: %{"logprobs" => logprobs}}]} =
      ChatOpenAI.call(chat, [message], [])

    # Streaming - logprobs appear on each delta chunk
    chat = ChatOpenAI.new!(%{model: "gpt-4o", stream: true, logprobs: true})
    # Each MessageDelta will have metadata: %{"logprobs" => ...}

    # logprobs contains the raw OpenAI response structure:
    # %{
    #   "content" => [
    #     %{
    #       "token" => "Hello",
    #       "logprob" => -0.0002,
    #       "bytes" => [72, 101, 108, 108, 111],
    #       "top_logprobs" => [
    #         %{"token" => "Hello", "logprob" => -0.0002, ...},
    #         %{"token" => "Hi", "logprob" => -8.53, ...},
    #         ...
    #       ]
    #     },
    #     ...
    #   ]
    # }

When `logprobs` is not enabled or the response contains no logprobs data,
`metadata` will be `nil`.

See the [OpenAI documentation](https://developers.openai.com/api/reference/resources/completions/methods/create)
for full details on the response structure.

## Azure OpenAI Support

To use `ChatOpenAI` with Microsoft's Azure hosted OpenAI models, the
`endpoint` must be overridden and the API key needs to be provided in some
way. The [MS Quickstart guide for REST
access](https://learn.microsoft.com/en-us/azure/ai-services/openai/chatgpt-quickstart?tabs=command-line%2Cjavascript-keyless%2Ctypescript-keyless%2Cpython-new&pivots=rest-api)
may be helpful.

In order to use it, you must have an Azure account and from the console, a
model must be deployed for your account. Use the Azure AI Foundry and Azure
OpenAI Service to deploy the model you want to use. The entire URL is used as
the `endpoint` and the provided `key` is used as the `api_key`.

The following is an example of setting up `ChatOpenAI` for use with an Azure
hosted model.

    endpoint = System.fetch_env!("AZURE_OPENAI_ENDPOINT")
    api_key = System.fetch_env!("AZURE_OPENAI_KEY")

    llm =
      ChatOpenAI.new!(%{
        endpoint: endpoint,
        api_key: api_key,
        seed: 0,
        temperature: 1,
        stream: false
      })

The URL itself specifies the model to use and the `model` attribute is
disregarded.

A fake example URL for the endpoint value:

`https://some-subdomain.cognitiveservices.azure.com/openai/deployments/gpt-4o-mini/chat/completions?api-version=2024-08-01-preview"`

## Service Tier

The `service_tier` option asks the provider to serve the request with a
particular processing tier, trading cost against latency:

    ChatOpenAI.new!(%{model: "gpt-5", service_tier: "priority"})

OpenAI accepts values such as `"auto"`, `"default"`, `"flex"`, `"scale"`,
`"priority"` and `"fast"`. OpenAI-compatible providers accept their own
subset, so the value is passed through without validation.

The tier that actually served the request can differ from the one requested.
When the provider reports it, it is kept in the token usage's `raw` map:

    %Message{metadata: %{usage: %TokenUsage{raw: %{"service_tier" => tier}}}} = message

When streaming, token usage (and the tier with it) is only reported when
`stream_options: %{include_usage: true}` is set. Some providers don't report a
tier at all.

## Token Limits

`max_tokens` sets the upper bound on generated tokens. It is sent as
`max_completion_tokens`, OpenAI's current name for the limit. OpenAI's
reasoning models require that name, and the limit includes reasoning tokens.

    ChatOpenAI.new!(%{model: "gpt-5", max_tokens: 4000})

Many OpenAI-compatible servers only accept the older `max_tokens` key and
ignore `max_completion_tokens`. `LangChain.ChatModels.ChatOpenAICompatible`
sends the limit as `max_tokens`.

## Additional Request Parameters

`extra_body` is a map of values merged into the request body last, so they
override anything the model computed. It reaches API parameters that
`ChatOpenAI` has no field for:

    ChatOpenAI.new!(%{
      model: "gpt-5",
      extra_body: %{"store" => true, "metadata" => %{"run" => "eval-42"}}
    })

- Keys may be strings or atoms. A key naming one the model already sends
  replaces that value, so `%{"n" => 2}` overrides the `n` field.
- A `nil` value removes the key from the body. `%{"n" => nil}` stops the
  always-sent `n` from going out.
- Nested maps are merged key by key. Any other value replaces the existing
  one.

Overriding `stream`, `messages`, `tools` or `model` can break response
handling. Headers and transport options belong in `req_config`, not
`extra_body`. See `LangChain.Utils.merge_extra_body/2` for the full rules.

## Reasoning Model Support

OpenAI's reasoning models (the o-series and the gpt-5 family) take a
`reasoning_effort` setting. To send it, set `:reasoning_mode` to `true`:

    model = ChatOpenAI.new!(%{model: "gpt-5", reasoning_mode: true, reasoning_effort: "low"})

Setting `reasoning_mode` to `true` does two things:

- Sends `:reasoning_effort` in the request. It sets how much time, and how
  many tokens, the model spends reasoning before it answers. OpenAI rejects
  the setting on models that do not reason. The accepted values depend on
  the model; across current models they are "none", "minimal", "low",
  "medium" (the default here), "high" and "xhigh".
- Sends system messages under the `:developer` role, OpenAI's convention for
  reasoning models. OpenAI's API also accepts `:system` on its current
  reasoning models and treats it as `:developer`.

### Returned Thinking

OpenAI does not return a reasoning model's thinking on this API. Some
OpenAI-compatible services return it in a `reasoning_content` field, which
is parsed into a `LangChain.Message.ContentPart` of type `:thinking`. See
`LangChain.ChatModels.ChatOpenAICompatible` for those services.

## Connection Retry Behavior

The `retry_count` option controls how many times a request is retried when
a pooled HTTP connection turns out to be stale (server closed it between
requests). This is a transport-level issue where retrying with a fresh
connection is the correct response.

**Only closed-connection errors are retried.** Timeouts, rate limits (429),
overloaded (529), authentication errors, and invalid requests all return
immediately -- they are not problems that a simple retry will fix.

| `retry_count` | Total HTTP requests |
|---|---|
| `0` | 1 (no retries) |
| `1` | 2 (1 initial + 1 retry) |
| `2` (default) | 3 (1 initial + 2 retries) |

Req's built-in HTTP retry is disabled to prevent the two retry layers from
compounding. See [GitHub issue #503](https://github.com/brainlid/langchain/issues/503).

When running LLM calls from a background job queue (e.g., Oban) that has its
own retry logic, set `retry_count: 0` so there are no hidden retries:

    ChatOpenAI.new!(%{model: "...", retry_count: 0})

# `t`
[🔗](https://github.com/brainlid/langchain/blob/v0.15.3/lib/chat_models/chat_open_ai.ex#L495)

```elixir
@type t() :: %LangChain.ChatModels.ChatOpenAI{
  api_key: term(),
  callbacks: term(),
  endpoint: term(),
  extra_body: term(),
  frequency_penalty: term(),
  json_response: term(),
  json_schema: term(),
  logprobs: term(),
  max_tokens: term(),
  model: term(),
  n: term(),
  org_id: term(),
  parallel_tool_calls: term(),
  reasoning_effort: term(),
  reasoning_mode: term(),
  receive_timeout: term(),
  req_config: term(),
  retry_count: term(),
  seed: term(),
  service_tier: term(),
  stream: term(),
  stream_options: term(),
  temperature: term(),
  tool_choice: term(),
  top_logprobs: term(),
  user: term(),
  verbose_api: term(),
  verbosity: term()
}
```

# `call`
[🔗](https://github.com/brainlid/langchain/blob/v0.15.3/lib/chat_models/chat_open_ai.ex#L738)

Calls the OpenAI API passing the ChatOpenAI struct with configuration, plus
either a simple message or the list of messages to act as the prompt.

Optionally pass in a list of tools available to the LLM for requesting
execution in response.

Optionally pass in a callback function that can be executed as data is
received from the API.

**NOTE:** This function *can* be used directly, but the primary interface
should be through `LangChain.Chains.LLMChain`. The `ChatOpenAI` module is more
focused on translating the `LangChain` data structures to and from the OpenAI
API.

Another benefit of using `LangChain.Chains.LLMChain` is that it combines the
storage of messages, adding tools, adding custom context that should be
passed to tools, and automatically applying `LangChain.MessageDelta`
structs as they are are received, then converting those to the full
`LangChain.Message` once fully complete.

# `content_part_for_api`
[🔗](https://github.com/brainlid/langchain/blob/v0.15.3/lib/chat_models/chat_open_ai.ex#L704)

Convert a ContentPart to the expected map of data for the OpenAI API.

# `content_parts_for_api`
[🔗](https://github.com/brainlid/langchain/blob/v0.15.3/lib/chat_models/chat_open_ai.ex#L697)

Convert a list of ContentParts to the expected map of data for the OpenAI API.

Thinking and unsupported parts are omitted. Both are response-side artifacts
with no request representation on this API surface. The omission is
unconditional so a conversation presents the same prompt prefix turn after
turn, which keeps prompt caching working.

# `decode_stream`
[🔗](https://github.com/brainlid/langchain/blob/v0.15.3/lib/chat_models/chat_open_ai.ex#L977)

```elixir
@spec decode_stream(
  {String.t(), String.t()},
  list()
) :: {[%{required(String.t()) => any()}], String.t()}
```

Decode a streamed response from an OpenAI-compatible server. Parses a string
of received content into an Elixir map data structure using string keys.

If a partial response was received, meaning the JSON text is split across
multiple data frames, then the incomplete portion is returned as-is in the
buffer. The function will be successively called, receiving the incomplete
buffer data from a previous call, and assembling it to parse.

# `for_api`
[🔗](https://github.com/brainlid/langchain/blob/v0.15.3/lib/chat_models/chat_open_ai.ex#L685)

```elixir
@spec for_api(
  struct(),
  LangChain.Message.t()
  | LangChain.PromptTemplate.t()
  | LangChain.Message.ToolCall.t()
  | LangChain.Message.ToolResult.t()
  | LangChain.Message.ContentPart.t()
  | LangChain.Function.t()
) :: %{required(String.t()) => any()} | [%{required(String.t()) => any()}]
```

Convert a LangChain Message-based structure to the expected map of data for
the OpenAI API. This happens within the context of the model configuration as
well. The additional context is needed to correctly convert a role to either
`:system` or `:developer`.

NOTE: The `ChatOpenAI` model's functions are reused in other modules. For this
reason, model is more generally defined as a struct.

# `for_api`
[🔗](https://github.com/brainlid/langchain/blob/v0.15.3/lib/chat_models/chat_open_ai.ex#L597)

```elixir
@spec for_api(
  t() | LangChain.Message.t() | LangChain.Function.t(),
  message :: [map()],
  LangChain.ChatModels.ChatModel.tools()
) :: %{required(atom()) => any()}
```

Return the params formatted for an API request.

# `new`
[🔗](https://github.com/brainlid/langchain/blob/v0.15.3/lib/chat_models/chat_open_ai.ex#L548)

```elixir
@spec new(attrs :: map()) :: {:ok, t()} | {:error, Ecto.Changeset.t()}
```

Setup a ChatOpenAI client configuration.

# `new!`
[🔗](https://github.com/brainlid/langchain/blob/v0.15.3/lib/chat_models/chat_open_ai.ex#L559)

```elixir
@spec new!(attrs :: map()) :: t() | no_return()
```

Setup a ChatOpenAI client configuration and return it or raise an error if invalid.

# `restore_from_map`
[🔗](https://github.com/brainlid/langchain/blob/v0.15.3/lib/chat_models/chat_open_ai.ex#L1065)

Restores the model from the config.

# `retry_on_fallback?`
[🔗](https://github.com/brainlid/langchain/blob/v0.15.3/lib/chat_models/chat_open_ai.ex#L1025)

```elixir
@spec retry_on_fallback?(LangChain.LangChainError.t()) :: boolean()
```

Determine if an error should be retried. If `true`, a fallback LLM may be
used. If `false`, the error is understood to be more fundamental with the
request rather than a service issue and it should not be retried or fallback
to another service.

# `serialize_config`
[🔗](https://github.com/brainlid/langchain/blob/v0.15.3/lib/chat_models/chat_open_ai.ex#L1036)

```elixir
@spec serialize_config(t()) :: %{required(String.t()) => any()}
```

Generate a config map that can later restore the model's configuration.

---

*Consult [api-reference.md](api-reference.md) for complete listing*