Represents the OpenAI ChatModel.

Parses and validates inputs for making a requests from the OpenAI Chat API.

Converts responses into more specialized LangChain data structures.

• https://github.com/openai/openai-cookbook/blob/main/examples/How_to_call_functions_with_chat_models.ipynb

OpenAI-compatible services

This module follows OpenAI's own API, including Azure OpenAI, and adopts OpenAI's changes as they ship. Services that offer an OpenAI-compatible endpoint (Cloudflare Workers AI, Groq, OpenRouter, vLLM, SGLang, Ollama, LM Studio and others) often do not support those changes, and many ignore or silently drop what they do not understand. For example, a system message sent under the developer role, as reasoning_mode does, never reaches a model served by SGLang or vLLM when the model's chat template has no developer branch.

Use LangChain.ChatModels.ChatOpenAICompatible for those services. It sends only the fields you set, always sends system messages as system, and never falls back to the global OpenAI API key.

ContentPart Types

OpenAI supports several types of content parts that can be combined in a single message:

Text Content

Basic text content is the default and most common type:

Message.new_user!("Hello, how are you?")

Image Content

OpenAI supports both base64-encoded images and image URLs:

# Using a base64 encoded image
Message.new_user!([
  ContentPart.text!("What's in this image?"),
  ContentPart.image!("base64_encoded_image_data", media: :jpg)
])

# Using an image URL
Message.new_user!([
  ContentPart.text!("Describe this image:"),
  ContentPart.image_url!("https://example.com/image.jpg")
])

For images, you can specify the detail level which affects token usage:

• detail: "low" - Lower resolution, fewer tokens
• detail: "high" - Higher resolution, more tokens
• detail: "auto" - Let the model decide

File Content

OpenAI supports both base64-encoded files and file IDs:

# Using a base64 encoded file
Message.new_user!([
  ContentPart.text!("Process this file:"),
  ContentPart.file!("base64_encoded_file_data",
    type: :base64,
    filename: "document.pdf"
  )
])

# Using a file ID (after uploading to OpenAI)
Message.new_user!([
  ContentPart.text!("Process this file:"),
  ContentPart.file!("file-1234", type: :file_id)
])

Callbacks

See the set of available callbacks: LangChain.Chains.ChainCallbacks

Rate Limit API Response Headers

OpenAI returns rate limit information in the response headers. Those can be accessed using the LLM callback on_llm_ratelimit_info like this:

handlers = %{
  on_llm_ratelimit_info: fn headers ->
    IO.inspect(headers)
  end
}

{:ok, chat} = ChatOpenAI.new(%{callbacks: [handlers]})

Handlers assigned to the model are fired by the model itself and receive only the event's argument. Handlers assigned to an LangChain.Chains.LLMChain receive the chain as an additional first argument:

handler = %{
  on_llm_ratelimit_info: fn _chain, headers ->
    IO.inspect(headers)
  end
}

%{llm: ChatOpenAI.new!(%{})}
|> LLMChain.new!()
|> LLMChain.add_callback(handler)

When a request is received, something similar to the following will be output to the console.

%{
  "x-ratelimit-limit-requests" => ["5000"],
  "x-ratelimit-limit-tokens" => ["160000"],
  "x-ratelimit-remaining-requests" => ["4999"],
  "x-ratelimit-remaining-tokens" => ["159973"],
  "x-ratelimit-reset-requests" => ["12ms"],
  "x-ratelimit-reset-tokens" => ["10ms"],
  "x-request-id" => ["req_1234"]
}

Token Usage

OpenAI returns token usage information as part of the response body. The LangChain.TokenUsage is added to the metadata of the LangChain.Message and LangChain.MessageDelta structs that are processed under the :usage key.

The OpenAI documentation instructs to provide the stream_options with the include_usage: true for the information to be provided.

chat = ChatOpenAI.new!(%{stream: true, stream_options: %{include_usage: true}})

The TokenUsage data is accumulated for MessageDelta structs and the final usage information will be on the LangChain.Message.

NOTE: Of special note is that the TokenUsage information is returned once for all "choices" in the response. The LangChain.TokenUsage data is added to each message, but if your usage requests multiple choices, you will see the same usage information for each choice but it is duplicated and only one response is meaningful.

Tool Choice

OpenAI's ChatGPT API supports forcing a tool to be used.

• https://platform.openai.com/docs/api-reference/chat/create#chat-create-tool_choice

This is supported through the tool_choice options. It takes a plain Elixir map to provide the configuration.

By default, the LLM will choose a tool call if a tool is available and it determines it is needed. That's the "auto" mode.

Parallel Tool Calls

By default, OpenAI models may decide to make multiple tool calls at once, including calling the same tool multiple times. You can limit this behavior by setting the parallel_tool_calls option to false.

Example

For the LLM's response to make a tool call of the "get_weather" function.

ChatOpenAI.new(%{
  model: "...",
  tool_choice: %{"type" => "function", "function" => %{"name" => "get_weather"}}
})

Log Probabilities

OpenAI can return log probability information for output tokens, which is useful for evaluating model confidence, building classifiers, or debugging token selection.

Enable with the logprobs option. Optionally set top_logprobs to receive the N most likely tokens (0-20) at each position:

chat = ChatOpenAI.new!(%{
  model: "gpt-4o",
  logprobs: true,
  top_logprobs: 3
})

When enabled, the logprobs data from the API response is placed in the metadata field under the "logprobs" key. This works for both non-streaming (LangChain.Message) and streaming (LangChain.MessageDelta) responses:

# Non-streaming
{:ok, [%Message{metadata: %{"logprobs" => logprobs}}]} =
  ChatOpenAI.call(chat, [message], [])

# Streaming - logprobs appear on each delta chunk
chat = ChatOpenAI.new!(%{model: "gpt-4o", stream: true, logprobs: true})
# Each MessageDelta will have metadata: %{"logprobs" => ...}

# logprobs contains the raw OpenAI response structure:
# %{
#   "content" => [
#     %{
#       "token" => "Hello",
#       "logprob" => -0.0002,
#       "bytes" => [72, 101, 108, 108, 111],
#       "top_logprobs" => [
#         %{"token" => "Hello", "logprob" => -0.0002, ...},
#         %{"token" => "Hi", "logprob" => -8.53, ...},
#         ...
#       ]
#     },
#     ...
#   ]
# }

When logprobs is not enabled or the response contains no logprobs data, metadata will be nil.

See the OpenAI documentation for full details on the response structure.

Azure OpenAI Support

To use ChatOpenAI with Microsoft's Azure hosted OpenAI models, the endpoint must be overridden and the API key needs to be provided in some way. The MS Quickstart guide for REST access may be helpful.

In order to use it, you must have an Azure account and from the console, a model must be deployed for your account. Use the Azure AI Foundry and Azure OpenAI Service to deploy the model you want to use. The entire URL is used as the endpoint and the provided key is used as the api_key.

The following is an example of setting up ChatOpenAI for use with an Azure hosted model.

endpoint = System.fetch_env!("AZURE_OPENAI_ENDPOINT")
api_key = System.fetch_env!("AZURE_OPENAI_KEY")

llm =
  ChatOpenAI.new!(%{
    endpoint: endpoint,
    api_key: api_key,
    seed: 0,
    temperature: 1,
    stream: false
  })

The URL itself specifies the model to use and the model attribute is disregarded.

A fake example URL for the endpoint value:

https://some-subdomain.cognitiveservices.azure.com/openai/deployments/gpt-4o-mini/chat/completions?api-version=2024-08-01-preview"

Service Tier

The service_tier option asks the provider to serve the request with a particular processing tier, trading cost against latency:

ChatOpenAI.new!(%{model: "gpt-5", service_tier: "priority"})

OpenAI accepts values such as "auto", "default", "flex", "scale", "priority" and "fast". OpenAI-compatible providers accept their own subset, so the value is passed through without validation.

The tier that actually served the request can differ from the one requested. When the provider reports it, it is kept in the token usage's raw map:

%Message{metadata: %{usage: %TokenUsage{raw: %{"service_tier" => tier}}}} = message

When streaming, token usage (and the tier with it) is only reported when stream_options: %{include_usage: true} is set. Some providers don't report a tier at all.

Token Limits

max_tokens sets the upper bound on generated tokens. It is sent as max_completion_tokens, OpenAI's current name for the limit. OpenAI's reasoning models require that name, and the limit includes reasoning tokens.

ChatOpenAI.new!(%{model: "gpt-5", max_tokens: 4000})

Many OpenAI-compatible servers only accept the older max_tokens key and ignore max_completion_tokens. LangChain.ChatModels.ChatOpenAICompatible sends the limit as max_tokens.

Additional Request Parameters

extra_body is a map of values merged into the request body last, so they override anything the model computed. It reaches API parameters that ChatOpenAI has no field for:

ChatOpenAI.new!(%{
  model: "gpt-5",
  extra_body: %{"store" => true, "metadata" => %{"run" => "eval-42"}}
})

• Keys may be strings or atoms. A key naming one the model already sends replaces that value, so %{"n" => 2} overrides the n field.
• A nil value removes the key from the body. %{"n" => nil} stops the always-sent n from going out.
• Nested maps are merged key by key. Any other value replaces the existing one.

Overriding stream, messages, tools or model can break response handling. Headers and transport options belong in req_config, not extra_body. See LangChain.Utils.merge_extra_body/2 for the full rules.

Reasoning Model Support

OpenAI's reasoning models (the o-series and the gpt-5 family) take a reasoning_effort setting. To send it, set :reasoning_mode to true:

model = ChatOpenAI.new!(%{model: "gpt-5", reasoning_mode: true, reasoning_effort: "low"})

Setting reasoning_mode to true does two things:

• Sends :reasoning_effort in the request. It sets how much time, and how many tokens, the model spends reasoning before it answers. OpenAI rejects the setting on models that do not reason. The accepted values depend on the model; across current models they are "none", "minimal", "low", "medium" (the default here), "high" and "xhigh".
• Sends system messages under the :developer role, OpenAI's convention for reasoning models. OpenAI's API also accepts :system on its current reasoning models and treats it as :developer.

Returned Thinking

OpenAI does not return a reasoning model's thinking on this API. Some OpenAI-compatible services return it in a reasoning_content field, which is parsed into a LangChain.Message.ContentPart of type :thinking. See LangChain.ChatModels.ChatOpenAICompatible for those services.

Connection Retry Behavior

The retry_count option controls how many times a request is retried when a pooled HTTP connection turns out to be stale (server closed it between requests). This is a transport-level issue where retrying with a fresh connection is the correct response.

Only closed-connection errors are retried. Timeouts, rate limits (429), overloaded (529), authentication errors, and invalid requests all return immediately -- they are not problems that a simple retry will fix.

Req's built-in HTTP retry is disabled to prevent the two retry layers from compounding. See GitHub issue #503.

When running LLM calls from a background job queue (e.g., Oban) that has its own retry logic, set retry_count: 0 so there are no hidden retries:

ChatOpenAI.new!(%{model: "...", retry_count: 0})