Models a ContentPart. ContentParts are now used for multi-modal support in
both messages and tool results. This enables richer responses, allowing text,
images, files, and thinking blocks to be combined in a single message or tool
result.
Types
:text- The message part is text.:image_url- The message part is a URL to an image.:image- The message part is image data that is base64 encoded text.:file- The message part is file data that is base64 encoded text.:file_url- The message part is a URL to a file.:thinking- A thinking block from a reasoning model like Anthropic.:unsupported- A part that is not supported but may need to be present. This includes Anthropic'sredacted_thinkingblock which has no value in being displayed because it is encrypted, but can be provided back to the LLM to maintain reasoning continuity. The specific parts of the data are stored in:options.:provider_item- An opaque block or item a provider produced that must go back to that provider unchanged, such as the blocks a tool search leaves in an assistant message. See "Provider items".:tool_reference- Names a tool for the provider to load into the model's context, placed in a tool result. The tool name is thecontent. See "Tool references".
Fields
:content- Text content.:options- Options are a keyword list of values that may be specific to the LLM for a particular message type. For example, multi-modal message (ones that include image data) use the:mediaoption to specify the mimetype information. Options may also contain key-value settings likecache_control: truefor models like Anthropic that support caching, orprompt_cache_breakpoint: %{mode: "explicit"}for supported OpenAI Responses input blocks. SeeLangChain.ChatModels.ChatOpenAIResponsesfor prompt caching configuration.When receiving content parts like with Anthropic Claude's thinking model, the options may contain LLM specific data that is recommended to be preserved like a
signatureorredacted_thinkingdata used by the LLM.A map of options is accepted and converted to a keyword list, so a part holds the same shape whichever way it was built and keeps it across a serialization round trip. The keys must be atoms; a map with a string key is rejected rather than converted, because turning caller-supplied strings into atoms would let untrusted input grow the atom table.
Narration and answers
Some models speak more than once in a single turn. Before calling a tool, a model may say what it is about to do ("I'll check the logs first"). That text is narration: a status update about work in progress, not a reply. The turn is not over until the model answers.
A part records which kind of utterance it is in the :utterance option:
"narration"- the model is describing work in progress and has more to do. Build one withnarration!/1."answer"- the model is replying. Build one withanswer!/1.nil(option absent) - unmarked. The provider did not say, and the part is treated as an answer.
Read the marker with utterance/1 or narration?/1 rather than from the
options directly. Chat models that support the distinction set the marker on
the parts they decode, and send it back to the provider when the message is
replayed as history, because a model that sees its own narration replayed as
an unlabelled reply behaves worse on later turns.
LangChain.Message.narration?/1 answers the question for a whole message,
and LangChain.Chains.LLMChain uses it to keep a turn open.
Provider items
Some provider responses contain blocks that only the provider understands
and that must be sent back exactly as they arrived, in the same position,
when the conversation continues. Tool search produces them: Anthropic's
server_tool_use and tool_search_tool_result blocks, and OpenAI's
tool_search_call and tool_search_output items. A conversation that
searched for tools once replays the search on every later request, which is
how the loaded tools stay loaded.
A :provider_item part holds one such block. Build one with
provider_item!/2:
:provideroption - the string name of the provider that produced it:"anthropic"or"openai". It names the provider's API, not the chat model, so any chat model that speaks that API replays the item. An itemLangChain.ChatModels.ChatAnthropicdecoded replays throughLangChain.ChatModels.ChatReqLLMon an Anthropic model, and the other way around. Likewise, an itemLangChain.ChatModels.ChatOpenAIResponsesdecoded replays throughChatReqLLMon an OpenAI model.:itemoption - the block itself, as a string-keyed map of JSON values.:content-nilfor a complete item. While a streamed item is still arriving, a chat model may accumulate a fragment of it here.
The chat model for the named provider sends the item back. Every other chat
model leaves it out of the request, and the text helpers such as
content_to_string/2 skip it. Keep provider items wherever the conversation
is stored: a stored conversation without them replays calls to tools whose
search is gone, which the provider may reject.
Tool references
A tool can load other tools for the model by returning :tool_reference
parts in its result, built with tool_reference!/1. This is client-side
tool search: each referenced tool must be declared to the provider, normally
as a deferred function (see "Tool search options" in LangChain.Function).
On a provider that loads referenced tools, the reference becomes the
provider's own reference block. Any other chat model sends every deferred
function as an ordinary tool already, so it renders the reference as a line
of text naming the tool (tool_reference_text/1).
The text helpers read a reference the same way. When asked for :text,
content_to_string/2 and parts_to_string/2 include the
"Tool available: <name>" line, so a tool result that only loads tools
still reads as something.
Image mime types
The :media option is used to specify the mime type of the image. Various
LLMs handle this differently or perhaps not at all.
Examples:
media: :jpg- turns into"image/jpeg"or"image/jpg", depending on what the LLM accepts.media: :png- turns into"image/png"media: "image/webp"- stays as"image/webp". Any specified string value is passed through unchanged. This allows for future formats to be supported quickly.- When omitted, the LLM may error or some will accept it but may require the
base64encoded content data to be prefixed with the mime type information. Basically, you must handle the content needs yourself.
Summary
Functions
Create a text ContentPart explicitly marked as an answer. Raises an exception if not valid.
Returns the citations for this content part, defaulting to empty list.
Convert "content" to a string. Content may be nil, a string, or a list of ContentParts.
Create a new ContentPart that contains a file encoded as base64 data.
Create a new ContentPart that contains a URL to an file. Raises an exception if not valid.
Returns true if this content part has any citations.
Create a new ContentPart that contains an image encoded as base64 data. Raises an exception if not valid.
Create a new ContentPart that contains a URL to an image. Raises an exception if not valid.
Merge two ContentPart structs for the same index in a MessageDelta. The
first ContentPart is the primary one that smaller deltas are merged into.
The primary is what is being accumulated.
Create a text ContentPart marked as narration: the model describing work in progress rather than answering. Raises an exception if not valid.
Return true when the part is marked as narration.
Build a new message and return an :ok/:error tuple with the result.
Build a new message and return it or raise an error if invalid.
Helper function for easily getting plain text from a list of ContentParts.
Create a :provider_item part holding a block that provider produced and
expects back unchanged. See "Provider items". Raises an exception if not
valid.
Return true when the part is a :provider_item produced by provider.
Return the block a :provider_item part holds, ready to send back.
Mark a part as "narration" or "answer". Any other value raises a
FunctionClauseError.
Return true when the part holds a model's reasoning rather than its output.
Sets an option on the last text part in a list of ContentParts. Returns the updated content parts.
Create a new ContentPart that contains text. Raises an exception if not valid.
Create a new ContentPart that contains thinking text. Raises an exception if not valid.
Create a :tool_reference part naming a tool for the provider to load. See
"Tool references". Raises an exception if not valid.
Render a :tool_reference part as the text a chat model without client-side
tool loading sends in its place.
Return the part's utterance marker: "narration", "answer", or nil when
the part is unmarked. A value other than the two recognized strings reads as
nil.
Types
Functions
Create a text ContentPart explicitly marked as an answer. Raises an exception if not valid.
@spec citations(t()) :: [LangChain.Message.Citation.t()]
Returns the citations for this content part, defaulting to empty list.
Convert "content" to a string. Content may be nil, a string, or a list of ContentParts.
Create a new ContentPart that contains a file encoded as base64 data.
Create a new ContentPart that contains a URL to an file. Raises an exception if not valid.
Returns true if this content part has any citations.
Create a new ContentPart that contains an image encoded as base64 data. Raises an exception if not valid.
Options
:media- Provide the "media type" for the image. Examples: "image/jpeg", "image/png", etc. ChatGPT does not require this but other LLMs may.:detail- if the LLM supports it, most images must be resized or cropped before given to the LLM for analysis. A detail option may specify the level detail of the image to present to the LLM. The higher the detail, the more tokens consumed. Currently only supported by OpenAI and the values of "low", "high", and "auto".
ChatGPT requires media type information to prefix the base64 content. Setting
the media: "image/jpeg" type will do that. Otherwise the data must be
provided with the required prefix.
Anthropic requires the media type information to be submitted as separate information with the JSON request. This media option provides an abstraction to normalize the behavior.
Create a new ContentPart that contains a URL to an image. Raises an exception if not valid.
Merge two ContentPart structs for the same index in a MessageDelta. The
first ContentPart is the primary one that smaller deltas are merged into.
The primary is what is being accumulated.
A set of ContentParts can be merged like this:
Enum.reduce(list_of_content_parts, nil, fn new_part, acc ->
ContentPart.merge_part(acc, new_part)
end)Create a text ContentPart marked as narration: the model describing work in progress rather than answering. Raises an exception if not valid.
Example
ContentPart.narration!("I'll check the logs first.")Return true when the part is marked as narration.
@spec new(attrs :: map()) :: {:ok, t()} | {:error, Ecto.Changeset.t()}
Build a new message and return an :ok/:error tuple with the result.
Build a new message and return it or raise an error if invalid.
Example
ContentPart.new!(%{type: :text, content: "Greetings!"})
ContentPart.new!(%{type: :image_url, content: "https://example.com/images/house.jpg"})
ContentPart.new!(%{type: :thinking, content: "I've been asked...", options: [signature: "SIGNATURE_DATA"]}
ContentPart.new!(%{type: :unsupported, content: "redacted_data", options: [type: "redacted_thinking"]}Helper function for easily getting plain text from a list of ContentParts.
This function processes a list of ContentParts and joins the text parts together using "
" characters. Only parts where type: :text are used, plus
:tool_reference parts, which read as the line tool_reference_text/1
renders. A tool result made only of references therefore still reads as
text. All other parts are ignored.
Examples
iex> parts = [
...> text!("Hello"),
...> image!("base64data"),
...> text!("world")
...> ]
iex> parts_to_string(parts)
"Hello\n\nworld"
iex> parts_to_string([])
nilCreate a :provider_item part holding a block that provider produced and
expects back unchanged. See "Provider items". Raises an exception if not
valid.
Example
ContentPart.provider_item!("anthropic", %{
"type" => "tool_search_tool_result",
"tool_use_id" => "srvtoolu_01ABC123",
"content" => %{"type" => "tool_search_tool_search_result", "tool_references" => []}
})Return true when the part is a :provider_item produced by provider.
Return the block a :provider_item part holds, ready to send back.
A streamed item can arrive as its opening block plus the block's "input"
in JSON fragments, which a chat model accumulates in the part's content.
When content holds a JSON object, it becomes the block's "input". A
complete item, with content: nil, is returned as stored.
Mark a part as "narration" or "answer". Any other value raises a
FunctionClauseError.
Return true when the part holds a model's reasoning rather than its output.
Reasoning arrives in several shapes, depending on the chat model that decoded it:
- any
:thinkingpart: Anthropic thinking,reasoning_contentfrom OpenAI-compatible services, and streamed OpenAI Responses reasoning items - an
:unsupportedpart whose:typeoption is"redacted_thinking"(Anthropic) or"reasoning"(a non-streamed OpenAI Responses item) - an
:unsupportedpart carrying:reasoning_details(LangChain.ChatModels.ChatReqLLM)
Provider items, such as the blocks a tool search leaves behind, are not reasoning.
Examples
iex> alias LangChain.Message.ContentPart
iex> ContentPart.reasoning?(ContentPart.thinking!("Let me think."))
true
iex> ContentPart.reasoning?(ContentPart.text!("Hello"))
falseSets an option on the last text part in a list of ContentParts. Returns the updated content parts.
Create a new ContentPart that contains text. Raises an exception if not valid.
Create a new ContentPart that contains thinking text. Raises an exception if not valid.
Create a :tool_reference part naming a tool for the provider to load. See
"Tool references". Raises an exception if not valid.
Example
ContentPart.tool_reference!("devices_list")Render a :tool_reference part as the text a chat model without client-side
tool loading sends in its place.
Return the part's utterance marker: "narration", "answer", or nil when
the part is unmarked. A value other than the two recognized strings reads as
nil.
Examples
iex> utterance(narration!("Looking that up."))
"narration"
iex> utterance(text!("Hello"))
nil