Skip to main content

ChatOpenAI

OpenAI is an artificial intelligence (AI) research laboratory.

This guide will help you getting started with ChatOpenAI chat models. For detailed documentation of all ChatOpenAI features and configurations head to the API reference.

Overview​

Integration details​

ClassPackageLocalSerializablePY supportPackage downloadsPackage latest
ChatOpenAI@langchain/openaiβŒβœ…βœ…NPM - DownloadsNPM - Version

Model features​

See the links in the table headers below for guides on how to use specific features.

Tool callingStructured outputJSON modeImage inputAudio inputVideo inputToken-level streamingToken usageLogprobs
βœ…βœ…βœ…βœ…βŒβŒβœ…βœ…βœ…

Setup​

To access OpenAI chat models you’ll need to create an OpenAI account, get an API key, and install the @langchain/openai integration package.

Credentials​

Head to OpenAI’s website to sign up for OpenAI and generate an API key. Once you’ve done this set the OPENAI_API_KEY environment variable:

export OPENAI_API_KEY="your-api-key"

If you want to get automated tracing of your model calls you can also set your LangSmith API key by uncommenting below:

# export LANGCHAIN_TRACING_V2="true"
# export LANGCHAIN_API_KEY="your-api-key"

Installation​

The LangChain ChatOpenAI integration lives in the @langchain/openai package:

yarn add @langchain/openai

Instantiation​

Now we can instantiate our model object and generate chat completions:

import { ChatOpenAI } from "@langchain/openai";

const llm = new ChatOpenAI({
model: "gpt-4o",
temperature: 0,
// other params...
});

Invocation​

const aiMsg = await llm.invoke([
[
"system",
"You are a helpful assistant that translates English to French. Translate the user sentence.",
],
["human", "I love programming."],
]);
aiMsg;
AIMessage {
"id": "chatcmpl-9rB4GvhlRb0x3hxupLBQYOKKmTxvV",
"content": "J'adore la programmation.",
"additional_kwargs": {},
"response_metadata": {
"tokenUsage": {
"completionTokens": 8,
"promptTokens": 31,
"totalTokens": 39
},
"finish_reason": "stop"
},
"tool_calls": [],
"invalid_tool_calls": [],
"usage_metadata": {
"input_tokens": 31,
"output_tokens": 8,
"total_tokens": 39
}
}
console.log(aiMsg.content);
J'adore la programmation.

Chaining​

We can chain our model with a prompt template like so:

import { ChatPromptTemplate } from "@langchain/core/prompts";

const prompt = ChatPromptTemplate.fromMessages([
[
"system",
"You are a helpful assistant that translates {input_language} to {output_language}.",
],
["human", "{input}"],
]);

const chain = prompt.pipe(llm);
await chain.invoke({
input_language: "English",
output_language: "German",
input: "I love programming.",
});
AIMessage {
"id": "chatcmpl-9rB4JD9rVBLzTuMee9AabulowEH0d",
"content": "Ich liebe das Programmieren.",
"additional_kwargs": {},
"response_metadata": {
"tokenUsage": {
"completionTokens": 6,
"promptTokens": 26,
"totalTokens": 32
},
"finish_reason": "stop"
},
"tool_calls": [],
"invalid_tool_calls": [],
"usage_metadata": {
"input_tokens": 26,
"output_tokens": 6,
"total_tokens": 32
}
}

Custom URLs​

You can customize the base URL the SDK sends requests to by passing a configuration parameter like this:

import { ChatOpenAI } from "@langchain/openai";

const llmWithCustomURL = new ChatOpenAI({
temperature: 0.9,
configuration: {
baseURL: "https://your_custom_url.com",
},
});

await llmWithCustomURL.invoke("Hi there!");

The configuration field also accepts other ClientOptions parameters accepted by the official SDK.

If you are hosting on Azure OpenAI, see the dedicated page instead.

Custom headers​

You can specify custom headers in the same configuration field:

import { ChatOpenAI } from "@langchain/openai";

const llmWithCustomHeaders = new ChatOpenAI({
temperature: 0.9,
configuration: {
defaultHeaders: {
Authorization: `Bearer SOME_CUSTOM_VALUE`,
},
},
});

await llmWithCustomHeaders.invoke("Hi there!");

Calling fine-tuned models​

You can call fine-tuned OpenAI models by passing in your corresponding modelName parameter.

This generally takes the form of ft:{OPENAI_MODEL_NAME}:{ORG_NAME}::{MODEL_ID}. For example:

import { ChatOpenAI } from "@langchain/openai";

const fineTunedLlm = new ChatOpenAI({
temperature: 0.9,
model: "ft:gpt-3.5-turbo-0613:{ORG_NAME}::{MODEL_ID}",
});

await fineTunedLlm.invoke("Hi there!");

Generation metadata​

If you need additional information like logprobs or token usage, these will be returned directly in the .invoke response within the response_metadata field on the message.

tip

Requires @langchain/core version >=0.1.48.

import { ChatOpenAI } from "@langchain/openai";

// See https://cookbook.openai.com/examples/using_logprobs for details
const llmWithLogprobs = new ChatOpenAI({
logprobs: true,
// topLogprobs: 5,
});

const responseMessageWithLogprobs = await llmWithLogprobs.invoke("Hi there!");
console.dir(responseMessageWithLogprobs.response_metadata.logprobs, {
depth: null,
});
{
content: [
{
token: 'Hello',
logprob: -0.0005151443,
bytes: [ 72, 101, 108, 108, 111 ],
top_logprobs: []
},
{
token: '!',
logprob: -0.00004334534,
bytes: [ 33 ],
top_logprobs: []
},
{
token: ' How',
logprob: -0.000035477897,
bytes: [ 32, 72, 111, 119 ],
top_logprobs: []
},
{
token: ' can',
logprob: -0.0006658526,
bytes: [ 32, 99, 97, 110 ],
top_logprobs: []
},
{
token: ' I',
logprob: -0.0000010280384,
bytes: [ 32, 73 ],
top_logprobs: []
},
{
token: ' assist',
logprob: -0.10124119,
bytes: [
32, 97, 115,
115, 105, 115,
116
],
top_logprobs: []
},
{
token: ' you',
logprob: -5.5122365e-7,
bytes: [ 32, 121, 111, 117 ],
top_logprobs: []
},
{
token: ' today',
logprob: -0.000052643223,
bytes: [ 32, 116, 111, 100, 97, 121 ],
top_logprobs: []
},
{
token: '?',
logprob: -0.000012352386,
bytes: [ 63 ],
top_logprobs: []
}
]
}

Tool calling​

Tool calling with OpenAI models works in a similar to other models. Additionally, the following guides have some information especially relevant to OpenAI:

strict: true​

Requires @langchain/openai >= 0.2.6

As of Aug 6, 2024, OpenAI supports a strict argument when calling tools that will enforce that the tool argument schema is respected by the model. See more here: https://platform.openai.com/docs/guides/function-calling

Note: If strict: true the tool definition will also be validated, and a subset of JSON schema are accepted. Crucially, schema cannot have optional args (those with default values). Read the full docs on what types of schema are supported here: https://platform.openai.com/docs/guides/structured-outputs/supported-schemas.

import { ChatOpenAI } from "@langchain/openai";
import { tool } from "@langchain/core/tools";
import { z } from "zod";

const weatherTool = tool((_) => "no-op", {
name: "get_current_weather",
description: "Get the current weather",
schema: z.object({
location: z.string(),
}),
});

const llmWithStrictTrue = new ChatOpenAI({
model: "gpt-4o",
}).bindTools([weatherTool], {
strict: true,
tool_choice: weatherTool.name,
});

// Although the question is not about the weather, it will call the tool with the correct arguments
// because we passed `tool_choice` and `strict: true`.
const strictTrueResult = await llmWithStrictTrue.invoke(
"What is 127862 times 12898 divided by 2?"
);

console.dir(strictTrueResult.tool_calls, { depth: null });
[
{
name: 'get_current_weather',
args: { location: 'Hanoi' },
type: 'tool_call',
id: 'call_aB85ybkLCoccpzqHquuJGH3d'
}
]

API reference​

For detailed documentation of all ChatOpenAI features and configurations head to the API reference: https://api.js.langchain.com/classes/langchain_openai.ChatOpenAI.html


Was this page helpful?


You can also leave detailed feedback on GitHub.