OpenAI adds generative text features and lowers prices

OpenAI is improving its text-generating models and lowering prices as generative AI competition increases.

Today, OpenAI released GPT-3.5-turbo and GPT-4, its latest text-generating AI with function calling. In a blog post, OpenAI explains that function calling lets developers describe programming functions to GPT-3.5-turbo and GPT-4 and have the models generate code to execute them.

Function calling can help chatbots call external tools, convert natural language into database queries, and extract structured data from text. “These models have been fine-tuned to both detect when a function needs to be called… and to respond with JSON that adheres to the function signature,” OpenAI writes. Developers can get structured model data more reliably with function calling.

OpenAI’s GPT-3.5-turbo has a much larger context window than function calling. The model considers the context window, measured in tokens, before generating additional text. Models with small context windows “forget” even recent conversations, veering off topic in problematic ways.

The new GPT-3.5-turbo has four times the context length (16,000 tokens) of the vanilla version at twice the price—$0.003 per 1,000 input tokens and $0.004 per 1,000 output tokens. OpenAI can process 20 pages of text at once, compared to Anthropic’s flagship model’s hundreds. (OpenAI is testing GPT-4 with a 32,000-token context window in limited release.)

GPT-3.5-turbo, the original version without the expanded context window, is 25% cheaper, according to OpenAI. Developers can use the model for $0.0015 per 1,000 input tokens and $0.002 per 1,000 output tokens, or 700 pages per dollar.

OpenAI’s popular text embedding model, text-embedding-ada-002, is also being discounted. Text embeddings are used for search and recommendations to rank results by relevance to a query string.

Text-embedding-ada-002 costs $0.0001 per 1,000 tokens, down 75%. As a startup that spends hundreds of millions of dollars on R&D and infrastructure, OpenAI’s system efficiency is a key focus.

After releasing GPT-4 in early March, OpenAI has indicated that incremental updates to existing models are its focus. At an Economic Times conference, CEO Sam Altman said OpenAI “has a lot of work to do” before training the successor to GPT-4.