OpenAI Function Calling

May 25, 2024 | seedling, permanent

tags :

OpenAI Function Calling #

Connect large language models to external tools.

In an API call, you can describe functions and have the model intelligently choose to output a JSON object containing arguments to call one or many functions.
The Chat Completions API does not call the function; instead, the model generates JSON that you can use to call the function in your code.

Common Use Cases #

https://platform.openai.com/docs/guides/function-calling/common-use-cases

Function calling allows you to more reliably get structured data back from the model. For example, you can:

Create assistants that answer questions by calling external APIs e.g. define functions like send_email(to: string, body: string), or get_current_weather(location: string, unit: ‘celsius’ | ‘fahrenheit’)
Convert natural language into API calls e.g. convert “Who are my top customers?” to get_customers(min_revenue: int, created_before: string, limit: int) and call your internal API
Extract structured data from text e.g. define a function called extract_data(name: string, birthday: string), or sql_query(query: string)

Supported Models #

ref

Not all model versions are trained with function calling data. Function calling is supported with the following models:

gpt-4,
gpt-4-turbo-preview,
gpt-4-0125-preview,
gpt-4-1106-preview,
gpt-4-0613,
gpt-3.5-turbo,
gpt-3.5-turbo-0125,
gpt-3.5-turbo-1106, and
gpt-3.5-turbo-0613

In addition, parallel function calls is supported on the following models:

gpt-4-turbo-preview,
gpt-4-0125-preview,
gpt-4-1106-preview,
gpt-3.5-turbo-0125, and
gpt-3.5-turbo-1106

Example #

From openai website #

ref

A good example from youtube tutorial #

https://github.com/daveebbelaar/langchain-experiments/blob/main/openai-functions/openai_function_calling.py

GPT-4o #

Code Examples #

jupyter notebook

OCR of Images #

2024-05-16_10-42-11_screenshot.png #

Discussion gpt-4-turbo- preview Gpt-40 Gpt-4-0125-preview gpt-4-turbo Optimization Approach Time per run (sec) 52.3 58.3 57.5 63.2 65.2 61.0 65.3 63.8 Time per run (sec) Accurecy 0.333 0.487 0.48 0.603 0.53 0.593 0.56 0.62 Accuracy Accuracy Accuracy No System Prompt Flattening Schema 144.3 0.353 205.5 0.533 213.4 0.54 198.4 0.64 230.0 0.587 316.2 0.673 244.8 0.64 212.7 0.686 198.5 0.36 0.487 0.533 0.633 0.607 0.66 0.553 0.707 0.36 0.527 0.553 0.633 0.553 0.633 0.64 0.70 Flattened Schema + Simple System Prompt Flattened Schema + Focused System Prompt Flattened Schema + Focused System Prompt + Function Name Flattened Schema + Focused System Prompt + Function Description Optimized Optimized Flattened Schema + Focused System Prompt containing Schema summary Flattened Schema + Focused System Prompt containing Schema summary Flattened Schema + Focused System Prompt containing Schema summary + Function Name Optimized Function Description Optimized 61.3 0.54 0.68 196.2 0.707 0.687 Flattened Schema + Focused System Prompt containing Schema summary Function and Parameter Descriptions Optimized 60.6 0.68 0.787 212.4 0.767 0.767 Flattened Schema + Focused System Prompt containing Schema summary Function and Parameter Descriptions Optimized + Function Call Flattened Schema + Focused System Prompt containing Schema summary Function and Parameter Descriptions Optimized + Function Parameter examples added 68.0 0.693 0.707 0.6 0.693 examples added 67.3 0.74 222.4 0.787 0.693 0.787 Function calling accuracy across different openai models