Function Calling

Avian's API is OpenAI compatible, including their libraries and schema.

Learn how to connect large language models to external tools using the Avian API.

Introduction

In an API call, you can describe functions and have the model intelligently choose to output a JSON object containing arguments to call one or many functions. The Chat Completions API does not call the function; instead, the model generates JSON that you can use to call the function in your code.

The latest models have been trained to both detect when a function should be called (depending on the input) and to respond with JSON that adheres to the function signature more closely than previous models. With this capability also comes potential risks. We strongly recommend building in user confirmation flows before taking actions that impact the world on behalf of users (sending an email, posting something online, making a purchase, etc).

Supported Models

Model Name	Function Calling	Parallel Function Calling
Meta-Llama-3.1-405B-Instruct	✅	✅

Common Use Cases

Function calling allows you to more reliably get structured data back from the model. For example, you can:

• Create assistants that answer questions by calling external APIs

  • e.g., define functions like send_email(to: string, body: string), or get_current_weather(location: string, unit: 'celsius' | 'fahrenheit')

• Convert natural language into API calls

  • e.g., convert "Who are my top customers?" to get_customers(min_revenue: int, created_before: string, limit: int) and call your internal API

• Extract structured data from text

  • e.g., define a function called extract_data(name: string, birthday: string), or sql_query(query: string)

...and much more!

The basic sequence of steps for function calling is as follows:

1. Call the model with the user query and a set of functions defined in the functions parameter.

2. The model can choose to call one or more functions; if so, the content will be a stringified JSON object adhering to your custom schema (note: the model may hallucinate parameters).

3. Parse the string into JSON in your code, and call your function with the provided arguments if they exist.

4. Call the model again by appending the function response as a new message, and let the model summarize the results back to the user.

Function Calling Behavior

The default behavior for tool_choice is tool_choice: "auto". This lets the model decide whether to call functions and, if so, which functions to call.

We offer three ways to customize the default behavior depending on your use case:

1. To force the model to always call one or more functions, you can set tool_choice: "required". The model will then select which function(s) to call.

2. To force the model to call only one specific function, you can set tool_choice: {"type": "function", "function": {"name": "my_function"}}.

3. To disable function calling and force the model to only generate a user-facing message, you can set tool_choice: "none".

Remember to replace "your-api-key-here" with your Avian API Key when using this code.

Simple Function Calling Example

import openai

# Initialize the OpenAI client with the Avian API
client = openai.Client(
    api_key="your-api-key-here",
    base_url="https://api.avian.io/v1"
)

# Define the function for getting current weather
def get_current_weather(location, unit='celsius'):
    return {
        "name": "get_current_weather",
        "description": "Get the current weather for a given location",
        "parameters": {
            "location": location,
            "unit": unit
        }
    }

# Function to call the OpenAI API with the function definition
def call_weather_function(location, unit='celsius'):
    response = client.chat_completions.create(
        model="Meta-Llama-3.1-405B-Instruct",
        messages=[
            {"role": "user", "content": f"Get the current weather for {location} in {unit}"}
        ],
        functions=[get_current_weather(location, unit)]
    )

    # Parse the JSON response
    function_call = response.choices[0].message.get('function_call', {})
    if function_call:
        function_name = function_call.get('name')
        parameters = function_call.get('arguments')
        print(f"Function to call: {function_name}")
        print(f"Parameters: {parameters}")
    else:
        print("No function called by the model.")

# Example usage
call_weather_function("New York", "celsius")

Parallel Function Calling Example

Parallel Function Calling

Parallel function calling is the model's ability to perform multiple function calls together, allowing the effects and results of these function calls to be resolved in parallel. This is especially useful if functions take a long time, and reduces round trips with the API. For example, the model may call functions to get the weather in 3 different locations at the same time, which will result in a message with 3 function calls in the tool_calls array, each with an id. To respond to these function calls, add 3 new messages to the conversation, each containing the result of one function call, with a tool_call_id referencing the id from tool_calls.

Parallel function calling can be disabled by passing parallel_tool_calls: false in the request. The model will only call one function at a time when parallel function calling is disabled.

Example: Invoking Multiple Function Calls in One Response

from openai import OpenAI
import json

client = OpenAI(
    base_url="https://api.avian.io/v1",
    api_key="your-api-key-here"
)

# Example dummy function hard coded to return the same weather
# In production, this could be your backend API or an external API
def get_current_weather(location, unit="fahrenheit"):
    """Get the current weather in a given location"""
    if "tokyo" in location.lower():
        return json.dumps({"location": "Tokyo", "temperature": "10", "unit": unit})
    elif "san francisco" in location.lower():
        return json.dumps({"location": "San Francisco", "temperature": "72", "unit": unit})
    elif "paris" in location.lower():
        return json.dumps({"location": "Paris", "temperature": "22", "unit": unit})
    else:
        return json.dumps({"location": location, "temperature": "unknown"})

def run_conversation():
    # Step 1: send the conversation and available functions to the model
    messages = [{"role": "user", "content": "What's the weather like in San Francisco, Tokyo, and Paris?"}]
    tools = [
        {
            "type": "function",
            "function": {
                "name": "get_current_weather",
                "description": "Get the current weather in a given location",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "location": {
                            "type": "string",
                            "description": "The city and state, e.g. San Francisco, CA",
                        },
                        "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
                    },
                    "required": ["location"],
                },
            },
        }
    ]
    response = client.chat.completions.create(
        model="Meta-Llama-3.1-405B-Instruct",
        messages=messages,
        tools=tools,
        tool_choice="auto",  # auto is default, but we'll be explicit
    )
    response_message = response.choices[0].message
    tool_calls = response_message.tool_calls
    # Step 2: check if the model wanted to call a function
    if tool_calls:
        # Step 3: call the function
        # Note: the JSON response may not always be valid; be sure to handle errors
        available_functions = {
            "get_current_weather": get_current_weather,
        }  # only one function in this example, but you can have multiple
        messages.append(response_message)  # extend conversation with assistant's reply
        # Step 4: send the info for each function call and function response to the model
        for tool_call in tool_calls:
            function_name = tool_call.function.name
            function_to_call = available_functions[function_name]
            function_args = json.loads(tool_call.function.arguments)
            function_response = function_to_call(
                location=function_args.get("location"),
                unit=function_args.get("unit"),
            )
            messages.append(
                {
                    "tool_call_id": tool_call.id,
                    "role": "tool",
                    "name": function_name,
                    "content": function_response,
                }
            )  # extend conversation with function response
        second_response = client.chat.completions.create(
            model="llama-3.1-405b-instruct",
            messages=messages,
        )  # get a new response from the model where it can see the function response
        return second_response

print(run_conversation())

This example demonstrates how to use the Avian API to perform function calling, including parallel function calling. The code defines a weather function and shows how to interact with the API to get responses based on function calls.