Skip to content

Instantly share code, notes, and snippets.

@sayap

sayap/tool-calls.py

Created March 7, 2026 00:35
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save sayap/33550a28a18f29081dcc0b832f675c48 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save sayap/33550a28a18f29081dcc0b832f675c48 to your computer and use it in GitHub Desktop.
Download ZIP
Test script for Q3C / Q3CN / Qwen3.5 tool calls
Raw
tool-calls.py
from openai import OpenAI
import json
client = OpenAI(base_url="http://127.0.0.1:8080/v1", api_key="dummy")
tools = [
{
"type": "function",
"function": {
"name": "get_time",
"description": "Get the current local time",
"parameters": {
"type": "object",
"properties": {},
"required": [],
},
},
},
{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get the current weather in a given city and state",
"parameters": {
"type": "object",
"properties": {
"city": {"type": "string", "description": "City, e.g. San Francisco"},
"unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
"state": {"type": "string", "description": "State, e.g. CA"},
},
"required": ["city"],
},
},
},
]
response = client.chat.completions.create(
model='dummy',
messages=[
{
"role": "user",
"content": "What's the weather like in San Francisco, CA, New York, NY, and Toronto, ON? Is it below 70 degree in any of the 3 cities? What's the time now btw?",
},
],
tools=tools,
tool_choice="auto",
logprobs=True,
top_logprobs=3,
stream=False,
)
choice = response.choices[0]
message = choice.message
logprobs = choice.logprobs.content
print('reasoning_content:')
print(message.reasoning_content)
print()
print('content:')
print(message.content)
print()
print('tool_calls:')
for tool_call in message.tool_calls:
print(tool_call.function)
print()
tool_call_started = False
worst_logprob = 0.0
for logprob in logprobs:
if not tool_call_started and logprob.token == '<tool_call>':
tool_call_started = True
if not tool_call_started:
continue
if logprob.logprob < worst_logprob:
worst_logprob = logprob.logprob
print(f'worst logprobs in tool_calls: {worst_logprob:.9f}')
@sayap
Copy link
Author

sayap commented Mar 7, 2026

Bad output when parallel tool calls are not allowed (pre ikawrakow/ik_llama.cpp#1300):

reasoning_content:
The user is asking for:
1. Weather in San Francisco, CA
2. Weather in New York, NY
3. Weather in Toronto, ON
4. Whether it's below 70 degrees in any of these cities
5. Current time

I need to make function calls to get this information. Let me check the available functions:
- get_time: Get the current local time
- get_weather: Get the current weather in a given city and state (requires city, optional state and unit)

For the weather calls, I need to be careful about the parameters:
- San Francisco, CA: city="San Francisco", state="CA"
- New York, NY: city="New York", state="NY"
- Toronto, ON: city="Toronto", state="ON"

I should use fahrenheit since the user mentioned "70 degree" which is typically fahrenheit in the US context.

Let me make all the function calls.

content:
None

tool_calls:
Function(arguments='{"city":"San Francisco","state":"CA"}', name='get_weather')

worst logprobs in tool_calls: -21.598724365

We can change the argument order in the tool definition to be city, state, unit, to bypass the strict ordering issue:

reasoning_content:
The user is asking for weather information for three cities:
1. San Francisco, CA
2. New York, NY  
3. Toronto, ON

They also want to know if it's below 70 degrees in any of these cities, and they want to know the current time.

I need to:
1. Get weather for San Francisco, CA
2. Get weather for New York, NY
3. Get weather for Toronto, ON
4. Get the current time

For the weather function, I need to provide city and state parameters. The user specified:
- San Francisco, CA
- New York, NY
- Toronto, ON

For the unit parameter, since they're asking about 70 degrees, they're likely referring to Fahrenheit (which is common in the US), so I'll use "fahrenheit" as the unit.

Let me make the function calls.

content:
None

tool_calls:
Function(arguments='{"city":"San Francisco","state":"CA","unit":"fahrenheit"}', name='get_weather')

worst logprobs in tool_calls: -23.443851471

The very bad logprob happened when the model wanted to continue with the 2nd tool call, but the grammar didn't allow it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment