Response Object

This page details the structure of the response objects returned by the Infuzu API. Understanding the structure and fields of these objects is crucial for effectively using the API and extracting the information you need.

Overview

Upon successfully making a request to the Infuzu API, you receive a JSON response containing various data points. The primary object is ChatCompletionsObject, which encapsulates the AI's response(s), metadata, and usage information. This page describes the structure of the ChatCompletionsObject and its related objects.

ChatCompletionsObject

The ChatCompletionsObject is the top-level object returned by the API. It provides an overview of the entire chat completion operation.

Fields

Field	Type	Description
`id`	string	Unique identifier for the chat completion.
`choices`	array of `ChatCompletionsChoiceObject`	Array containing the different choices (responses) provided by the API.
`created`	int	Timestamp of when the completion was created (in Unix epoch format).
`model`	string	The model used for generating the completion (a list of models can be found in the pricing list).
`service_tier`	string	Service tier associated with the request, if any.
`system_fingerprint`	string	Fingerprint of the system that generated the response.
`object`	string	Type of object, usually `"chat.completion"`.
`usage`	object of `ChatCompletionsUsageObject`	Usage statistics related to the request.

Example

{
    "id": "chatcmpl-xxxxxxxxxxxxx",
    "choices": [
        {
            "finish_reason": "stop",
            "index": 0,
            "message": {
                "content": "Hello! I am doing well, thank you for asking. How can I assist you today?",
                "role": "assistant"
            }
        }
    ],
    "created": 1677652288,
    "object": "chat.completion",
    "usage": {
        "prompt_tokens": 15,
        "completion_tokens": 20,
        "total_tokens": 35
    }
}

ChatCompletionsChoiceObject

Each ChatCompletionsChoiceObject represents a single response option provided by the API. The Infuzu API might return multiple choices based on the model configuration (imsn parameter).

Fields

Field	Type	Description
`finish_reason`	string	Reason for finishing the completion. Can be `"stop"` (normal completion) or `"error"` (if an error occurred).
`index`	int	Index of the choice in the list of choices.
`message`	object of `ChatCompletionsChoiceMessageObject`	Contains the actual message content.
`logprobs`	object of `ChatCompletionsChoiceLogprobsObject`	(Optional) Log probabilities of the tokens in the message.
`model`	object of `ChatCompletionsChoiceModelObject`	Model that generated this choice
`error`	object of `ChatCompletionsChoiceErrorObject`	(Optional) Contains error information if an error occurred.
`latency`	object of `ChatCompletionsChoiceLatencyObject`	Latency statistics related to the response.

Example

{
    "finish_reason": "stop",
    "index": 0,
    "message": {
        "role": "assistant",
        "content": "Hello! I am doing well, thank you for asking. How can I assist you today?"
    },
    "model": {
       "ref": "modelA",
       "rank": 1
    }, 
    "latency": {
       "start_latency": 250,
       "end_latency": 500
    }
}

ChatCompletionsChoiceMessageObject

This object contains the actual message content and role.

Fields

Field	Type	Description
`role`	string	The role of the message sender (e.g., `"assistant"`, `"user"`).
`content`	string	The content of the message.

Example

{
    "role": "assistant",
    "content": "Hello! I am doing well, thank you for asking. How can I assist you today?"
}

ChatCompletionsChoiceLogprobsObject (Optional)

This object contains log probabilities for the tokens in the message. It's only included if requested in the request parameters.

Fields

Field	Type	Description
`content`	array of `ChatCompletionsChoiceLogprobsItemObject`	Array of log probabilities for the content tokens.
`refusal`	array of `ChatCompletionsChoiceLogprobsItemObject`	Array of log probabilities for refusal tokens, if the model refused to respond.

Example

{
    "content": [
        {
            "token": "Hello",
            "logprob": -0.5,
            "bytes": [72, 101, 108, 108, 111]
        }
    ]
}

ChatCompletionsChoiceLogprobsItemObject (Optional)

This object contains log probability information for a single token.

Fields

Field	Type	Description
`token`	string	The token.
`logprob`	float	The log probability of the token.
`bytes`	array of int	Byte representation of the token.
`top_logprobs`	array of `ChatCompletionsChoiceLogprobsItemTopLogprobsObject`	Array of the most probable tokens at this position.

ChatCompletionsChoiceLogprobsItemTopLogprobsObject (Optional)

This object contains information about the top log probabilities for a token.

Fields

Field	Type	Description
`token`	string	The token.
`logprob`	float	The log probability of the token.
`bytes`	array of int	Byte representation of the token.

ChatCompletionsUsageObject

This object provides usage statistics for the request, including token counts.

Fields

Field	Type	Description
`prompt_tokens`	int	Number of tokens in the prompt.
`completion_tokens`	int	Number of tokens in the generated completion.
`total_tokens`	int	Total number of tokens used (prompt + completion).

Example

{
    "prompt_tokens": 15,
    "completion_tokens": 20,
    "total_tokens": 35
}

ChatCompletionsChoiceModelObject

This object describes the model used to generate the ChatCompletionsChoiceObject.

Fields

Field	Type	Description
`ref`	string	The name or reference of the model that generated the completion
`rank`	int	The rank of the model chosen for generation, based on Intelligent Model Selection

Example

{
   "ref": "modelA",
   "rank": 1
}

ChatCompletionsChoiceErrorObject

This object describes any error that may have occurred during the generation of a specific ChatCompletionsChoiceObject.

Fields

Field	Type	Description
`code`	string	A code describing the type of error that occurred
`message`	string	A human-readable error message

Example

{
   "code": "model_unavailable",
   "message": "The requested model is currently unavailable"
}

ChatCompletionsChoiceLatencyObject

This object contains information about the measured latency.

Fields

Field	Type	Description
`start_latency`	int	The time it took for the model to start generating the output in milliseconds
`end_latency`	int	The total time for the model to generate the entire output in milliseconds

Example

{
   "start_latency": 250,
   "end_latency": 500
}

Code Examples

Here’s how you can access response data in Python:

import os
from infuzu import create_chat_completion, ChatCompletionsHandlerRequestMessage

api_key = os.getenv("INFUZU_API_KEY")
if not api_key:
    raise ValueError("API key required.")

messages = [
    ChatCompletionsHandlerRequestMessage(
        role="user",
        content="Hello, how are you?"
    )
]

response = create_chat_completion(
    messages=messages,
    api_key=api_key,
)

print("Response ID:", response.id)
print("Created Time:", response.created)
print("Model Used:", response.model)
print("Prompt Tokens:", response.usage.prompt_tokens)
print("Completion Tokens:", response.usage.completion_tokens)
print("Total Tokens:", response.usage.total_tokens)

if response.choices and response.choices[0].message:
    print("Assistant Reply:", response.choices[0].message.content)
else:
    print("No message content returned.")

By understanding these response objects, you can efficiently parse the API’s output and build applications that leverage the power of intelligent model selection.

Last modified: 25 February 2025