Input Params

chat_context:json

The current chat context, the detailed description of the AKI.IO chat context format.

text_context:string

Alternative to the chat context a string based context prompt can be set, please note that most instruct based llms require a model specific syntax as input to respond.

chat_output_format:string

The desired format of the chat output, available options are: "chatml", "raw" and "json". A detailed explanation of the llm chat output formats please read here.

temperature:float

The Temperature sampling parameter in the range 0.0 - 1.0

top_k:int

The Top-K sampling parameter in the range 0..1000

top_p:float

The Top-P sampling parameter in the range 0.0 - 1.0

max_gen_tokens:int

Limit the response to generate maximum number of tokens

wait_for_result:boolean

For blocking API call set to "true". For a streaming response set to "false".
Read more about the powerful AKI.IO streaming responses
Find a more detailed explanation of the input parameters here

Output Progress

text:[string|json]

The current generate chat response in the desired chat_output_format.

num_generated_tokens:integer

The current generated number of tokens.

current_context_length:integer

The current length of the current chat context in tokens.

Output

text:[string|json]

The complete generate chat response in the desired chat_output_format.

num_generated_tokens:integer

Total number of generated tokens in this request.

prompt_length:integer

The length of the given chat_context aka prompt in tokens.

current_context_length:integer

The total length of the current chat context in tokens.

max_seq_len:integer

The maximum context length in tokens this endpoint is capable to process.

success:string

"true" in case the request could be processed successful

error:string

The error message in case success is "false".

For more information and examples about how to use this API endpoint please read here.