Input Params
chat_context:json
The current chat context, the detailed description of the AKI.IO chat context format.
text_context:string
Alternative to the chat context a string based context prompt can be set, please note that most instruct based llms require a model specific syntax as input to respond.
chat_output_format:string
The desired format of the chat output, available options are: "chatml", "raw" and "json".
A detailed explanation of the llm chat output formats please read here.
temperature:float
The Temperature sampling parameter in the range 0.0 - 1.0
top_k:int
The Top-K sampling parameter in the range 0..1000
top_p:float
The Top-P sampling parameter in the range 0.0 - 1.0
max_gen_tokens:int
Limit the response to generate maximum number of tokens
wait_for_result:boolean
For blocking API call set to "true". For a streaming response set to "false".
Read more about the powerful AKI.IO streaming responses
Find a more detailed explanation of the input parameters here
Output Progress
text:[string|json]
The current generate chat response in the desired chat_output_format.
num_generated_tokens:integer
The current generated number of tokens.
current_context_length:integer
The current length of the current chat context in tokens.
Output
text:[string|json]
The complete generate chat response in the desired chat_output_format.
num_generated_tokens:integer
Total number of generated tokens in this request.
prompt_length:integer
The length of the given chat_context aka prompt in tokens.
current_context_length:integer
The total length of the current chat context in tokens.
max_seq_len:integer
The maximum context length in tokens this endpoint is capable to process.
success:string
"true" in case the request could be processed successful
error:string
The error message in case success is "false".
For more information and examples about how to use this API endpoint please read here.
