This API lets you get the LLM & Generative AI Usage Logs for both Co-Pilot and Dynamic Conversation features.
| Method | POST |
| End Point | https://{{host}}/api/1.1/public/bot/{{botId}}/getLLMUsageLogs |
| Content Type | application/json |
| Authorization | auth: {{JWT}}
|
| API Scope |
|
Note: You can access records spanning up to a 90-day timeframe with one request.
Query Parameters
| Parameter | Required/Optional | Description |
|---|---|---|
| host | Required | The environment URL. For example, https://bots.kore.ai |
| botId | Required | Bot ID or Stream ID. You can access it from the General Settings page of the bot. |
Sample Request
--header 'auth: eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwiYXBwSWQiOiJjcy03ZWMxNmFjZS03ZGNmLTU3MjQtYjM5NS1hYTA4YmRmYTAwMzMifQ.n_Es9ZBsiCYzpzsfN4p4I1SlHK05gewJFyqAIngr4Qg' \
--header 'Content-Type: application/json' \
--data '{
"dateFrom": "2024-03-07",
"dateTo":"2024-03-09",
"limit":"1",
"skip" : "5",
"isDeveloper": true,
"channel" : "msteams",
"featureName" : ["dynamicEntity"],
"taskId" : "dg-d4924db4-b325-5b4d-ae51-aa5c7be41d4f",
"channelUserIds":["29:1gmw8z03rvk6njl6k7ohtdh2v9zubxiip7kvu1yiek_qri4grpmd0k_d1yjlpzbj40wk1am9dphqkoiwatzwttw"],
"userIds" : ["u-40b3eafc-a5aa-55f2-83e8-cf4d0fb1de07"],
"sort" : {
"field" : "Time Taken",
"order" : "desc"
}
}'Body Parameters
| Parameter | Required/Optional | Description |
|---|---|---|
| dateFrom | Required | The timestamp from which the records are displayed. |
| dateTo | Required | The timestamp till which the records are displayed. |
| limit | Required | The number of records to be displayed on each page. |
| channel | Optional | The channels to be considered for the metrics. |
| channelUIds | Optional | The user ids (channel specified) to be included in the metrics. |
| isDeveloper | Optional | To include developer metrics. Set to true or false. |
| taskId | Optional | To filter records based on the task ids. |
| userId | Optional | To filter records based on user id. |
| sort | Optional | To sort the results. |
Sample Response
[
{
"Prompt Name": "Default",
"Integration": "OpenAI",
"start Date": "2024-03-08T06:38:54.008Z",
"End Date": "2024-03-08T06:38:58.386Z",
"Time Taken": 4378,
"statusCode": 200,
"Bot ID": "st-73bfdb2f-9101-55e6-b4c8-3f568a6ea8e0",
"User ID": "u-40b3eafc-a5aa-55f2-83e8-cf4d0fb1de07",
"Feature Name ": "GenAI Node",
"Model Name": "GPT-4",
"Channel Name": "msteams",
"Description": "Order pizza-GenAINode0002",
"task Id": "dg-200ad1ff-8db8-5219-810e-4ee3800d212c",
"Status": "Success",
"Payload Details": {
"Prompt Name": "Default",
"Request Payload": {
"model": "gpt-4",
"temperature": 0.5,
"max_tokens": 1500,
"top_p": 1,
"frequency_penalty": 0,
"presence_penalty": 0,
"messages": [
{
"role": "system",
"content": "You are a virtual assistant representing an enterprise business, and so you have to act professionally at all times. You do not participate or respond to any abusive language or indulge in any conversation that does not represent enterprise business.\nAct like pizza ordering site. For the instructions that the user provides, you have to process the instructions. Here are the rules that you are supposed to follow: \nRule 0:\n Each pizza can only have maximum of 3 toppings \nRule 1:\n List of entities you need to capture from user are number of pizzas, toppings and size.. You need to capture all these entitites. \nRule 2:\n Capture entities from the user and the conversation history. \nRule 3:\n Collect the unfulfilled entities or check if the user wants to update a collected entity. \nRule 4:\n If user has provided the required value for any of the required entities, then do not prompt for it again. \nRule 5:\n Generate appropriate prompt to the end user to collect the information for the unfulfilled entities. \nRule 6:\n In the output return JSON must containing {\"bot\"://next prompt , \"analysis\"://Summarization of the entire conversation with all details.,\"entities\"://array of objects,\"conv_status\"://The status of the conversation. \"ongoing\" or \"ended\".} \nRule 7:\n When returning the result return a json, stringified and formatting removed. \nRule 8:\n Once all the entities details are captured generate conv_status as 'ended'. When the flow is to be continued, generate conv_status as 'ongoing' \nRule 9:\n Keep the prompts and messages voice friendly. \nRule 10:\n If there are mutiple entities, return entitites in format of array of objects. Where each object is entity key values pairs. \nRule 11:\n Ignore any out of topic user/assistant conversations and always collect ALL the mentioned entities. \n\n "
},
{
"role": "user",
"content": "Order pizza"
},
{
"role": "assistant",
"content": "{\"bot\":\"Sure, I can help with that. How many pizzas would you like to order?\",\"analysis\":\"The user initiated the pizza ordering process but didn't provide any details.\",\"entities\":[],\"conv_status\":\"ongoing\"}"
},
{
"role": "user",
"content": "5"
},
{
"role": "assistant",
"content": "{\"bot\":\"Great, 5 pizzas it is. What size would you like these pizzas to be?\",\"analysis\":\"The user has ordered 5 pizzas but has not specified the size or toppings.\",\"entities\":[{\"number_of_pizzas\":\"5\"}],\"conv_status\":\"ongoing\"}"
},
{
"role": "user",
"content": "Medium Size"
},
{
"role": "assistant",
"content": "{\"bot\":\"Perfect, 5 medium-sized pizzas. Now, could you please specify the toppings for your pizzas? Remember, each pizza can have a maximum of 3 toppings.\",\"analysis\":\"The user has ordered 5 medium-sized pizzas but has not specified the toppings.\",\"entities\":[{\"number_of_pizzas\":\"5\"},{\"size\":\"Medium\"}],\"conv_status\":\"ongoing\"}"
},
{
"role": "user",
"content": "okay\n\n{"
}
]
},
"Response Payload": {
"id": "chatcmpl-90OAYDFFkbjgMxhMxKukwqklT6SWO",
"object": "chat.completion",
"created": 1709879934,
"model": "gpt-4-0613",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "{\"bot\":\"I'm sorry, but your message seems incomplete. Could you please specify the toppings for your pizzas? Remember, each pizza can have a maximum of 3 toppings.\",\"analysis\":\"The user's response was incomplete. The toppings for the pizzas are still needed.\",\"entities\":[{\"number_of_pizzas\":\"5\"},{\"size\":\"Medium\"}],\"conv_status\":\"ongoing\"}"
},
"logprobs": null,
"finish_reason": "stop"
}
],
"usage": {
"prompt_tokens": 614,
"completion_tokens": 74,
"total_tokens": 688
},
"system_fingerprint": null
},
"Request Tokens": 614,
"Response Tokens": 74
}
}
]