The LLM and Gen AI Logs on the XO Platform provide detailed information about requests sent to LLMs and the corresponding responses. The logs include data on features accessing the LLMs, response generation time, payload details, tokens used, and more. It enables bot designers to track and compare usage across various LLM features and refine prompts and settings to boost performance and user experience.
The log analysis focuses on the following key areas:
- Request-response dynamics: Analysis of request-response dynamics between user prompts and model responses offers insights into prompt and model performance in specific scenarios.
- Payload details: Analyzing the payload data exchanged during interactions allows for effective monitoring and optimization of advanced AI functionalities.
To access the logs, Go to Analyze > LLM and Gen AI Logs. Click any record to view the log summary and payload details.
Field Description
You can sort the data by either Newest to Oldest or Oldest to Newest. Click the click to view the Summary and Payload Details.
Response TokensResponse tokens for a Large Language Model (LLM) are the pieces of generated output, like words or punctuation, showing the model’s response. These tokens make up the structured parts of the LLM’s text, making it easier to understand and analyze.
| Fields | Description |
|---|---|
| Date & Time | The timestamp of the call made to the LLM. |
| User ID | The distinct identifier of the end user engaged in the conversation.
You can view the metrics based on the Kore User id or Channel User Id. Channel-specific ids are shown only for the users who have interacted with the VA during the selected period. |
| Feature | The XO Platform feature (Co-Pilot and Dynamic conversation features) is making calls to the LLM models. |
| Description | Extra details about the node and task name linked to the feature. |
| Model | The Large Language Model to which the request was made. |
| Language | The language in which the conversation occurred. If it is a multi-lingual VA, you can select specific languages to filter the conversations that occurred in those languages. The page shows the conversations that happen in all enabled languages by default. |
| Time Taken | Time taken by the LLM to generate the response. |
| Status | Status of a call made to the LLM. “Success” or “Failure”. |
| User | Bot designer or end user who made a call to the LLM. |
| Integration Type | Type of integration used (e.g. System/Custom). |
| Prompt Name | Prompt used with the model and running at the node/task level. The pre-built prompts are named ‘Default. |
| Channel | The communication channel or platform used for the interaction with LLM. |
| Session ID | Identifier for the session. |
| Request Payload | The request payload sent to a Large Language Model (LLM) is the user’s input or question, along with any extra details needed for the model to give a good response. |
| Response Payload | The Large Language Model (LLM) produces a response payload as its answer to the input it receives. It’s in the text format and contains additional information required to present the response. This payload helps developers use the model’s output effectively. |
| Request Tokens | Request tokens for a Language Model (LLM) are the individual parts of input text, like words or punctuation, given to the model to create a response. These tokens are the basis for the model’s understanding and its output generation. |
| Response Tokens | Response tokens for a Large Language Model (LLM) are the pieces of generated output, like words or punctuation, showing the model’s response. These tokens make up the structured parts of the LLM’s text, making it easier to understand and analyze. |
| Stream | If the stream = true, then the response is delivered incrementally, token by token in real-time. |
Filter Criteria
The LLM and GenAI logs data can be viewed based on the selected filter criteria. Learn more.