Preventing Conversation Drift in LLM-based Chatbots Using ChatGPT
Let’s use session with ChatML to handle conversation drift
Maintaining accurate and interesting interactions is essential for a successful user experience as chatbots continue to gain popularity across different industries. However, chatbots frequently experience conversation drift, in which over time, their responses become less pertinent and comprehensible. Users may find it annoying and may also reduce the chatbot’s efficiency.
A simple and straightforward method can be applied to stop conversation drift and maintain the focus of the chatbot convo. We will examine this method in this post and show how it works using ChatGPT, a highly robust large language model (LLM) that has recently gained a lot of popularity in different specializations. This article will only cover practical implementations, as there are already several posts on the technical aspects of ChatGPT.
TL;DR: The source code for this project can be found in this repository.
GPT-3 vs. ChatGPT
For the sake of clarity, ChatGPT is not exactly the same as GPT-3, even though it uses GPT-3 models as its backbone. The training strategy of ChatGPT goes a bit further to incorporate the concept of supervised and reinforcement learning. Particularly, OpenAI used reinforcement learning from human feedback (RLHF), which leverages Human AI trainers to provide feedbacks, which is used to improve the performance of the model. This approach essentially allows the model to be “rewarded” for correct responses and “penalized” for incorrect responses, just like how we train a pet or teach a child.
Human input is used in reinforcement learning to teach the AI model which behaviors are preferred and which ones should be avoided. From this feedback, the model modifies its behavior in an effort to maximize rewards and decrease penalties. This procedure enables the model to improve its interactions with people over time by learning from its mistakes. Though the technique looks simple on paper, it can be difficult to implement successfully in practice.
Building a Chatbot with ChatGPT Model
Now, proceeding with our implementation, we will rely on some of the functions implemented in my previous article on fine-tuning the GPT-3 model. As opposed to using GPT-3 to build a chatbot, we will use the ChatGPT model, which is simply referred to as gpt-3.5-turbo on the model page of OpenAI.
In order to run an inference with the model, we will use the following function:
It can be observed from the code that the prompt style is somewhat different from text-davinci-003(GPT-3) . We now have the capability of using Chat Markup Language (ChatML) which enables parsing of JSON as input chat message instead of unstructured text in the previous API.
In addition, we implement a flask API, to serve the model:
We carry out some tests with the function implemented above, by calling the API:
Initial Question and Response:
input: {
"text": "tell me about linked list"
}
DSA:{
"message": "A linked list is a linear data structure consisting
of a sequence of nodes, where each node contains a data element
and a pointer to the next node in the sequence. Unlike an array,
a linked list does not have a fixed size and can dynamically grow
or shrink. Linked lists are efficient in terms of insertion and
deletion operations but accessing an element at a particular index
requires traversing the list from the beginning."
}Follow-up Question and Response:
input: {
"text": "how can we use it"
}
DSA: {
"message": "You can use DSA to ask questions and receive concise answers
on topics related to data structures and algorithms. You might
use that information to improve your coding skills, prepare
for technical interviews, or better understand how to approach
programming problems in general."
}It is obvious that the model is unable to relate the follow-up question to the previous one, which is simply termed conversation drift in chatbots.
Handling Conversation Drift
The new ChatML functionality is exactly what we will be exploiting to manage drift in the chatbot conversation.
In order to address this, we used a naive session concept and append follow-up conversations to old ones, while using the session to manage when the model should continue a conversation or kick off a new one. The implementation is as follows:
And we implement its flask API as:
Now, let’s test the model again, by adding the required session_id:
Initial Question and Response:
input: {
"text": "tell me about linked list",
"session_id": "KJN=87aGNq"
}
DSA: {
"message": "A linked list is a linear data structure that consists of a
sequence of elements, each of which stores a reference
(or pointer) to an object and a reference to the next element
in the sequence. The elements are not necessarily stored in
contiguous memory locations, and so linked lists can grow or
shrink dynamically at runtime. Common operations performed on
linked lists include insertion, deletion, traversal, and search.",
}Follow-up Question and Response:
input: {
"text": "how can we use it ",
"session_id": "KJN=87aGNq"
}
DSA: {
"message": "We can use linked lists to implement various data structures
such as Stacks, Queues, Hash tables and Graphs. Linked lists
are also used in applications like train route databases,
music playlists, and text editors. They can also be used to
implement dynamic memory allocation, where memory is allocated
and deallocated during program execution as needed.
Furthermore, linked lists are fundamental for understanding
algorithm complexities and improving problem-solving abilities.",
}We can now notice that the conversation is continued from the initial one, as long as the session_id is not changed. However, it is worth noting that the maximum number of tokens acceptable in this model is 4096 hence, we can experience some errors once this is exceeded, in which case the session_id has to be changed to start a new conversation. The source code for this project can be found in this repository.
