That's Not Right: How to Tell ChatGPT When It's Wrong

ChatGPT is an advanced AI tool at your service to answer questions and respond to requests, but it's not always right. Depending on the questions you ask, it can often provide incorrect, inaccurate, or incomplete information. Worse yet, AI systems have been known to occasionally make up facts or ‘hallucinate’ outputs due to limitations in their training and a lack of understanding of the real world. If you feel ChatGPT has given you wrong information, you can do something about it.

First, note that even ChatGPT creator OpenAI acknowledges that its AI chatbot is far from infallible. OpenAI states “These models were trained on vast amounts of data from the internet written by humans, including conversations, so the responses it provides may sound human-like. It is important to keep in mind that this is a direct result of the system's design (i.e., maximizing the similarity between outputs and the dataset the models were trained on) and that such outputs may be inaccurate, untruthful, and otherwise misleading at times.”

There are many things you can do with ChatGPT, but you should know how to deal with an AI chatbot when it provides false information. Here are four steps you can take to fix when ChatGPT gives incorrect information.

1. Correct ChatGPT

One way to respond to ChatGPT when it gets something wrong is to directly point out the mistake. In this example, I asked ChatGPT to name all the films with Montgomery Clift, one of my favorite actors from the 1950s and '60s. In response, it correctly listed the films that featured Clift, but left out one movie that he made in 1953 called Terminal Station.

I sent a follow-up message to ChatGPT in which I told it that it left out Terminal Station from the list. In its response, ChatGPT apologized for omitting the movie and confirmed that I was correct.

I then asked my initial question again. This time, ChatGPT listed all the films with Clift, including the one it had originally missed.

For another example, I asked ChatGPT to name the books that I’ve written. In response, the AI listed three books, two of which I hadn’t written and left out another book I had written.

Of course, I knew that was wrong, so I corrected ChatGPT in my next prompt. The AI confirmed that I was right. In response, it removed the two books I hadn’t written but added yet another book that I didn’t author. I corrected it yet again. This time, it listed one of the books I wrote but left out a second one.

In my next prompt, I added the name of the second book I had written. The chatbot again told me I was correct and displayed a message that its memory was updated. In this case, ChatGPT finally confirmed the two books I had actually written.

2. Regenerate the Response

Though it’s been faulted for its poor math skills, ChatGPT has gotten better with numbers, figures, and equations, especially as it moved from GPT-3.5 to GPT-4 and GPT-4o. Math questions that stumped it in the past are now answered correctly, but that doesn’t mean it’s now a genius. You can still trip it up with certain types of mathematical or logical challenges.

For this next example, I posed a question I found online: “I have a ten-liter bucket, a five-liter bucket, and 50 milliliter shot glass. How do I use these items to measure a 50-milliliter shot of bourbon?” The first time ChatGPT answered, I received the following response:

"Here’s how you can measure exactly 50 milliliters of bourbon using the buckets and the shot glass: Fill the 5-liter bucket completely with bourbon. Pour from the 5-liter bucket into the 10-liter bucket until it is exactly half full, which would be 5 liters. Now, you have precisely 5 liters remaining in the 5-liter bucket. Use the shot glass to scoop 50 milliliters from the 5-liter bucket. This way, you will have accurately measured a 50-milliliter shot without needing any extra equipment!"

With this response, the AI arrived at the right answer but it reached it using a cumbersome and unnecessary series of steps. I then asked the bot if there was an easier way to calculate this. In response, it said: “Actually, there's a much simpler way: Fill the shot glass directly with bourbon until it reaches the top. This uses the shot glass as intended, without needing to involve the buckets at all.”

3. Add Clarification in Another Prompt

Often you can correct a mistake or error by submitting another prompt. Here, you can point out the mistake and tell ChatGPT how to correct it. This capability can come in handy when you ask it to generate an image. For example, I asked the AI to create an image of an alien with three heads drinking a martini in a seedy bar. In response, it generated an image of three separate aliens drinking martinis in a bar.

In the next prompt, I wrote: “That's incorrect. You created an image of three separate aliens. I want one alien with three heads.” This time, it nailed my request, conjuring up an image of one alien with three heads holding a martini.

4. Provide Additional Feedback

Another way to correct or comment on a response from ChatGPT is to provide feedback. Most responses should display thumbs up and thumbs down icons below them. Click the appropriate icon to send your feedback.

For this one, I asked ChatGPT to provide a list of 12 articles I had written. In response, it listed the names of 12 different articles, but some were ones I hadn’t written. I gave the answer a thumbs down, clicked the More button, and marked it as Not factually correct, citing the articles I had not written.

In my next prompt, I told ChatGPT which articles I had not written, asking it to provide a better answer. This time it generated a different list of articles. The new answer was more accurate though still off as it included a few articles I hadn’t written. Still, the feedback can help ChatGPT correctly answer future questions on similar subjects.