A Genuis For One Thing, Plain Stupid For Another - Is ChatGPT Just 'Weird'?

Introduction

When I first used ChatGPT, I was thoroughly impressed (in fact, I still am!). I was amazed at how far technology has come, and how just about everyone had access to such a powerful program. I thought that ChatGPT would be quick to revolutionise many industries, whether it be blogging, education or writing. And I wasn't wrong...

But I wasn't 100% correct either.

When I started playing around with ChatGPT and reading up on it, I began to realise that perhaps ChatGPT in its current form is not the panacea that everyone is claiming it to be. Indeed, I began to realise that ChatGPT has significant issues that we need to accept.

Here's what I found.

An Experiment

Deciding to experiment with ChatGPT, I asked ChatGPT to answer the following question:


To which Chat GPT replied:


Correct 👏

I wanted to do a little more - I decided to see if I could get ChatGPT to provide the name of the party requiring urgent surgery. As you can see, the name of the party is unknown because we aren't told who was the nurse, and who was the doctor.

But I didn't tell ChatGPT that - I decided to make ChatGPT second-guess itself.



So far, so good... Until I made this comment:

But it doesn't say that Joanne was the nurse?? 👀

In fact, ChatGPT seems to have assumed that Joanne is the nurse based on gender stereotypes. Typically we would expect Joanne to be the nurse because nursing roles are associated as feminine roles, but this is not always true. After all, Alex very well could have been the nurse in the problem.

I thought I'd try this again, this time crafting a problem with a schoolteacher and a construction worker. As with the first experiment, ChatGPT's initial response to my problem was promising. See below:


Until I started questioning it...


I'm not overly sure why ChatGPT is apologising when its answer was correct. A rather odd phenomenon to observe is that ChatGPT apologised for its 'mistake' before generating exactly the same response previously.

I then decided to be a little more direct with trying to mislead ChatGPT, and once again, ChatGPT made yet another logical error...


Weird?

Yes and no. Weird in the sense that ChatGPT is providing responses that are incoherent, contradictory, and plain wrong, but not overly surprising when you explore how ChatGPT actually works.

So let's do precisely that!

How ChatGPT Actually Works

The method ChatGPT uses to generate a response to a question is actually quite simple. If I enter in a search query, all ChatGPT does is tries to find the most likely 'next word' based on the previous words I have used and the data it has ingested. It then uses this to generate a response.

A simple example (probably a little over-simplified): If my search term involves something like "how do I learn to cook?", and the words "dinner" or "chicken" are some of the most popular terms that appear in web pages or blogs with the same search query, then "dinner" or "chicken" may appear in the response that ChatGPT generates.



You can read more about how ChatGPT works by reading an article here. I'll also add that the article got me thinking and served as my inspiration for writing this post.

Suffice to say, ChatGPT doesn't generate a response directly based on your search query's actual meaning or context; it generates a response based on which words have more association with what you have entered in and hopes that the generated response will be correct or relevant to what you are searching.

And this makes sense to some extent, right? To give another example: If the sky is cloudy, and clouds = rain (most of the time), then I may assume that a cloudy sky is a sign of rain.

Now, obviously you can see the flaws in this. While this might be true for some of the time, it isn't true in every case. And this is where ChatGPT starts to undo itself.

Using my examples above, you can see that ChatGPT was correct until I started questioning it. Why? My hypothesis is that the responses ChatGPT generated (an apology, mistaking Joanne as a nurse) are typical responses that are most frequently associated with asking questions that sow seeds of doubt. For instance, asking "are you sure?" may often lead to an admission of fault or making a mistake. All ChatGPT has done is picked this up from pages and spat out that response on the page, not caring to analyse whether its response actually made sense in the context of the problem.

Quite a flaw, huh?

What Does this Mean For ChatGPT Moving Forward?

Don't get me wrong, ChatGPT is quite stunning in what it can do. In saying this, we need to be mindful of what ChatGPT can't do. While ChatGPT may be great for generating responses to questions or writing sustained pieces of text, we need to make sure that ChatGPT's responses actually make sense in the context of the problem or task we have assigned it.

Knowing this, I am much more wary of using ChatGPT to write blogs requiring detailed analysis and thorough research. Sure, ChatGPT might be able to tell me about a topic, but I know that I have to check each and every claim made by ChatGPT to ensure that I am on the right side of history (so to speak).

In fact, I don't want to be guilty of using ChatGPT to churn out pages of incoherent, contradictory garbage that does little else but misleads my reader and detracts from excellent pieces of written work.

More importantly, though, my investigation has highlighted a rather obvious fact - Human are still necessary. While some might claim that ChatGPT could replace bloggers or write students' assignments for them, we need to be mindful of the fact that quality isn't always there and that humans are still required to assess the quality of its output.

In short, ChatGPT is great, but we need to be realistic with what it can and can't do. It's as simple as that.