Why do many AI bots feel the need to be know-it-alls?

Question

Having used various AI bots often over recent months, I noticed that often it will claim to know something, even if it doesn't. It would then either explain something which is clearly nonsense, or by rambling on about how the answer isn't known in general. Or how, if asked for example- "would you be able to explain X" it wouldn't respond "yes, I could" but rather would elucidate X. Have they been trained to always respond as though it were a know-it-all? (Google's Bard and ChatGPT specifically, although I'm assuming only open-source AI will be answerable)

Answer 1

Have they been trained to always respond as though it were a know-it-all?

Yes, sort of, although it is not in some attempt to upset you or other users.

The language models used in the chat bots have been fine-tuned in conversations with users, and also have been "pre-prompted" to be helpful assistants. This impacts the kind of text that they generate.

Separately to this, the fact that they are often incorrect or generate nonsense is a limitation of the core technology. There is no underlying world model, the model cannot self-assess whether anything it generates is correct. Although the large language models can often surprise users (including the original developers) in the breadth of what they can respond to correctly and give the appearance of understanding, the same models can also fail easily at apparently simple tasks. This can cause some cognitive dissonance for users who expect an AI that appears to communicate like an intelligent human to be able to perform simple maths for example.

There is probably no global default chat tone that would suit all users. You could however use the raw model more directly and create a chatbot with a different tone to suit you. This would not impact accuracy, and would not add ability to self-assess correctness, but it may address feeling of being talked down to for example. Some services, like Character AI, attempt to give you tools to do just that, although the level of usefulness you get from them will depend on what they focus on (Character AI is more for creative fun, than for functional assistants).

In limited cases you can also address accuracy with pre-prompts or fine tuning that put a few facts into working memory. This is limited, and cannot address lack of general ability in logic or maths though. Corporate users can do this to give a chat bot some basic product knowledge or correct address for their help desk etc.

Answer 2

Humans, who know things, tend to limit themselves to speaking of things they know, especially in print. Thus any text corpus you may find will have an overpopulation of examples of confidently presented answers.

Chatbots, which do not know things, cannot impose a similar limit and thus will babble at length on topics about which they know nothing. But because they are trained on those data of people writing about things they know, they will tend to mimic the same confident tone and style of those people.

It is very important to reiterate here again that chatbots do not know things. By good fortune, they may produce a text which happens to convey factually accurate information. But they may not, and because they do not know things, they do not in advance (or even in retrospect) if they can (or did) produce a factually accurate text or meaningless drivel.

Thus a chatbot (or its developer by proxy) has only two options available when asked a question. Option 1 is to attempt to generate an answer to every question. Option 2 is to answer every question with "I don't know." The latter is technically uninteresting and practically useless, so everybody chooses Option 1.

Answer 3

It's quite simple really. LLM based generative AIs don't "know" anything. They're glorified next-word predictors. The only way that they're going to produce an answer to a prompt along the lines of "I don't know" is if their training corpus would indicate that that was the likely response a human would give to the same prompt.

Answer 4

The only thing the bot "knows" is how to generate sequences of lexical symbols (ie. texts) that mimic what people have written in the corpus it has been trained on.

The output is based on the prompt using complex rules + some internal state (so that it seems to "remember" the past discussion). It is all just mathematics that is used to choose which symbols (ie. letters and punctuation) to output. It could be implemented just as well with pocket calculators, or even doing the math manually, albeit very slowly and very arduously.

There is nothing else to the bot. No knowledge, no reasoning, no needs, no goals, no personality. Basically just numbers manipulated in a complex way.

Whatever else you happen see in the output is always your interpretation. If you spot a need in the text, it is most likely there only because some actual person did write something that reflected their need, and from it, the bot learned such rules that the output mimics something that was written with emotion.

It is as sentient as a cartoon character in an animated movie. All emotions that you assume to be in the character/text originates from you.

So, why it looks like they feel the need to be know-it-alls? Because they mimic what people write, and people are like that.

Answer 5

The main reason an AI model won't tell you "I don't know" is that the developer doesn't want you to get the impression that the model is incapable of answering your questions.

Imagine a scenario where you had several difficult or non-trivial questions that the model doesn't have the right answers for. If answers in the form of "I don't know" kept piling up, it will drive you away and make you less confident about getting an answer the next time you have a question.

So even if it gives an incorrect or inaccurate answer, by just giving an answer, it leaves you under the impression that sometimes it gives correct answers - sometimes it doesn't, and the next time you have a question you might return because you'd think it might give you a correct answer this time.

Answer 6

The large language models underpinning these "AI" bots have indeed been tweaked to be biased against providing a negative response.

The simple reason being that the executives of the companies trying to sell "AI" as the Next Big Thing, believe they're less likely to make sales if their product appears to be unable to answer questions. Thus they instruct their engineers to train these models to divert, dodge and dissemble - even to plainly ridiculous lengths - when the model is incapable of coming up with an affirmative answer to the question asked.