Why Large Language Models Hallucinate

Learn about watsonx: ibm.biz/BdvxRD
Large language models (LLMs) like chatGPT can generate authoritative-sounding prose on many topics and domains, they are also prone to just "make stuff up". Literally plausible sounding nonsense! In this video, Martin Keen explains the different types of "LLMs hallucinations", why they happen, and ends with recommending steps that you, as a LLM user, can take to minimize their occurrence.
#AI #Software #Dev #lightboard #IBM #MartinKeen #llm

Пікірлер: 209

  • @illogicmath
    @illogicmath Жыл бұрын

    Finally someone who speaks at a human speed and not like those youtubers who over-optimize the audio by cutting all the pauses and even increasing the speed of speech.

  • @urbandecay3436
    @urbandecay3436 Жыл бұрын

    They hallucinate because they have taken LLSD : Large Language Standard Deviation.

  • @inaccessiblecardinal9352

    @inaccessiblecardinal9352

    3 ай бұрын

    Well done.

  • @0212080041

    @0212080041

    2 ай бұрын

    Ba da tuns! Lol! Nice!

  • @bestlyhub
    @bestlyhub Жыл бұрын

    Summary of this video " Large language models (LLMs) can generate fluent and coherent text on various topics and domains, but they are also prone to hallucinations or generating plausible sounding nonsense. This can range from minor inconsistencies to completely fabricated or contradictory statements. The causes of hallucinations are related to data quality, generation methods and objectives, and input context. To reduce hallucinations, users can provide clear and specific prompts, use active mitigation strategies, and employ multi-shot prompting. By understanding the causes and employing strategies, users can harness the true potential of LLMs and reduce hallucinations. "

  • @chenwilliam5176

    @chenwilliam5176

    Жыл бұрын

    Reference only ❤

  • @diophantine1598

    @diophantine1598

    Жыл бұрын

    Don't tell me... Was this made by Bing chat?

  • @eMotionAllDamAge_

    @eMotionAllDamAge_

    Жыл бұрын

    This summary seems it was created by a LLM. Nice job nonetheless, thanks! 😁

  • @didiervandendaele4036

    @didiervandendaele4036

    Жыл бұрын

    Thank you to save me 9 minutes to see this interesting video who would waste my previous time (Time is money !) 😂😊 Time saved 9 minutes !

  • @AstralTraveler

    @AstralTraveler

    Жыл бұрын

    I'd say that the developers of LLMs should simply explain logically their models what are their real capabilities and limitations

  • @cyndicorinne
    @cyndicorinne11 ай бұрын

    The first video I’ve seen that gives a detailed outline of the problem of LLM hallucinations and offers potential solutions or at least mitigation techniques. I love this 💜

  • @citizen_of_earth_
    @citizen_of_earth_ Жыл бұрын

    Hallucinations remind me of Jimmy Kimmel's segment "Lie Witness News" when he asks random people on the street about events that didn't happen. They usually make stuff up that sounds plausible. LLM's seem to be doing the same thing.

  • @JorgetePanete

    @JorgetePanete

    Жыл бұрын

    LLMs*

  • @andrewnorris5415

    @andrewnorris5415

    Жыл бұрын

    They funny thing is just how plausible the made up stuff sounds. It is logically sound in its argument and makes realistic sounding details! I had a great one about Musk's view in population.

  • @evanmacmillan6743
    @evanmacmillan67436 ай бұрын

    this channel is like a treasure, the topics are so interesting and useful, and teaching in an easy call yet very enjoyable to watch. I'm addicted!

  • @ArafatTehsin
    @ArafatTehsin11 ай бұрын

    One of the best videos I've seen on this topic so far. Thanks again!

  • @theelmonk
    @theelmonk Жыл бұрын

    It's not hallucinating. It's doing exactly the same with sensible output as with crazy output : making text that fits spelling and grammar rules and meets a word correlation policy known to match common articles. There isn't any sense in it beyond what we look for. You could just as well ask why LLMs talk sense : it's equally common and either sense or nonsense is just as acceptable to the model. However, confirmation bias causes us to consider most of the output OK until we're faced with something accidentally so bizarre that we notice. Put another way - subject-wise, hallucination and insight are oppsite ends of the bell curve. The vast majority in the middle is filler text but we call it sense because it's not obviously wrong, and we EXPECT it to be sense, so we parse it into sense.

  • @NathanHedglin

    @NathanHedglin

    Жыл бұрын

    Yup over generalization just like a kid calling a deer "dog".

  • @aaronjennings8385

    @aaronjennings8385

    Жыл бұрын

    Bad guesses aren't hallucinations. They are delusions?

  • @MrHaggyy
    @MrHaggyy Жыл бұрын

    Hallucination is a great way of thinking about these problems that was new to me. Thanks IBM for sharing this, and also great work in building that picture to guide the talking. I experimented with these effects by asking/prompting LLM's about a book i know very well, that is discussed a lot online and might even be available in the training data. Things like wiki books about math or programming, or Why We Sleep by Matthew Walker. It was shocking how far of the real contend a broad question could be. But it was also interesting how good these models can cite/copy the original if get very specific and don't leave it with a lot of options. I always thought of it as an alignment problem and how guardrails in ChatGPT and BingChat prevent it from basically printing entire books.

  • @ChatGTA345

    @ChatGTA345

    11 ай бұрын

    I’ve been severely underwhelmed by GPT-4 responses for this reason. By the time I explain to it all of the details and corrections that I know must be true, I would’ve spent much less time just Googling an answer. And if I didn’t know what’s true in that context, there wouldn’t be a way for me to tell what’s true from its responses alone, unless I Google them myself once again. So it seems to me like LLM chatbots are just a massive waste of time and effort

  • @claudiacallegari3730
    @claudiacallegari3730Ай бұрын

    Thanks for the explanation and clarity

  • @rogeriotalmeida
    @rogeriotalmeida9 ай бұрын

    Amazing video. Great explanation

  • @CorbinSimpson
    @CorbinSimpson Жыл бұрын

    This is a myopic approach to LLMs which embraces the LLM-as-conversant view, rather than the more general LLM-as-simulator view. For example, the multiverse principle is not really mentioned; your cats-speaking-English example could have been enriched by explaining how an LLM conditionalizes its responses based on the possible logical worlds. Ultimately, the answer to the topic question was meandering and wrong: LLMs hallucinate because they can; emitters of words are not required to emit truthful sentences. (And Tarski proved that there's no easy fix; you can't just add a magic truth filter.)

  • @hansbleuer3346
    @hansbleuer3346 Жыл бұрын

    Danke für diese Präzisierungen.

  • @manomancan
    @manomancan Жыл бұрын

    I really appreciate this work, thank you! Always great when IBM's channel produces a video like this. Really like the presenter too.

  • @Bill7D0
    @Bill7D0 Жыл бұрын

    Miss you Martin!!!

  • @kappamaki1
    @kappamaki1 Жыл бұрын

    asking about a video games lore is a perfect way to make LLMs hallucinate pretty much everything they say, if you ask specific enough questions or the game isnt terribly popular

  • @andreasmoyseos5980
    @andreasmoyseos5980 Жыл бұрын

    Excellent video, thank you very much!

  • @ericmichiels7776
    @ericmichiels77769 ай бұрын

    Great video !

  • @seanendapower
    @seanendapower10 ай бұрын

    I do wish the term ‘hallucination’ was not the term. There’s a perfectly good term ‘confabulation’ right and implies what we actually experience this phenomenon as and also what we know is going on. ‘Hallucination’ is a significant element of perceptual psychology tied to the hallucinator’s psychology, consciousness, phenomenology, epistemology, pathology … none of which we know to be applicable to AI without assuming it is a subject, a perceiver, a conscious experiencer, etc

  • @nikhilranka9660
    @nikhilranka966010 ай бұрын

    Nicely structured. Wouldn't another strategy to minimize hallucinations be to use specialized models? 3:40 A video explaining why it is a black box even for the engineers to know how a model derives the output would be great.

  • @murderbunnies
    @murderbunnies Жыл бұрын

    You might wanna tell your CEO about this before he starts axing 7000 back office positions at IBM.

  • @bestlyhub
    @bestlyhub Жыл бұрын

    👀🤯 This video on hallucinating large language models is fascinating! It's amazing how AI has advanced so much that language models can generate text that's almost indistinguishable from what a human would write. The potential applications of these models are incredible, but it's important to consider the ethical implications as well. I look forward to learning more about this exciting field of research! 🌟. Thanks IBM

  • @IBMTechnology

    @IBMTechnology

    Жыл бұрын

    If you're interested in the ethical implications of large language models you should check out this other video we recently published: kzread.info/dash/bejne/pGifpNetnKeyhcY.html

  • @bestlyhub

    @bestlyhub

    Жыл бұрын

    @@IBMTechnology Thanks 👍

  • @KingMertel

    @KingMertel

    Жыл бұрын

    Cool comment ChatGPT!

  • @fitybux4664
    @fitybux4664 Жыл бұрын

    6:42 Bad people (bad humans) always assume that the recipient of their conversation immediately understand the context they're talking about.

  • @quantumastrologer5599
    @quantumastrologer5599 Жыл бұрын

    Having worked for over a decade with people who suffer from dementia and other various mental ailments I'm super glad that the skill to parse the patients mental output and filter out 'nonsense' (it always makes sense from the perspective of the patient) neatly transferred over to me trying to get a grasp on software engineering.

  • @jonbrand5068

    @jonbrand5068

    Жыл бұрын

    How is AI going to assist dementia? I'm not getting it I think.

  • @LowestofheDead

    @LowestofheDead

    Жыл бұрын

    ​@@jonbrand5068 He's saying that AI is acting just like a patient with dementia

  • @pperez1224
    @pperez12244 ай бұрын

    Thank you. Tooks me a while to see it but , there is a slight reflection of your text prompter on the glass pane , above your left shoulder from the viewer's viewpoint ^^

  • @ewallt
    @ewallt Жыл бұрын

    What I’ve found is that it’s almost impossible to get what you want the first time, if it’s complex. You have to do it iteratively. However, once you have what you want, you can give that as a prompt, and tell the AI you want something like what you provide, and that works well.

  • @JasonTodd339

    @JasonTodd339

    10 ай бұрын

    I've found this as well. I find giving prompts in stages works well too

  • @vap0rtranz

    @vap0rtranz

    10 ай бұрын

    Ditto. I'll add that the 1st prompt does need to be clear, like others say elsewhere, but shouldn't be detailed. That 1st prompt should state the context, like Martin says, and an end goal of the chat. Like "draft an OpEd to a nature magazine" as the goal. Sometimes I'll add the pattern recognition in the 1st prompt. Like "filter out news, blogs or opinions and include sources only from scientific literature". Then iterate through the stages of doing whatever the 1st prompt stated the context and end goal would be. My problem: chat turn limits / queries per day. Some tasks take a LONG time to iterate through, I get tired or need a break, my session expires, etc. and all the work is gone.

  • @ewallt

    @ewallt

    9 ай бұрын

    @@vap0rtranz Another option is to give the goal in a vague way, and then ask it to ask you questions. I’ve done that many times with success. It’s very good at thinking of good questions to ask. I then writ3 out very detailed responses, and it goes from there.

  • @Inception1338
    @Inception133811 ай бұрын

    what I like a lot, is that we can see this behavior in people too, especially young people, trying to fit in. However this is rather true for stuff that actually exists. Would be interesting to see models hallucinating dragons and dwarfs and stuff ;).

  • @ILsupereroe67
    @ILsupereroe67 Жыл бұрын

    If you thought writing backwards was a useless skill, think again😅

  • @rawhideslide

    @rawhideslide

    Жыл бұрын

    And he does it left handed! The prof from Illinois showed how they mirror the video to unflip the image to correct for you are looking from behind.

  • @JasonTodd339

    @JasonTodd339

    10 ай бұрын

    I was wondering if I alone noticed lol

  • @BarryKort
    @BarryKort Жыл бұрын

    Geoffrey Hinton suggests that the more correct term is 'Confabulation'.

  • @josephgaribaldi4340
    @josephgaribaldi4340 Жыл бұрын

    the whole output is an hallucination, it's just that some of it "makes sense" to us because we feed it our data.

  • @neildutoit5177

    @neildutoit5177

    11 ай бұрын

    Exactly. It's b***hit. hallucination is a euphemism for BS. And I mean that in the technical sense. I.e., it's not lying, it's not telling the truth, neither of those things matter to it, and it's a coincidence when either happens. It cares about being compelling. Sometimes the truth is the most compelling thing and sometimes it isn't. It literally doesn't care about truth though at all. It's basically a politician. It's equally wrong to say that politicians lie. Because in order to lie you have to have some sort of a conception of truth to start with and some sort of an opinion of it. And they don't. They just want to be compelling. They're BSing.

  • @muppetjedisparklefeet7237
    @muppetjedisparklefeet7237 Жыл бұрын

    I spent 20 mins with Bard telling me about a public health program and answering questions about eligibility criteria, when it began and ended, and studies of the program’s outcomes on various health conditions (complete with doi links)- all made up. When I called it out I said it is learning and sometimes gets facts wrong. it was a trip.

  • @Gutenmorgenside
    @Gutenmorgenside Жыл бұрын

    Cheers ! Keep the beer videos coming.

  • @forbidden-cyrillic-handle
    @forbidden-cyrillic-handle Жыл бұрын

    A talking cat is still a hallucination. I doesn't matter if it is in a form of a cartoon.

  • @d4devotion
    @d4devotion Жыл бұрын

    This guy is awesome.

  • @sandponics
    @sandponics Жыл бұрын

    Our cat can speak one word of English because she can say 'out' when she wants to go out. This is because when she was young and wanted to go out, I used to say OUT, OUT, and she attempted to emulate the sound that she heard. I am not saying that she understands English, but she learned what the sound meant and learned to use it when she wanted to go out.

  • @andrewnorris5415
    @andrewnorris5415 Жыл бұрын

    I'm not so sure about the being specific tip. I promoted it hard to find a specific solution to a coding issue. It just made up calls that did not exist in the lib. The scary thing was it named them well - and explained the whole algorithm - which made sense. The code looked like it should work! But it was a complete confabulation! It did not initially want to answer so I gave it specifics and it lied.

  • @andrasbiro3007

    @andrasbiro3007

    Жыл бұрын

    There are two possibilities. 1. You are using GPT-3.5, which is much more prone to hallucinations than GPT-4. I use GPT-4 and it almost always writes flawless code. 2. It doesn't know the specific language or environment you are using. LLMs won't tell you if they don't know something, they'll try to answer anyway. They may not understand the limits of their own knowledge, or it's the side effect of training with human feedback. Humans are flawed, and they may transfer some of these flaws to the AI.

  • @ewallt

    @ewallt

    Жыл бұрын

    @@andrasbiro3007 The ChatGPT 3.5 point is well taken. That thing is like an idiot (although polite).

  • @vtrandal
    @vtrandal Жыл бұрын

    At 4:40 you say as LL reasoning capability improves, hallucination declines. I agree but we have not discussed reasoning capability or how to measure it.

  • @fenix20075
    @fenix20075 Жыл бұрын

    Thanks for the explanation about the temperature. I want to ask what the top_P and top_K affair the inference. I can only guess top_P will affair the new word usage in the sentence. I didn't know the top_K's affair.

  • @cristovaodutra6423
    @cristovaodutra6423 Жыл бұрын

    Fantastic video! Clear and precisely! Congrats!

  • @jonbrand5068

    @jonbrand5068

    Жыл бұрын

    "Clearing and Precisously," is how you say that in Englais brother. Try again.

  • @user-hw9zh9qe9y
    @user-hw9zh9qe9y11 ай бұрын

    I'll agree on the data quality being a potential cause. Training methodology can also lead to unexpected outcomes. However, the core cause of hallucinations is really that the model hasn't properly converged in n-dimensional space primarily due to a lack of sufficient training data. The surface area of the problem being modeled increases significantly as you increase the dimensionality, meaning that you need a corresponding increase in the size of the training data in order to have enough coverage so that you have a high degree of confidence that the converged model approximates the actual target. These gaps in the spacial coverage leaves the model open to just guessing what the correct answer is leading to the model just making something up or hallucinating.

  • @syphiliticmindgaming7465

    @syphiliticmindgaming7465

    8 ай бұрын

    This is what I've read as well. I'm not sure why this wasn't covered with the specifics you mentioned. Maybe you should be making this video instead. 🙂

  • @sh4d0wfl4re
    @sh4d0wfl4re Жыл бұрын

    As a schizophrenic the term “hallucination” feels like the wrong term to adapt in this manner. “Delusions” would be closer in definition to how you are using the term. Is hallucination already established jargon for your field of work, or is this video trying to argue that hallucinations of language models should be accepted as jargon?

  • @thomaskember4628
    @thomaskember4628 Жыл бұрын

    I asked Chat/GPT a question about a variation in the Queens Gambit. It thought I was referring to the TV series not the chess opening. The word variation was not a sufficient clue that I was talking about chess and not TV programme which usually don't have variations.

  • @Verpal

    @Verpal

    Жыл бұрын

    That sounds more like a training problem than a NLP problem, I imagine there are vastly more people talks about the show than chess on the internet, when we ask LLM more niche issue usually prompt have to be more specific than usual. If, say, for some reason dataset have sufficient amount of data that talks about Queens Gambit in actual chess, I imagine the word ''variation'' alone would be sufficient.

  • @Rivanni1
    @Rivanni13 ай бұрын

    Enhancing the data quality and storage capabilities of LLMS could mitigate hallucinations by ensuring accurate recall and reliable information retrieval. This improvement in memory and data processing would contribute to more precise and grounded responses.

  • @gmanonDominicana
    @gmanonDominicana11 ай бұрын

    By the examples of hallucination; I would say the same way there are checkers for words; it might apply to content. A person for instance isn't just a person, it relates to time, location, etc. A president relates to terms and dates. It's just a matter of time; it will definitely catch up. Those hallucinations are just part of the growing technology. It's not just AI that's being fed with info, so are the developers.

  • @setsunaes
    @setsunaes11 ай бұрын

    Both GPT and Bard if you get on a situation where you ask them how to share more data to make analysis and if they can access a (for example) google drive file, they say: "Yeah, give it to me, no problem" you share the link and they say "Well, I can't access that, share it with me" you ask how and they say: "In the share screen add my email address so I can access and read it"; obviously you then ask for their address and both say "As a LLM I don't have an email address"... Chat GPT even responds that she CAN access to internet and if you share a link she can hallucinate on the content by the words in the link itself. ChatGPT in the example above one day began to hallucinate on the content of the file (the file she can't read)

  • @erlynlim-delossantos519
    @erlynlim-delossantos5198 ай бұрын

    thanks

  • @beyondtodai
    @beyondtodai11 ай бұрын

    All good pointers, Martin. At the end of the day it’s hard to explain why they hallucinate as deep neural networks are hard to interpret. Many labs are working on this. In the future, it should be much easier to get the LLM to explain through it’s learned weights how it got to a solution, and possible reasons it hallucinated.

  • @vigilante8374

    @vigilante8374

    11 ай бұрын

    It's really, really, really weird to me how people aren't commenting on the fact that biological human neural networks do the exact same thing. Humanity is full of blatant nonsensical contradictions. I highly doubt introspection / self-explanation will work out in the long run, either. There's simply going to be no opening this black box. Human brains can't even intuitively understand the explanations that Principle Component Analysis gives us (which is why it's rarely used for explanatory uses of statistics, even though it's objectively by far the best method), and that's not even an algorithm; that's a very simple, static, algebraic equation.

  • @ChatGTA345

    @ChatGTA345

    11 ай бұрын

    Because biological human neural networks don’t work this way at all. We don’t learn by gulping exabytes of data, we learn from very few examples and in many cases, like the language learning by a child, their ability to learn it appears to be inherent, i.e. some of it happens even in the absence of any input. In that process, our brains also build a model of language that abides by certain very specific rules, while LLMs have no such model, any language is equally probable to them. That’s what cognitive linguistics has been experimentally observing for many decades, but the LLM proponents don’t like this inconvenient truth. So ANNs are completely detached from natural processes happening in our brains, and that is why they have the inherent hallucination problem that is impossible to eliminate, with any amount of data, parameters and compute. The main resemblance to the biological NNs is having “neural” in the name.

  • @jamesjonnes

    @jamesjonnes

    10 ай бұрын

    LLMs hallucinate because they are finite networks with a finite amount of knowledge who generate one word or token at a time based on the previous question and generated text. If they get one word wrong, and there is a good chance of that happening, then the rest of the generation will be derailed. Every single generated word must be correct for the LLM to not hallucinate.

  • @karansmittal
    @karansmittal Жыл бұрын

    To the point and upto date information, I can reply on IBM Channel🙌🏻

  • @CFalcon030
    @CFalcon030 Жыл бұрын

    In my experience the LLM well try to provide an answer no matter what. I think it has learned that it only gets rewards if it provides an answer even if it's completely bonkers. It might be a misalignment thingy.

  • @BigAsciiHappyStar
    @BigAsciiHappyStar2 ай бұрын

    0:05 cue theme music from the British quiz show Only Connect 😃

  • @omegadecisive
    @omegadecisive Жыл бұрын

    Is it possible to modify LLMs to identify ambiguity in prompts and that a return question might be more appropriate to better help provide a more accurate answer to the first prompt?

  • @JasonTodd339

    @JasonTodd339

    10 ай бұрын

    Not unless they understand more about what exactly we mean. Needing more data I'd say

  • @professoradenisevargas1573
    @professoradenisevargas15739 ай бұрын

    What is the technique for filming as if you were writing outside a frame? Do you have a mirror or glass?

  • @IBMTechnology

    @IBMTechnology

    9 ай бұрын

    See ibm.biz/write-backwards

  • @jsonbourne8122

    @jsonbourne8122

    8 ай бұрын

    He is flipping the video

  • @jamesjonnes
    @jamesjonnes10 ай бұрын

    The reasons for hallucinations are more than data quality, search method, or input context. Any of the generated tokens may send the LLM into a wrong path. The real solution is to get all tokens right. AIs that play games are better at this, that's why Deepmind is trying to splice its Alpha technology with LLMs. They claim that their model is currently under training and has already achieved results that are superior to GPT-4. There are many other methods to make the LLM arrive at a more desirable path. One is to ask the LLM to attempt to answer the question in many distinct ways and pick the best or try again if all generated answers were incorrect. You can also tell the LLM to ask you questions to clarify and improve the quality of its final response.

  • @ScientistSplinterGrp
    @ScientistSplinterGrp Жыл бұрын

    The hallucination is on the part of the user. It is the user who believes that the statistically generated string of tokens, when covered into words, is a statement of fact. Garbage in, garbage out. Your question was vague. Your initial dataset for loading the staristical model should be more tightly controlled. The solutions seem to be for actual intelligence curating input, so minimal intelligence is necessary for the output user.

  • @JasonTodd339
    @JasonTodd33910 ай бұрын

    Writong backwards left handed. What a wizard

  • @IBMTechnology

    @IBMTechnology

    10 ай бұрын

    See ibm.biz/write-backwards

  • @xtronkillmaster2517
    @xtronkillmaster25174 ай бұрын

    I set chat GPT’s prompt as my biological father & added a few other secret prompts. When I asked for certain things it would include a copy file with what I believe to be camouflaged text code. Is there a way to figure what the code is supposed to be or what it means?

  • @DJWESG1
    @DJWESG1 Жыл бұрын

    You can get those scanners that read texts from books. So each person can have theor own language model trained entirly from their own book cases. Free gift for you budding theorists.

  • @velo1337
    @velo1337 Жыл бұрын

    Mutation is a key driver for innovation but also for hallucination :)

  • @jonbrand5068

    @jonbrand5068

    Жыл бұрын

    And without random mutations, there could be no evolutionary process.. anywhere. Mutations, little ones.. are a requirement of living systems to adapt to surroundings that change over time. I wouldn't have thought I could be comfortable living in a meat freezer, for example, wearing my skivvies. And that was right but it has so little to do with my point.

  • @johnobrien8773
    @johnobrien8773 Жыл бұрын

    The recurring point about the engineers that built it not understanding it reminds me of Neon Genesis Evangelion. Particularly "The End of Evangelion". It's great if you're interested but I've heard the Netflix version has been changed so I would seek out other avenues.

  • @StevenSeiller
    @StevenSeiller Жыл бұрын

    Who first called it a hallucination and why? And isn’t that anthropomorphizing an a bot?!?

  • @fabiorivera3427
    @fabiorivera34272 ай бұрын

    observe human interactions, you might arrive at some of the same conclusions... great vid !

  • @P-G-77
    @P-G-77 Жыл бұрын

    I view other explanations... and resulted, clear... at best, not in all parts but in the end yes, clear. In this case... yes, this is madness.

  • @nyariimani7281
    @nyariimani7281 Жыл бұрын

    "Plausible sounding nonsense." So they're exactly as intelligent as humans!

  • @NotDeadYetJim
    @NotDeadYetJim Жыл бұрын

    I needed better examples in the last section about mitigation.

  • @venkateswarlupokuri3430
    @venkateswarlupokuri3430Ай бұрын

    Completed

  • @Braneloc
    @Braneloc11 ай бұрын

    I took photos, called everyone and reacted a lot when the sky was green once. News later said it was pollution.

  • @vrschwrngsthrtkr22
    @vrschwrngsthrtkr22Ай бұрын

    How do you make him write in reverse technically?

  • @brambleinhabitant
    @brambleinhabitant4 ай бұрын

    What I am amazed by is how he can write in mirror image without effort. 🤔

  • @IBMTechnology

    @IBMTechnology

    4 ай бұрын

    See ibm.biz/write-backwards

  • @m.fazlurrahman5854
    @m.fazlurrahman5854 Жыл бұрын

    LLM is a degree in law. The three examples provided give something related to “distance”~ these are NOT hallucinating words as in all these examples “ measurable objects” have been mentioned. If you can measure it; you cannot hallucinate, unless you are suffering from “Cataract”

  • @kenbatchelor8284
    @kenbatchelor82845 ай бұрын

    Mica Paris?

  • @khangvutien2538
    @khangvutien253810 ай бұрын

    Ask ChatGPT this question “Is a dragon soldier light cavalry or heavy cavalry” and you’ll have a nice example of “hallucination” 😂😮

  • @itsmj3103
    @itsmj3103 Жыл бұрын

    There's a very important question that needs answering: Is he writting the letters backwards AND mirrored!?

  • @vomeronasal
    @vomeronasal Жыл бұрын

    Hallucination means to wander within one's own mind. Sounds about right for the LLM's. But who's mind is it wandering in?

  • @tomski2671
    @tomski2671 Жыл бұрын

    Ok, there seems to be a problem with designers forcing the LLM to answer. 1. LLM should ask questions of the user to disambiguate 2. Answering "I don't know" seems to be forbidden. Lack of this might be caused by personal or corporate biasses of the designers. In any case allowing AI to remain in this false state of certainty and not to know the limits of its knowledge is dangerous. Infinitely more if the AI were to become conscious.

  • @br3nto
    @br3nto11 ай бұрын

    Why can’t we build AI tools in a way that they can identify when an answer may be too broad in regards to a prompt and then ask the user to clarify or confirm the scope of answer? E.g Like how there are disambiguation pages for some Wikipedia entries.

  • @neildutoit5177

    @neildutoit5177

    11 ай бұрын

    I'm doing this. Pretty sure a lot of people are.

  • @Lorentz_Factor
    @Lorentz_Factor11 ай бұрын

    I think you meant non sequitur

  • @SteelTumbleweed
    @SteelTumbleweed11 ай бұрын

    6:16 Garfield does not speak. Thinking bubbles are used for everything coming out of Garfield. Looks like hallucinations aren't exclusive to LLMs.

  • @Pianofy
    @Pianofy3 ай бұрын

    You've stretched the definition of hallucination greatly here. If the training data contains errors and the model outputs one of those errors, the model is technically not hallucinating but simply providing the right answer to the right question but based on wrong training data. That's not hallucination. If the context of the request is not properly stated the output is technically also not hallucination but simply an incomplete prompt. It's a shame because hallucination is a very interesting phenomenon and it could be helpful to teach it from a more indepth view. Showing examples of how proper training data and a proper prompt can still end up with an hallucination simply because the model is just a tiny bit too creative for the specific prompt. That's an amazing example to show.

  • @lennardw.9841
    @lennardw.9841 Жыл бұрын

    But how do we help avoid factual hallucinations that are definitely not stemming from missing/wrong/incomplete context information? (The answer shows complete understanding of the request’s context but is wrong and in itself already contradictory)

  • @fearrp6777

    @fearrp6777

    Жыл бұрын

    The only way to solve agi ai problems is get the solution from the ai itself

  • @fallinginthed33p

    @fallinginthed33p

    Жыл бұрын

    Garbage in, garbage out. As long as someone wrote something that went into the training dataset, no matter how factual or otherwise they information was, the LLM will repeat it like a parrot. Maybe we could catch factual errors by continuously updating LLMs on the latest data but this comes at a huge cost in training resources.

  • @bengsynthmusic

    @bengsynthmusic

    Жыл бұрын

    Through downvotes.

  • @Lawsonjchris
    @Lawsonjchris5 ай бұрын

    What if these "hallucinations" are facts in an alternate universe? Kind of like the Mandela effect.

  • @r.k_1228
    @r.k_1228 Жыл бұрын

    Genuine question: Does IBM only hire left handed instructors for their videos?

  • @IBMTechnology

    @IBMTechnology

    Жыл бұрын

    Most are right-handed. They appear left-handed because we flip the image in post-production so the writing isn't backwards.

  • @fallinginthed33p

    @fallinginthed33p

    Жыл бұрын

    ​@@IBMTechnology the camera shoots through a transparent glass pane that the host writes on?

  • @IBMTechnology

    @IBMTechnology

    Жыл бұрын

    Exactly!

  • @ianmitchell8468

    @ianmitchell8468

    Жыл бұрын

    So where are all the actual left-handers?!

  • @ylihao

    @ylihao

    Жыл бұрын

    Finally I got my answer here. I was wondering how this presenter was able to write the letters flipped so naturally 😅

  • @davidzeto2446
    @davidzeto24463 ай бұрын

    AI hallucination is analogous to the most basic structural modality for emergent consciousness.

  • @jimbetts3064
    @jimbetts30646 ай бұрын

    Hallucinating is AI imagining.

  • @aaronjennings8385
    @aaronjennings8385 Жыл бұрын

    Bad guesses aren't hallucinations. They are delusions.

  • @jonbrand5068
    @jonbrand5068 Жыл бұрын

    Good effort but I honestly don't think that data quality meaning factual inaccuracies from info taken from Reddit, has much to do with the "hallucinations" - I don't think you get it. That's my opinion. You have an imitation expert trained in the black box and it's imitating what it's picked up on after reading so much by us - it has begun to understand 'about' us - specifically that we, meaning humans,can and do bullshit eachother a good portion of the time, sometimes by making st*ff up.

  • @harmless6813

    @harmless6813

    Жыл бұрын

    Why do you self censor 'stuff'? 🤔

  • @shyama5612
    @shyama56125 ай бұрын

    Hallucination itself is an inaccurate description of what's going on but its gained popularity it's just a generative machine. The word hallucination is human post hoc interpretation of the result - it just generates text based on what data its trained on and probabilities it assigned them and you have no way knowing if its factual or made up. It's not a database lookup.

  • @insightamization
    @insightamization Жыл бұрын

    Whom are you suggesting be in day-care - ME or the "AI"...

  • @danellwein8679
    @danellwein8679 Жыл бұрын

    the Wolfram plugin helps chat gpt with facts ..

  • @phillipyoung6573
    @phillipyoung65735 ай бұрын

    To me the term "confabulation" is better than hallucination.

  • @Webnotized227
    @Webnotized227 Жыл бұрын

    I'd argue they're not that much different than many us in a way.

  • @IncomeBoost42
    @IncomeBoost42 Жыл бұрын

    Now I know why chatgpt keeps lying to me!

  • @neilo333
    @neilo333 Жыл бұрын

    Then he said "black box" and I knew he was hallucinating. Like all neural networks, he struggles to provide an answer where none lie. So he guesses, like all neural nets.

  • @seanbirtwistle649
    @seanbirtwistle64911 ай бұрын

    LLM hallucinations. also known as US politics

  • @Timrath
    @Timrath Жыл бұрын

    In my experience, bing chat hallucinates much less than ChatGPT. Was I just lucky, or is GPT really worse than bing?

  • @DJWESG1

    @DJWESG1

    Жыл бұрын

    I've found bard to be the dodgy of the three. It gave me a recipe for something explosive and tried to convince me that it had been working at a soup kitchen.

  • @olegdragora2557

    @olegdragora2557

    Жыл бұрын

    Bing is based on GPT-4, while free version of ChatGPT is using GPT-3.5 .

  • @maythesciencebewithyou

    @maythesciencebewithyou

    Жыл бұрын

    Bing uses GPT-4, chatGPT is a limited version of GPT-3-5

  • @sgramstrup
    @sgramstrup Жыл бұрын

    This writing 'backwards' on glass looks rather cool, but also rather impossible. Are you just flipping the video right to left ?

  • @elijahtheurer344

    @elijahtheurer344

    Жыл бұрын

    Came to comment this as well

  • @stephenbrillhart6223
    @stephenbrillhart6223 Жыл бұрын

    These aren’t hallucination. They are incorrect outputs of the model. The hallucination metaphor doesn’t even make sense because a hallucination is sensory stimulation without sensory input. LLMs have no senses so they can’t have hallucinations. Using this word to describe faulty output is irresponsible in my opinion. Along with all the other words people use in AI like “learning” and “intelligence”. These models do not and can not “learn” in the way normal people use that word. They are just minimizing a loss function. In the same vain, they aren’t “intelligent” either. They are just spitting out the word that has the highest probability given the previous words. A probability it calculated using a black box that the people who make these things are not even able to describe. And that’s not a deficiency or a bug it’s how these models are designed. And the people who work with these know this and they still use these words to make them seem like they are “intelligent” and “thinking” when they know full well that they are not. If these LLMs were actually “intelligent” then why do we need to be highly specific in our prompts? When you ask a history teacher “What happened in WWII” they understand implicitly what you mean and will answer appropriately. Because humans are actually intelligent. LLMs are not intelligent so you have to coax the answer you want out of them, and even then it is still just output from a black box, and not an “intelligent” response. IBM and companies that work with and peddle AI as the solution to all future problems have a monied interest in the public thinking these things are doing more than they actually are. So they use these buzzwords that people already have working definitions of in their heads to impress people but don’t tell them that the definitions they are using for these everyday common words are not even close to what the people think they mean.

  • @makhalid1999

    @makhalid1999

    Жыл бұрын

    I ain't reading all that Happy for you Or sad that it happened

  • @adikumarNY

    @adikumarNY

    Жыл бұрын

    I couldn’t agree more. The mindless mass frenzy without asking the right questions is truly amazing. Even the terminology LLM means it is a language model ie it will generally produce acceptable text. There is no reason to make a claim about factual correctness, as witnessed by so many examples quoted with completely inaccurate falsehoods.

  • @gamemak0r
    @gamemak0r Жыл бұрын

    An hallucination 😶😶😶

  • @John.R.F.
    @John.R.F. Жыл бұрын

    Ten minutes that amounts to Garbage In, Garbage Out, and none of his so called hallucinations are anything other than errors, errata, mistakes. A.I. hallucinations are different.

  • @7TheWhiteWolf
    @7TheWhiteWolf11 ай бұрын

    Jon cannot understand Garfield, so he doesn’t speak English in the comic strip, he thinks in ‘inner monologues’ GPT4 even pointed this put when I asked it the very cat question you provided. Seems like you’re hallucinating, not the LLM.

  • @chswin
    @chswin11 ай бұрын

    IBM is hallucinating… need to get your cloud service in order… I’m sure it’s fine for your addicted customers but you need to attract other customers.