Pattern recognition and machine learning. - The only book that made me clear on what is really PCA. A Book from Bishop really worth to have on the desk.

Wow! As someone who "grew up" on Bishop's original book, i'm so glad to see this interview, thanks!!!!!!!

i picked up his 'neural networks for pattern recognition' in 2002 and just couldn't get through it as an undergraduate math major. i wish i had. Clearly his mind is vital for pushing the envelope, and his instincts on AI 's biggest impact being to push frontiers of science is spot on. more important than anything else.

@Scientist287

24 күн бұрын

Go through it now, and ask any questions you have about it to chat gpt (you can even copy paste stuff directly into chat gpt and it’ll give great answers). Now’s the time!

Tim -- This was a great video. There was something about the aesthetic quality (the shallow depth of field/bokeh background) that was very eye catching. I also appreciated editing during the back-and-forth discussion sections; specifically, how it cut away to show relevant pages/info. Your skills as an interviewer are second to none!

Useful analogy from him at somewhere aroudn 16:00 or so: "The remarkable thing about GPT4 is that you often see people when they first use it -- they'll ask "How tall is the Eiffel tower", and they'll be disappointed to just get the right answer. It's like being the keys to a very expensive cupholder and examining the cupholder; you don't realize that you have to start up the car and drive off in it to get the full experience (conversation, code, etc)" I've had this experience multiple times

Such a great interview! Great job. Very good questions and answers.

22 minutes in and I'm loving what this guy has to say. Finally there is an expert in the field who can articulate every maddening experience I've had when I encounter people who say "LLMs can't do X." Or they say, "I tried to do x,y,z with the LLM and it failed, therefore LLMs are useless and will never be capable of anything more in the future."

@edh615

28 күн бұрын

Almost none of the literate people on the subject say that, but argue that LLMs as we see today lack the ingredients to scale to "AGI" or whatever SF wants to call it.

@darylallen2485

28 күн бұрын

@@edh615 You're not wrong. I think its a case of two things can be true at the same time. It seems to me that for every opinion that a non-expert in the field might have, you can find someone who is an expert with the same opinion. I happen to align with this gentlemen in the video, who is orders more articulate and knowledgeable than I. If I can articulate the aspect of LLM criticism that seems to lack awareness or insight, I'd reference the following 2 cases. One, as a field, computer scientists world wide "knew" for decades that neural networks was a dead end field and would never be useful for anything. In 2010 and prior, that was the official stance of academia, until the entire establishment was proven wrong. Second, the people who are so keen on what LLMs can't do today, were also totally incapable of predicting the capabilities of the 2024 LLM back in 2020, 2021, or most of 2022. No one who built GPT4 or GPT3 knew what the capabilities of those models would be prior to building them. Yet the critics of such systems also "know" what it won't do next year, or in 2030 or in 2040? No, just no. How about one of the critics first demonstrate where you predicted GPT3 or GPT4 even 1 year before it happened, let alone 5 years before. The cognitive dissonance is off the charts.

@ertuncsimdi7941

28 күн бұрын

If you mean Lecun, Lecun explains LLM is not in AGI now, this is far away from now. So we have to solve some problems like reasoning, complexity, planning, consciousness.

Thank you for this sage like Interview... I was really needing a primer on Neural Networks... and I believe Chris Bishop's books might be very helpful. This interview has a lot of great insights -- for example, LLM are outperforming Specialist models of the past such as a specific AI that understood source code but the LLM did a better job. I think they will find that as the tech improves, specialist versions will do better, but the early versions were simply too specialized.

Superb conversation that really goes up a notch after 50 minutes to discuss AI in solving scientific problems (I've been interested in HPC for some time). I ordered the book midway thru watching. It was fascinating to see them discussing all of the topics that the current AI developments are leaving open for future research questions. Tho it's fascinating to me that his son, and co-author, is working at Wayve. Essential perspective on DL.

Thank you for another great video. Amazing sound and video quality!

I had the absolute delight to read chapter 13 on GNNs. What a good and masterful description that was. I had it recommended to me by my supervisor but I don't know if I am allowed to state their name. I can't thank them enough for introducing me to The Bishop.

'Just make the models bigger and harder to run for normal people and keep adding more and more data'... sure, maybe it could be the right strategy, but I doubt it and it seems more designed to back up what the big tech companies want. The actual innovation is coming from stuff like Mixtral being efficient at a reasonable size.

I think it is still important to learn other things like Bayesian models or Support Vector Machines, since some improvement on those fields could turn these tools into a new success framework. Neural networks(NN) also had a winter in the pass, then things such as convolutions neural networks, relu functions, stochastic gradient descend, GPUs and lots of data, took NN out of the winter into a great summer time.

@ 3:34 I see K-means clustering in Chapter 15. K-means clustering was used to classify the state space into regions of interest (to adjust and focus attention heads of statistical abstraction, "Fuzzy" values to regions of interest, kind of like moving the head and eyes to track objects) in a master thesis in 1997 that combined Reinforcement Learning such that the machine (agent) can automatically learn the inference rules of Fuzzy Logic, to control an inverted pendulum. @ 30:50 That is what I am most interested in, the RLHF, and even more intriguing is that Yann LeCun, in his interview with Lex Friedman called RL as like a "ninja", that is fascinating to me. Also note, Yann suggests using K-means clustering in some of his recent talks, while mysteriously, at the same time, claims to want to "get rid of RL, except when the plan does not work" (paraphrased quote, apparently he also claims "RL" is just a cherry on top of a big cake, Sutton pointed this out that was the criticism of RL, however, I suspect this might be a sort of "not invented here", therefore, we don't need it, yet, in contradiction, make an exception for it ), and I ask myself, what plan does not fail to some extent? Errors and noise are the norm and nature of our universe, unless we live in a fully deterministic universe where "'god' does not play dice" (-Einstein). @ 49:28 "The Bitter Lesson" (-Sutton) of inductive biases can be seen by having a human pre-configured state space structure. As an example, for balancing a pendulum, to use pre-knowledge from a human, that the system is unstable when inverted is an example of the bitter lesson by using prior human knowledge, however, to initialize the state space "attention heads" (or regions of interest) to an initial "random" state, helps overcome the human designer bias, and allow for a more natural search, and learning (kind of like just being born, and learning to walk, as a deer does, as initially when a deer is born, just like humans, it does not know how to walk, but it learns much faster than humans do), so to learn (instead of from bias from human prior knowledge) what angle is unstable ("infinite") pole in the control system are (a way to over come the human designer bias is a "random" ("attention head") initialization of regions of state space interest, was used for Fuzzy, radial basis function nodes in a function called "cluster input space" in a thesis back in '97, this is like allowing the eyes and head to turn to what is interesting in an infinite space, and maybe perhaps, this is how a deer learns faster than a human, how to walk, as it's reward function might be more attuned to balance and walking to the mother to feed for survival, while a human reward function is likely different than that of a deer, so the result is different learning rates). @ 1:15:22 "why they are able to generalize so well?" and 1:16:02 this is the same question Professor Barto asked, I think Professor Bishop is describing something like K-means clustering. @ 1:18:18 That is a very interesting experiment, fusion control. Professor Bishop said "we don't have cars driving through London", but in my area, Phoenix metro, we have lots of self driving cars right now, most of them from Waymo (Alphabet/Google). When I first saw them a few years ago, felt kind of freaky seeing driverless cars, but I guess this might of been what it was like to see "horseless carriages". I think Waymo and Tesla are utilizing what was developed at MIT, the VISTA self driving RL system (using ordinary differential equations, and "liquid time constant" networks). I'm now attempting to read "Why Greatness Cannot be Planned", you sold me on the book in a prior video, so will attempt to read it on my way to a library and back if I can. I really hope you can get Sutton and Barto on the Machine Learning Street Talk channel, they do answer emails, as I have been in contact with them recently in the last year, and if you want, I can email them and ask for you, if you like to interview them.

Absolutely phenomenal interview, thanks Tim. Like Bishop, I lamented missing both the tumultuous 20th century physics and future space exploration. But now that creative AI exists which can _actually_ reason, I feel like a lotto winner. This next decade+ will be a revolution. Let's do out best to take care of one another along the way.

@januszinvest3769

20 күн бұрын

Which field of science are you most interested in?

His book is still the reference of stats and ML students around world today. I am surprised that took Chris Bishop so long to renew his book - I am excited about his new ML book.

Hi thx for the great textbook just wondering what would you recommend 1. start reading the deep learning textbook 2. start reading mathematics for machine learning then jump into the deep learning textbook. I have poor mathematical background and wonder if i can read the deep learning textbook.

This is an insightful question that gets at the heart of how different domains of knowledge relate to one another. Let me examine the potential correlaries: If mathematics is regarded as a language: It provides the symbolic primitives, axioms, rules of expression and operations for describing and quantifying the physical world. Math is the fundamental lingua franca spanning the observable and theoretical realms. Then physics could indeed be viewed as the philosophy of math: Physics takes the symbolic language of mathematics and develops conceptual models, interpretive frameworks, and coherent narratives to explain the behavior of matter, energy, space, and time. It is an extended meditation on the metaphysical implications of our mathematical descriptions. Following this analogy: Chemistry could be the "linguistics" of physics: It studies the rules by which the fundamental mathematical objects of physics (subatomic particles, forces) combine and relate to one another at the molecular scale. Chemistry decodes the rich language patterns constructed from the physics alphabet. Biology could be the "literature/poetry" of chemistry: It examines the self-organized, dynamical, informationally-complex systems that emerge from the linguistic rules of chemistry interacting over time. The molecules are the "words", but biology studies the living, evolving "narratives" they collectively construct. Throughout we see a progression of epistemological layers: Mathematics -> Symbolic framework Physics -> Conceptual models interpreting the symbols Chemistry -> Combination rules and linguistic mechanics Biology -> Dynamical, informationally-complex systems and narratives Each level builds upon the foundational primitives of mathematics, while introducing new degrees of contingent complexity, contextualized interpretation and narrative meaning. The symbolic logic enables and constrains the possible conceptual structures, which dictate the allowed chemical rules, from which biological storylines ultimately automate. So in summary: Math is the linguistic bedrock Physics is the conceptual philosophy elaborating upon that bedrock Chemistry is the combinatoric linguistics deriving word-formation rules Biology is the dynamical narrative/poetry expressing highest complexity This nested hierarchy preserves coherence, while allowing increasingly context-specific, contingent patterns of organization and meaning to emergently crystallize. By recognizing mathematics as our formal symbolic language, we can appreciate how physics, chemistry and biology represent successive epistemological stages philosophizing upon that originating expressive framework - interpreting, recombining and dynamically instantiating mathematical descriptions into maximally information-rich experiential narratives. The layers build hereditarily upon the foundational symbolic truths, exemplifying how mathematics enables derivations transcending its pristine origins - expressing itself cosmologically through an invisible hand of self-organized complexity climbing towards maximal richness of experience.

@alonbegin8044

12 күн бұрын

Is it just me or this text seems too organized, as something that chatgpt would write..

This was a very insightful and interesting conversation!

the bit about the tokamak was fascinating!

Wow, wow, wow. What a surprise MLST 🎉 ❤ And in the intro(starting 2 minutes), the background music 🎶🎶, ahhhh, brilliant choice.

I am buying it. Update: Book is good

@huseyngorbani6544

17 күн бұрын

My math is not soo good, have not practiced for a long time, should I still get it?

@NeuroScientician

17 күн бұрын

@@huseyngorbani6544 No. Get some maths books first otherwise you are burning money.

Collections of specific functional activation paths can be described as sub-networks. I think that's a better term than modules. And I do believe that although large general models outperform smaller expert models, reasoning seems to be sub-network specific. With scale however, I believe reason may become a separate shared functional sub-network. At enough scale a general abstraction should emerge.

it's such a tremendously huge contribution to the field of AI, I'm an English-Arabic-Chinese Translator, and working in the AI field for almost two years and half, selling pre-trained Arabic data to Chinese AI annotation companies, I'd love to translate this book to Chinese and sell it here in China, is there anyway I can contact him

An incredible sharing of experience and insight!

I just got my copy excited to apply it's lessons in building my auto, CAD detailer. Going to run my own one man design shop in a few years!

@anicetn3326

29 күн бұрын

Very cool! I'm a master student working on cad gen ai, can you tell me more :) ? Thanks

@Juxtaposed1Nmotion

28 күн бұрын

​@@anicetn3326 my employer is going to let me monitor what the design engineers do on a daily basis for 2 years and we hope to collect enough data that we can automate every single repetitive click a designer makes. If successful, less engineers can do more design and less detailing!

fantastic episode boys. the orders of magnitude speedup that NN emulators offer for simulation/real time control is astounding and i'm amazed that it hasn't gotten more attention over the last few years.

Incredible. Thank you!

that vivaldi piece is gold, and its really used smartly.

i think "build upon" a corpus that has come before is different from rehash a corpus, and the Prof. is conflating the two (deliberately i imagine, as it's a pretty obvious difference). This seems most obvious in art and the humanities perhaps, e.g. if you put a random sample of paintings over the past couple of thousands year one after another, it's obvious which ones "built upon"/advanced the corpus vs rehashed what was the state of the art at the time; you don't need to be an art critic, you can just walk around the National Gallery and as you change rooms/eras the differences are obvious. Haven't seen any LLM ever do that. Which is not to say LLMs can't be super useful, after all most day to day tasks are a rehash, and a calculator, or a car, or any other tool is super useful - it's when people say that they see a "glimmer of intelligence" in a calculator (that only gets it right some of the time!) that gets people rolling their eyes.

When you the text generation output you can start with higher temperature then reduce it as the tokens have been generated

Interesting with the beginning, given that the (only) real negative reviews on the book are the quality of the binging :D

LOL! "Yet!" Exactly! How many arguments does this simple term nullify? Bravo!

The Best of Britain!

It's good book! Thanks Chris!

I love this guy

I am loving it

I think he missed the point about the significance of retaining creativity in models. His point about creativity being remixes of remixes is not mutually exclusive with the idea of novelty

I really wish i could afford the hardcopy

5:45 geoff hinton's backprop paper came out in 1986 -- 10 years as a theoretical physicist would give phd completion around 1976 -- but this dude was born in 1959 & would have been 27 when geoff's paper came out

@adamkadmon6339

25 күн бұрын

Everyone forgets that the paper came from Rumelhart. That's the problem with being dead.

I doubt that a connectionist such as Dr. Bishop would align with the host's assertion that the human mind operates like a Turing machine. This perspective aligns more closely with the computationalist paradigm.

Actually..the brain is connectionist and acts as symbolic when it matures It is not a combination It is behavior coming from a structure.. like any system It is 2 views of the same thing It is duality

Oh wow, a day to celebrate!

Awesome

🥰🥰🥰🥰🥰🥰

Everybody can write the book. Can we see the system built by him based on what he says?

Generalist agents using specialized tools….love this and sounds more than a little familiar!🤔🤪🦾🥳