Honestly man, you are doing an absolute great job with these interviews. Plenty of people like myself are hungry for these technical nuggets of information, and want to go beyond the superficial questions that most interviewers ask people in the AI field. Keep going!

Modern Programming languages are based around a fairly raw representation of context free grammars, invented by the linguist Noam Chomsky. It would therefore be expected, that by putting in programming languages, a model would pick up on the aspects of context free grammars that are decent approximations to syntax of natural language. So I would hesitate to put that forth as an example of transfer learning, given programming languages exist as they do as representations of ideal natural languages.

The uproar and the wide cut at 1:28 is hilarious

at 2:25 there's reference to a paper that shows that if you train on math you get better at coding. what's that paper?

Love this and love the small shade to LeCun.

Am I the only one that thinks Dwarkesh looks like younger version of Leto Atreides?

Can you share the links to the papers referenced?

Wow, uploaded 6 seconds ago. I'm premiering some of the top Anthropic insight!

@user-wy4dt2kc3m

2 ай бұрын

I dont care about u

Who are these guys? I never heard of them.

@lance3301

2 ай бұрын

Trenton Bricken is with Anthropic, Sholto Douglas is with DeepMind.

@hummuswithpitta

2 ай бұрын

Takes longer to type that dumb question here than Google their names.

@JimLloyd1

2 ай бұрын

I was happy to see that question asked and answered as it saved me the time to go google them.

@dg-ov4cf

2 ай бұрын

ah yes, Trenten and Sholto, the stalwart old dynamic duo of nitwit charlatans, back on the show to no doubt drum up yet another salvo of nonsensical sequiturs and debunked, a priori chest-beating - at which we empiric-minded individuals of a higher epistemic sophistication must calmly roll our eyes, release an unbemused sigh, and take a moment of our valuable time to once again dispel, nonplussed... i would implore any fair-minded listener to take their inventive (however risibly ill-conceived and fanciful) analogies with LESS than a grain of salt. tempting as their ideas may be to imbibe unquestioningly - you'll notice that these sorts of miracle peddlers never seem so inclined to likewise offer equal time's elucidation as to *where* exactly these oh-so seemingly innocent lines of rhetoric lead, once taken a bit farther down the philosophical road... that's a deduction through which I'll trustingly refrain from holding your hand.

@dg-ov4cf

2 ай бұрын

ah yes, Trenten and Sholto, the stalwart old dynamic duo of nitwit charlatans, back on the show to no doubt drum up yet another salvo of nonsensical sequiturs and debunked, a priori chest-beating - at which we empiric-minded individuals of a higher epistemic sophistication must calmly roll our eyes, liberate but an unbemused sigh, and take a begrudging moment of our valuable time to once again dispel, nonplussed and stoic as ever... i would implore any fair-minded listener to take their inventive (however risibly ill-conceived and fanciful) analogies with LESS than the proverbial grain of salt. tempting as their grandiose ideas may be to imbibe unquestioningly - you'll notice that these sorts of miracle peddlers never seem so inclined in their sophistry to likewise offer an equal time's elucidation as to *where* exactly these oh-so seemingly innocent lines of rhetoric lead, once taken a bit farther down the philosophical runway... the product of its "takeoff"? that's a deduction through which I'll trustingly refrain from holding your hand.

@Dwarkesh Patel, its not that crazy if you have strong intuitions about why transfer learning works. I'd love to tell you about fractal machine learning.

It should be enshrined in law that smart people can't look good. It's simply unfair to have both.

@DistortedV12

2 ай бұрын

None of them are like stellar ML researchers in terms of citations. Look like they got to where they are from networking

@hayekianman

2 ай бұрын

one is a podcaster. others are users of ml models

@mikestaub

2 ай бұрын

I'm fine with smart people being attractive, as long as they have no artistic talents. Then they should be in a separate tax bracket.

@therainman7777

2 ай бұрын

@@hayekianmanWhat are you talking about? His guests are AI researchers at Google DeepMind and Anthropic, which are two of the best AI organizations on the planet. They’re hardcore researchers and engineers. They’re “users of ML models”? No, sorry. That’s not true at all.

@therainman7777

2 ай бұрын

@@DistortedV12You do realize that citation count is not the only metric for gauging the quality of a researcher/engineer? Not everyone is in academia, and not everyone’s goal in their career is to publish research that gets the maximum number of citations. These guys work in the private sector, so much of what they work on is never going to be published, and much of what they do publish is going to be published in general terms only, so as to not give away their company’s IP. As a result of course they’re going to be cited less. To say it’s “just the result of networking” frankly sounds like envy and/or cope; I will trust the judgment of hiring managers at two of the top AI firms on Earth over the judgment of a commenter on KZread who is basing his assessment on a video he watched. Unbelievable levels of ignorance in these comments.

can you cite the papers in your description (links)? would be helpful for reference tbh.. great interview nonetheless. get LeCun on and pick his brain.

How come he doesn't have a million views ... it's time for the general public to understand AI better

@darylallen2485

2 ай бұрын

I don't think most humans are ready to accept the idea of an apex intellect that isn't human. The mental contortions that people go through to "prove" that LLMs aren't reasoning in some capacity is, in my honest opinion, emotionally driven, rather than objective and logically based. Knowing the basis for resistance helps to understand what arguments will and won't work. Emotionally motivated people, in general, don't find objective logical arguments persuasive.

@dg-ov4cf

2 ай бұрын

a lamentable state of affairs it is indeed - you see, however, in the wake of many a sleepless night spent reeling in nocturnal bedlam, ever-haunted by the harrowingly grim prospects of variously inauspicious destinies once relegated to the strongbox of fiction - i fear not *for* the deplorable, unthinking masses in their haphazard, drudging (if not outright futile) path toward the inevitable conclusions upon which you and I have no doubt converged independently - it has most unfortunately come to pass that my once-utilitarian woes of trepidation arise no longer from the naïve, altruistic notions of Bentham's folly - rather, owing to many a midnight epiphany had betwixt months past, these anxieties would now be better served by the forlorn, unwelcomedly negotiated *inverse* characterization - that is, being moreso *of* the aforementioned hordes of thoughtless serfs and cantankerous luddites, once these harsh eventualities will have (in the fullness of time) come to dawn upon their feeble, insectoid intellects, knowing as I all too well do of their profound collective ignorance, which they seem to carry faithfully (and with stern resoluteness) to the bitter end.

@dg-ov4cf

2 ай бұрын

a lamentable state of affairs it is indeed - you see, however, in the wake of many a sleepless night spent reeling in nocturnal bedlam, ever-haunted by the harrowingly grim prospects of variously inauspicious destinies once relegated to the strongbox of fiction - i fear not *for* the deplorable, unthinking masses in their haphazard, drudging (if not outright futile) path toward the inevitable conclusions upon which you and I have no doubt converged independently - it has most unfortunately come to pass that my once-utilitarian woes of trepidation arise no longer from the naïve, altruistic notions of Bentham's folly - rather, owing to many a midnight epiphany had betwixt months past, these anxieties would now be better served by the forlorn, unwelcomedly negotiated *inverse* characterization - that is, being moreso *of* the aforementioned hordes of thoughtless serfs and cantankerous luddites, once these harsh eventualities will have, in the fullness of time, come to dawn upon their feeble, insectoid intellects, knowing as I all too well do of their profound collective ignorance, which they seem to carry faithfully (and with stern resoluteness) to the bitter end.

@lolololo-cx4dp

2 ай бұрын

​@@darylallen2485I think you are describing your self

@darylallen2485

Ай бұрын

@@lolololo-cx4dp you've figured me out. Congrats.

Deacon actually claims that language has been evolving for at least 2 million years, not tens of thousands. IMO you are right to say tens of thousands, as there is only unambiguous evidence of symbolic thought (by his admission) starting ~50 kya. It could have been millions of years earlier, but then what are all of the genetic and cultural changes in the last 50 kya? Seems like they are more than window dressing, or at least have to put some probability mass on the shorter timeline.

@dg-ov4cf

2 ай бұрын

hmmph. an amusing, if somewhat tortured, conjecture - but why, dear friend, would we hazard to seize upon so gaunt a figure as mr. deacon's measly two million? far be it from me to consider wholly invalid the very premise of your flawed inquiry - for who, pray tell, might be so brave as to put forth a rhetorically sound line of reasoning in negation of the obvious - that is, the genesis of this nebulous communicative medium which we (so steadfast in our collective penchant for recursivity) call by the term "language" is, in fact, ultimately but a mere desultory extension of a far more primitive anatomical substrate which indeed blessed our lineage more than some 500 million years before - that being, of course, the very development of the the oral apparatus itself. indeed, i dare say this "deacon" fellow - estimable an academic as he very well may be - might do well to consider limiting the so wantonly distributed scope of his professed expertise to that which pertains more strictly to matters of the clergy... sigh... but of course, lest i bear the risk of summary judgment for want of due professional regard i dare clarify the previous slight, pernicious in bite as it may come across, to be made *wholly* in jest - though the poignancies of my monologue as extolled above stand wholly without caveat...

@doggedinterlocutor

2 ай бұрын

hmmph. an amusing, if somewhat tortured, conjecture - but why, dear friend, would we hazard to seize upon so gaunt a figure as mr. deacon's measly two million? far be it from me to consider wholly invalid the very premise of your flawed inquiry - for who, pray tell, might be so brave as to put forth a rhetorically sound line of reasoning in negation of the obvious - that is, the genesis of this nebulous communicative medium which we (so steadfast in our collective penchant for recursivity) know by the term "language" is, in fact, ultimately but a mere desultory extension of a far more primitive anatomical substrate which indeed blessed our lineage some 500 million years before - that being, of course, the very development of the the oral apparatus itself. indeed, i dare say this "deacon" fellow - estimable an academic as he very well may be - might do well to consider limiting the so wantonly distributed scope of his professed expertise to that which pertains more strictly to matters of the clergy... sigh... but of course, at the risk of summary judgment for want of due regard I dare clarify the previous slight to be in jest - though the poignancies of my monologue as extolled above stand without caveat...

@doggedinterlocutor

2 ай бұрын

hmmph. an amusing, if somewhat tortured, conjecture - but why, dear friend, would we hazard to seize upon so gaunt a figure as mr. deacon's measly two million? far be it from me to consider wholly invalid the very premise of your flawed inquiry - for who, pray tell, might be so brave as to put forth a rhetorically sound line of reasoning in negation of the obvious - that is, the genesis of this nebulous communicative medium which we (so steadfast in our collective penchant for recursivity) know by the term "language" is, in fact, ultimately but a mere desultory extension of a far more primitive anatomical substrate which indeed blessed our lineage some 500 million years before - that being, of course, the very development of the the oral apparatus itself. indeed, i dare say this "deacon" fellow - estimable an academic as he very well may be - might do well to consider limiting the so wantonly distributed scope of his professed expertise to that which pertains more strictly to matters of the clergy... sigh... but of course, at the risk of summary judgment for want of due regard I dare clarify the previous slight to be in jest - though the poignancies of my monologue as extolled above stand without caveat...

@dg-ov4cf

2 ай бұрын

hmmph. an amusing, if somewhat tortured, conjecture - but why, dear friend, would we hazard to seize upon so gaunt a figure as mr. deacon's measly two million? far be it from me to consider wholly invalid the very premise of your flawed inquiry - for who, pray tell, might be so brave as to put forth a rhetorically sound line of reasoning in negation of the obvious - that is, the genesis of this nebulous communicative medium which we (so steadfast in our collective penchant for recursivity) know by the term "language" is, in fact, ultimately but a mere desultory extension of a far more primitive anatomical substrate which indeed blessed our lineage some 500 million years before - that being, of course, the very development of the the oral apparatus itself. indeed, i dare say this "deacon" fellow - estimable an academic as he very well may be - might do well to consider limiting the so wantonly distributed scope of his professed expertise to that which pertains more strictly to matters of the clergy... sigh... but of course, at the risk of summary judgment for want of due regard I dare clarify the previous slight to be in jest - though the poignancies of my monologue as extolled above stand without caveat...

OK; then you have to explain the process by which that happens. Otherwise we're into the magic territory!

I had no idea Jim Halpert had anti-aging treaments and got into AI research

for the algo

Gotta wait til Thursday!?

Who are the guests?

We shan’t forget LLMs are dead Frankenstein brains. Outside inference it’s not doing anything.

tmne gujarati avde che?

👍

sweet

hmmm idk

Nost handsome Indian Ive seen

Epic hair.

4:15. That’s not learning. That’s still statistically derived information. If you can’t see that, how are you working in this field? You’re projecting humanness onto a machine.

@darylallen2485

2 ай бұрын

Can you elaborate how the explanation for 3:33 isn't learning? My understanding from this clip is, they explain the circumstance of a board game to an LLM (sequence of moves in the game). Then the LLM has neurons that correlate to a mental model of the board, as explained to it? You're saying thats not an example of learning? Or you're saying the sequence of moves was in the training data? Or you have a 3rd claim that I've completely misunderstood?

@MrMichiel1983

2 ай бұрын

@@darylallen2485 I would say that this is not well defined model building, but still merely parroting. The comment was about generalization, and that can still be parroted if overfitted rather than learned. In the argument itself I did not spot anything to assume it was not overfitting.

@generichuman_

2 ай бұрын

Based on your last sentence, your definition of "learning" is that which only humans can do, which is an incredibly useless definition of learning. It's always amusing to me how faceless people on KZread can make substanceless comments like this with such confidence.

@andybaldman

2 ай бұрын

@@darylallen2485 The output is statistically derived from the input training data set. Regardless of how you word it, that's ultimately all that is happening in the system, because that's all it can do.

@andybaldman

2 ай бұрын

@@generichuman_ Strawman. I never said that. What I said is that he's interpreting humanness in a system that is only producing a simulacrum of it. That's not to say a non-human system can never be capable of learning. But there's a difference between memory, statistical inference, and the things that humans can actually do. The first two are more mechanistic, while what humans do is a lot more generalized. It's likely related to the fact that we are made up of agential material (i.e., our cells have intelligence, goal-seeking ability, and information-processing capabilities that the passive transistors in a computer do not.) Somehow that intelligence gets aggregated by how we are architected, in ways that we have not yet reproduced in a non-human system.

bad reasoning seeing things that aren't there

stop saying "like" all the time, it's exhausting.