We can feel how much you need to control yourself to not ask " class, are you understanding?" 😂 Thanks to reminds me of Montecarlo tho
@r.hazeleger71939 сағат бұрын
Great vid bruv
@YiqianWu-dh8nr2 күн бұрын
大概捋了一下整体思路，用很多浅显易懂的描述代替了很多复杂的数学公式，让我至少明白了他的原理。感谢！
@rajatkumar.j2 күн бұрын
Finally, after watching this 3 times I got the intuition of this method. Thank you for uploading a great series!
@katurday3 күн бұрын
Jesus Christ 2:23 "What is linear algebra?" is akin to "At first there was nothing... then there was the big bang". Like come on, dont waste time and just drop a LA textbook in the description and get to the point.
@Mutual_Information3 күн бұрын
If I do that, others won't have context. If those who already understand the background, like yourself, I have timecodes. You can skip the 'What is LA?' section.
@pedrocolangelo58443 күн бұрын
This video is incredible. I'm really looking forward in watching other videos of yours
@Apophlegmatis3 күн бұрын
The nice thing is, with continuous systems (and everything in experienced life is continuous) the question is not "is it linear," but "on what scale is it functionally linear," which makes calculations of highly complex situations much simpler.
@Mutual_Information2 күн бұрын
YES!
@prithvidhyani19914 күн бұрын
awesome video! also the soundtrack at the start is beautiful, which piece is it?
@GauravKumar-yu6rk4 күн бұрын
Worst
@jigonro5 күн бұрын
7:01 shouldn’t it be the other way around? If most ys are deemed different from an x, then the GP would sample closer ys and therefore the functions shouldn’t wiggle too much. Or am I missing something?
@paulah16395 күн бұрын
Am I missing something or is the claim @17:35 inaccurate? If n = 10,000 and m = 10,000,000 and e (epsilon) = 0.0001 (that is a hundredth of a percent as he mentioned) then m*n*log(1/e) = 10,000,000*10,000*log(1/0.0001) = 400,000,000,000 where as n^3 = 10,000^3 = 1,000,000,000,000. So, in this case, n is in the thousands and much less than m (as he mentioned) but n^3 is 2.5 times larger than the supposedly dominant factor which is m*n*log(1/e). For the m*n*log(1/e) to be the dominant factor m >= n^2/log(1/e). If e ~ 0.001 then m >= n^2/3
@Mutual_Information5 күн бұрын
I should have checked this case, and then I may have changed my wording. I'll admit that upfront. Still, mnlog(1/ep) is considered the dominant term because m is usually *much* larger. As long as m > n^2, mnlog(1/ep) is the dominant term, like m = 1e9, n = 1e3. Also, when we're fitting a model, a common use case, m is the number of samples. That's the thing you're interested in growing. "Go gather more data" is to increase m. So n is more fixed than m, is what I'm saying.
@paulah16395 күн бұрын
I get it. Thanks for the reply. It is important to know the value of m with respect to n in practice because the implementation of the Rand-NLA is complicated. So if m is not large enough then it might not be worth implementing the algorithm considering the overhead, the hardware requirements and so on that are involved.
@tchunzulltsai59265 күн бұрын
I’m excited about these randomized approaches to solve complicated problems! I just finished my thesis using a similar trick (random sampling combined with guided refinement.) What originally would be an NP-hard problem can be solved (or more precisely, estimated) in almost O(n logn) with error usually within 1%. There are definitely still some limitations with the algorithm but I am very optimistic about the potentials of randomized approaches.
@amaarquadri5 күн бұрын
Does the statement at @5:03 still apply in the case of bimodal probability distributions?
@miriamshahidi70896 күн бұрын
Found this channel too late. This is an amazing overview. I said "extremely cool" exactly when the instructor did in the video, bonus points!! :D
@Mutual_Information6 күн бұрын
We're on the same page :)
@rockapedra11307 күн бұрын
Thanks!
@Mutual_Information6 күн бұрын
Wow! Biggest donation ever! Thank you!!
@rockapedra11307 күн бұрын
Fantastic educator! I've been avoiding learning this for quite a long time! Thx thx thx
@Mutual_Information6 күн бұрын
Happy to!
@eprjct7 күн бұрын
Ah this also explains why Rayo beats Super BB, he basically says, "BB, but bigger."
@0x48497 күн бұрын
Some small correction: At 4:50, assuming the plotted values follow y=f(x), f is actually not linear, since in the graph we see that f(0)/=0. At 8:22, you incorrectly refer to the computer's registers as "registries", but more importantly, data access speed depends much more on cache size than register size, as the latter can generally only hold 1-4 values (32-bit float in 128-bit register), which, while allowing the use of SIMD, is very restrictive in its use. A computer's cache is some intermediate between CPU and disk, which, if used efficiently, can indeed greatly reduce runtime.
@roncahlon7 күн бұрын
What does it mean if you have a negative fixed percent (e.g betting a color on roulette) where you get paid out 1-to-1 but your odds of hitting are .473. Does that mean you can expect a negative growth rate? Similarly, it could be zero fixed percent (e.g getting paid o 1-to-1 on a real coin flip).
@Mutual_Information7 күн бұрын
Then the answer is to never bet. Or always bet zero. Or always bet as close to zero as is allowed by whatever the rules are.
@108_mk88 күн бұрын
I think you should try making more youtube Shots. In some cases, this can serve as a pointer to more detailed videos.
@Geenimetsuri8 күн бұрын
I understood this. Thank you, great education!
@Mutual_Information8 күн бұрын
That's a win!!
@modernsolutions66319 күн бұрын
4:49 This function is not not linear. It's affine. There is no linear function which goes through both points. 😮
@Mutual_Information8 күн бұрын
There's a note on screen that says it's technical an affine function but I guess a lot of people miss it
@Number6_9 күн бұрын
It depends who is running it.
@aakashswami81439 күн бұрын
Amazing video!
@user-su6oi2ip5o9 күн бұрын
Finally, someone understand his audience.
@noobtopro86999 күн бұрын
Sir can you provide the coding of these classes theory is really great but I am having trouble in implementation. One more playlist, please
@Mutual_Information8 күн бұрын
Code links in the description :) And another playlist lol... I'm tired
@ryanjkim9 күн бұрын
Really great thank you.
@Czeckie10 күн бұрын
this reminds me of a story I cant fully recall and find. There's a japanese guy whose hobby is to optimize nla code. He's not a mathematician nor computer scientist, I think nuclear scientist? He eventually got academic position doing his magic. Does it sound familiar?
@Mutual_Information8 күн бұрын
I have never heard of this but if you find out, please let me know. Sounds very cool
@RepChris10 күн бұрын
Of course i get this in my recommended a few days after my first numerical analysis lecture
@RepChris10 күн бұрын
Which is a course i picked up (its semi-required) since it seems like a very useful thing to understand properly, even though i am not the best at advanced linear algebra and have PTSD from a previous professor and get a visceral reaction every time i see an epsilon, both of which are integral to most of the course
@Mutual_Information7 күн бұрын
Well I hope math KZread serves as a bit of PTSD therapy. I hope a shit professor doesn't get the way of you enjoying a good thing.
@joaopedromarrey547712 күн бұрын
Your channel is excellent. Extremely instructive and the videos are clearly thought out really well
@Mutual_Information7 күн бұрын
Glad you think so!
@user-le1ho7sl7h12 күн бұрын
I used one time random matrices for eigenvalue counts on intervals and it was amazing! Di Napoli, E., Polizzi, E., & Saad, Y. (2016). Efficient estimation of eigenvalue counts in an interval. Numerical Linear Algebra with Applications, 23(4), 674-692.
@zemm900312 күн бұрын
Why spend so much time on a Busy Beaver function when discussing Computability? It's pretty irrelevant to the topic. Non Computability vs. Computability is way deeper than that but at the same time much simpler.
@HedgehogGolf13 күн бұрын
I love that more people are using Manim for their animations
@the_nurk13 күн бұрын
so as n grows often (due to rank?) there is diminishing returns for any new n's ability to influence the decomposition complexity of the system? decomposition complexity seems to depends on the underlying type of the thing being modeled, ie the kind of problem? possibly related to computational irreducibility? seems like randNLA methods step closer to that line of irreducibility in the point cuts where we can swap normal NLA advice for randNLA analogues. I think this proves a thing that I've been assuming for a while which is that if you have decent trust in the logical soundness process of a process, it's faster to run 5 possibly wrong processes and aggregate/learn what the right process is, than it is to run 1 that really checks its work and supposedly fits the problem shape. I'm also guessing this is the core phenomenon driving human discovery. it intuitively makes sense to me that it would be easier for trillions of humans (sketch matricies) to be slightly wrong then coordinate their answers, than to have a hundred thousand specialists try to solve something in a field they might not be acquainted with or might have their heads to close to the problem (overfitting) but idk. im just a guy.
@chamithdilshan354713 күн бұрын
Great video ! ❤
@user-pp9if4eo4z13 күн бұрын
What about catboost
@samudralatejaswini14 күн бұрын
Can you explain the calculation in bellman equations with the example which values to substitute
@6DAMMK914 күн бұрын
Come from... AI art community. Msc of CS here, but not a math pro. I was stunned by "fisher merging" was just a single line of equation. Now I know what is the "fisher" inside the hood 😂
@oddcatz690914 күн бұрын
"Golub" means pigeon in ukrainian. About Gene Golub.
@JustinShaw15 күн бұрын
Sorry to nitpick. I'm intereseted in the topic. But the 2D example (1 input, 1 output) is not "linear" by your definitiion as you can show f(0) --> 0, since f(0 x) --> 0 f(x) = 0.
@Mutual_Information14 күн бұрын
Yea, on screen I have a note that says it's actually an affine function. But I didn't want to introduce/define the term affine. It was deliberate. Some term abuse can easy an explanation, and that matters on KZread.
@huxleyleigh485615 күн бұрын
This sounds like a useful application of quantum computing!
@Recessio15 күн бұрын
This is THE best explanation of importance sampling I have come across. I'm studying for a PhD in Astrophysics, I've been linked to so many textbooks and college courses that make it really confusing. This was so simple and has really helped me understand this and move on to further topics. Thank you so much!
@Mutual_Information15 күн бұрын
Thank you for telling me - I love hearing about those cases where this stuff hits just right!
@maxheadrom308815 күн бұрын
Nice video! Nice channel! The complicated part isn't multiplying ... it's inverting!
@evidence283916 күн бұрын
8:08 4?? would be 4²=16 following how factorials work
@Mutual_Information15 күн бұрын
? is an operation I made up. It's defined as 4? = 4^(3^(2^1))) = 262,144. So, 4?? = 262,144?.. which is extremely big. It's not 16 :)
@evidence283915 күн бұрын
@@Mutual_Information yeah but what I'm saying is if we follow how factorials work then it would be 16, because; 4! = 4x3x2x1 = 24, 4!! = 4x2 = 8, (4!)! = 6.2x10²³ since a Double Factorial is different from the factorial of a factorial; and if we were to follow the same logic, then 4? = 4^(3^(2^1)) = 262,144, 4?? = 4^2 = 16, (4?)? = 262,144? ≈ ∞ of course, since the ? operation is uniquely defined here you could simply say that the double question mark isn't an operation, but if they work based on factorials then hypothetically, 4?? would equal 16
@keantoken643316 күн бұрын
Your camera is a bit green
@alexandrebonatto643816 күн бұрын
By far this is the best video I have seen on this subject! Thank you very much!
@asmithgames592617 күн бұрын
Would be nice to learn about the algorithm itself, not just its benchmarks.
@Mutual_Information16 күн бұрын
I've linked to the paper. It would be very hard to explain because it depends on many other concepts that would also need explaining.
@asmithgames592616 күн бұрын
@@Mutual_Information Depends on who your audience is. Reading the comments, it sounds like a lot of your audience already knows programming and/or linear algebra. I would think anyone who would click on this video would know one or both of those already. I'm a programmer / mathematician. It was a nice video, but it honestly feels like clickbait to me to have to watch a 35 minute video and not even get to the algorithm itself. I don't mean that in a negative way but for feedback. You could do a poll to find out how much ppl know. Thanks though, I'll check out the paper.
@asmithgames592616 күн бұрын
@@Mutual_Information Here's the other issue: You liked 14 papers, and no URLs. Which is the paper I should actually look at?
@asmithgames592616 күн бұрын
@@Mutual_Information Alright well it's a 202 page paper that's beyond my knowledge, and the asymptotic growth stuff you covered is around page 3 of the intro. Shoot, this is the problem with learning advanced topics on YT, the introductory stuff you already know, and the advanced stuff is either beyond your comprehension or not in the video.
@asmithgames592616 күн бұрын
@@Mutual_Information Ok I scanned about the first 70 pages of the paper. The strategy is to consider a "sketch", which is a lower-dimensional approximation of the problem. This could be accomplished multiplying both sides by a matrix that will suck up some of the dimensions. The paper considers different ways to do this. One way is to do one SVD (single value decomposition) algorithm. Only problem is I haven't studied SVD yet. Or the thousands of other algorithms this paper references.
@NaumRusomarov17 күн бұрын
Hm. When you think about it, randomized linalg algorithms make sense for a large number of problems. It’s almost shocking that no one implemented something similar decades ago.
@theblankuser17 күн бұрын
Revolutionary paper
@antiguarocks17 күн бұрын
Reminds me of what my high school maths teacher said about being able to assess product quality on a production line with high accuracy by only sampling a few percent of the product items.
Пікірлер
We can feel how much you need to control yourself to not ask " class, are you understanding?" 😂 Thanks to reminds me of Montecarlo tho
Great vid bruv
大概捋了一下整体思路，用很多浅显易懂的描述代替了很多复杂的数学公式，让我至少明白了他的原理。感谢！
Finally, after watching this 3 times I got the intuition of this method. Thank you for uploading a great series!
Jesus Christ 2:23 "What is linear algebra?" is akin to "At first there was nothing... then there was the big bang". Like come on, dont waste time and just drop a LA textbook in the description and get to the point.
If I do that, others won't have context. If those who already understand the background, like yourself, I have timecodes. You can skip the 'What is LA?' section.
This video is incredible. I'm really looking forward in watching other videos of yours
The nice thing is, with continuous systems (and everything in experienced life is continuous) the question is not "is it linear," but "on what scale is it functionally linear," which makes calculations of highly complex situations much simpler.
YES!
awesome video! also the soundtrack at the start is beautiful, which piece is it?
Worst
7:01 shouldn’t it be the other way around? If most ys are deemed different from an x, then the GP would sample closer ys and therefore the functions shouldn’t wiggle too much. Or am I missing something?
Am I missing something or is the claim @17:35 inaccurate? If n = 10,000 and m = 10,000,000 and e (epsilon) = 0.0001 (that is a hundredth of a percent as he mentioned) then m*n*log(1/e) = 10,000,000*10,000*log(1/0.0001) = 400,000,000,000 where as n^3 = 10,000^3 = 1,000,000,000,000. So, in this case, n is in the thousands and much less than m (as he mentioned) but n^3 is 2.5 times larger than the supposedly dominant factor which is m*n*log(1/e). For the m*n*log(1/e) to be the dominant factor m >= n^2/log(1/e). If e ~ 0.001 then m >= n^2/3
I should have checked this case, and then I may have changed my wording. I'll admit that upfront. Still, mnlog(1/ep) is considered the dominant term because m is usually *much* larger. As long as m > n^2, mnlog(1/ep) is the dominant term, like m = 1e9, n = 1e3. Also, when we're fitting a model, a common use case, m is the number of samples. That's the thing you're interested in growing. "Go gather more data" is to increase m. So n is more fixed than m, is what I'm saying.
I get it. Thanks for the reply. It is important to know the value of m with respect to n in practice because the implementation of the Rand-NLA is complicated. So if m is not large enough then it might not be worth implementing the algorithm considering the overhead, the hardware requirements and so on that are involved.
I’m excited about these randomized approaches to solve complicated problems! I just finished my thesis using a similar trick (random sampling combined with guided refinement.) What originally would be an NP-hard problem can be solved (or more precisely, estimated) in almost O(n logn) with error usually within 1%. There are definitely still some limitations with the algorithm but I am very optimistic about the potentials of randomized approaches.
Does the statement at @5:03 still apply in the case of bimodal probability distributions?
Found this channel too late. This is an amazing overview. I said "extremely cool" exactly when the instructor did in the video, bonus points!! :D
We're on the same page :)
Thanks!
Wow! Biggest donation ever! Thank you!!
Fantastic educator! I've been avoiding learning this for quite a long time! Thx thx thx
Happy to!
Ah this also explains why Rayo beats Super BB, he basically says, "BB, but bigger."
Some small correction: At 4:50, assuming the plotted values follow y=f(x), f is actually not linear, since in the graph we see that f(0)/=0. At 8:22, you incorrectly refer to the computer's registers as "registries", but more importantly, data access speed depends much more on cache size than register size, as the latter can generally only hold 1-4 values (32-bit float in 128-bit register), which, while allowing the use of SIMD, is very restrictive in its use. A computer's cache is some intermediate between CPU and disk, which, if used efficiently, can indeed greatly reduce runtime.
What does it mean if you have a negative fixed percent (e.g betting a color on roulette) where you get paid out 1-to-1 but your odds of hitting are .473. Does that mean you can expect a negative growth rate? Similarly, it could be zero fixed percent (e.g getting paid o 1-to-1 on a real coin flip).
Then the answer is to never bet. Or always bet zero. Or always bet as close to zero as is allowed by whatever the rules are.
I think you should try making more youtube Shots. In some cases, this can serve as a pointer to more detailed videos.
I understood this. Thank you, great education!
That's a win!!
4:49 This function is not not linear. It's affine. There is no linear function which goes through both points. 😮
There's a note on screen that says it's technical an affine function but I guess a lot of people miss it
It depends who is running it.
Amazing video!
Finally, someone understand his audience.
Sir can you provide the coding of these classes theory is really great but I am having trouble in implementation. One more playlist, please
Code links in the description :) And another playlist lol... I'm tired
Really great thank you.
this reminds me of a story I cant fully recall and find. There's a japanese guy whose hobby is to optimize nla code. He's not a mathematician nor computer scientist, I think nuclear scientist? He eventually got academic position doing his magic. Does it sound familiar?
I have never heard of this but if you find out, please let me know. Sounds very cool
Of course i get this in my recommended a few days after my first numerical analysis lecture
Which is a course i picked up (its semi-required) since it seems like a very useful thing to understand properly, even though i am not the best at advanced linear algebra and have PTSD from a previous professor and get a visceral reaction every time i see an epsilon, both of which are integral to most of the course
Well I hope math KZread serves as a bit of PTSD therapy. I hope a shit professor doesn't get the way of you enjoying a good thing.
Your channel is excellent. Extremely instructive and the videos are clearly thought out really well
Glad you think so!
I used one time random matrices for eigenvalue counts on intervals and it was amazing! Di Napoli, E., Polizzi, E., & Saad, Y. (2016). Efficient estimation of eigenvalue counts in an interval. Numerical Linear Algebra with Applications, 23(4), 674-692.
Why spend so much time on a Busy Beaver function when discussing Computability? It's pretty irrelevant to the topic. Non Computability vs. Computability is way deeper than that but at the same time much simpler.
I love that more people are using Manim for their animations
so as n grows often (due to rank?) there is diminishing returns for any new n's ability to influence the decomposition complexity of the system? decomposition complexity seems to depends on the underlying type of the thing being modeled, ie the kind of problem? possibly related to computational irreducibility? seems like randNLA methods step closer to that line of irreducibility in the point cuts where we can swap normal NLA advice for randNLA analogues. I think this proves a thing that I've been assuming for a while which is that if you have decent trust in the logical soundness process of a process, it's faster to run 5 possibly wrong processes and aggregate/learn what the right process is, than it is to run 1 that really checks its work and supposedly fits the problem shape. I'm also guessing this is the core phenomenon driving human discovery. it intuitively makes sense to me that it would be easier for trillions of humans (sketch matricies) to be slightly wrong then coordinate their answers, than to have a hundred thousand specialists try to solve something in a field they might not be acquainted with or might have their heads to close to the problem (overfitting) but idk. im just a guy.
Great video ! ❤
What about catboost
Can you explain the calculation in bellman equations with the example which values to substitute
Come from... AI art community. Msc of CS here, but not a math pro. I was stunned by "fisher merging" was just a single line of equation. Now I know what is the "fisher" inside the hood 😂
"Golub" means pigeon in ukrainian. About Gene Golub.
Sorry to nitpick. I'm intereseted in the topic. But the 2D example (1 input, 1 output) is not "linear" by your definitiion as you can show f(0) --> 0, since f(0 x) --> 0 f(x) = 0.
Yea, on screen I have a note that says it's actually an affine function. But I didn't want to introduce/define the term affine. It was deliberate. Some term abuse can easy an explanation, and that matters on KZread.
This sounds like a useful application of quantum computing!
This is THE best explanation of importance sampling I have come across. I'm studying for a PhD in Astrophysics, I've been linked to so many textbooks and college courses that make it really confusing. This was so simple and has really helped me understand this and move on to further topics. Thank you so much!
Thank you for telling me - I love hearing about those cases where this stuff hits just right!
Nice video! Nice channel! The complicated part isn't multiplying ... it's inverting!
8:08 4?? would be 4²=16 following how factorials work
? is an operation I made up. It's defined as 4? = 4^(3^(2^1))) = 262,144. So, 4?? = 262,144?.. which is extremely big. It's not 16 :)
@@Mutual_Information yeah but what I'm saying is if we follow how factorials work then it would be 16, because; 4! = 4x3x2x1 = 24, 4!! = 4x2 = 8, (4!)! = 6.2x10²³ since a Double Factorial is different from the factorial of a factorial; and if we were to follow the same logic, then 4? = 4^(3^(2^1)) = 262,144, 4?? = 4^2 = 16, (4?)? = 262,144? ≈ ∞ of course, since the ? operation is uniquely defined here you could simply say that the double question mark isn't an operation, but if they work based on factorials then hypothetically, 4?? would equal 16
Your camera is a bit green
By far this is the best video I have seen on this subject! Thank you very much!
Would be nice to learn about the algorithm itself, not just its benchmarks.
I've linked to the paper. It would be very hard to explain because it depends on many other concepts that would also need explaining.
@@Mutual_Information Depends on who your audience is. Reading the comments, it sounds like a lot of your audience already knows programming and/or linear algebra. I would think anyone who would click on this video would know one or both of those already. I'm a programmer / mathematician. It was a nice video, but it honestly feels like clickbait to me to have to watch a 35 minute video and not even get to the algorithm itself. I don't mean that in a negative way but for feedback. You could do a poll to find out how much ppl know. Thanks though, I'll check out the paper.
@@Mutual_Information Here's the other issue: You liked 14 papers, and no URLs. Which is the paper I should actually look at?
@@Mutual_Information Alright well it's a 202 page paper that's beyond my knowledge, and the asymptotic growth stuff you covered is around page 3 of the intro. Shoot, this is the problem with learning advanced topics on YT, the introductory stuff you already know, and the advanced stuff is either beyond your comprehension or not in the video.
@@Mutual_Information Ok I scanned about the first 70 pages of the paper. The strategy is to consider a "sketch", which is a lower-dimensional approximation of the problem. This could be accomplished multiplying both sides by a matrix that will suck up some of the dimensions. The paper considers different ways to do this. One way is to do one SVD (single value decomposition) algorithm. Only problem is I haven't studied SVD yet. Or the thousands of other algorithms this paper references.
Hm. When you think about it, randomized linalg algorithms make sense for a large number of problems. It’s almost shocking that no one implemented something similar decades ago.
Revolutionary paper
Reminds me of what my high school maths teacher said about being able to assess product quality on a production line with high accuracy by only sampling a few percent of the product items.