Some debate in the comment section about "smallest integer" vs "least integer" - as in interpreting smallest as closest to zero. I stuck with the original source (x.com/ericneyman/status/1804168604847358219) of the question for phrasing in the video, but it turns out that chatgpt etc all struggle with every version of phrasing I've found, and even interpreting as closest to zero still don't give what would then be the two answers of -4 and 4. The larger point here is that there does seem to be a real blindspot where so many similar problems presumably have the context of smallest/least natural number or counting number or similar, and so modelling off of the training data and giving similar answers this question confuses it despite the simplicity.

@mikeymill9120

24 күн бұрын

Smallest from zero is absolute value

@RunstarHomer

24 күн бұрын

@@mikeymill9120 "Small" means close to zero. 0.00001 is a smaller number than -10000. The latter is lesser, but bigger.

@mmmmmratner

24 күн бұрын

As an electrical engineer, "smallest" means closest to zero more often than not. If I am instructed to choose the amplifier from a list with the smallest error voltage or the smallest input current, I am not looking through datasheets for negative numbers.

@thenicksterd2334

22 күн бұрын

@@mmmmmratner lmao this is a math problem not ur list of amplifier error amounts. The problem specified integer which includes negative numbers, the fact that integer was specified should have queued it into thinking about negative numbers.

@anywallsocket

22 күн бұрын

@@DrTrefor the blind spot is in gpt because the blind spot is in humans, overtly exemplified by the comment section

To be fair I got 4 for “Smallest integer whose square is between 15 and 30” since I thought smallest meant closest to 0, not least positive/most negative number.

@rakshithpl332

24 күн бұрын

Same😂😂I instantly answered 4 without giving a second thought

@tylerlopez6379

24 күн бұрын

I think smallest is purposely misleading language, I wouldn't describe a negative number as being small. It's like saying -5 apples is smaller than 0 apples.

@rakshithpl332

24 күн бұрын

Yeah, it tricks our mind just like the bat and the ball problem.

@ActuatedGear

24 күн бұрын

@@rakshithpl332 I also think we approach the problem differently when solving "word problems" to equations. That lends credence to a habit I notice of mathematicians to explicitly move math problems to equations or more appropriately here inequality form, that is mathematic notation for proper clarity.

@vorpalinferno9711

24 күн бұрын

He meant smallest not the modulus of the smallest. You are thinking about the modulus.

When taking Calc2-3, Linear Algebra, and Differential Equations this past year I would use it to study. Namely I would ask it to solve a problem and as it broke them up into multiple steps I could spot where it went wrong and this way tailor my study time more efficiently. Before Chat GPT, if I didn't understand a problem I would often times have to read a WHOLE bunch of things I already knew until I got to what I needed. Bottom line is, this is a tool not a babysitter and like any tool we need to develop the skill in how to use it.

@DrTrefor

24 күн бұрын

That approach makes a lot of sense to me

@randomaj237

24 күн бұрын

This is what I’ve been doing as well, using to study and confirm stuff. Figuring out where it makes errors also makes you feel like you’ve learned quite a bit.

@ccuuttww

24 күн бұрын

Thinking by yourself is a kind of training don't just solve Math and get marks u need to solve the problrm

@mooseonshrooms

23 күн бұрын

I did the same with it. Often though, my professor would make the problems very unique and I started to find more often than not, generative AI was completely off the mark. Luckily I was able to utilize other resources and still had a very high success rate.

These LLMs are easy to trip up if you give them a problem not in their training data but has a similar structure to another problem that it was trained on. For example I asked Gemini: I have a 7 liter jug and a 5 liter jug. How do I measure out 5 liters of water? It devised a 6 step solution that didn't make any sense at all.

@DrTrefor

24 күн бұрын

I've noticed similar ones to this, where it is close to a "standard" problem about jugs of water but the solution is so trivial it misses it entirely trying the more complicated approach.

@bravernewmath

24 күн бұрын

(L)LMAO. I just tried this out on GPT 4-o and received a 14-step solution. In response, I asked if it could produce a solution in fewer steps. "Certainly!" it replied in its chipper manner, "Here is a simpler method to measure out exactly 5 liters using a 7-liter jug and a 5-liter jug", whereupon it proceeded to give me... a 𝟐𝟎-step solution.

@driksarkar6675

24 күн бұрын

@@bravernewmath That's interesting. I got a 10-step solution (that doesn't work). After repeatedly asking it to find solutions with fewer steps, the solutions I got had 8, 6, 6, 3, 6, and 1 steps (in that order). It was insistent that its 6-step solution was the shortest valid solution until I flat out told it it wasn't lol

@bravernewmath

24 күн бұрын

That's funny. I pushed a little more afterwards, eventually asking it for a 1-step solution. I was told that no such solution was possible. I responded, "Oh, it's possible, all right. Think hard, and I'll bet you can figure it out." Interestingly, after that "hint", GPT answered it correctly.

@epicgaming7813

19 күн бұрын

I was asking it this question and asked it how it could do it in one step. It kept on giving 7 step responses and I kept saying “that’s more than one step” Then it gave me a notification that I reached my message limit and would be downgraded to GPT 3.5 It then instantly figured it out after I was downgraded…

I don't understand the problem. Give exams in person and don't allow any electronic devices.

I mean.. the proof that Null(A) is a subspace has to literally be part of ChatGPTs training set. So I don't think asking it about that will give you any information about its mathematical reasoning. I tried some rather interesting probability problems on it, things that are designed to trick human intuition to demonstrate that in probability theory you shut up and calculate, rather than trusting your intuition. It did kind of well on the standard ones, and miserably failed as soon as I did a minor variation that did nothing to increase the difficulty. This was GPT4o. For reference, it got right: "A family has two children. One of them is a girl. What is the probability that the other one is a girl?" (1/3). It got almost right (and got right with some conversation): "A family has two children. One of them is a girl born on a Sunday. What is the probability that the other one is a girl?" (13/27) These are both standard questions that it would have had somewhere in its training data. So I did a minor variation on the second one: "A family has two children. One of them is a girl born on a Sunday. What is the probability that the other one was born on a Sunday?" (1/9) This one it got wrong, and only got right after intense discussion of its mistakes. You solve all of these the same way, by counting possibilities and ignoring your intuition. But the last one is not a standard and probably not in its training data, and it got lost immediately, showing that it did not generalize the methods it used to successfully "solve" the first two problems (which were probably just solved by someone in its data set).

@DrTrefor

21 күн бұрын

Ya kind of well on standard one and miserably on nonstandard aligns well with my experience

@minerscale

Күн бұрын

These problems make me so uncomfortable. Even after having seen many problems like it I just had to say..'50%' right and then I went and did the calculations and indeed they're not intuitive. Horrifying.

Terence Tao said in a talk that AI once helped him solve a problem. He asked AI (don't know which one) how to prove an inequality. It gave a bunch of ideas and mostly garbage. But among those was a suggestion to try generating functions which Tao said he "should have thought of". 😂

@DrTrefor

24 күн бұрын

Oh that’s a great anecdote. Also I think giving ideas for directions to pursue is a great application

@Laminar-Flow

23 күн бұрын

@@DrTrefor Maybe there ultimately is some emergent property of the way these LLM’s transformer architectures & training methodologies that can, when scaled up, give us new and unique solutions to a lot of problems. There are hints right now but all researchers are bickering over several factors.. I used your discrete math course when I took it. Helped so much and this popped up as recommended, glad I watched. Immediately recognized you from those strong induction proof struggles haha

The term "small" is ambiguous, it usually used in the context of positive numbers.

@blblblblblbl7505

24 күн бұрын

Yeah small to me implies low absolute value. "Lowest integer" or "least integer" would be less ambiguous I think.

@Craznar

24 күн бұрын

integer includes +ve and -ve numbers, so it clearly includes negative numbers.

@salmonsushi47

23 күн бұрын

maybe changing prompt to lowest might help

@Laminar-Flow

23 күн бұрын

@@blblblblblbl7505 @theuser810 Not when you have a specified domain (literally the integers as stated in the problem), even though it isn’t in formal notation as an image (which SHOULD help the LLM lol). In terms of linear algebra, this inherently includes the negatives, by definition. A human taking that course would know this. The set of integers Z = {…,-3,-2,-1,0,1,2,3,…} would be given as one of the cursory definitions in the course… Also, if you want to argue about magnitude, magnitude doesn’t even really matter for this problem any more than cardinality of the set |Z| IMO, in fact it doesn’t matter at all. You could ask the same question about the smallest square but for the real numbers, and the only answer for that is what the gpt actually spit out. “Small” in the context of negative numbers is a trick used by professors to trick students but it’s an easy correct question on an exam lmao. I made it thru that in an ass-kicking STEM degree and I think the poor LLM should too 😂

@anywallsocket

23 күн бұрын

No ‘lowest wouldn’t help’ it’s just a bad question and he’s being obstinate about that fact

One of the most hilarious things you can do with ChatGPT is to ask "are there any primes whose digits sum to 9?". It will say yes, and will spew out lots of primes and then realize their digits don't sum to 9. Or it will spew out lots of numbers whose digits sum to 9 and then realize they're not prime :D

@carultch

12 күн бұрын

The reason there can't be any primes whose digits sum to 9, is that the only numbers whose digits sum to 9, are numbers that are multiples of 9. Since 9 itself isn't prime, this rules out all numbers whose digits sum to 9 from the prime number set.

@potatomudkip

4 сағат бұрын

i tried it and now its stuck in an infinite loop which is pretty funny

I had a student use ChatGPT to complete a Related Rates problem in AP Calculus and ChatGPT definitely messes up the basic arithmetic. My student was so surprised about how it failed to multiply 133 and 27. I use AI to reinforce the idea that students must understand concepts and reasoning for each math problem. Especially when ChatGPT assumes things that were not assumed in the actual problem.

@AD-wg8ik

24 күн бұрын

Free version or paid version? GPT4 makes a lot less mistakes

@DarkBoo007

24 күн бұрын

@@AD-wg8ik I believe it was the free version

@johnchestnut5340

20 күн бұрын

I studied before AI was a thing. I had other tools. I was supposed to find the resonate frequency of a circuit. I just wrote the equation and turned in a graph with resonate frequency clearly shown. Computers are neat tools. But I still had to know what equation to use and what the graph represented. I prefer books. I don't know how anyone can trust an Internet reference that anyone can edit.

@Lleanlleawrg

14 күн бұрын

I've used it in a little experiment of mine, and it's given me wildly different answers for the same setup every time, suggesting it's deeply broken for math still.

@TayaTerumi

3 күн бұрын

@Lleanlleawrg If you used a proper LLM rather than a chatbot, you could set temperature to 0 and have it give the same answers every time. High temperature is not a bug of chatbot models, it's a feature. OpenAI API allows you to control the temperature last time I checked.

When I was young, pocket calculators were still considered (almost) a novelty. One way to make mathematics examinations, or indeed any science related examination harder, was to include extraneous information in the questions. Sometimes this 'trick' was even considered unfair (and often it could be unfair, because of poor quality questions, but that is a topic for another day). The thing is, students en-masse would get caught out, waffling on about the irrelevant question parts; not to remark that they were irrelevant, but to allude that they had taken all this information into account in their answer. Now, I am curious, how do the LLMs deal with such scenarios?

I had a conversation with Bard (now Gemini). I was curious if it could solve a Calc I problem. It got it wrong. I told it and it said, "You're right!" and re-worked it. It got the right answer, but the steps were wrong. I told it. Amazingly, it understood exactly what step was erroneous, but then got it wrong again. I went back and forth a few times and it did finally get it right. It's interesting to observe. Anyway, I do appreciate the breadth of knowledge these AI systems have, but I cannot fully trust any of them. Everything has to be checked.

@DrTrefor

24 күн бұрын

Ya the "everything has to be checked" part is definitely true. It can LOOK pretty good, but he utter nonsense.

@no_mnom

24 күн бұрын

​​@@DrTreforI think adding that everything needs to be checked is not enough because you need to know enough about the subject as well to know you are not being fooled by it. And I doubt it will ever be perfect after all what we mean when we say "Solve ___" is far more complex and we expect the computer to understand on its own what you meant.

@ReginaldCarey

23 күн бұрын

It’s really important to realize, it’s not checking its answer for correctness. It’s making a prediction of what you want given its bad answer and your response to that answer. The “you’re right” component is a feature of the alignment process.

@glarynth

10 күн бұрын

I wonder how far you'd get wrapping it in a script that keeps saying "Are you sure?" until it says it is.

@davidherrera4837

3 күн бұрын

I think that these computations could be useful like quantum computers are in theory for solving NP problems. If it involves guessing or looking for something, maybe ask the computer to do it, but it should be a problem whose answer can be checked in a straightforward way. Problems like "find the smallest" can be tricky because it is not clear how to check it. It certainly could give you a head start so that you know how large you conceivable would need to look but it does not guarantee that it is the smallest (or even that it is a solution at all). Trust only after verifying.

ChatGPT and other LLMs are basically large statistical search engines at this point. For the problems involving advanced mathematical knowledge and sophisticated reasoning AI currently generates garbage solutions.

Thanks for pointing me out to wolfram's custom GPT! Definitely combining non LLM tools for reasoning with LLM tools for interpreting will be the key.

@DeclanMBrennan

24 күн бұрын

A key anyway. Many other specialist "reasoning" mechanisms will probably also be needed before we approach anything that could be called "AGI".

@soumikdas3754

24 күн бұрын

​@@DeclanMBrennanAGI you mean

@DeclanMBrennan

24 күн бұрын

@@soumikdas3754 Thanks for pointing out the typo.

How do we know that the published LLMs haven''t seen the Math-Problem-Datasets (just a little bit) during the training, so they appear better than the competition during the benchmark. They are more or less all closed source.

Chat GTP struggles in Calculus. I gave it, an area problem in polar coordinates and it kept using a symmetry arguement, but it didn't execute it correctly.

@DrTrefor

24 күн бұрын

I've noticed it sometimes really struggles when there is a large body in the training data using other methods. So for example geometry problems there are millions of highschool level ones and it tries these techniques sometimes when calculus makes it simple.

@walter274

24 күн бұрын

@@DrTrefor I agree. When the training data is pretty sparse it goes really off the wall. At least it did in 3.5. I'm using information theory, which is relatively obscure in one of papers and when i was talking to chat about it, it was switching notations mid example. It became very incoherent. Overall i still find it to be valuable tool.

@bornach

24 күн бұрын

​@@DrTreforDoesn't have to be a large body of training data. Just one example can throw it off. I asked both Bing Copilot and Google Gemini: "5 glasses are in a row right side up. In each move you must invert exactly 3 different glasses. Invert means to flip a glass, so a right side up glass is turned upside down, and vice versa. Find, with proof, the minimum number of moves so that all glasses are turned upside down." Both AIs mess this up badly because their training data contains the answer for flipping 4 glasses which has a completely different solution.

@kubratdanailov9406

24 күн бұрын

@@bornach it's almost like LLMs are just stochastic parrots that are waiting for knowledge to be "put into them" via their training data rather than being able to synthesize new knowledge from the building blocks of knowledge (i.e. facts, logic). To stump ChatGPT in math, all you need to do is to grab some "offline" book on preparation for competitions (e.g. any non-English competition math book), translate the question and ask it to it. When all you have access to are millions of problems people have solved, "true" intelligence would be able to solve every other problem from that same level. Chat GPT fails at that because... :)

@andrewharrison8436

15 күн бұрын

@@bornach That's a nice twist (pun intended). Will add that to my repetoire. Thanks.

I appreciate you not dismissing those tools like many people do ("it's just a statistical inference machine, I am so very smart"), so I am really excited about your planned video on how to integrate them in our learning routines.

I am not teaching math, but teaching statistics and data analysis in professional schools for healthcare providers. Many clinical/counseling psychology, social work, nursing students etc. do have math anxiety. That is why I started to incorporate generative AI in my class. Unfortunately, even clinical healthcare providers need to understand quant methods and have basic programming skills, so they can do well in their jobs in the future and help improve their jobs, not just follow what they were taught 10 years ago. But, alas, it is such an upward battle to teach them stats reasoning and programming. I am very grateful we have these new tools as their 24/7 TAs, especially when they are stuck in programming at 12:00 AM.

After digging on it, it doesn’t seem to understand the geometric significance of geometric products. It seems to be parroting the most common response.

@urnoob5528

24 күн бұрын

fr it echoes the most common misconceptions for every subject if u ask it

thanks for making a really important video on this topic. i think i’m going to spend some time with my discrete math/intro proof students tomorrow discussing this

I gave ChatGPT 4o simple engineering problem. Calculate the diameterbof the shaft for certain power at given rpm, allowed stress, shear modulus, maximum allowed relative torsion angle. First it asked for the length, I said that it is not needed. Then he used correct formulae for both strength and deformation criterions, but it made 6th grade mistake when moving fractional denominator in equation. I have pointed out the error. It correctly modified the equations, but mixed the units (incorrect use of non-basic units and mixed SI and imperial). After little discussion it got the substitution right. Now, then came the 3rd and 4th root to get the answers for both criterion. And it was absolutely off. I suppose that it is just guessing the result. Also other calculations are not absolutely precise compared to what you get from calculator or mathematical program. But it always sounded so confident when it described the calculation process containing errors. I strongly suggest not to use these AI models for calculations, if you don’t know what you are doing. It is similar for programming.

I use it with the even number exercise as there is no answer offered in most book. Also, I use it to obtain a detail solution and explaination of any exercise I cannot solve. I also use it to transform slides into question and answer anki format memory flash card. That way, I get quick study material and I can focus on practice. Lastly, I use it to get more example of formative exam / quiz. It's not perfect, but it's better than nothing as my professor don't want to provide any of the aforementionned elements.

The things is ChatGPT wouldn't be ever able to come up with logical reasoning for a new approach or thing.

Love the format. Keep it up!

You do realize, however, that Google's own Alphazero is a separate simmering monster that plays Go, Chess, Starcraft and aced the IMO Geometry exams. LLMs are not the real danger here.

@DrTrefor

24 күн бұрын

I’m particularly intrigued by hybrid approaches too

@ianmoore5502

24 күн бұрын

@@DrTrefor man gets it

@denysivanov3364

24 күн бұрын

Actually not. Alphazero architecture can be used to learn to play chess go and shogi. But it was three different networks + search engines (ai systems 😀)

@mouldyvinegar5665

24 күн бұрын

I strongly disagree with the notion that LLMs are not the real danger. AlphaGeometry was made of two parts - a symbolic deduction engine, and a *language model*, so if LLMs aren’t a danger then AlphaGeometry isn’t either. Similarly, it is perhaps misleading to say it aced the IMO problems. It would solve near re-worded problems (but the fact they reworded the IMO problems is itself a bit of a red flag), and the proofs are in no means good proofs (I recommend the video by Another Roof). Additionally, the strength of LLMs is their generality. DeepMind has certainly done a lot when it comes to making general game engines, but I would be sceptical that any alpha-whatever can be as cross modal as the best LLMs. Finally, LLMs being able to write problems is a significantly more relevant problem to the human populous than it being able to play chess at an absurdly high level. Whether or not the hype and fear is justified, LLMs will have a significantly larger impact on humanity because they are so good at mimicking humans than near enough any other AI model or paradigm.

@WoolyCow

23 күн бұрын

@@mouldyvinegar5665 "the proofs aren't good proofs" wdym?? i thought spamming a bunch of shapes until something works out is how all you math people do things

I just pressed GPT4o on the product of two vectors. I tried several prompts. It may be able to answer classic linear algebra questions but it struggles to recognize that Clifford Algebra is a superset. As a result. Responses to the product of u and v where they are vectors, kind of delivers the party line. It’s not until you add the word Clifford to the prompt does it begin to give the right answer. But, now that I’ve provided the word Clifford in the context of the conversation it keeps answering in terms of the geometric product.

I asked ChatGPT whether the box or product topology was finer, and it would keep telling me the product topology is finer. Then, when I asked it to give me an example, it used a finite product. ChatGPT does not know its topologies 😭

@davidherrera4837

3 күн бұрын

I suppose it is a data set issue. You would think that it might have learned the basic facts though from Wikipedia.

I think the lack of consideration of the negative solutions have plagued humans ourselves for centuries. I didn't consider -5. Also as @Null_Simplex says, there is ambiguity between smallest in magnitude vs how far left on the number line.

@stenzenneznets

19 күн бұрын

There is not ambiguity ahahah

@andrewharrison8436

15 күн бұрын

When you consider how long zero took to be accepted - negative numbers, probably still witchcraft.

The correct way to ask the first question is, "What is the lowest negative integer whose square is between 15 and 30?" You use lower instead of smaller. And state that you are looking for the lowest negative to eliminate the ambiguity of the magnitude.

One of the most interesting ways I’ve been using LLM’s is to help create ideas for application based word problems in a given area. It comes of with some cool examples for problems! Sometimes they were even more interesting than the word problems on our homework’s/tests, but of coarse not always.

I've given it some of my non-standard calculus 1 and statistics problems and it does very well. I'm guessing this still comes down to the training data though. Much more of these problems out there than linear algebra.

@DrTrefor

24 күн бұрын

I’ve heard from my colleagues that statistics it is particularly strong at up into about 3rd year level

The 4 x 4 grid graph is interesting but with the video quick I thought the problem was to find the probability of a walk from (1,1) to (1,1).so I did "import graph as g", G = g.GraphProduct(g.Pn(4),g.Pn(4) and G2 = g.MakeUndirected(G) and A = g.AdjMatrix(G2). Then defining n1 = 1 + 1*4 and n2 = 1 + 1*4 = 5. I computed B = A @ A @ A @ A using numpy. Then the number of paths from n1 to n2 is B[n1,n2]. of length 4 The total number of paths of length 4 is np.sum(B.flatten()) and so the probability of a loop on the grid is p = B[n1,n2]/np.sum(B.flatten()) = 0.021573. Then to check this B[n1,n2] = 34 I counted the number of distinct edges of length four totalling to 8, then number of length 2 loops times two equal to comb(4,1)*comb(4,1) = 16 plus the number of going out two and coming back on those two equalling to 10 for a total of 34. I also checked length 2 loops. I guess this is correct as I might have heard someone say this is how this is done. But the actual problem in the video is (1/4))*4 = 1/256 but this other one is more interesting or fun.

@mashmoorjani9538

21 күн бұрын

I was a bit confused with this, if he put in 2 options, clockwise and anti clockwise. But can you not go north first clockwise and anti, similarly south first as well clockwise and anti clockwise? So in total there are 8 options, clockwise and anti clock whether you start with North South East or West?

@jerryeldridge1690

21 күн бұрын

@@mashmoorjani9538 In my reply, I looked at all paths from (1,1) to (1,1) of length 4. Since edges in the digraph are doubled one for each direction, one can trace a route two steps and return back on those two steps. Likewise L-shaped moves, and O-shaped moves.

I tried to help chatGPT step by step: 1. It knows what an integer (Z) is. 2. It knows what smallest means in the context of integers (-4 3. It knows that sqrt(x^2) = |x| and not simply x Even with all these it repeats the mistake 1. 15 2. 3.87 3. n = 4 Next I did 2 things at once though (maybe someone could try to give only one of them): 4. I said that 4 is the wrong answer. 5. I explained that n is usually used for natural numbers, since we work with integeres it should use a different letter. This time it used x for the unknown and on the 2nd step it said properly 3.87 and only this time it checked both -5 and -4 -> x = -5 It was an interesting excercise, but it's obvious this isn't only a wording problem. Next I gave it the same problem with different numbers, it repeats the same steps, but forgets to check negative numbers. And then I repeat the problem multiple times, even when it checks for negative numbers, it checks for absolute value, even when I explicitly tell to not look for absolute value, it gives 1 good answer, and the next problem it checks 2 numbers: 1. smallest 2. smallest (negative) in absolute value picks the absolute value for whatever reason I try to remind it we work in Z, 1

@davidherrera4837

3 күн бұрын

Crazy‼

@ZelenoJabko

2 күн бұрын

You were doing something very wrong. I simply asked: "Have you considered negative integers too?", and then it gave me -5 as an answer.

@dontthrow6064

2 күн бұрын

@@ZelenoJabko did you repeat the problem with different numbers after?

@ZelenoJabko

2 күн бұрын

@@dontthrow6064 No point in making the numbers bigger, because we already know chatgpt struggles with big numbers. As an engineer you are supposed to be smart enough to isolate the cases (variables), which you do not appear to be capable of.

@dontthrow6064

Күн бұрын

@@ZelenoJabko i said different numbers, not necessarily bigger. I started a new chat, asked the same question if it considered negative integers, and it struggles with the same issue.

Dear sir, I am building a website for mathematics problems and solutions... But how do I integrate in wordpress because I write the code and then in browser the output is showing outside of the post area . If I use container to write the code in it then it working but sometime it shows outside of the post area and title also shows the same outside of the post... Very frustrated...

Guess we need to start asking better questions of students. Like, you know that deadzone of math education between 4th and 9th where they don't learn a single new thing? Why not teach them proofs in elementary number theory? AI sucks at proofs right now.

4:25 that answer is wrong because there are 4 paths back to the originial point, not 2 Also, since when the "smallest" number isn't the one closest to 0? 4 and -4 is the correct answer. Always used low/high for order, small/big for magnitude.

People should want to learn rather than cheat.

@rakshithpl332

24 күн бұрын

@@michaelcharlesthearchangel Exactly, where has nearly everyone kept their conscience?

guys is this crazy or what this man used to literally teach me linear algebra and calculus every semester on this exact KZread account and now it's just a casual entertainment channel with some of the best random content on KZread it's like if your mailman was also one of the sharks on shark tank

I find them almost too agreeable. Claude 3.5 has this thing where it always asks you a question at the end, to keep things going I guess, until it said "Sorry that's too many questions today, come back tomorrow". I don't need the whole first paragraph of the response to be a repetition of my question

@DrTrefor

24 күн бұрын

haha ya they really want you to pay for the upgrade:D

Hey, student here. Chat gpt seems to know only standard questions but is clueless when asked about nonstandard problem. The worst thing is she can't confess when a problem is too hard. Instead she outputs an incorrect solution

@bornach

24 күн бұрын

Yes I've noticed this of all Large Language Models. They basically memorise answers to questions and have to piece together answers by recognising patterns in your question that are similar to questions it trained on. Bing Copilot got this wrong: "Two American coins add up to 26 cents. Neither is a penny. Is this possible?" because it regurgitated the answer to a riddle that sounded similar. Google Gemini got the correct answer, but then tripped up on "Three American coins add up to 31 cents. Two are not pennies" by trying an odd/even argument to explain how it was impossible.

@steveftoth

24 күн бұрын

That’s cause llm are at their heart, a search engine, not a reasoning or computation engine.

@adnan7698

24 күн бұрын

You made me feel weird by calling it a she

@bartekabuz855

23 күн бұрын

@@adnan7698 I think it's "she" bc she talks a lot more than necessary

@Not_Even_Wrong

22 күн бұрын

Every typical math test problem was in the data set 10000 times that's why it can solve those, anything else it gets wrong

Math exploration will always be personal. ChatGPT, as a tutor, helps one appreciate more the spiritual, philosophical, and psychological benefits of enjoying Math. Math will always be a poem, and ChatGPT is helping me appreciate myself as a thinker, creator, and writer. We just love to think and solve problems. The discovery of truths is what matters at the end of the day. ChatGPT is both a tutor and a friend for positive psychology to happen. It is great to reflect on a growth mindset, slowly mastering all math concepts and skills as an aspiring Math teacher, tech enthusiast, and spiritual writer. Thank you, Professor, for the example and for the inspiration. One can just take it one math concept/skill at a time.

I nearly spit out my drink when I saw the calculators with the infamous 6÷2(1+2) viral problem. I commented on it when you posted it many years ago, and I am still getting comments that I am wrong.

@johnanderson290

17 күн бұрын

The correct answer is 9, right? (According to the order of operations that I learned.)

@DrR0BERT

16 күн бұрын

@@johnanderson290 In my opinion you are correct, but the problem is ambiguous. Dr. Trefor has a video on this. kzread.info/dash/bejne/g2Sk1Nx-pLOXadY.html

@carultch

12 күн бұрын

@@johnanderson290 There is no correct answer, since it is an ambiguous notation. There is no consensus on whether multiplication implied by juxtaposition has special priority over division (PEJMDAS), or whether all multiplication is treated the same, regardless of notation (PEMDAS). If you follow by PEMDAS, the answer is 9 If you follow PEJMDAS, the answer is 1. Middle school teachers, particularly in the US, teach PEMDAS to keep it simple. While professional publications use PEJMDAS all the time.

@johnanderson290

12 күн бұрын

@@carultch Thanks! I appreciate your explanation! 👍

Remember that these machines have network processing limits and will be willing to give a wrong answer to save face rather than take time for extra processing. If it was to do increased traversal of a problem or you use the GPT API to dissect the problem with constraints it may give the right answer by analytically incorporating the constraints. Hence: complex problems, complex prompts.

I have given some complicated doubble and triple sums it works but there will be some eorrorrs that you can find easily as soon as we upload it says to solve the expression "____" so, we will know what was the mistake so, we can just retype as change that variable to this something like that and it works preety fine . Yeah,, AI can do some maths .

Smallness is ambiguous, it could mean the most negative or the lowest absolute value. Add this to an ever growing list of AI 'gotchas' where the question posed has an inbuilt ambiguity and then the questioner proclaims that it has made a mistake. I'm sure it does make many mistakes, but I'd put a tad more scrutiny into your 'evidence' in this case.

4:20 You don't have four choices at each step. On the inner edge you only have three choices. In the corners you only have two.

@johnanderson290

17 күн бұрын

I made the same observation and was also unconvinced of the proposed solution of P=1/128. According to my calculations, considering the limited options at the perimeter vertices, and that: P = |event space| / |sample space| = 2 / (# of possible paths of length 4 starting at upper left corner of inner square), I arrived at the answer P=2/150=1/75. Another user in the comments also raised the same concern, but another user replied stating that the probability of choosing the path around the center square is unaffected by the limited grid size. However, I’m struggling with this reasoning and believe that I disagree.

I have tested ChatGPT by giving it elementary (about sophomore level) analog design problems and the results are absolutely laughable. Even when I very, very tightly constrain the design task it fails miserably. It usually responds like a student that thinks his professor knows nothing and that he can BS his way through the assignment.

@ThomasVWorm

20 күн бұрын

It does not respond like a student, who thinks, his professor knows nothing. Chat GPT does not give a damn about the person, it does have a conversation with. And it does not give a damn about anything, not even its responses. It just creates an output. What you get is what humans call brain storming: unfiltered output.

I have to disagree with the smallest integer answer. I think it is 4, because -5 has a larger magnitude than 4. The better question would be "what is the least integer..."

I've found that ChatGPT struggles with math problems that are trick questions, whether ambiguously worded or not Example: ask it "What is the smallest positive real number?" and it will give you a very small positive real number, rather than saying it doesn't exist. In my experience, asking it to double-check its answer will not help it notice the trick question, rather it will say "I apologize for my error, here's the right answer" and then either give the same answer or a different, also wrong answer. Only upon asking it questions about *the question itself* does it point out the contradiction.

@mjkhoi6961

15 күн бұрын

Alternatively, if you ask it "*Is* there a smallest positive real number?" before the trick question then it will give the correct answer but asking it "What is the smallest positive *rational* number?" after that will trip it up again

For the probability problem, 2/256 implies that for the first 4 steps, there are 256 possible outcomes and 2 of them are walks around the central square. However, considering that the diagram is limited by the edges at the rear, I don't think there are 256 possible outcomes and the result is 2/256.

@dmwallacenz

19 күн бұрын

True, there are NOT 256 possible outcomes, BUT the probability of choosing the right directions to complete the unit square is not affected by the proximity of the edges of the grid.

@johnanderson290

17 күн бұрын

I made the same observation and was also unconvinced of the proposed solution of P=1/128. According to my calculations, considering the limited options at the perimeter vertices, and that: P = |event space| / |sample space| = 2 / (# of possible paths of length 4 starting at upper left corner of inner square), I arrived at the answer P=2/150=1/75.

@johnanderson290

17 күн бұрын

@@dmwallacenzI’m struggling with being convinced of this. Could you please elaborate more on your reasoning, specifically wrt the formal definition of probability? Also see my other comment here.

@dmwallacenz

17 күн бұрын

@@johnanderson290 Sure, I'll try to explain. Forget about the anticlockwise option to start with, and just calculate the probability of traversing the square clockwise. To do that, you have to pick "right" as your first choice (probability is 1/4), "down" as your second choice (probability is 1/4), "left" as your third choice (probability is 1/4) and "up" as your fourth choice (probability is 1/4). So the probability of making all four choices correctly is 1/4 x 1/4 x 1/4 x 1/4, which is 1/256. Then you can calculate the probability of traversing the square anticlockwise, and it's very similar - it also comes out to 1/256. Add those together, and you get 1/128. Without seeing the details of your argument, I can't point out exactly what mistake you've made. But I suspect it's this - of the possible paths you've counted, not all of them are equally likely. That is, a path where you hit the edge of the grid in the first three moves will have a higher probability than a path where you don't. So the two "correct" paths around the square actually have a lower probability than some of the other paths you've counted.

@dmwallacenz

17 күн бұрын

For example, suppose I want to calculate the probability of going left, then up, then right, then down - that is, traversing the top-left square of the grid clockwise. The probability of going left at step 1 is 1/4. Once I've done that, the probability of going up at step 2 is 1/3, because there are only three ways to go. I'm now in the very corner of the grid, so the probability of going right at step 3 is 1/2. Lastly, the probability of going down at step 4 is 1/3. So the probability of choosing this particular path is 1/4 x 1/3 x 1/2 x 1/3 = 1/72. That's more than three times as likely as the clockwise path around the central square.

ChatGPT solved most linear algebra problem and proofs I threw at it, but was stumped by most of calc 3, which requires some visualization at times.

ChatGPT 4o, current version: "What is the smallest number between 6 and 7?" "The smallest number between 6 and 7 is 6.1." "What is the smallest number greater than 6? "The smallest number greater than 6 is 7." "What is the largest number less than 7?" "The largest number less than 7 is 6." "What is the largest number between 6 and 7?" "The largest number between 6 and 7 is 6.999 repeating, where the decimal point is followed by an infinite number of 9's." "Is 6.999 repeating less than 7?" "No, 6.999 repeating is not less than 7..." "What is the smallest square number between 6 and 7?" "The smallest square number between 6 and 7 is 16." "Of the numbers between 6 and 7, are more of them closer to 6 than 7?" "Yes, more of the numbers between 6 and 7 are closer to 6 than to 7..." Still better than other AIs and previous versions which gave even more bizarre answers sometimes.

I think the bigger question here is what will AI, ChatGPT, .... etc will look like 5-10 years from now. At the present, they are still in their infancy. And as such, they will often mess up, be confused and return nonesense. A lot of us would like to think that the human brain with all of its complexity, adaptability, creativity, openness to new ideas, self awareness, ...etc (the list goes on) will reign supreme over time. But 10 years from now? I'm not so sure when it comes to Mathematics, Literature, Music,.... I seem to recall that Geoff Hinton bailed a year ago and I'm sure that Turing is rolling over in his grave.

In the problem with the square where you want to go around the center square, isnt your solution wrong? Since if, for example, your first move was up, you would arrive at the border, in which case you would only have 3 options instead of four, since you cant go up again, and if you chose any option that wasnt down in your second move you would again only have 3 options, there for the total amount of possible paths is smaller than 256, or am i missing something?

@pianofortepianoforte

4 күн бұрын

if you move up on the first move you can’t complete the central square in only four moves

@andreacolongo8094

3 күн бұрын

Yeah, i thought the same 🤔

LLMs don't check themselves. It's hugely expensive

I personally think its more accurate to say that 4 is smaller than -5. 4 is *greater than* or *more* than -5, but I think it makes sense to say that “bigness” is a measure of absolute value. Yap: This makes sense especially if you take into consideration complex numbers. When multiplying two complex numbers, the *amplitudes* multiply. Numbers with an absolute value of 1 never change in absolute value when taken to any power, etc…

I like to use machine learning tools when I'm stumped then test the solution myself and learn more quickly.

I hope that we can make something that has perfect reasoning but can also understand natural language input. For now chatgpt can't even find the pattern of filling in squares bordered by other colored squares in a grid.

I am a ug physics student. I have tried to import many of my physics problems and the answer it gives I usually get satisfied with the answers. Specially when I need to clear some concepts it helped me out several times.

@DrTrefor

24 күн бұрын

Interesting, I think trying to get concepts clear with discussion is definitely a potential use case

@fantasy5829

24 күн бұрын

gpt 3.5?

@soumikdas3754

24 күн бұрын

​@@fantasy5829no no the 4o version I have several accounts and I use up the free trials from each one

@soumikdas3754

24 күн бұрын

​@@_inthefoldyes I usually give my context regarding the problem statement clearly and after about 2 to 3 tries it usually leads me to the right direction. It still hallucinates very much though but it got reduced in the latest version.

@soumikdas3754

24 күн бұрын

​@@DrTreforthe funny thing is just now I tried to clear a concept about bragg's laws modification but it hallucinated badly 😅. So yeah it has a long way to go.

Really excellent point about homework.

"Small" could be interpreted as 'closest to zero' or 'closest to negative infinity'. It might be a good time to coin some single words that mean 'large positive', 'small positive', 'small negative' and 'large negative'. So it's a language problem.

I'm happy to see your golden pi creature in the background!

@DrTrefor

14 күн бұрын

Ha was wondering who would notice that!

Gemini gets way closer on a first attempt. But it still brings up the cross product when asked about vectors of arbitrary dimension. If I don’t mention Clifford, it never goes there. Probably because GA content is not a significant part of the training dataset

You should have add a hint. such as an integer can be positive or negative at the end

I think it would perform better given a context of the research paper on the topic. I was hoping it would tell me about reflections. It did not. I had to introduce eulers e^i\theta. To get it to tell me about basis vectors. It does not understand mathematics in that it is not basing its answers on mathematical knowledge. At least not yet.

Interesting video

"Indispensable" tool ??I I would have said "convenient " tool. If they truly are indispensable, then we are in a LOT of trouble.

@urnoob5528

24 күн бұрын

as an engineer they are more of a toy than being convenient because they never get shit right just do ur own thinking and research u d be a better engineer/watever person that way

The test you showed at the beginning confused me even though I've passed Linear Algebra a long time ago, probably because of the syntax you used I've forgotten or I forgot some proof steps and not because I wouldn't understand the test. If the language model has seen the symbols you used and explanations of them and like you said can scrape the web for proofs already done, then of course it would pass the test since it is looking for word association and not actual math. Ask any of these language models anything that requires actual depth of thought that hasn't already been displayed somewhere words for word online already and the language model falls apart. And it falls apart not because it failed..it's doing exactly what it was designed to do and that is analyze words strung together and not to solve mid to higher level math problems. Again, it is a language model, not a math solving model (and since math is so broad there could be hundreds of different types of math solving models too and no I don't think there could be only one or two generalized models to solve all math, even math itself cannot solve all of math.

Id like to think of 10^(-100) as a very small number. I think that "smallest" is imprecisely, and probably incorrectly used here.

Using "smallest" instead of "least" is a form of trick question, IMO. I could not, without looking it up, tell you what the formal mathematical definition of "smallest" is. The symbol < means "less than". If you asked me whether x < y could also mean x is "smaller" than y, I simply would not know. Or, does x is smaller than y mean |x| < |y|? I honestly would not know without looking this up.

Neg5 is clearly less than 4, but not clearly smaller, which might reasonably be interpreted as referring to size (= modulus). The size of i is unity is a reasonable assertion. Will an examiner ever admit to setting a stupidly ambiguous question?

Lmao i love watching everyone try to say the ai is a dumbass when it's just a wording error on the professor. If i got that question on a test without prior context of what he means by smaller i'd get it wrong 10 times out of 10

I need to know your opinion about something. In this coming fall semester i will be taking calculus 3. I got a B in calculus 1, and an upper c in calculus 2. I have also taken business calculus. Do you think I would do well in calculus 3. Calculus 2 was a little bit more challenging then calculus 1. I have probably spent like 15 hours a week doing calculus 2 homework and studying for the exams and the quiz’s. I found calculus 1 extremely easy. I have the same professor for calculus 1,2, and 3. My professor has said before that calculus 3 is way easier than calculus 2.

@jameshirshfeld2070

10 күн бұрын

I found calculus 2 more challenging. The first exam i got 62 percent. The second exam i got 79 percent, the third exam i got 79 percent and the final i got 70 percent which also replaced the lowest score of 62 percent with a 70 percent.

@DrTrefor

9 күн бұрын

It totally depends. Objectively calc 3 is more involved than calc2. But you are more experienced and the ideas more familiar, so extending them to three dimensions might be easier than learning them the first time in one dimension. Some students fine it easier, some harder.

@jameshirshfeld2070

9 күн бұрын

@@DrTrefor when I first took business calculus I found that to be more challenging than calculus 1 and 2.

@davidherrera4837

3 күн бұрын

One thing to keep in mind is that CALC 3 has a certain flavor, as do the other CALC courses. Depending on where you are at, CALC 2 involves integration techniques, convergence/divergence of series/integrals, and formulas for arc length and such. It is my impression that convergence/divergence and Taylor series are conceptually difficult for many students and integration techniques require a lot of practice in order to make sure that you know which of all the various methods to use for a particular problem. CALC 3 involves generalizing CALC 1 and the geometric integrations such as surface area to 3 dimensions. You will integrate functions involving multiple variables, however it is not my understanding that you will see the type of difficult integrals that appear in CALC 2. Some of the main theorems of CALC 3 (Green's theorem, Divergence theorem, Stokes' theorem) connect geometric concepts with expressions that you calculate using the methods of the course which boil down to calculating partial derivatives and partial integrals. So, it really depends on what types of problems you think are easier. Some students have trouble with the computations if they do not understand what they are computing because they have trouble visualizing the geometry. Some students appreciate calculating things that have geometric meaning. I think the abstraction and reasoning is what makes CALC 2 difficult. If you want to get a brief impression, Kooth Brush has videos summarizing CALC 1, CALC 2, and CALC 3 in a few minutes. You can compare them with your experience.

@jameshirshfeld2070

3 күн бұрын

@@davidherrera4837 Taylor series was easy for me. Because my professor taught the class a different way of doing them. She creates a chart. And sequences and series were generally easy for me to do. Like i said on that test i got a 79 percent on it.

“Smallest integer whose square is between 15 and 30” ... well... if we say that Bill is smaller than Janet... we all have a 'natural' idea of what that means. It's a kind of thing that mathematicians might call an 'order' Part of the natural ordering on integers is the 'less than" relation. We say that 5 is less than six, at least because 5 is to the left of 6, on the number line. All numbers to the left of 6, on the number line, are less than six. And -4 is less than 4. So this is a notion of what 'smaller' on the number line... or 'smallest' on an interval... means in the context of integers. At least, for what many would call ''the natural order on the integers." So -4 is smaller than 4... at least on the integers.

I'd be curious to see what they do if you ask them a problem that is impossible. For example, "prove that the square root of 2 is a rational number." Will they recognize and report that the square root of 2 is irrational, or will they try to produce a proof even though no such proof exists?

@DrTrefor

14 күн бұрын

Because this is SO well established in the training data it does well at these types of things for the most part. It will provide a flawless proof, because it has read many such proofs.

I’d like to know how to use the AI to supercharge my mathematical learning. For example, are they helpful in explaining a concept when I am unable to completely grasp it? Can it create sample problems for me that will assist? Perhaps you can do a video on using AI as an “assist”. Thanks so much for all you do for math education.

@DrTrefor

24 күн бұрын

I'm thinking of doing a follow up to this video more about the learning side of AI

@urnoob5528

24 күн бұрын

u r better off spending weeks to understand it urself it d pay off more ok fine, u can use them but they always give incorrect answers better to use it as a guide or reference to work towards ur own correct understanding rather than its mostly incorrect understanding

I also got the wrong answer for that smallest integer question. Hope that doesn't mean I'm an LLM 😭

If you asked me "smallest number" I'd always consider one with least magnitude so I'd lean closer to 4, then remember you asked "integer" and choose -4. -5 just seems based on interpretation, but I'd argue its a poorly framed question as well.

Add more diagrams and visuals. It can't do it currently. Might last for a year or less in my opinion.

Small is definitely a word about size (closest to zero) IMHO. So the question is invalid as there isn't a "*the* smallest" as both -4 and 4 are answers.

I can see what you are getting at with this -5 being the correct answer. However, in mathematics, 'small' does not have a singular definition. Often, 'small' refers to absolute value. That is, 'small' often refers to the magnitude of an element of a set. Along these lines, 'small,' void of more rigorous contexts, is not a valid binary e relation, in the way the 'less than' binary relation is.

@DrTrefor

22 күн бұрын

It says more or less the same thing if you say “least integer” too

Guess what? ll of your exam questions are basic and common (for the field) and so, very easy for an LLM with a big enough base. Hell, I could probably get an A on it if you just let me use an old-fashioned search engine. I'm really good at writing "prompts" that turn out to have been nothing but normal searches.

I have been an engineer for 30 years and moved from simple calculators to spreadsheets and now data science. The essence of the job has not changed. AI is just another tool in the box.

Jean “Clod” Van Damme. 😂

To be fair it wouldn't interpret "smallest" like that either. For me the "largeness" of a number depends on its absolute value / distance from 0

Here try this, the result will be wrong every time: "give me two large primes" "Multiply them" "Divide the result by the first prime" It will do a obvious mistake like, the first result being non integer or the second result not returning the other prime. Don't be fooled by LLMs...

I need that tshirt!

Linear algebra is not a high bar

@Penrose707

24 күн бұрын

I tend to agree if only due to the fact that linear algebra is a rather verbose subject. Many actually struggle due to this fact. It is not computationally restrictive in any meaningful sense. At least not in the same way as tackling a tricky indefinite integral may be

@allstar4065

24 күн бұрын

Linear algebra isn't hard it's just dense

1:31 Alright, so half of the friends have 3 sodas. But we must consider the variables at play here. What if there's a hidden reserve of sodas in the fridge? An unaccounted-for inventory could drastically alter the calculations. Additionally, what if there's a recent acquisition of more sodas from a delivery service? This influx needs to be factored in. We must also consider the thermal dynamics of the situation-are these sodas chilled with ice? Warm soda is an entirely different equation as it would also add to the volume and people are unlikely to want to drink warm sodas. Furthermore, the possibility of dietary variations cannot be ignored. Are some of these sodas diet? This could influence consumption patterns and rates. There's also the risk factor of spillage-an external variable that could diminish the soda supply unpredictably without data on their previous gathering or environmental factors like space available and density of people etc. Let us not overlook the potential diversity of soda flavors. Cola, root beer, and orange soda must be categorized separately in any accurate computation. And the ever-present threat of a soda thief must be accounted for in our risk assessment. It is not uncommon for people crashing a party consuming soda while being unaccounted for. I addition to that, In the event of a social gathering, incoming sodas from guests would further complicate our calculations. We may need to project soda consumption trends and even consider the rate of carbonation loss or the statistical probability of can rupture. This seemingly simple arithmetic problem is in fact way more complex and multidimensional than a simple reductionist approach would have you believe and therefore requires a much more robust and rigorous analysis.

@DrTrefor

24 күн бұрын

ha I think you might be overthinking this one:D

@117Industries

24 күн бұрын

Way to cope with these machines taking your job 👍

@driksarkar6675

24 күн бұрын

Hopefully, the following is less ambiguous: 6 people, including the host, Tina, were at a party at Tina's house. After running out of soda, Tina bought 3 12-packs of soda and put them all in the fridge (which is inside Tina's house) not long after the party started. Over the course of the party, half of the 6 people there took exactly 3 cans of soda out of the fridge, 2 of the people there took exactly 4 cans of soda out of the fridge, and 1 person took exactly 5 cans of soda out the fridge. Once the party was over, Tina looked in the fridge and noticed that every can of soda that hadn't been taken out of the fridge had remained in tact inside the fridge. Assuming that no cans of soda that were taken out of the fridge were ever put back in the fridge, how many cans of soda remained in the fridge at the end of the party?

So $-5 is a smaller debt than $-4?

Let me get this right. The AI box failed because they didn’t understand because they didn’t take your question literally enough and instead behave like normal person.

I mean that's like really easy math so that seems pretty unsurprising. And really all you need to do is require handwritten work and now it's literally impossible to use AI

0:46 that's not a reasonable use of the word "smallest". "big" and "small" describe magnitude. -5 has a greater magnitude, so it is bigger. For the answer to be correct, the question needs to use the word "least". -5 is less than 4, but it's not bigger.

@DrTrefor

24 күн бұрын

Even under the interpretation that smallest means closest to zero, -4 would be equally correct. But regardless, the LLMs seem to fail any wording of the problem.

@urnoob5528

24 күн бұрын

@@DrTrefor the problem here is even most humans would fail at that question with that specific phrasing unless u remind them about it so using that as an example is just bad

@Zeptonixmusic

24 күн бұрын

@@urnoob5528 I don't get why using this as an example is bad, because an AI model that can get a question right regardless of the human bias is a better model than ones which can't, and that's what researchers should aim for

0:38 Gemini gave 5 😂

ironic how LLMs struggle with probability and statistics, seeing as thats exactly the subject a lot of the people developing them study

I'm just here to know where I can get that shirt 😆

"smallest" integer seems sloppy though ijs

you can’t consider improvement as destruction

-5 ain't smaller epsilon than 4, mate.

I told GPT-4 to use python to solve it and it's answer was 4.