Full podcast episode: kzread.info/dash/bejne/lZidpo9yfbDWkpM.html Lex Fridman podcast channel: kzread.info Guest bio: Andrej Karpathy is a legendary AI researcher, engineer, and educator. He's the former director of AI at Tesla, a founding member of OpenAI, and an educator at Stanford.

@Av-vd3wk

Жыл бұрын

You know, I applaud his genius and all but here’s the problem. The VAST majority of potential problems and .9999 edge cases here are because of the “unknown knowns and, the unknown, unknowns” - HOWEVER, and I know this has been proposed many times before. This problem is an INDUSTRY problem and will only be solved by ALL manufacturers agreeing to an automotive standard whereby, similar to something like Bluetooth, there’s a chipset/mass-made, short-range transponder on every vehicle which all talk to each other to CLEARLY dictate each vehicles intentions.

@TheElectrocar

Жыл бұрын

Will autopilot get better with no radar, sure. But right now its a liability since my 2021 Model 3 was updated to not use radar. I live out in AZ, we don't get much change in season, but AP loves to phantom brake now on perfect roads with minor inclines or declines. Even traffic aware cruise control loves to phatom brake now. It is astonishing how Tesla gets to a point where AP works well and then they repeat the cycle of trying to get back to that state after making yet another change wheather thats because of some hardware change or a code change. We just want the damn thing to work.

@luisdussan9947

Жыл бұрын

The reasoning by Andrej is basically apologetic. The reasoning here as i see it is basically, lets create an autonomous vehicle, which requires sensors, but lets not put the sensors required to do it. And so now lets go and list all the reasons why you wouldn't want to use anything but cameras, no matter how silly they sound to back up our faulty conclusion. If you can't get more important information out of Radar, and ultrasonics, and lidar, then it's because you don't know how to do it. Not because it can't be done. To date NOBODY ELSE outside of Tesla believes camera only is a good way to go.

@luisdussan9947

Жыл бұрын

Since 2016, Tesla has claimed that all its vehicles produced going forward have “all the needed hardware” to become self-driving with future software updates. What's Andrej's position on that?

I drove my Model 3 and Model Y with radar in foggy conditions, and they were able to easily pickup cars and pedestrians before I was able to see them. They were also able to identify cars past the car directly in front of me, which makes a significant difference in braking with heavy traffic. Now with FSD beta, I've lost both capabilities since Tesla disabled the radar. It is a noticeable step backward in capability, and removal of the ultrasonic sensors are going to result in the same.

@miklschmidt

Жыл бұрын

Completely agree. My autopilot now reduces speed (most of the time to like 60% of the speed limit) at least 2 or 3 times on my commute to work. Too much glare on the road? "I'm blind!". Fog? "I'm bliiiiind". Not having ultrasonic sensors in Denmark is a huge mistake, it makes for a worse product.

@CapnSnackbeard

Жыл бұрын

Wait.... you paid for them. They tuened them off. Where is your discount?

@miklschmidt

Жыл бұрын

@@CapnSnackbeard You make an excellent point.

@CapnSnackbeard

Жыл бұрын

@@miklschmidt maybe people effected can announce to Tesla that they will be "disabling the functionality" of their payments a couple months early.

@TinkCSA

Жыл бұрын

Setting aside the FSD, you’ve paid for a car that has radar and USS functionalities. How can making them inaccessible be legal?

At least one of my Tesla cameras are blocked or blinded on every single journey I take. Sun through the trees, rain, fog, mud, snow, condensation...etc. The auto wipers use the camera instead of a simple rain sensor and are terrible in the rain - so bad I constantly have to press the stalk to clear the windscreen. I get what they are doing, but it's just not ready for prime time.

@torchbox

Жыл бұрын

Same issue with my 2022 MYP- had both of my pillar cameras checked out and they said the constant blind or disabled warning is normal and nothing wrong with my cameras. The auto rain wipers is worst of any car I’ve ever owned.

@RetroExhibitCollective

Жыл бұрын

Same everytime “mutiple camera blocked” lol

@astronemir

Жыл бұрын

More features isn’t always better. These sensors could be making the overall model worse

@conchobar

Жыл бұрын

@@astronemir More Features aren't always better, but more sensors is always better. Teslas went from Cameras and ultrasonic sensors, to just cameras. Only a fanboy would attempt to make an argument that cameras alone are better in any way.

@hayden8693

Жыл бұрын

@@torchbox my jeeps auto wipers are much worse

In my car, driving into directly very bright sunlight causes the camera systems to not work, with a warning displayed to the driver - YET the radar system is unaffected. It still works regardless of sun/rain/fog, and can keep pace with traffic, auto-brake, and accurately measure the distance to vehicles when cameras are unable to expose correctly.

@wg4476

Жыл бұрын

yeah, but can radar see traffic light colors and be used to correctly label and recognize objects? If not, then it is practically unusable for FSD

@LightRayTracer

Жыл бұрын

@@wg4476 Yes, radars since their invention have been able to classify objects with very little processing power.

@nolimitsouldier09

Жыл бұрын

@@wg4476 chances are, if the car has active cruise control radar, it also has lane keep assist cameras

@Navhkrin

Жыл бұрын

I worked in 3D Object Detection myself. While I understand their justificiations, improved accuracy gained from LiDAR is way too high to simply ignore. Yes it adds cost, but is is necessary evil. And it will provide extremely valuable real life data to further tune the cars.

@Uncool-vn5vz

Жыл бұрын

@@wg4476 It’s like saying that a person just needs eyesight, s/he can forgo sense of smell, taste, touch to be fully functional & capable.

Whenever someone changes/reframes your question: ( 0:20 ), they are being evasive. "Does it make your perception harder or easier?" was the question. He did not answer the question. Instead, he responded by explaining that there were too many sensors, they were costly and there was a bloat of sensor information. A simple challenge to his response would be: "Is the data you are receiving now, equal or superior to what you received before?". If he cannot give a clear "Yes" then all the cutbacks are to the detriment of safety and consequently highly irresponsible. ( 1:47 ) "The delta was not massive" is what he said when they compared a full sensor suite compared to a reduced sensor suite. By the way he spoke, and by the nature of the sensors removed, I would assume that the vehicle with reduced sensors was on the "harder perception" side of the delta gap. Hence why he evaded the question. Phantom braking skyrocketed after radar was removed and Tesla is still trying to completely solve the issue. They clearly hadn't properly tested the efficacy of the sensor removal and when issues came up, there was nothing to fall back on. You do not cut costs when you do not have an adequate replacement (which Tesla clearly didn't), and you do not cut costs without proper R & D testing. You play it safe when it comes to safety. Why do you play it safe? Because lives are on the line. If Tesla is going to play in the autonomous driving space, they had better have a much more serious attitude when it comes to safety. Either that, or give up on this whole "full self driving" gimmick.

@luisdussan9947

Жыл бұрын

That's a great observation. lol

@md.mohaiminulislam9618

Жыл бұрын

it comes down to the whole fsd meaning, they are trying to mimic human accuracy of a good driver to achieve fsd, not fsd in any conditions to go from one point to another avoiding all obstacles in the way.

@5893MrWilson

Жыл бұрын

@@md.mohaiminulislam9618 also another thing to remember is there will be a mix of human and autonomous cars on the road for a very long time. Does extra sensor capability really matter when you are stuck in traffic with humans who only use vision and their neural net.

@5893MrWilson

Жыл бұрын

"s more serious attitude when it comes to safety" you do realize that Tesla's are the safest cars on the road and that there has not been a single fatality from FSD beta of which there are 100,000 users and it has been out for over a year. You clearly don't understand why radar was a problem. Radar reflects strongly off of metal objects such as manhole covers and steel bridges. In these situations the radar was telling the car to engage emergency braking which can lead to accidents and a unpleasant driving experience. The phantom braking got worse briefly after Tesla removed radar but this was necessary to rebuild the neural net to rely on vision only. This shows how little you understand Tesla's software. You can't just have a perfect vision system and then turn off radar because Tesla needs the cars to interact with the world using vision only to improve the new vision only nueral net. The nueral net can only improve after radar is taken away. They could have done a beta program but then it would have taken a lot longer to get the software back up to where it was with radar.

@Shackkobe

Жыл бұрын

@@5893MrWilson Where do I begin… “ Tesla's are the safest cars on the road and that there has not been a single fatality from FSD beta of which there are 100,000 users and it has been out for over a year. ” What would be your standard for safety sir? Fatalities? Collisions? Near misses? It seems that you have chosen an extremely low bar for ranking Teslas as the safest cars on the road. I wouldn’t put you on my safety team for sure. Please note, every time there is an incident with FSD (and there are many, do your research), Tesla points to their FSD fine print and blames the driver for not paying enough attention. This little loophole allows individuals like you to claim that Teslas with FSD are the safest cars on the road, because technically, whenever anything goes wrong it is ALWAYS the driver’s fault. “ Radar reflects strongly off of metal objects such as manhole covers and steel bridges. In these situations the radar was telling the car to engage emergency braking which can lead to accidents and a unpleasant driving experience ” Interesting, that sounds a lot like phantom breaking… How is it possible then, that removing the cause of the problem (radar) causes the very problem you are trying to prevent to become worse? You’re not making any sense right now my good friend. By the way, Andrej Karpathy never stated this was the reason radar was removed. But I may be wrong… you may understand more about Tesla’s software than even he does. “ They could have done a beta program but then it would have taken a lot longer to get the software back up to where it was with radar. “ Really? You mean the R&D on the software was incomplete and it was released anyway? Are you implying that Tesla used the 100,000 FSD users as guinea pigs in order to get its broken software up to speed on public roads? Wow… Teslas… FSD… safest cars on the road…

Like - “Trying to have a conversation in a night club” it’s virtually impossible, however the deaf do it so easily with sign language..No distractions!

@Tom_H

Жыл бұрын

How does that "conversation" continue in an unlit room? The dismissive attitude around the automotive edge cases helps explain Tesla's recurring problems with sensational FSD Fail headlines.

@ItsBondoBaby

Жыл бұрын

@@Tom_H do you make a habit of driving around in complete darkness?

@Tom_H

Жыл бұрын

@@ItsBondoBaby Hundreds of thousands of truck drivers drive in the darkness daily. They also drive in the rain and in snow day and night. Not so much habit, as part of the job.

@Marcus-up5wk

Жыл бұрын

@@Tom_H Radar/ LiDAR doesn’t work in rain or snow but vision can..

@Tom_H

Жыл бұрын

@@Marcus-up5wk By definition, rain and snow are "impaired visibility" conditions. Dark hotel parking garages also thwart vision only mobility. There are tradeoffs regardless whether you go vision only or add LiDAR, Ultrasonics and/or RADAR. Occam's Razor favors the simple solution, but driving, like life, is often complicated.

I feel that was more an operational answer than an engineering one .. I still feel that depth perception of vision alone is unreliable...

@chickenchopfriedrice

Жыл бұрын

you still dont get it. their decisions are backed by data and experience, not “feel”

@SiddharthKulkarniN

Жыл бұрын

@@chickenchopfriedrice I am not saying they are doing wrong. I can't, I am no expert. But this answer in isolation wasn't convincing. It was focussed towards simplifying production and not making FSD safe.

@chris12456

Жыл бұрын

Do you feel that your own depth perception using only your vision is unreliable?

@SiddharthKulkarniN

Жыл бұрын

@@chris12456 Yeah. Hence humans cause so many accidents. Point of FSD should be better and not equal to humans. Again, I could be wrong and safety inclined.

@chris12456

Жыл бұрын

@@SiddharthKulkarniN I agree that humans are a major cause of accidents. I don’t agree that’s it’s because of issues with our vision. More because of distractions, impairment, not driving to the conditions. You can add all kinds of sensors to a car, but if they don’t add value what’s the point? Tesla have access to the sensors. They used them and worked with them. And then they drop them but upgrade the cameras. Maybe they realised something. Time will tell if they are taking the right approach.

"the sensors are a liability to you cause they cost us a lot of money" great answer...

@tashtant

Жыл бұрын

Thank you!

@UTUBE3JC

Жыл бұрын

Exactly haha

@TheHuxely

Жыл бұрын

I guess the response would be something like “more sensors would have been prohibitively expensive and therefore significantly delay full self driving and therefore lots more people would die due to the necessity of driving themselves around”. This is what it looks like when an issue is complex and either side could argue endlessly. And do, obviously.

@Neurona73

Жыл бұрын

According to Munro they are $114 what a great saving in a 70k car. They just want more profits.

@kevins3646

Жыл бұрын

I don't think he answered it well, what he was saying is that the AI engine is getting more info if u have lidar and vision. It makes decision making difficult, which one do u consider vision or lidar when u have conflicting info.

Thanks to this change, autopilot no longer works in heavy rain, light fog, or snow. I used to exclusively use autopilot in these conditions because I knew the radar could see what I couldn’t and now autopilot just shuts off… way to go guys. You’ll never make full autonomous if you give up in slightly adverse weather. I don’t see why you would limit machines to only the sensors and wavelengths that people can see when you could make them superhuman. Especially when tesla cameras are already far worse than human eyes (they consistently misread speed limit signs).

@jumboshrimps4498

Жыл бұрын

Because you shouldn't be using autopilot in those conditions. It's a liability. Tesla made the right move to remove it.

@thomasweyermann5848

Жыл бұрын

I said “autopilot” for simplicity, but I really meant automatic cruise control (no steering) and you don’t even need to have that turned on. It even applies to emergency braking. Nearly ALL major auto manufacturers provide this safety feature, so it’s not a liability. Even if you’re still in total control, most cars will emergency brake if you screw up. If you’re only relying on visible spectrum cameras, this system can fundamentally only be as good as the best humans in low visibility. It’s not about using tech to be careless, it’s about improving our standards of safety.

@spicemelange1077

Жыл бұрын

@@jumboshrimps4498 FSD... ?

@galanoth17

Жыл бұрын

@@jumboshrimps4498 wasn't tesla supposed to work on driverless cars and trucks, taxi fleet etc?. How the hell are you going to get that if you can't have auto pilot in all weather conditions

@ericp4573

3 ай бұрын

@@jumboshrimps4498these Tesla fanboys are all jokes 😂

Every time Andrej says 'entropy' you have to take a drink

@Av-vd3wk

Жыл бұрын

Digest this: Life is Entropy. Reality is even worse.

@hieroglyph321

Жыл бұрын

What day is it today?

@doitbeforeyoudieful

Жыл бұрын

I guess it depends on entropy

@FredroStarr12

Жыл бұрын

The answer to not having redundant sensing? cause entropy

@lumberjackdreamer6267

Жыл бұрын

That guy learned a new word and he really wants to use it over and over… entropy, entropy, entropy!!

I work on ADAS functions and cutting out RADAR seems insane. I can see going for camera over LiDAR but RADAR is good, cheap and easy and it adds information cameras can't(see through fog, obstacles, etc.). Sure object fusion can be hard, but I'd rather work with a noisy hint that there could be an obstacle, especially when e.g. on a highway, than just not know it at all...

@THEMithrandir09

Жыл бұрын

Also saying vision is enough to drive would only be true if there are 0 accidents with human drivers. There's not, so you just don't know if vision is enough or not. This is a cost cutting move, possibly risking lives.

@connoisseurofcookies2047

Жыл бұрын

I mean if you can achieve human-level or above performance with camera-only input first and then you'd integrate radar tech on top of that afterwards to improve the internal model I'd understand the decision but to completely rule out radar integration seems a bit irrational.

@THEMithrandir09

Жыл бұрын

@@connoisseurofcookies2047 The thing is, radar is just the more logical starting point. It does everything that cameras do, but better, safe for reading street signs and light signals. It isn't affected by weather, smog, dirt, or even other vehicles blocking vision. And computationally it's also way easier to use, because most automotive radars already give you a 3D object model, whereas with cameras getting that out of multiple camerastreams is computationally very expensive, and even then much of it is neural network guess-work, which is subject to adverserial attacks as well as just random pretty much unpredictable failures. They talk about noise and entropy but completely ignore how a camera channel is MUCH noisier than radar by virtue of containing orders of magnitude more information. Basing perception on vision with an option to improve it later is like basing your lunch on making mayonnaise and once you got that you can have it as is or improve it later with other ingredients like fries or meat. Imho, fully autonomous driving is too hard to work in an individual transport model, which by itself is super outdated anyway. Make cars that are safe but cost 100 grand and encourage carsharing and pooling and focus on public transport. If you don't self driving cars will only make traffic worse, because they'll drive around empty a lot.

@steve.k4735

Жыл бұрын

Andrej Karpathy ...He authored and was the primary instructor of the first deep learning course at Stanford, CS 231n: Convolutional Neural Networks for Visual Recognition .. he is literally the man who at a leading world class university run the course on computer vision recognition yet on this set of comments you will have a vast number of people who are SURE that lidar / Radar is better / necessary and will happily tell you why they are right this is the dunning-Kruger effect in action

@THEMithrandir09

Жыл бұрын

@Steve.K47 You have no idea what you are talking about and are just simping to an authority that desperately tries to rationalize a stupid decision by a boss who knows nothing. It doesn't matter who he is if what he's saying is objectively wrong. RADAR just gives you a clean 3D object list on CAN that you can read and use, video you have to do expensive perception first before you can guess about the existence of some objects but finding out their distance is another hard problem to do from camera only. That costs lots of energy and is imprecise. And the reason they do it is because humans do it, which isn't a good reason to begin with.

I just wish that Tesla achieved the same functionality before removing the sensors.

@happyjohn1656

Жыл бұрын

This is why I believe it wasn't a voluntary change 7:57 PM 11/1/2022

@CarlosElPeruacho

Жыл бұрын

I think he explained pretty well the reasoning behind that. Time is finite, and taking time away from Primary systems to maintain redundant support systems basically ensures that your progress with the primary system will suffer. It's been my experience that such things suffer further from inefficiency when the tasks are separated into different groups, and increased complexity further hinders all interdepartmental work and communication. Now consider this, dojo is processing all of the data that comes in from every car, adding more sensors means that the workload for dojo per car is increased, and not in a manner that would be ultimately beneficial to the end goal. If the cameras are at a point now where they can do even 90% of the work on their own, removing that data from the dojo pipeline means accelerated progress on the machine learning side, not just the increase in efficiency on the human side of the equation. I believe ultimately this will be seen as the correct decision.

@Tuck_Frump

Жыл бұрын

@@CarlosElPeruacho it's interesting seeing how a scientific mind and nonscientific mind works.

@diddybopper2052

Жыл бұрын

@@happyjohn1656 what do you think it was then?

@diddybopper2052

Жыл бұрын

@@Tuck_Frump you mean a scientific mind, and a pragmatic mind. Tesla can only survive for so long with the current story of ‘it’s coming soon, but pay thousands for it now anyway’ with all this. Can you imagine Tim Cook saying here’s an iPhone that can project a holographic image from its camera. Then when you buy the iPhone Tim says we’ll it’s coming in a software update in the future. They announced a charging pad a few years ago that was supposed to be able to charge a watch, AirPods and a phone… did they release it? NO, it didn’t work so guess what? They pulled it. They didn’t charge everyone three grand to have something added to their product that isn’t even possible, in the hope that it will come later.

Andrej fast forwards his own speech to save time. 😳😂🤣🤓

@theoneandonlymeshe9174

Жыл бұрын

I had to check my playback speed 😂

@Davoudi

Жыл бұрын

F**** greatful for that!

@cristiankin5420

Жыл бұрын

I had to check my video wasn't playing at 1.5x.

@mubumubu5975

Жыл бұрын

then there's me watching at 2 times speed

@cheikhounadjigal2105

Жыл бұрын

Ben Shapiro 😅😂😂

I have a friend who works for Mercedes. They are extremely excited to see Tesla take a step back so they can take on Tesla. 😂

@fabriceharris3364

Жыл бұрын

For real ? they are really saying that in close doors at mercedes ?

@Nitidus

Жыл бұрын

@@fabriceharris3364 I suppose it was said in a humourous way.

@hayden8693

Жыл бұрын

Did you watch the video? This is not a step back

@jonathangwynne1917

Жыл бұрын

If Mercedes engineers think this is a step back, that's one of the reasons Mercedes is behind.

I have had my Tesla for four years and frequently have my cameras blinded by the sun or sending error messages when it’s dark because they can’t see in the dark. I live in the country and have to reverse up to my charger in the pitch black. My sensors are essential.

@talhaakram

Жыл бұрын

they are essential for on road safety too, two motorcyclists have died already because of Tesla autopilot relying on CV alone.

@testing2517

Жыл бұрын

Buy a light

@SL3DApps

Жыл бұрын

Technology is constantly evolving. The new cameras most likely have basic night vision capabilities and also some form of shaded glass or brightness adjustment to block out light like sun/car brights.

@_PatrickO

Жыл бұрын

​@@talhaakram Motorcyclists opt into turning fender benders into deadly accidents when they choose to ride a motorcycle. No one is at fault but the motorcyclist who chose to give up safety on purpose. They get hit all the time, it simply gets national coverage just because a tesla is involved.

@mikethiem3628

Жыл бұрын

@@talhaakram link?

Isn't redundancy a good thing when it comes to safety?

@spicemelange1077

Жыл бұрын

yes, but $$$

@Iamjustkiddings

Ай бұрын

Imagine gaining 0.002% of safety at $15000 additional cost and additional processing time

Imagine if airplane companies took this approach. "No we got rid of the backup engines as it's one more thing our engineers had to worry about. Gotta simplify the system dude"

@5893MrWilson

Жыл бұрын

Absurdly bad analogy. Tesla's have 8 cameras. 5 of them are forward facing with significant overlap. There is basically a zero percent chance that multiple could fail at once but even then the car can slow down and pull over or come to a stop.

@daijoubu4529

Жыл бұрын

@@5893MrWilson it's just a matter of time before they cut down the amount of cameras 😂

@5893MrWilson

Жыл бұрын

@@daijoubu4529 I mean humans do drive with two fairly low resolution cameras and a biological nueral net but... the number of cameras isn't really that important. Does the car have a 360deg view, is the resolution/frame rate high enough, and do we have the camera angles that the driving computer needs.

@MaltePersike

Жыл бұрын

How well thought out this was can be judged by comparing the auto pilot functionality before and after the change. Before: * 150 km/h max speed regardless of weather and light conditions * wiper function unaffected * high beam function unaffected After * 140 km/h max speed in optimal conditions, often way lower * wipers switch to auto and go crazy even in dry conditions * high beam switches to auto and is way too aggressive So, yeah, no issue. In addition, do you not see the idiocy underlying the whole argument? It's like saying that a human with only one mode of sensory input (say, vision) will perform better in a range of everyday tasks than a human who has all five senses available. The problem lies not in the noise in the second channel. The problem is deficient software engineering that is unable to integrate this second channel in a beneficial way.

@5893MrWilson

Жыл бұрын

​@@MaltePersike FSD beta on vision only drives far better now than any version of FSD with radar. Debating which sensors or which combination of sensors is pointless when good AI software will eventually be able to drive at a super human level with only vision. Humans have 5 senses but we only use vision to drive.

I think he probably forgot to mention entropy

@ericp4573

3 ай бұрын

@@bloedblarreI think he has a tism

Removal of ultrasonic for parking was a silly move, the front cameras can't see the front of the bonnet properly... If there is a low level object it will not detect it.

Yes, my BMW i3 with an optical sensor decision ladder outs from functioning whenever a motorcycle rider cuts in front of my car. My previous Prius which is radar based never exhibited this problem of new small object introduction.

@pitech4446

Жыл бұрын

cuz you have shitty software made by green hill garbage software. don't even compare that with Tesla FSD

@stevechance150

Жыл бұрын

I want a self driving car that can see the deer standing in front of my car, IN THE FOG. Cameras aren't going to provide that.

@marcbungener1877

Жыл бұрын

are you certain the i3 doesn’t cumulate vision and radar ? my old 2008 550i had only radar but I thought after that they went radar+vision ?

I watched Andrej discuss this same topic on a different KZread tesla fanboy video. (I wish I could remember the channel) In that interview he gives a totally different answer saying it isn't about cost but actually about making a better product. Very interesting to see that he doesn't dare try to pull that kind of bullshit with Lex.

@Evghenios79

Жыл бұрын

we know it's an inferior product without the sensors the numbers on the road will tell the true tale in the next 10-20 years

@hundredsac

Жыл бұрын

You were likely misunderstanding because you are not smart lol 😂

@burts6896

Жыл бұрын

Unfortunately Andrej is BS'ing Lex as well; Andrej's arguments are blatantly one sided. He over emphasizes the problems and costs of ultrasonic sensors while overlooking the benefits of sensor fusion, particularly in environments were vision cameras don't perform relatively well.

@coolcat23

Жыл бұрын

He still gave bullshit answers to Lex, though. The parts of the questions he did not evade, he answered vaguely and dishonestly.

What about FOG? Genuine question. Fog around here is a HUGE concern, wouldn't radar massively increase safety in such conditions? EDIT: In my personal opinion the end goal should be to be WAY better than humans. Look up IMC, IFR and CAT III related to aircrafts. Autonomous systems there are immensely better than humans, making things like landing a 300 seats aircraft with ZERO visibility absolutely routine, I'd like to see a similar level of "superhuman" capabilities for cars also, but I understand that may be still 20 years down the line.

@AndrosYang

Жыл бұрын

Their interest is the match humans not to be better than it.

@user2C47

Жыл бұрын

Self-driving systems should ideally refuse to operate when visibility becomes too low.

@wanfredzwick2173

Жыл бұрын

1. Cameras with AI seeing better than humans in fog or darkness 2. If a human can't look far enough, he slows down. Self driving cars can do the same

@CL-gq3no

Жыл бұрын

@@wanfredzwick2173 , to add to your first point, cameras are able to see wavelengths that humans can't see. IR and UV for example. According to Tesla, their cameras are already able to see better in poor weather/lighting than humans so that helps make their vision only approach more feasible.

@Livinghighandwise

Жыл бұрын

The cameras see through fog better than humans do, and if the conditions are bad enough, autopilot will force control back to the driver.

Did he ever even mention what the customer might want? They’ve left the customer without any parking sensors, which are clearly essential to many people. Who knows when Tesla Vision might actually arrive. Insanely bad decision.

@pawsup93

Жыл бұрын

Wtf are u on about, teslas are full of proximity sensors and parking cameras.

@pawsup93

Жыл бұрын

Hes talking about lidar and radars not being necessary to solving autonomous driving so they got rid of them

@DanielBulygin

Жыл бұрын

@@pawsup93 nope. Not anymore. They are dropping them completely and leaving only the cameras

@CL-gq3no

Жыл бұрын

@@DanielBulygin, by the time those new vehicles without ultrasonic sensors get delivered to customers they will probably already have the Tesla vision update released. At least in any significant volume.

@Livinghighandwise

Жыл бұрын

You realize that Tesla's use it's cameras as the parking sensors right?

When the rest of the industry bets on lidars, then the cost would go down dramatically. In fact lidars have gone from $10K down to $1K very quickly. When it costs $100, then suddenly the cost benefit function will become very different.

Im surprised no one is talking about the infrared camera. It works at low resolution, works at night and is compatible with vision models

@parihar786

Жыл бұрын

You are right. These are a AI PR folks who have just consumed too much info without any real insights and now womitting that all around

@mazimadu

Жыл бұрын

@@parihar786 in college we attempted to create a dense point cloud using a monocular camera system. That means create a 3D map using a single camera. The conclusion was to not do that, but use and infrared depth sensor, or 2 cameras since that is the acceptable means of creating point clouds. If they are using a vision model and a camera setup anyway, why not have one low res infrared camera for night time driving, instead of spending billions updating the vision model to recognise things at night?

@theondono

Жыл бұрын

Because for this scenario IR cameras are almost worthless. IR cameras will easily be saturated by reflections, incoming sunlight and are hard to build models for in the street, because hot stuff confuses the hell out of the camera. Redundancy is about picking a sensor that is *strong* in situations where your main sensor is weak. IR cameras suffer *the same* problems as conventional cameras, so that’s useless as a redundancy.

@Danuxsy

Жыл бұрын

wdym it works? It quite clearly doesn't work since people have to intervene quite often even in 2022, many years after Elon promised self driving.

@zanzeit

Жыл бұрын

Also it causes entropy and stomach bloating

Tesla is gambling right now. But they’re not gambling with their money, they’re gambling with yours. They get rid of sensors with the hope that they will work out a vision only solution. Ok, then get the camera system working first, then remove all of the other sensors. That would require them to put their money where their mouth is…. And give them an incentive to finish faster.

I've seen this similar delineation in higher end robot vacuums: Roomba vacuums typically only use camera-based systems and are not very precise or accurate in darkness and bump and bang into things, while even lower end purely lidar robots are very precise as long as objects are in line with their lidar beam. The newest and best robots use a combination of lidar for positional/location accuracy as well as cameras for obstacle recognition and avoidance. Yes optical is probably "sufficient" a majority of the time, but average human vision is also a liability versus radar/lidar and is a big reason for many accidents on the road today. I was just looking at a Tesla since we have so many deer and animals run across the roads, purely optical isn't good enough.

Although I agree additional sensors add bloat and complexity, I want redundant systems in my car. Computer vision isn't perfect and has environmental limitations. Having lidar is useful for refining the CV model and validating the distance calculation are accurate. It's useful from a training perspective and helps you get to a mostly CV approach. Plus that lidar data can be used for virtual driving in simulation. There are use cases you can't do in real life for ethical and moral reasons. Instead you should get traffic reports and build your simulation environment to test edge cases you can't collect with your fleet.

@Shadi2

Жыл бұрын

exactly. it can act as a failsafe to avoid issues of driving into something optical doesnt detect as an obstacle/barrier like a tipped over truck. it doesn't need to be used for navigation or anything else, just barrier/obstacle detection as an emergency.

@shanewalker3273

Жыл бұрын

Yeah, he's just saying that to move as fast as possible and have to balance resources, spending more engineers on the vision problem than a bunch of engineers maintaining stuff you don't need will yield more future value and growth. The earlier self driving cars come the less people will die.

@woolfel

Жыл бұрын

@@shanewalker3273 I get that is how Tesla thinks. In 20+ years working in IT, I've seen this type of thinking first hand countless times. Sometimes first to market is critical to success. Sometimes getting it "right" is more important. Each person get to decide which product they want and how much risk they are comfortable with.

@Pyriold

Жыл бұрын

I always wondered what would happen if lidar would be on most cars. Wouldn't it be pretty hard to distinguish laser beam returns send by my own car from those of all the other cars? I tend to think of lidar as a technology that's not really good for mass use.

@woolfel

Жыл бұрын

@@Pyriold good points. If we're talking about older gen lidar I believe the literature has documented numerous issues. Newer gen lidar systems "seem" to have resolved many of those issues and are in field tests with some car brands. Until we get a thorough set of public results proving it is really solves noise issues, it's should be taken with a huge grain of salt. There's also 4th gen radar that claims to solve many of the phantom issues from things like manhole covers, freeway bridges and other environmental noise.

My 2018 tesla vision based rain sensor is still garbage. It's so annoying to manually trigger the wiper. It is always a topic of conversation in the car that overall degrades our view of quality of the car.

How good is a normal camera vs ultrasonic, lidar and radar in fog or rain environment?

@LAKXx

Жыл бұрын

Way worse, naturally.

A vital parameter at least for European market is the functional safety aspect. Without the sensor system the redundancy , reliability and fallback alternatives are affected. I am curious to hear or read how Tesla manages to fullfill the requirements of European safety standards.

@frankcb11

Жыл бұрын

Give the European what they want and give us the scraps and bs experiments

@seb.

Жыл бұрын

well they likely didn't describe their system as level 3+ to EU authorities... so I guess they don't need any of these redundancy as level 2 ADAS

@chrisd997

Жыл бұрын

@@seb. that is for sure , the problem is when they sell it as level 5 and at the same time playing with words without mentioning the exact level

So… sensors are removed from the front bumper… how does the camera at the top of the windscreen see what’s right in front of the car? Would require additional cameras right? So didn’t that just get more complex?

Sensors boil down to cost. Curious how the delta was calculated to arrive at a minimal advantage to the additional sonar and radar sensors.

@theastuteangler

Жыл бұрын

Failure analysis

@ricknash3055

Жыл бұрын

@@theastuteangler FSD is already L4... ? Lets call it what it is.

@ricknash3055

Жыл бұрын

@Adam Failed engineering is also very costly particularly if it takes human lives. Karpathy never mentions safety as the prime directive with L4. See Boeing 737 Max 8 for failed engineering example. That's concerning Ill take a pass on such arrogant hubris.

@ricknash3055

Жыл бұрын

@Adam Most modern vehicles currently have impact avoidance using sonar sensors. I don't see removing them and disabling that feature until they perfect it with the cameras as life saving. Let's admit that Tesla's actions are not for saving lives but for saving costs.

@luisdussan9947

Жыл бұрын

@@ricknash3055 L1?

respectfully disagree with this - only relying on one type of sensor is a huge risk, because it will be terrible at perceiving certain obstacles that radar/lidar could be doing much better, also adding redundancy improves safety in case a camera fails…

@TheMikernet

Жыл бұрын

Also...optical sensors in bad weather / fog? This sounds like a terrible idea.

@hundredsac

Жыл бұрын

Adding redundancy lol please tell us more mr expert lmao 🤣

@hundredsac

Жыл бұрын

Lol we need more compute so let’s bog things down with redundancies omg lol bro, humans have two eyes. Let’s just add a wider variety of sensors, throw in my LiDAR, little bit a radar. We can definitely get all things inputs to perfectly synchronize with each other in making split second decisions. Did I mention we have enough compute? Lmao 🤣 I’m dead bro

@hundredsac

Жыл бұрын

@@TheMikernet lol 😂

@TheMikernet

Жыл бұрын

​@@hundredsac The beauty of things like lidar are ability to see through rain and fog. The best thing about adaptive cruise control for me is being able to set it during heavy rain and not worry about my follow distance with the car ahead or a stopped car in front appearing out of nowhere in fog. I honestly think Tesla has just given up on supporting anything outside of fair weather conditions, given how much trouble they are having even in fair weather..so, out with the sensors.

It’s seems a mistake to reduce the capabilities without the sensors. Seems they stopped doing it because they “had to worry about it”. Simplifying a system that literally has lives at stake when it could result in some percentage of safety seems off. I’d pay for a 1% increase in my chance of survival let alone for that of my family.

The cameras built for self-driving cars will be perfect eyeballs for The Terminator

BMW i3 optical sensor very nice BUT encased in an unheated windshield space is unusable in the AM do to condensation blocking it’s view. In addition, optical sensor is useless heading into a blinding sunrise or sunset. Previous cat a Toyota Prius used radar and the safe distance following was absolutely trouble free.

Lex: Tesla is removing sensors. Does this make the perception problem harder or easier? Elons minion: entropy. Cost. Cost. Entropy. Complex. System. Cost. Entropy

@helloworld101

Жыл бұрын

Karpathy is a minion? Lol

@Syncopia

Жыл бұрын

It's one side of a manufacturing coin.

Happy he dealt with the crutch that some have for CM level HD maps for autonomy. Reminds me of a meeting I was in where I talked to client team about traffic light and sign system... they said they didn't need it because they knew where lights would be within a CM with their HD maps. I actually laughed and said you're going to maintain a map of every traffic light in the world and adjust every time the wind blows. 😊 They answered, we only need to do what will please the investors. BAM!

I actually think the government should mandate radar based collision avoidance or breaking. It can be a totally separate system to the FSD

The elimination of all the radar sensors and going with full cameras on the tesla reminds me of how my rear view camera is covered in snow for 6 months a year rendering it useless.....

I'm not an expert at all so I could be mistaken, but the scale of data that Tesla is working with and the lack of geofencing probably means that they weigh their cost VS benefits of sensors differently to Waymo. Lidar/radar is maybe helpful in inclement weather if you're using HD maps in a geofenced area but how would you scale it to Tesla speed? They are working with more way data accross a larger area with more edge cases than Waymo and Cruise have to deal with.

@rogerstarkey5390

Жыл бұрын

The problem is not just "scaling" a high dev 3D environment, but as he said. "maintenance" of that system requires huge *and constant* data transfer. . Various methods(?) 1) Store a map of a local area on the vehicle, then if a destination outside that area is set in the vehicle, download a "corridor" to that point. Problems? Data transfer reliability (poor signal? If you're in a data deadspot, you're "lost") . Millions of vehicles constantly uploading large amounts of data (bandwidth) . 2 way transfer(!) Which vehicles update changes, and how often? . Data accuracy (constant update and checks at the "mothership") ..... 2) Store a "countrywide" map in the vehicle. Problems? HUGE storage in the car. Storage needs capacity and compute power. That requires energy. That's a problem. . Plus the same data update/ transfer issues. . On the other hand, the Tesla "problem"? An "edge case" is a "classification" of problem, a general type of situation. Once "the system" knows the general situation and teaches the vehicles to evaluate "on the fly" the vehicle will run through the grade options and choose the correct sequence wherever it recognises that situation. It's the same way humans do it. Learn the rules, Adapt.

@Dggb2345

Жыл бұрын

Do glad you brought this up as my earlier comment regarding BMW i3 optical sensor vs Toyota Prius radar didn’t mention the Toyotas ability to see thru fog vs the BMW’s total lack of ability regarding same.

@zwillx3953

Жыл бұрын

well Lidar certainly sucks in dust, rain, fog, snow, and any refraction/reflection situation lol. Why do you think they chose Phoenix? Full sun and very dry. Lidar's biggest weakness is weather. (Radar the opposite)

@luisdussan9947

Жыл бұрын

it also probably means they should not be doing what they are doing, by your analysis. Which is probably the right conclusion.

Aka, there is so much additional complexity thanks to the sensors, going without and hoping that it’s possible could save money. Or if it doesn’t, It could sink the entire company

And yet, 10 years into this project, they haven't put cameras to left and right of the front bumper. Tesla in FSD constantly struggle with getting onto busy roads, when they come out from between parked cars or any other tight corner.

@CL-gq3no

Жыл бұрын

The forward facing side cameras are exactly in line with the heads of human drivers. How do human drivers pull out onto busy roads without laying head first on the hood of the car?

@Assywalker

Жыл бұрын

@@CL-gq3no That's the point, they don't. They get into dangerous guessing games, just like the current FSD system which often ends up being very cautious and slow.

@CL-gq3no

Жыл бұрын

@@Assywalker, are you even old enough to drive? Your response makes zero sense.

@Assywalker

Жыл бұрын

@@CL-gq3no And that's the moment where I politely leave the conversation. Good day, sir.

@MajorPickleSwag

Жыл бұрын

@@CL-gq3no his response made perfect sense 😂

How long will it take to restore the functionality just using cameras that they are currently taking away in auto pilot?

Vision is necessary, but is it sufficient? What about audio info eg people shouting? What about the sound of a siren in the distance?

Is sole vision really sufficient? Humans also understand relations between events, for example if a football rolls on the road, then a kid might jump in front of the car chasing after it. Or even some more time-spread events. How do you go about modelling these relationships?

@jakecoventry9004

Жыл бұрын

Yes, one of the biggest problems with Tesla systems (and others) is that they don't anticipate. When I am in self driving on the motorway I can see brake lights on the horizon, so I can change the way I drive. The Tesla doesn't anticipate this and drives as if nothing is happening up ahead - only to realise at the last minute. It's the worst adaptive cruise I have ever used.

What I want to know is how do you sense what's close and down low directly in front or in rear of the vehicle without the ultrasonic sensors?

@philliptemple9841

Жыл бұрын

As you approach it, the AI is building a virtual model of the world in front. Ultrasonic sensors can't detect if there is something low in front. The camera will be able to know what's there and how high it is, but beyond a certain distance will no longer be real time eg if you go close, pause long enough, then a cat could run between the object in front and the wheels. Personally I prefer the without ultrasonic model as there is more chance of me scraping a wheel arch than an object suddenly moving into that small blind spot. Of course I'd like to see it working in the real world before my ultrasonic sensors get disabled! Phillip.

@luisdussan9947

Жыл бұрын

i think you mean how will the vehicle detect its surroundings from a dead park initial condition, and that ultrasonics are required for that. Yes you are right. but you do have surround view cameras as well. However one is a passive system (requires ambient light) the other is an active system (brings its own signal to the party). Tesla obviously assumes a well lit, high contrast, environment.

@philliptemple9841

Жыл бұрын

@@luisdussan9947 If the car is towed to a different location then yes it will be 'blind' if up against a wall. I guess if there is a mismatch between its memory and what it sees then you will get an error message. You'll have to do it the old fashioned way. Phillip.

@luisdussan9947

Жыл бұрын

@@philliptemple9841 i understad what you are saying but would you agree that's not the only situation. Any situation where the camera lacks light or contrast would be a situation where it couldn't even make the determination that there is a mismatch.

@philliptemple9841

Жыл бұрын

@@luisdussan9947 It's possible but I've yet to see the distance measure in lux camera only. I'd just be guessing. I use my reversing camera in the dark all the time. Saying things are not possible without empirical evidence doesn't seem smart. Phillip.

How is the cameras system working with a bunch of dirt on lenses?

So basically Tesla wanted to save money by removing the other sensors, got it.

Also, by ditching lidars, you have to train a whole new set of networks for depth estimation from 2D image. That itself is not trivial and requires more resources in research as well as on board of the car, plus it will never be as accurate as the lidar. It just seems like a step back from multiple points of view. The only pro reason is the reduced manufacturing cost.

@Argosh

Жыл бұрын

This. All companies that do self driving cars use them. Sadly Tesla is into self driving death traps...

Not sure... but it sounds like the main motivation behind removing Sensors is...cost. This is not really a good overall strategy for consumers, even if it is good tor the profit margin of the company...

The Tesla vision is blinded by sunsets and Winter low Sun angle driving anywhere in North America that experiences Winter. My sensors are constantly messaging me that they are “blinded” this time of year. Never had that happen with radar.

@fabriceharris3364

Жыл бұрын

Have the same issue at least once a week.

This goes with EVERY sensor with the vehicle, back-up sensors, cameras, etc. Ultrasonic and Radar sensors provide a high cost marking up the vehicle cost hence why it is more meant for luxury vehicles with features compared to a Tesla that is more full of tech.

@MisTomis

Жыл бұрын

So how much is price of tesla dropping because of not using other sensors? Oh yeah, that's right , it ain't going down

@TheGamerShowsYou

Жыл бұрын

@@MisTomis *facepalm* your comment has nothing to do with what I said. Also NO current EV is actually going down, this isn't a Tesla thing kiddo' welcome to the world of high gas prices and tax reductions for electric vehicles.

@Mickitao

Жыл бұрын

@@MisTomis you should leave KZread and work harder to not have to complain about prices like a hobo

@bluetoad2668

Жыл бұрын

@@MisTomis take a course in economics 101 before you comment on Tesla's pricing. Tesla is a for profit public company - it has a fiduciary responsibility to sell it's products at the market clearing price which is a function of demand, supply and production costs.

@CL-gq3no

Жыл бұрын

@@MisTomis, as others have pointed out, Tesla's prices won't come down with costs because of supply and demand (aka capitalism). However, this is a good example of how Tesla is constantly working to decrease the costs of their products. When needed, Tesla will be able to lower prices and still maintain a healthy profit while many of their competitors will struggle to avoid losses or even bankruptcy.

One funny thing about Lex asking if an AI will say they want to rephrase the question, there is a very strong suspicion that a lot of professional diffusion models actually are rephrasing peoples prompts before feeding them to the models. You feed it a quote like "cat" and before actually feeding it into the model, it will convert to "cat highly detailed by artist XYZ" etc which is 10 times longer. The user just thinks the model is amazing, when the next model which is "honest" appears to give worse results.

@Dggb2345

Жыл бұрын

I’m always noticing just how good our brains are at this type of nuance which is trivial for us yet so far as I know impossible for machine learning algorithms.

@testing2517

Жыл бұрын

"reframe" and "rephrase" are two very different things

@varoon5

Жыл бұрын

Sort of but not exactly. They are not changing the prompt they are adding “context” to it, based upon previous activity.

Holy crap I’m glad I got the last model year with radar. Helps me a ton, can’t imagine driving without them

@FlorinArjocu

Жыл бұрын

Unfortunately they will probabaly disable your radar with software updates as the software will be developed only around cameras. It is pure stupidity, you will have no real assistamce at night, driving against the sun, in fog, in heavy rain, in snow, when a bit dirty etc.

@stormdancer1910

Жыл бұрын

The radar sensor get's deactivated with the new update... even if your car has one it does not use it anymore.

@freund_hein

Жыл бұрын

@@stormdancer1910 I think that is very rude. You pay for something that gets disabled with a software update, but would still function. It’s really not acceptable.

I wonder if they can do like "transition" glass for the cameras, similar to people's glasses. Might help with glare.

I guess, delta of seatbelts is very low as well, probably we need to get rid of it. Best part is no part

@Finnspin_unicycles

Жыл бұрын

Pretty much what I thought. I don't know what metrics they used to determine performance, but a few percent could be the difference between running over a cyclist and not doing that..

The cameras are only going to work in a very tight window, unless the cameras gets eyelids. They need to be able to clear water and ice from their vision to see good. Even then they suck at all the things humans suck at like seeing through fog or visual distortions. Maybe they will add other forms back in once they get good with just cameras.

I get the idea of simplifying, but anything can be overdone, even oversimplification. Cameras are great for clear days but if FSD is aiming to be truly autonomous then cameras for clear days and then in heavy rain or obstructing weather then the radar should be an option that turns on by itself. I get that its more tech into the car, but it may be inevitable if truly autonomy is being sought.

Still wondering how the cameras will work in so many different conditions. Rain, sun glare. Ice.

@notthatkindofsam

Жыл бұрын

Here's the thing.. they don't :/

Much respect to him and I have no doubt he's right about the future. But right now I'm afraid to use autosteer on my Model 3 because the auto-high-beams are trash.

@CL-gq3no

Жыл бұрын

Can't you disable auto-high-beams? I have it turned off. What am I missing?

@Dima1415

Жыл бұрын

@@CL-gq3no has to be enabled for auto steer

@Kawboy65

Жыл бұрын

@@Dima1415 you can immediately click auto high beams off when you enable autosteer. It pissed me off until I realized you can still turn them off.

@notthatkindofsam

Жыл бұрын

@@Kawboy65 Yes, but you have to turn them off each time

@neale3

Жыл бұрын

@@CL-gq3no he is saying the car can't even get the auto high beams working correctly, why would he trust auto steer if it can't turn the brightness of lights up or down correctly

Interesting take. A fleet of cars on road is more important than a few cars with all the sensors.

@timbehrens9678

Жыл бұрын

The take rate for the FSD isn't stellar. Most people are pretty reluctant to pay 10k+ for a very questionable privilege to serve as an unpaid test driver liable for any Tesla fuckups. Especially when the product has been promised and not delivered since 5 years.

@FacadeMan

Жыл бұрын

@@timbehrens9678 I totally agree. But does the user of a non FSD car have an option to disable the camera data uploads to Tesla servers? I’m under the assumption that irrespective of wether the car can drive itself, it’s the video feed of the car’s surroundings that Tesla wants. Later they build the world based on this live data and let it’s AI drive in this simulated environment. Please do correct me if I’m misunderstanding something.

@timbehrens9678

Жыл бұрын

@@FacadeMan The are no live feeds being uploaded to Tesla. No cellular network has bandwidth for that. Tesla collects so-called telemetry and short video sequences if something went wrong or was a close call. They do that since 2019. It is obviously useful and important information, but it didn't help them to be first at Level 3.

Imagine that line of reasoning applied to aviation. Navigating in clouds at night would be impossible. But lets not add sensors because "hey, humans don't have radio navigation devices in their heads"

That guy learned a new word and he really wants to use it over and over… entropy, entropy, entropy!!

I heard costs…but I’ve used eyes, radar, laser, nvg’s, FLIR; all are benefits and make detection of targets better. I can’t see through fog, radar can….Tesla can’t see through fog either now. Basically, the technology can’t synthesize all the info from various sensors yet, at least not cheaply…so no sensors other than vis. At least, that’s what I heard…maybe I’m wrong.

@Mickitao

Жыл бұрын

How many times throughout a month you drive with fog?

@sardoniclaugh9646

Жыл бұрын

You can already see. The computer cannot. We need to teach it to see and it's easier to teach AI to see than to teach it to see, hear, read braille and sign language at the same time. I hope you understand the problem. Once the AI has vision we can teach it to use radar and all the other tools as a secondary tool but teaching it all at once makes a difficult problem even more difficult. Seeing is a very complex task, our brains do a very large amount of calculations subconsciously to allow us to map our surroundings instantaneously using only photons that impact our retina. We recognize shapes, complete them and make all sorts of assumptions in a split second when we look ahead at a road. For instance we only have depth perception for the things we focus on, both our eyes have to be pointing at an object for us to "feel" how far or close it is using parallax . Our periphery vision has no depth but our brains assume a small cow is far away and a large human is close for example. This is a complicated process and AI still has not mastered it.

@jorgesmith2000

Жыл бұрын

DMV: if you can’t see the road, then it’s not safe to drive.

@tenzinpassang4812

Жыл бұрын

kzread.infonBVD2iIUUls?feature=share

@bluetoad2668

Жыл бұрын

@@sardoniclaugh9646 that said, an AI driving with exactly the same safety level as a human will NOT be acceptable, the AI must be an order of magnitude safer - that's just how we are when we automate things like this.

So basically its worse but cheaper (especially when producing at scale)... exactly what I thought.

@jaad9848

Жыл бұрын

pretty much

what about more camera's/higher resolution? How to solve camera only in snow/heavy rain?

Very interesting interview!

Glad to see they are finally following George Hotz / Comma AI’s approach to the problem

@PetWanties

Жыл бұрын

I really want to see george and andrej having lunch and discussing AI

@sgttomas

Жыл бұрын

@@PetWanties yes!

@GreenPartyHat

Жыл бұрын

Comma AI and Tesla are the only ones working towards real self driving.

@SpyderRacing

Жыл бұрын

@@GreenPartyHat wut? Chevy Cruise and Google Waymo are already doing real self driving.

@GreenPartyHat

Жыл бұрын

@@SpyderRacing They do premapped driving. Its not self driving. If a Waymo went down a unmapped road it wouldn't know what do.

Here's the quick answer : TO MAKE MORE PROFIT

Oh let’s only use three wheels because it’ll ease the supply chain. I have a Y with ultrasonic sensors but no radar. It sucks compared to my 2018 Leaf, which was better at lane centering and adaptive cruise control, and the top-down camera view was much more useful when navigating tight spaces.

I use the all around view cameras along with the ultrasonic and radar sensors. The sensors can offer additional safety where they can be set up to prevent you from accidentally touching your vehicle against the one in front or back. When parking my vehicle I don't find the sensors and radar distracting. For example if a dog or cat, or a small child runs behind or in front of your vehicle while parking the ultrasonic and radar sensors will pick this up and apply the breaks while sounding a warning a lot faster than a human can react to. When driving an SUV or a truck the cameras and sensors are an excellent safety addition. The blind spots are large with these types of vehicles. Better to visually see around the vehicle with the cameras than guess.

Never in a million years would I have thought that having a camera, radar and ultrasonic together is less effective than just having a camera lol. Something sounds a bit off but I’m no expert in tech so I’ll leave it at that

@CDX7

Жыл бұрын

It’s less effective in aggregate when considering all aspects including the sourcing, manufacturing, installation, and software layer handling of all the sensors. Let’s say car 1 has version 1 of the US sensor and car 2 has version 2 and each retrieves data slightly differently how does that impact the model? Unless it’s 100% needed then it’s just bloat keeping us from the goal. Humans drive with only vision. There’s no evidence that these sensors are needed to drive autonomously.

@gjermundnorumbugge7373

Жыл бұрын

This has actually been clear for a pretty long time if you have been following this matter. It's just like he said. Humans can drive with only vision, there is no reason a computer cannot. The computer will be much better at it. It is very simple in fact. Other car manufacturers are so far behind in so many ways it is just crazy, and this is one of them. No other company is collecting the data, and this is the main problem. Data collection

@zodiacfml

Жыл бұрын

definitely less effective. until now, the cars without USS still don't have park assist and it will not work well in extremely rare cases where USS will easily do. this high risk taken by Tesla engineers, they removed a feature without a reduction car in price or making it a choice to the buyer.

@solarpoweredafricanvegansp178

Жыл бұрын

@@zodiacfml yea I agree

@KriegsterZ

Жыл бұрын

Do you need LiDAR as a human to drive a vehicle effectively? We don’t. It is absolutely possible to teach a machine to do the same with visual only.

The approach applies to Weapons Systems also... 🙂😎👍

@nickduplaga507

Жыл бұрын

Expensive now, cheep soon, but no competition = still expensive.

@Buggabones

Жыл бұрын

@@nickduplaga507 Cheap* Cheep is what birds do

@jt-vj4sl

Жыл бұрын

Just camera guided missiles. Forget about IR, Laser or Radar guided. Just tell Putin tankers not to spill smoke when your missiles inflight

@remasher

Жыл бұрын

It applies to all systems really.

Have you ever been driving in low sun or in rain where there is spray from trucks and other vehicles? There are often times vision is impaired, so sure we all use vision to drive and it works most of the time, but sensors that don't get blinded at the same time is fundamentally a good thing. Supply chain should not be a factor in a system that's supposed to drive me and my family safely (eventually). Towing the company line on this one....

As a model 3 driver, they really need to add more cameras for this to work.

@fabriceharris3364

Жыл бұрын

even with more cameras it won't work. Cameras can't do the job when there is fog or a lot of rain.

I disagree. The USS should be disabled when driving at speed, but switched on when parking. The cameras are no good for seeing low level objects directly in front of the car about 50cm away. Someone uploaded a video showing the line of site of the Tesla camera when parked. It’s doesn’t compare to USS

@meriofrog

Жыл бұрын

Here comes the expert... If you actually listened, its more than just functionality, sensors cost a lot for what they offer, plus they bloat the system.

@tundeuk

Жыл бұрын

@@meriofrog Here comes the Tesla fan boy. If you listen to what tKarpathy said and what I wrote, you would understand that the bloat to the system is caused when in FSD at speed. I recommend disabling USS when in high speed velocity, but to enable it when in parking mode. There’s a clear difference.

@meriofrog

Жыл бұрын

@@tundeuk USSs are not used by FSD, what WAS used by FSD is the big and expensive radar, if you ACTUALLY listened, you would've known he talked about both radar and USSs. USSs were and are used exactly like you said, that's why they are also called parking sensors

@kkulkulkan5472

Жыл бұрын

It is still a crutch. We have been driving without USS for decades. It’s a nice to have option while parking. Half of my USS in my old car don’t even work and I didn’t bother replacing it. Hopefully my kids don’t rely on it to park when they learn to drive.

@tundeuk

Жыл бұрын

@@kkulkulkan5472 “A nice to have” as you say. That’s the point many customers would like to have parking sensors as an assistant which current Tesla hardware would find difficult in edge cases. It all boils down to the field of view when stationary.

Sorry to say but Elon Musk, normally giving the impression that he is a true engineer, has painted himself into a corner when it comes to what sensors that will be needed in order to obtain FSD. The strategy to only use photopic vision (cameras) will eventually prove being a historical mistake. Maybe, but only maybe, it will be possible to make it work in California, in perfect weather conditions, but there are many places in the world where the weather conditions aren't very much like Californias. You will need sensors like Lidar to be able to detect what is happening around the vehicle even in bad weather conditions like pitch black, rain and snow. This means that Tesla sooner or later will lose the momentum they have right now in the FSD battle.

@OzVicBitter

Жыл бұрын

LIDAR is no good in rain and snow. Humans are no good in pitch black unless they have headlights... Just like cameras.

@codelife9184

Жыл бұрын

Camera's are a no brainer for FSD. Humans do it with only eyes, ofc we dont need any other sensors. They render the envirement in vector space making it essentially the same as ladar.

@pdorism

Жыл бұрын

@@codelife9184 our eyes are so much better than cameras tho. We have huge dynamic range. A camera will saturate if there's too much contrast unless it's one of those high-end, expensive cameras. In which case, you might as well fork out for the radar and lidar

@codelife9184

Жыл бұрын

@@pdorism this is not really a debate. It has already been proven. The path is crystal clear.

@pdorism

Жыл бұрын

@@codelife9184 is too

How about depth perception from a monocular camera. What of low light, bright light, obscured image sensor, foggy weather, rain. What if objects of a similar colour to the background? Cameras alone are not going to allow competent lane keep, distance measurement, obstacle detection.

@CL-gq3no

Жыл бұрын

Cameras alone are already doing those things. The thing that is missing is understanding the right thing to do in an environment that is often inconsistent, poorly constructed, and full of humans doing unpredictable things. All of that is an AI problem. No amount of extra sensors is going to solve those problems.

I'd love to grab a beer with these guys and chat

@Av-vd3wk

Жыл бұрын

about?

@remasher

Жыл бұрын

Prob it would be too mentally exhausting.

@demoticshadow2494

Жыл бұрын

@@Av-vd3wk the way Andrej approaches problems there is no telling where the conversation would go and on the spot no specific questions come to mind but his way of thinking gets me thinking and from that good questions just happen. Would be fun to bounce ideas about my feild off him for insight, we deal with some very basic algorithms for presence absences of components in manufacturing but have false rejections from time to time, a fresh set of eyes never hurts

well, humans are not just a pair of eyes either, so reducing everything to a computer vision problem for the sake of simplicity will get you only that far. what about audio or proprioception? they for sure play their part in driving as well

@CL-gq3no

Жыл бұрын

Audio is something they will possibly need to consider at some point for sirens mostly, but that has little to do with the radar, lidar, or ultrasonic sensors being discussed. Arguably, flashing lights would be enough though since the car sees 360 degrees all the time, unlike a person that needs an audible queue to know to look around for the flashing lights. Proprioception is already covered by accelerometers, torque sensors, wheel position encoders, etc.

@heeroyuy298

Жыл бұрын

I think I can drive just fine in video games without proprioception. Audio helps me to hear other cars but if everything is electric not so much.

I dont mind losing radar and ultrasonic - but how about regular sonics - I'd like to think the AutoPilot would get an "input" if the people inside the car start to scream ?

@johnpapiewski7022

Жыл бұрын

Sirens too

My question is how well does only using visual work in the fog, and visibility is only 15 ft or so?

@bluetoad2668

Жыл бұрын

About as well as it does for a human driver. As visibility reduces humans either slow down or become far more likely to have an accident.

@Dggb2345

Жыл бұрын

BMW i3 optical system in fog is useless. Prius radar system is fully functional and frankly far far superior in all driving situations

@user2C47

Жыл бұрын

If visibility is 15", GET OFF THE ROAD!

@whatusernameis5295

Жыл бұрын

@@bluetoad2668 actually it's far better for a camera as they aren't restricted to the visible light spectrum like humans are

@bluetoad2668

Жыл бұрын

@@Dggb2345 so you drive using radar when you can't see the road ahead clearly? Is that what you are saying?

How does entropy translate into a system like self driving?

Ford dropped reverse sensors due to inconvenient malfunctions. If only 1 of the 5 sensors on the bumper broke, then they all fault and not work properly. The camera is much more reliable so long as there isn't a delay with it.

@chillphil967

Жыл бұрын

Didn't hear about that one yet, but either way the sensor doesn't have smarts to turn other things on / off intelligently. That's a bcm / vehicle level responsibility, to respond correctly to the presence of faults. Sensor is simple. Fail safe was probably "one out all out" by mistake, and now they're just doing cost save lol. Also. There are 2 software safety recalls at ford rn for the backup camera system. And one investigation related to materials fogging 😳

@Rob02150

Жыл бұрын

@@chillphil967 you should have seen when we had severe delays with reverse cameras in the first version of MyTouch in the Taurus in very early 2010s. You'd pop in reverse and see a static camera image for about .75 of a second before it became an actual live feed. We actually had a car pass by that didn't even show up until it was mostly past the camera. The issue was well known and still allowed to go through with the promise of software updates fixing it. Which they did, but still didn't mind putting the defect out there.

@ChipMIK

Жыл бұрын

So you have at least 5 cameras vs. 1 radar and you believe the failurerate is less on cameras? BTW. I just have had a ranger here today to replace one of my cameras today...2 weeks without even cruisecontrol du to moisture...I never had a radar fail on me ;-)

@TheFPSPower

Жыл бұрын

That's just a programmer mistake at Ford, the sensors are independent and output their own distance value each, if there is a fault the others shouldn't stop working at all, it should just indicate to the user 1 of them isn't working.

Love this. Very few know how to properly quantify the total cost of problem statement and proposed solutions. The single drive for simplicity is so refreshing 😊

@glenwoodriverresidentsgrou136

Жыл бұрын

It’s also myopic and stupid. More data is better than less data. See post above.

how can I do my phone they have camera.

Their "AI" either only work off of video data, or radar data, they just didn't develop something that references both，and such project to incorporate both sensors has not yet been founded.

@tiro0oO5

Жыл бұрын

It is obviosly a topic in the industry …

I like how Lex was audibly annoyed when Andrej told him he pretty much asked a dumb question, and would re-frame it, never did and then never answered it.

Very interesting point of how dependencies and additions cause entropy and trickle down to make things harder. This will probably be one of the key reasons how Tesla will stay competitive which is a very novel way for car manufacturers. Will definitely be examined in lots of business studies in the future.

@sgttomas

Жыл бұрын

Very few business leaders will comprehend their decisions but, like with chess grand masters, Tesla can help a few more than any one else

@aalvarez2914

Жыл бұрын

@@sgttomas they’re just saving money at the cost of peoples’ lives. It’s obviously a cost-cutting decision he’s rationalizing, that’s all.

@slimconfucius

Жыл бұрын

@@aalvarez2914 at the cost of lives? How about you just drive your car instead of trusting it to drive for you.

@2204JCM

Жыл бұрын

@@aalvarez2914 I seriously doubt that. Tesla is committed to fully autonomous driving APPROVAL. This is at the very top of Elons list of things to accomplish. If removing the sensors was working against this they wouldn’t have done it period.

I think a tesla recently killed a motorcyclist at night on the freeway because the vision system thought the low rear lights meant an object further away, so the car accelerated into the motorcycle. Radar and lidar are probably essential.

@michaelharrison1093

Жыл бұрын

As stated in the video - every sensor has any associated cost. But what was not mentioned that every decision to remove a sensor like LIDAR also has a cost. I guess the cost of killing the occasional motorcycle rider is not that high on the list of cost considerations?

I worked in object/scene detection using computer vision. I own a Tesla, and I think they are making a mistake going vision-only. This is “human” thinking (“it’s good enough for humans”), where as a robot can be built to see better than human. This includes lidar, radar and hyperspectral imagery. For processing today, ok, vision is needed for sure. But …. the future robot (car) will do better than human, a robot will not be constrained to our visual spectrum.

But if you look mercedes radar sensor for sale two hundred bucks, if mercedes have used radar for years whats the deal?

@gjermundnorumbugge7373

Жыл бұрын

Mercedes has no clue what they are doing

@bengsynthmusic

Жыл бұрын

Tesla is at the forefront of AI not Benzie.

@nickduplaga507

Жыл бұрын

@@bengsynthmusic Cost of sensor.

@bengsynthmusic

Жыл бұрын

He just explained that it's not just the cost but that the sensors introduce noise and bulk data that needed to be eliminated.

@nickduplaga507

Жыл бұрын

@@bengsynthmusic One time cost (improving data filtering), and allows driving in a fog, blizzard, or sudden downpour with low visibility, especially at highway speeds with a upcoming accident. When cars are crashing, and causing casualties they will return to more sensors.

Meanwhile Mercedes offers level 3 autonomy. With radar.

@rogerstarkey5390

Жыл бұрын

Let's see how it advances.

@samsam2235

Жыл бұрын

@@rogerstarkey5390 Didn't want to hate on Tesla, btw. Just sayin that radar might not be that bad at all and that there are different approaches.

@slayerhuh404

Жыл бұрын

@@samsam2235 You got any videos of mercedes driving themselves across entire states?

@bluetoad2668

Жыл бұрын

@@slayerhuh404 the level 3 Merc had such a low speed it's basically only useful in a slow moving traffic queue.

Жыл бұрын

not really in any way useful L3 though.

All I've heard is "having fewer components in the system makes it cheaper and faster to manufacture". I am no expert in this field, but relying on a single data entry point like the camera, doesn't sound very reliable to me. You can make the camera the primary means by which the car navigates its surroundings, sure, but then there needs to be some way that the program can verify that what the camera thinks it's seeing, is indeed, accurate. Having secondary and tertiary data entry points into the program like the lidar and sonar sensors I would think, would be the way to make the system more reliable?

As long as the Ames room optical illusions works, sonar/lidar will be required

The point about solely vision is that camera tech we have today is nowhere near close to good as a our eyes in terms of fidelity and dynamic range. That's the issue these other sensors sort of solves. Also, i thought the whole goal was to become better drivers than humans. So having a radar which basically helps you monitor 2 cars in front of you for potential crash is a huge deal

@JohnnosaurusREX

Жыл бұрын

It really doesn't matter. Are blind people allowed to drive? No. If you cannot see, you cannot drive. It's that simple. I live in germany, but I'm sure simmilar laws aply to every other country. The first paragraph regarding speed tells you area only allowed to drive, so that you can stop in the amount of distance you can see. On a narrow road in half that distance. And the first paragraph regarding distance is that your distance must allways allow to safely stop if the car in front of you comes to a sudden stop. Along with the very first paragraph of the german driving laws, to always stay on the cautios side, nearly every single crash out there is because someone disregarded one or more of these basic laws.

@ThomasAlex

Жыл бұрын

@@JohnnosaurusREX did you mean to reply to someone else, because i absolutely do not understand what you are trying to convey here with respect to my comment

Despite all their simplicity, there is still no tesla being sold that can drive on it's own. Drivers are still responsible. one the one had he is saying data, data, data (that's what tesla is good at), and then additional data from sensors is somehow bad?

@bluetoad2668

Жыл бұрын

Yes, it is bad. Subtle differences eh?

@CL-gq3no

Жыл бұрын

"Signal to noise ratio" is a term that applies here. Collecting lots of high quality data is great. Collecting lots of conflicting or low quality data does more harm than good. I'm a data warehouse software engineer so I know this all too well.

@spicemelange1077

Жыл бұрын

@@CL-gq3no What is sufficient in one application may not be in another

It seems to me that we’re asking a lot from limited processing availability in these cars. A thought that I can’t shake is what if a national roadway sensor system could be part of our infrastructure. These embedded sensors would actually be more like repeating transmitter beacons to orientate all capable systems. Computers would have a lot less load if it merely received location data from beacons rather than processing of raw location data obtained from multiple sources.