All credit to comma.ai team for uploading original high quality video on their official kzread.info | comma.ai/shop/products/three ($2199) | comma.ai/jobs | Buy comma.ai/shop (Merch) | Follow the official kzread.info for more videos. Stay up to date by following twitter.com/comma_ai | Support comma.ai at comma.ai/shop. Chapters: 00:00:00 Intro Harald Schäfer 00:00:39 What is a real self driving car 00:02:21 The classical approach to self driving 00:03:54 Classical perception 00:04:41 Classical planning 00:08:02 The flaws of classical approach to self driving 00:09:17 Remembering how to drive 00:10:02 Why is end to end better 00:11:23 The new approach to self driving 00:11:55 Is all starts with data 00:12:45 So much diversity 00:14:12 What is the output? 00:14:44 Localization 00:15:18 GNSS 00:17:08 Sensor fusion 00:19:58 Time to train 00:20:48 Validation 00:21:48 Quote about software engineering 00:22:09 Simulator 00:24:16 Problem: cloning 00:27:10 Solution: train in simulator 00:28:25 Testing Testing Testing 00:29:33 Does model know it's speeds? 00:29:47 Is model accurate? 00:30:05 Does it drive well? 00:31:09 Does model predict swerve 00:32:05 Problem 2: Desire where do you want to go? 00:33:19 End-to-end that works 00:33:57 Technical difficulties 00:34:28 End-to-end lane changes 00:34:40 Technical difficulties 00:36:22 Some questions 00:36:27 Training the model components 00:37:06 End-to-end lane changes 00:37:53 End-to-end lane less 00:38:49 Longitudinal end-to-end 00:39:24 End-to-end forward collision warning 00:40:10 End-to-end stop lines 00:40:48 What is next for comma research? 00:41:32 Q&A 00:42:11 Driving without GPS data 00:43:22 Good data for training 00:44:35 Bad stop sign human driving data 00:45:10 Training out unwanted behavior 00:46:19 End-to-end lateral control 00:47:03 Explaining the model behavior 00:48:37 Larger training models and edge cases 00:49:44 Driver alertness 00:50:49 Community contributions 00:52:15 Evolution of end-to-end approach 00:53:31 Bounding boxes 00:54:40 Adding more complexity 00:55:20 VR sensor fusion 00:56:00 How big does the models have to get 00:57:00 End-to-end not fully end-to-end 00:58:20 Hardware limitations for Level 5 00:59:13 Data visualization of the model 01:00:02 Driving data utilization and benefits for the user 01:00:42 Path prediction on curvature 01:01:43 Pedestrian crossings 01:02:15 Lane changes and blind spot monitoring 01:02:47 Simulator scaling to more complex scenarios 01:03:37 Self-supervised learning 01:04:07 Hand labeling and comma-pencil usecase 01:05:19 Model mistakes and failure prediction 01:06:09 Artificial general intelligence 01:06:49 Failures of localizer 01:08:08 Model reliability and scaling to city driving 01:08:56 Differences between drivers in different cities 01:09:18 Humans teaching the car how to drive 01:10:00 End-to-end lateral planning model cheating 01:10:46 Training process and incremental learning 01:11:58 George Hotz More GPUs 01:12:02 Biggest limiting factor of improvement 01:13:11 Visual geometry lane lines bug how it was fixed 01:13:49 Harald's birthday

@scoobyphd

2 жыл бұрын

Damn. I live in San Diego and would have loved to cover this.

You thought the 30 under 30 list was rough but Harald is leading a successful open-source self-driving project and he’s barely 2 years old! My nephews could barely eat without putting most of their purée in their hair at that age.

@MrEmbrance

2 жыл бұрын

What are you talking about ?

Look at this. Free 3 years of googling and countless Jupyter notebooks on cool machine learning ideas all in one video. Imagine googling this and this guy just puts everything in one video. Nice.

This is just some another level insight for people interested in self driving and best part is it's free. Now we're in 2021;) and thanks a lot Comma !

I like how the audience pretty much ignored the whole talk for the Q&A

Belated Happy Birthday Harald. Cool talk btw, thanks.

there we goo boys

My take home from this is that the AI trained with beer goggles on

Would be interesting to know how far a super human driving agent could navigate blind from one snapshot of it current position, velocity, location, inferred object physics/behaviour, conditions and other inputs using its "imagination" before it fails, to see what margins exist in various scenarios and how much more anticipatory such systems can be compared to consensus human driving models which are bereft of these electronic data points.

45:52 Could the model be fine tuned to the way a particular driver drives?

Do they use one model for all countries? Or different for snow/tropical or left/right laned countries? If they use one that's really cool.

@fitybux4664

Жыл бұрын

Given there are limited resources aboard the comma devices, I think the efficientnet should be fine tuned based on the country you're in. (And it should be further fine tuned based on the driver.) It will be tricky to know how many layers to retrain to gain some additional accuracy.

This is like what the Tesla team is doing, but it seems to be able to do less things. I'm grateful to see that others are trying hard to make this a publicly available solution though. Nobody should privately possess this kind of technology to sell it. Great talk overall. I think it shows just how complex this open problem is.

@andrasbiro3007

2 жыл бұрын

As a Tesla investor, I'm perfectly fine with them having a monopoly.

@m4d_al3x

2 жыл бұрын

@@andrasbiro3007 same lol

When he means the "classical approach" he means the solution Tesla uses?

@kamalmuradov6731

2 жыл бұрын

No, they use essentially the same approach

@andrasbiro3007

2 жыл бұрын

​@@kamalmuradov6731 Not the same. This is a single neural net that takes raw sensor input and spits out driving commands. Tesla's system is far more complex, it has many stages and modules, some are even hand coded. There are similarities too, that differentiate these to from the rest, like sourcing data from customers and not using lidar. The ends state of the two systems will likely be the same, but they are taking different approaches. Both has it's advantages and disadvantages.

Very intelligent for a 2 year old.

@moracabanas

Жыл бұрын

Some people pointing that and I am not getting it

@asan1091

Жыл бұрын

@@moracabanas Sorry I forgot the context of my comment, and the video is too long to watch it again, my only guess is that I'm talking about a 2 year old AI maybe Idk

6:09

This was hard to watch. These questions were rough.

Is he German?

@coop4476

2 жыл бұрын

That last name looks swiss to me

@sheetboeh

2 жыл бұрын

Last name is very much German, but i think hes dutch actually.

@grekiki

2 жыл бұрын

@@sheetboeh .be, but close

Comma has Herald the German. They will win. He is worth 20x Tesla engineers.

@andrasbiro3007

2 жыл бұрын

Not even close. Tesla has some of the top AI experts. Andrej Karpathy for example. And Tesla is far ahead and has far more resources, so they will win, that's a given. Even George Hotz says Tesla already won this game.

@moracabanas

Жыл бұрын

​@@andrasbiro3007 Well that's fair but IMHO being able to get more than 50% profit researching this "for fun" and open sourcing it for people, and also giving us the opportunity to buy comma 3 hardware for under $2k with weekly updates and pretty reliable level 2 system in pre 1.0. is more than insane. I mean Hotz is just now taking some time off comma making livestreams on Twitch porting tinygrad to every possible SoC which could be candidate for running it. They coded tinygrad now used on comma and it outperforms the native Qualcomm ML framework. The last day I've seen him porting tensorflow and tflite models to tinigrad for Google coral in a few hours what the fuck.

the questions are terrible