DensePose - 3D Machine Vision

Can machine vision map humans from videos to 3D Models? Yes! DensePose is a new architecture by the team at Facebook AI research that does just that. It uses a convolutional network with some special features like region of interest pooling and cascading to make this happen. It was also trained on a newly created labeled dataset that mapped human poses to 3D models. The team open sourced the dataset but not the code, but using the details in the paper we can recreate their results. I'll explain how it works in this video.
Code for this video:
github.com/llSourcell/3D_Pose...
Please Subscribe! And like. And comment. That's what keeps me going.
Want more education? Connect with me here:
Twitter: / sirajraval
Facebook: / sirajology
instagram: / sirajraval
More learning resources:
arxiv.org/abs/1802.00434
densepose.org/
• Realtime Multi-Person ...
• Dong Zhang- Human Pose...
• Human Pose Estimation ...
• Video
Join us in the Wizards Slack channel:
wizards.herokuapp.com/
And please support me on Patreon:
www.patreon.com/user?u=3191693
Signup for my newsletter for exciting updates in the field of AI:
goo.gl/FZzJ5w
Hit the Join button above to sign up to become a member of my channel for access to exclusive content! Join my AI community: chatgptschool.io/ Sign up for my AI Sports betting Bot, WagerGPT! (500 spots available):
www.wagergpt.co

Пікірлер: 159

  • @jasonwee7580
    @jasonwee75806 жыл бұрын

    I’m grateful to know that mega-corporations like Facebook would be willing to share their dataset which required thousands of hours of annotation work. Thanks Facebook and Siraj!

  • @olfmombach260

    @olfmombach260

    6 жыл бұрын

    Though you don't know how much Facebook keeps to them

  • @SirajRaval

    @SirajRaval

    6 жыл бұрын

    thanks!

  • @AnArchaicSoul

    @AnArchaicSoul

    4 жыл бұрын

    @@gkcs4636 Precisely!

  • @saeedarabi5542
    @saeedarabi55426 жыл бұрын

    Perfect man, PERFECT. Keep on awesome work.

  • @SirajFlorida
    @SirajFlorida6 жыл бұрын

    Wow, that's awesome.

  • @rabiielbeji8541
    @rabiielbeji85416 жыл бұрын

    I thought that we need always stereo-vision for better 3D reconstruction, But Densepose will change my point of view, thank you Siraj

  • @satibel

    @satibel

    6 жыл бұрын

    if you see a picture, you can represent in your head how things are, so an AI can do the same. Of course it could be a lot better with 2 cameras, but 1 is enough.

  • @MrDanielphillis

    @MrDanielphillis

    6 жыл бұрын

    i think approximations must be made if you only have one view...but with video containing a moving camera, you have many viewpoints to improve 3D geometry

  • @ED-TwoZeroNine
    @ED-TwoZeroNine6 жыл бұрын

    Thanks for sharing, Sirajology :)

  • @inlustrolearningprivatelim4868
    @inlustrolearningprivatelim48686 жыл бұрын

    Awesome as always Siraj☺️👍You're an inspiration

  • @leowiniecki9264
    @leowiniecki92646 жыл бұрын

    Thank you so much for this video! 3D reconstruction tech is moving so fast. That projective geometry tho... :)

  • @rahulvanmali1406
    @rahulvanmali14066 жыл бұрын

    Buffing up! Siraj

  • @edeneden97
    @edeneden976 жыл бұрын

    Great content lately! Keep it up

  • @michaelyadchuk4989
    @michaelyadchuk49896 жыл бұрын

    OMG!!!! Using this technique, I can turn New York City into a knuckles army!!! #douknodawei

  • @AhmadM-on-Google
    @AhmadM-on-Google6 жыл бұрын

    woah this one is really interesting thanks for explaining, i gotta present this paper :p

  • @ccerrato147
    @ccerrato1476 жыл бұрын

    Damn! This is IMPRESSIVE!!!

  • @AmericanOtter
    @AmericanOtter6 жыл бұрын

    This is fascinating... And terrifying

  • @normanheckscher
    @normanheckscher6 жыл бұрын

    Love ya work Siraj.

  • @SpiridonovRU
    @SpiridonovRU6 жыл бұрын

    Greetings from Russia! Very interesting

  • @AntonioKowatsch
    @AntonioKowatsch5 жыл бұрын

    Well, it's been 9 months and they still haven't published the source code.

  • @alialtaf3412
    @alialtaf34126 жыл бұрын

    This is so good for motion capture. Imagine Indie developers benefiting from this technology and then creating quality games without investing millions to produce game.. So indie developers can also create AAA quality games with the help of this tech.

  • @RAJATTHEPAGAL
    @RAJATTHEPAGAL6 жыл бұрын

    Read a few days back. Awesome stuff o.O ....

  • @larryteslaspacexboringlawr739
    @larryteslaspacexboringlawr7396 жыл бұрын

    thank you for machine vision video

  • 6 жыл бұрын

    Wow, nice!

  • @gorannovaks
    @gorannovaks6 жыл бұрын

    Hey there, Siraj, so when it becomes possible to capture motion from image sequence / video and transfer it into digital skeleton with all constraints, morphing, weight painting for right deformation?

  • @ashfaqmulla874
    @ashfaqmulla8746 жыл бұрын

    Hey Siraj, your work is great. please make a video, which explains study plan, to study computer vision from scratch. I don't have any prior knowledge about it.

  • @nassav3
    @nassav36 жыл бұрын

    Hello Siraj! Nice, vid!

  • @aj-tt9qh
    @aj-tt9qh3 жыл бұрын

    Great video

  • @MajidAli-xu5lk
    @MajidAli-xu5lk6 жыл бұрын

    thanks a lot, the world going so fast

  • @masoudmasoumimoghaddam3832
    @masoudmasoumimoghaddam38326 жыл бұрын

    Siraj, You are perfect. This video sounds like transfer learning. Would you make another video explaining transfer learning.

  • @radhikajain5098
    @radhikajain50986 жыл бұрын

    Awesome man !!!

  • @user-ih1dx6wn9c
    @user-ih1dx6wn9c4 жыл бұрын

    is someone know how to use it for classification of movements like standing or sitting?

  • @KnThSelf2ThSelfBTrue
    @KnThSelf2ThSelfBTrue6 жыл бұрын

    Is this the real Siraj?

  • @dranzerashi
    @dranzerashi6 жыл бұрын

    Can we use similar 3d annotations for detecting say vehicles from images as 3d objects in place of LIDAR

  • @drawwithpeter6950
    @drawwithpeter69505 жыл бұрын

    I may have missed this, but does this work in real-time from live video footage?

  • @anuragmishra1315
    @anuragmishra13156 жыл бұрын

    Kewl!

  • @yakupgorur
    @yakupgorur5 жыл бұрын

    just awesomeee

  • @nimishverma7892
    @nimishverma78926 жыл бұрын

    Hey Siraj, Kinda offtopic in regard to this video. Can you do a video on how the Snapchat's CrowdSurf feature work?(It lets you watch a concert with different perspective, and the audio remains ) I have been trying to study how it works, like the root technical part. TIA

  • @unboxwithaakash
    @unboxwithaakash6 жыл бұрын

    Who to follow for computer vision in twitter? Thanks for video 😄

  • @atrumluminarium
    @atrumluminarium6 жыл бұрын

    Siraj, will you ever be taking a look at the Earth Mover's distance (I think the technical term is the Wasserstein distance/metric) and Barycenters applied to ML?

  • @mmkwazir
    @mmkwazir6 жыл бұрын

    Great video. Would suggest changing name to reflect the true content.

  • @maitreeable
    @maitreeable5 жыл бұрын

    How can you make 3d character from 2d video at the first of this clip video. Thanks

  • @w.maximilliandejohnsonbour725
    @w.maximilliandejohnsonbour7256 жыл бұрын

    Interesting...!!!!!.

  • @gshan994
    @gshan9946 жыл бұрын

    This was great. Do u have anytime worked with raspberry pi and computer vision?

  • @MrChatmoon
    @MrChatmoon4 жыл бұрын

    excellent. Thx you for sharing. I am designing, making and prototyping a kinetic sculpture of flock flying birds. I would love to find any material that can help to transform a 2d flying birg (video) into a 3d skeleton/wireframe model. Any suggestion?

  • @vencendoonline
    @vencendoonline5 жыл бұрын

    Do you think that Raspberry Pi 4 has computational power enough to detect bodies with DensePose?

  • @ZerofeverOfficial
    @ZerofeverOfficial5 жыл бұрын

    I wonder if remembering just the vertexs of things in a video instead of the whole video would be good for video compression, just rebuild the video on the end of the line (the client) i.e. (insert some algorithm remembering pertinent colors, peoples clothing and such, but not buildings, grass, trees, concrete, colors an AI would know)

  • @jayantgoel5567
    @jayantgoel55676 жыл бұрын

    How do u manage to learn so much...plz make a vedio about you..

  • @hammadshaikhha

    @hammadshaikhha

    6 жыл бұрын

    I think he has some videos about how he learns and reads research papers already.

  • @Diggy77

    @Diggy77

    6 жыл бұрын

    He`s an AI Robot :))

  • @abunickabhi
    @abunickabhi6 жыл бұрын

    Yay

  • @xuzhenlei5949
    @xuzhenlei59496 жыл бұрын

    nice!

  • @serhattural3168
    @serhattural31686 жыл бұрын

    Nice share. But Machine Vision title is irrelevant

  • @hassanmahmood7284
    @hassanmahmood72845 жыл бұрын

    very nice . You put good efforts. I am amazed how you do this. So consistently so Good....... may God (the One who creates everything in universes, multiverses whatever where ever its, the One who is not bounded by time and space, the One who is all powerful and merciful) bless you.

  • @harshitaarora6319
    @harshitaarora63196 жыл бұрын

    Wow, this seems like an awesome idea. One application of DensePose that comes to mind is projecting dresses, shoes or pretty much anything (converted from their 2D image, I guess) on people so they don't have to order something and try it out and return it because of size issues, among others. Could improve online shopping experience a LOT.

  • @harshasrivatsa9070

    @harshasrivatsa9070

    6 жыл бұрын

    I think Amazon Echo Look has the application feature of taking pictures of a person and giving recommendations of dresses and how the person will look in it.

  • @michaelbaumgarn4733

    @michaelbaumgarn4733

    6 жыл бұрын

    Your story makes me so sad. I saw the post on hackernews and after a quick google search I found this collection of all the posts: knowyourmeme.com/memes/events/harshita-aroras-crypto-price-tracker-app-controversy#comments_form On the one side there is a lot of people apparently harassing you, which is not ok, but on the other hand there is also people blindly defending you despite the screenshots indicating that you did indeed lie. I found this thread to be the most unbiased about your case: www.reddit.com/r/programming/comments/7vggpq/16_year_old_app_developer_plagiarist_or_prodigy/ I would wish people could have an open investigation about this. I would be fine if it turns out you didnt lie, but so far it looks more like that you did do lie. This will have no effect on my life and on the world in general - we all will have forgotten about this in about 3 months. But I just want you to know that people like you make me sad and I think your contribution to the world is negative. Overa

  • @videossoma
    @videossoma6 жыл бұрын

    I am working on NLP. I need to build an app that reads bunch of documents and answers questions related to the documents. please help how to do this. Please make a video on it

  • @sniperdaoud
    @sniperdaoud6 жыл бұрын

    you are the best ;)

  • @crosstuck
    @crosstuck6 жыл бұрын

    Thanks for being awesomeness #Tyler #TeamTyler @TeamTyl3r

  • @abdelhaksaouli8802
    @abdelhaksaouli880211 ай бұрын

    funny thing, i'm working on torso detection project, i was like pose estimation state of art method give only 4 point, why i don't build something that give the whole front torso, so i started to look for data to annotate it and apply keypoint regression. and i found densepose x')

  • @OwenIverson
    @OwenIverson6 жыл бұрын

    Systems of oppression never had it so good! This tech and the policy that will follow needs to be a major talking point with all politicians.

  • @AviPars
    @AviPars6 жыл бұрын

    Epic

  • @bruhaspati560
    @bruhaspati5606 жыл бұрын

    It suraj!!

  • @olfmombach260
    @olfmombach2606 жыл бұрын

    I know some of these words

  • @myspacetimesaucegoog5632
    @myspacetimesaucegoog56326 жыл бұрын

    For speeding 3d animation this sounds like a dream come true. But then is animation itself going to rapidly lose it's charm? The power of this for Big Brother surveillance is also a double edged sword for human freedom. The importance of democratising this type of tech is immense and Siraj is doing a superb job in this regard. Phew!

  • @evanperrygiblin
    @evanperrygiblin6 жыл бұрын

    Ive been using Winmorph recently to morph faces, I know if I just wait a couple of years there will be an app that will do the whole thing for me... and my avatar

  • @radhikajain5098
    @radhikajain50986 жыл бұрын

    Can you also have some videos on openvx ?

  • @6lack5ushi
    @6lack5ushi6 жыл бұрын

    This is getting crazy! how similar is densepose's ROI align method to YOLO?

  • @danieli4055
    @danieli40556 жыл бұрын

    dem moves doh

  • @oscarchampion5842
    @oscarchampion58426 жыл бұрын

    Deep fakes 2.0 is coming

  • @KanalMcLP

    @KanalMcLP

    6 жыл бұрын

    Oscar Champion current version is actually 2.1, so maybe 5.0

  • @oscarchampion5842

    @oscarchampion5842

    6 жыл бұрын

    It’s just a matter of time till they add body swapping and we can swap Nicolas Cages entire body

  • @satibel

    @satibel

    6 жыл бұрын

    or you can animate his body however you want, pornstars will need a new job XD.

  • @skotch_izolentovich

    @skotch_izolentovich

    5 жыл бұрын

    Porn of future is coming!

  • @rageshantony2182

    @rageshantony2182

    5 жыл бұрын

    Sorry . I can't understand ..

  • @thegamechanger7157
    @thegamechanger71572 жыл бұрын

    OK the pose estimation will come. But it requires some technical systems

  • @leander8986
    @leander89866 жыл бұрын

    If you were completely new to machine learning and wanted to get started, how would you proceed?

  • @satibel

    @satibel

    6 жыл бұрын

    I'd say get a problem that can be solved using ML, and solve it. Something like an AI for sudoku, or whatever game you like could be an idea.

  • @satibel

    @satibel

    6 жыл бұрын

    by the way, jabrils did a great 4 episode series of videos on creating an AI using a genetic algorithm for a video game he made. kzread.info/dash/bejne/jIxmqtuuZbrVdso.html

  • @inspectorcrud
    @inspectorcrud6 жыл бұрын

    Would be great if we could render old movies like some like it hot into 3D

  • @isacscjr

    @isacscjr

    6 жыл бұрын

    true, and re-render it in 4k

  • @rushikeshdarge6115
    @rushikeshdarge61156 жыл бұрын

    how to execute this code ? HELP???

  • @marektluczek9531
    @marektluczek95313 жыл бұрын

    What's the source of 1:20 video fragment?

  • @batosato
    @batosato6 жыл бұрын

    Is it possible to use this method to generate 3D model of buildings?

  • @gshan994

    @gshan994

    6 жыл бұрын

    batosato this was specifically used for body detection and 3d modelling of humans but if u use a dataset of buildings then it can create a 3d model of building

  • @batosato

    @batosato

    6 жыл бұрын

    Great. Is there a dataset of buildings? I reckon training the algorithm again on a single GPU would take ages!

  • @MolemanAnimation

    @MolemanAnimation

    6 жыл бұрын

    if you wanna create 3d models from a set of images look up "photogrammetry"

  • @satibel

    @satibel

    6 жыл бұрын

    you should be able to get lidar datasets and combine them with a picture to generate a depth map, maybe not exactly what you want, but good enough. buildings may be harder than humans, because they have different shapes, from a skyscraper, to a castle, they change a lot more than humans who don't have more than 2 arms, 2 legs, a torso and a head. if you concentrate on skyscrapers, you can approximate them as a box, which should be quite easy to do though.

  • @MrDanielphillis

    @MrDanielphillis

    6 жыл бұрын

    in conjunction with google street view - that should be doable for Machine Learning generated 3D models of buildings

  • @Ruhgtfo
    @Ruhgtfo4 жыл бұрын

    can make another tutorial how to make Amazon Go store alike feature computer vision example~ :D

  • @myselfremade
    @myselfremade6 жыл бұрын

    Siraj! you're up late!, or early..

  • @zabique
    @zabique6 жыл бұрын

    Oh, hi Raj

  • @JayRodge
    @JayRodge6 жыл бұрын

    Notification Squad

  • @tuanva6484
    @tuanva64846 жыл бұрын

    hay đó em trai

  • @chrischappell7643

    @chrischappell7643

    6 жыл бұрын

    ồ.... ra cao nhân là anh trai của siral à ... hè hè

  • @tuanva6484

    @tuanva6484

    6 жыл бұрын

    thằng em họ mà bác :3

  • @maroben225
    @maroben2255 жыл бұрын

    Is the source code available ?

  • @md.rijoanrabbi99
    @md.rijoanrabbi995 жыл бұрын

    How you know all of those man?? how it possible?

  • @caner19959595
    @caner199595956 жыл бұрын

    RoIPool or RoIAlign?

  • @akashnagaraj3954
    @akashnagaraj39545 жыл бұрын

    what is the papers name?

  • @ferdiansyahzhultriza9926
    @ferdiansyahzhultriza99266 жыл бұрын

    Technology has grown up more beautiful than meg on family guy. There's another lib called openpose, did you already see that?

  • @srikumarsastry7473
    @srikumarsastry74736 жыл бұрын

    Amazon GO?

  • @zhongzhongclock
    @zhongzhongclock6 жыл бұрын

    Oh! It's a new monster in technology world. Lots of fake videos could be more easily created. It's good or bad? I don't know.

  • @darogajee3286
    @darogajee32866 жыл бұрын

    Have some consistency... Make some playlist of your videos....

  • @DerekFolan
    @DerekFolan5 жыл бұрын

    Need more colours built into computers. Six digits hexadecimal is limited.

  • @peacemaker7953
    @peacemaker79536 жыл бұрын

    Dear Siraj this is very heavy for loading on mobile devices !!

  • @wheelbegood
    @wheelbegood6 жыл бұрын

    this is all very great and I'm very interested in this kind of technology but I have a moral problem with that - because surveillance of us citizens is getting much easier for big companies and the government I don't think that this is a good evolution.. Here we have it, the old problem of the double-edged sword :/ I think in the next year(s) we should think of the ethics behind AI very carefully and we should try to maintain our privacy as a citizen

  • @saminchowdhury7995
    @saminchowdhury79956 жыл бұрын

    These AI inventions always makes me demotivated. Can we still start something unique and meaningful given we live in a world where Google and Facebook comes up with groundbreaking ideas.

  • @apoorvwatsky

    @apoorvwatsky

    6 жыл бұрын

    Samin Chowdhury I kind of feel the same dude. Siraj what do you think?

  • @notaras1985

    @notaras1985

    6 жыл бұрын

    thats bogus. true problems are food scarcity, injustice, and energetic crisis. thats what they must solve. all of us

  • @MarkJay

    @MarkJay

    6 жыл бұрын

    of course you can. Yes the AI groups at Facebook, Google, Stanford, CMU are putting out impressive stuff, but it's all open source so anyone can study their work. Yes it will take a lot of time and learning but don't be discouraged, you can add value and maybe even create your own new improved model

  • @schrodingerscat3912

    @schrodingerscat3912

    6 жыл бұрын

    DeepMind started outside of Google. You can still be creative and develop original products that will be of value whether you work for a tech giant or not. Look at Siraj, he's enlightening and inspiring people to engage and pursue this field. That's just as groundbreaking as the ideas that will come later.

  • @SirajRaval

    @SirajRaval

    6 жыл бұрын

    vertical not horizontal products

  • @Y3nnick
    @Y3nnick6 жыл бұрын

    Can you ask Elon Musk if he's willing to do your next KZread Video #DeepFakes :P

  • @kaushikraghupathrunitechie
    @kaushikraghupathrunitechie4 жыл бұрын

    3D Human Pose Estimation demo I made: kzread.info/dash/bejne/i6OssNWNh9jHeJs.html

  • @KanalMcLP
    @KanalMcLP6 жыл бұрын

    Can't they make a break in AI inventions while I am writing a paper for school on it?

  • @satibel

    @satibel

    6 жыл бұрын

    you can also accept that your paper will be obsolete by the time it comes out :p

  • @cooky123
    @cooky1236 жыл бұрын

    Hey Stop dude, You're uploading too much videos. You're insane!

  • @rushikeshdarge6115
    @rushikeshdarge61156 жыл бұрын

    please!! code is not working? can anybody help me to execute this code?

  • @RiteshKumarMaurya
    @RiteshKumarMaurya6 жыл бұрын

    First...

  • @kunal2010cool
    @kunal2010cool6 жыл бұрын

    Difficult !

  • @urgisjot
    @urgisjot5 жыл бұрын

    Siraj could you do an explanation of how to use Vnect? gvv.mpi-inf.mpg.de/projects/VNect/ github.com/timctho/VNect-tensorflow

  • @aswathm78
    @aswathm786 жыл бұрын

    Hell No . Its like people actually took their inspiration from Black Mirror

  • @imshafay
    @imshafay6 жыл бұрын

    The technology is too fast mate tooo fast.

  • @intissarkhalifa5999
    @intissarkhalifa59996 жыл бұрын

    Hi Siraj i send you already a mail and i am waiting to your response

  • @Krommandant
    @Krommandant6 жыл бұрын

    Your green screen is bleeding onto your hands. What's up with that?

  • @YOYO-dl6fe
    @YOYO-dl6fe6 жыл бұрын

    Can I learn ai with class 8 maths I m pro in programming

  • @hammadshaikhha

    @hammadshaikhha

    6 жыл бұрын

    You can began by programming the ML algorithms using tensorflow or scikit learn in python for now. When you learn more math you can look into understanding how the algorithms work.

  • @YOYO-dl6fe

    @YOYO-dl6fe

    6 жыл бұрын

    hammad shaikh thanks very good suggestion u r great

  • @Pushpak4u
    @Pushpak4u6 жыл бұрын

    1st