Open Source Motion Capture for Autonomous Drones

A low cost ($20), open source motion capture system with millimeter level precision at room-scale. Used to localize autonomous drones inside a room.
GitHub repo with code & 3D files: github.com/jyjblrd/Mocap-Drones
Check out my blog post for more info on the drones, cameras, etc: joshuabird.com/blog/post/moca...

Пікірлер: 540

  • @emovard3n
    @emovard3n6 ай бұрын

    Wow, you definitely deserve more views. Very cool project, also making it open source means a lot

  • @moosesnWoop

    @moosesnWoop

    5 ай бұрын

    by making it open source, the Future thanks you.

  • @deweytann

    @deweytann

    5 ай бұрын

    Open source gives you wings

  • @0hleg

    @0hleg

    5 ай бұрын

    @@moosesnWoopand the person that actually eventually tunes it a bit and monetizes it.

  • @sabih1677
    @sabih16775 ай бұрын

    Very rarely a maker actually shares the math and science behind their projects. You sir have earned my subscription. Cant wait to see what you make next

  • @dowesschule

    @dowesschule

    5 ай бұрын

    there actually is a lot of youtube videos accompanying research papers, it's just that they don't scream and jump-cut, so the algorithm won't serve them. but many GitHub pages do have embedded yt videos similar to this one.

  • @andrewferguson6901

    @andrewferguson6901

    5 ай бұрын

    Because it's reproducible, that makes it science!

  • @elishmuel1976

    @elishmuel1976

    5 ай бұрын

    Same!

  • @googleyoutubechannel8554
    @googleyoutubechannel85545 ай бұрын

    A college student in his dorm room... this is amazing, precise indoor location has been a constant challenge

  • @Keyboardfanatiker

    @Keyboardfanatiker

    Ай бұрын

    Using ultra-wideband for ranging shouldn't be all that difficult actually

  • @googleyoutubechannel8554

    @googleyoutubechannel8554

    Ай бұрын

    @@Keyboardfanatiker "actually"

  • @Keyboardfanatiker

    @Keyboardfanatiker

    Ай бұрын

    ​@@googleyoutubechannel8554 I'm using Qorvo DW3000 boards for an LED controller game to enable spatial interactions

  • @tom-stein
    @tom-stein5 ай бұрын

    Someone should give this guy access to a workshop: Very nice work!

  • @connerwwise2315
    @connerwwise23155 ай бұрын

    I’m a senior graduating in Unmanned Aerial Systems and for my senior project I used an ESP32 and an ELRS transmitter to fly drones using any Bluetooth controller. And this project makes mine look like child’s play.

  • @technicalactivity

    @technicalactivity

    Ай бұрын

    that degree has been useless since I got one in 2015.. the problem is the real drone work is not there yet unless you have military experience

  • @Peter-898
    @Peter-8985 ай бұрын

    Massively impressive project, very well done & I'm glad to see your work shared. Hope we see more work spring up around it!

  • @Clark-Mills
    @Clark-Mills5 ай бұрын

    Very nice work, quite an achievement. Congrat's! :) And a huge thank-you for documenting it on YT so well and sharing the project.

  • @Legonigel
    @Legonigel5 ай бұрын

    I'm blown away by the work you put into this! Simply amazing!

  • @mumblety
    @mumblety5 ай бұрын

    Wow! I wasn't expecting you to actually explain how the tracking worked! It's really cool that you did! Thanks!

  • @iainmackenzieUK
    @iainmackenzieUK5 ай бұрын

    Your motivation, dedication and clear communication are more inspiring than the results.

  • @kagisopilane4088
    @kagisopilane40885 ай бұрын

    This is incredible! I just checked some of your other projects as well. All of them just lit a fire in me. I recently got access to a makerspace so I definitely want to work on something like this. Big Fan all the way from South Africa.

  • @DeanGadberryYT
    @DeanGadberryYT3 ай бұрын

    Thank you for your contribution to the open-source community! This is an impressive project. I am excited to see how your ingenuity is used in the world in the years to come. Please know that many of us are inspired!

  • @cachecaver
    @cachecaver6 ай бұрын

    Nice project that's pretty amazing. Better yet nice explanation of all the algorithms you used to piece it together. You have a bright future.

  • @JosOsborne
    @JosOsborne5 ай бұрын

    Absolutely amazing. You are brilliant and energetic and resourceful and an excellent communicator and I cannot wait to see what you do next.

  • @austindale3129
    @austindale31295 ай бұрын

    Really impressive, and incredibly well explained for such a complex and deeply technical project! I think my favorite aspect is your resourcefulness in utilizing the cheap cameras and floppy disk filter (MacGuiver would be proud)!!! All around inspiring work!

  • @pipsqueak2009
    @pipsqueak20093 ай бұрын

    Super impressive. And fantastic that you've open sourced the project so that others can learn and benefit from it.

  • @fixifaxhd772
    @fixifaxhd7725 ай бұрын

    Love the clip where you literally just toss the drone on the floor. Crazy project. Props on completing it!

  • @theluckybead
    @theluckybead6 ай бұрын

    Bravo, well done. This just poked human knowledge as a whole a bit further in both science and art. Thank you for both doing this experiment and documenting it and sharing it in such an endearing way. You worked extremely hard on this and it inspired me to model the way you shared your work. Bravo bravo.

  • @roland_does_things
    @roland_does_things6 ай бұрын

    Really cool project! Love that it all works in your dorm!

  • @AerialWaviator
    @AerialWaviator5 ай бұрын

    Such an impressive integrated system of multiple projects. The camera conversion project with custom IR filters is amazing in itself. What's fascinating is having 3 IR LEDs allows not only allows tracking position in space, but could enable determination of pitch, row, and yaw angles, so offers a number of possible options to modify and refine ... possibility leading to controlling a drone swarm. This video/project may be enough to gain sponsorship for a dedicated internship project next summer. (one focused fun job, instead of two)

  • @jamesbowe3
    @jamesbowe35 ай бұрын

    Joshua, what a great project and nicely presented! I will definitely share this with my students next semester. Thank you.

  • @cottsak
    @cottsak5 ай бұрын

    As a full stack web dev whose day job is mostly just LOB apps, and a hobbyist understanding of drones and embedded programming (I’ve done a bit of esp stuff) this is really impressive! Just looking at your react front end with the stream data blows my mind a bit hey. Amazing work!!

  • @user-vj6ec5do4d
    @user-vj6ec5do4d2 ай бұрын

    Thanks Joshua for sharing the maths and explaining your project in a very understable way. I thought I know a lot, you taught me different! Great job!

  • @dmitrysozinov2391
    @dmitrysozinov23915 ай бұрын

    Maaan, what a great project! I saw something similar made with high-speed cameras (probably costing thousands of dollars) AND YOU MADE THIS AWESOME THING for price of a meal, you are amazing!

  • @arpita1shrivas
    @arpita1shrivas5 ай бұрын

    how is this man so criminally underrated

  • @mehmetnabiduru8779
    @mehmetnabiduru87795 ай бұрын

    Amazing project and it's obvious that a lot of effort has gone into it nice work ​

  • @casperlehmann
    @casperlehmannАй бұрын

    I've had this video open in my browser for four months now, and I just can't get myself to close it. So freaking cool.

  • @pjforde1978
    @pjforde19785 ай бұрын

    Genuinely impressive work. I'm reminded of a demo I saw years ago in which a drone would recognize that it was running low on juice, so it could land on a pad which would swap out the battery pack, like the Tesla S did for 15 minutes in a simpler time. Anyhow, even more than getting it all working (which is a commendable feat) the real signal this project sends to someone who knows what they are looking at is that you're someone who can take a complicated problem that crosses multiple technical domains and then iterate on each aspect until you achieve an outcome. I would hire you in a heartbeat if I was in a position to do so.

  • @MKUMBRA
    @MKUMBRA5 ай бұрын

    Bro is an undergrad student and already better than SDEs with 10 YOE... he should be making 500k easily

  • @sirdiealot53

    @sirdiealot53

    5 ай бұрын

    More like 500m as CEO of his own company.

  • @davidcahan

    @davidcahan

    5 ай бұрын

    Prolly will b soon

  • @abyszero8620

    @abyszero8620

    5 ай бұрын

    Depends how much of the system he made himself. There's not much glory in using premade hardware with premade software to do premade tasks in a controlled setting.

  • @andrewferguson6901

    @andrewferguson6901

    5 ай бұрын

    It's so thoroughly researched, followed to conclusion, and presented so cleanly. Plus.. reproducible. This is a legitimate publication

  • @francefarms

    @francefarms

    5 ай бұрын

    Maybe it is open source?

  • @tunnelcat4592
    @tunnelcat45925 ай бұрын

    We've been doing exactly this with a single camera for head tracking in the flight sim community for a decade. First time I've seen someone make a proper mocap system out of it!

  • @felixman9691
    @felixman96915 ай бұрын

    I love your projects. Thanks for sharing this one! I was super surprised about the high frame rate on the sony camera. Really just so cool to see people like you sharing this stuff, I have a project I’m working on that I am gonna release for the community that I hope is useful and good too :)

  • @uncharteduty
    @uncharteduty5 ай бұрын

    This is an incredible video. I love it. This is a project I would love to tackle myself in the future.

  • @nicholasgrippo1754
    @nicholasgrippo17545 ай бұрын

    Wow! This is basically the same principle of VR tracking with base stations applied to drones. This is awesome

  • @mr.kybernaut6687
    @mr.kybernaut66875 ай бұрын

    Very cool work! I'm an robotics engineer, PHD, and flying drones in my free time. I thought about exactly the same project, but lack the time. Very impressive, considering you are still studying. Keep up the good work. I see an upcoming excellent engineer / software developer. Don't underestimate the open paths that lie in front of you. If you keep up this pace, you can become whatever you want.

  • @brycenarcisomercines5036
    @brycenarcisomercines50365 ай бұрын

    This is amazing work, explained with great clarity as well!

  • @md.sharifulislamrifat1597
    @md.sharifulislamrifat15975 ай бұрын

    That is very amazing ! And it means a lot to make the project open source....definitely gonna try it out...hats off...

  • @Taylachteam
    @Taylachteam6 ай бұрын

    I subscribed and will be following your journey every step of the way. please continue making this content!

  • @bryanoverbey
    @bryanoverbey5 ай бұрын

    Great work. Not easy to navigate all of those different protocols and stitch something like this together. Also great triangulation algorithm work. We need more bright and talented engineers like yourself. Thanks for making the project open source so that others can be inspired and have some assistance to build something great too

  • @hseinb
    @hseinb3 ай бұрын

    This is amazing, I can imagine it has lots of applications, congrats

  • @natonion1154
    @natonion11545 ай бұрын

    Super amazed by the tracking system. Off-the-shelf solutions are super expensive, but you managed to pull it off with just a lower budget. I'm saving this to potentially replicate it. Thank you for your hardwork!

  • @anjiraj2899

    @anjiraj2899

    5 ай бұрын

    do you know the name of the flight controler?

  • @draggador
    @draggador5 ай бұрын

    This is wonderful! You've my support in your endeavours!

  • @HectorGarciadeMarina
    @HectorGarciadeMarina5 ай бұрын

    It is a great project showing how to do more WITH LESS. Outstanding job! Engineering is about this; these are your resources, and squeeze the maximum from them.

  • @James_Rivers
    @James_Rivers5 ай бұрын

    This is crazy impressive, great work dude

  • @TestDetails
    @TestDetails5 ай бұрын

    you're truly a savant with your erudition, fantastic job!!! As well, appreciate your selflessness in sharing your source code.

  • @DashieDasher
    @DashieDasher5 ай бұрын

    Honestly I'm amazed you got it flying that well at all with that monstrous chain of PID loops

  • @sUASNews
    @sUASNewsАй бұрын

    Wow, just wow excellent work Joshua

  • @circuitguy9750
    @circuitguy97506 ай бұрын

    Wow! That's quite impressive. Thanks for sharing!

  • @brenogibson3538
    @brenogibson35382 ай бұрын

    amazing project! The camera pose logic will help me a lot in a different project I'm working right now. Thank you very much for sharing

  • @DonCarlione973
    @DonCarlione9735 ай бұрын

    Thats awesome bro. I love this new drone technology and where people are taking it to! 👍🏻👍🏻

  • @conall5434
    @conall54345 ай бұрын

    This is a super cool project dude! Good work!

  • @marsgizmo
    @marsgizmo3 ай бұрын

    inspiring project! 👏😎

  • @tomhannigan2234

    @tomhannigan2234

    2 ай бұрын

    terrifying

  • @tomasg.8418
    @tomasg.84185 ай бұрын

    Joshua Bird thank you very much. It's very impressive how simple and brilliant idea it is :)

  • @olasoderlind5685
    @olasoderlind56855 ай бұрын

    inpresive .. nice of you to show your drone craches

  • @8x13b
    @8x13b3 ай бұрын

    This has incredible applications in the VR world! Especially for full body tracking!

  • @gevaisemaginn4396
    @gevaisemaginn43963 ай бұрын

    this is a huge benefit for indoor drone use, as gps wont work as great under buildings. in the future drones will bring you food to you or any item you would like. really cool bro!

  • @robertviragh6527
    @robertviragh65275 ай бұрын

    Thank you for sharing this video. I especially like the systems-level thinking and diagram, and good job to get those complicated nested PID loops working. The math shown for bundle-adjustment minimizing reprojection error is something I didn't know before, it is very useful, I appreciated learning it from the video. If you need to track more than 6 points and they start to overlap you could try to synchronize timers so they're not all on all at once. The esp32 has a real time clock, so you may be able to synchronize several drones to be down to the millisecond with each other (you synchronize it with a physical pin you keep reading, then physically connect them all and synchronize them with one signal at the same time), and then turn them on and off in a certain pattern (so once you have synchronized the seconds the ones that are on for 000-005 ms after each second is the first light, the one that is on 005-010 is the second light, and so forth, and once you've got them all the pattern repeats - they turn on at their assigned time. Think of it like one of them shows up on the hour for five minutes, one each :05 to :10 past the hour, and so forth, only it's milliseconds not minutes and also it repeats, they just don't fire at the same time. It works as long as you can synchronize the clocks and they're high-resolution enough. This way they drones are way easier to tell apart. The number of drones you can handle one at a time this way depends on your camera's framerate. This works because speed of light is really fast so as long as you synchronize ahead of time and have a good real time clock you can figure out which is which, and you do mention the camera has a really high frame-rate. The CV system has a latency in it so you should have one of the lights turn on on exactly on the second every second for a different number of milliseconds from the others, so by looking for that length you can figure out what that latency is. This will require you to track how long the beacons have been on but it should not be too difficult. The CV system's latency is fixed usually, so you should be able to do this. Try it. For milspec applications they could use a super high resolution timer and synchronize real time clocks down to the nanoseconds (my DM's are open, DoD) and keep track of up to tens of thousands of drones through high resolution cameras, but milliseconds is good for your $20 system and could get you to twenty drones or so without the spots interfering with each other and without adding too much latency. I'd love to know if you are able to try it, let me know.

  • @cheythompson740
    @cheythompson7405 ай бұрын

    So a couple bucks in hardware but a good few thousand in knowledge

  • @infocentrousmajac
    @infocentrousmajac5 ай бұрын

    Excellent video! Looking forward to see more content from you. I will attempt to understand and play with your source code. I have few ideas for application and use cases. Great work and even better use of highly sophisticated math. Impressive!

  • @1pierce158
    @1pierce1585 ай бұрын

    You making it look simple that’s how you know you did a good job

  • @shitboxlife
    @shitboxlife5 ай бұрын

    Look at you and your amazing creation! Someone give this man a proper lab stat!!!

  • @KevinDiVico
    @KevinDiVico5 ай бұрын

    Bravo! that is amazing work. You have inspired me to tinker with drones again. :-) look forward to seeing more of your work in the future.

  • @drewmiller9036
    @drewmiller90365 ай бұрын

    This is amazing! Deserves more attention

  • @thedronedojo
    @thedronedojo4 ай бұрын

    This is some extremely impressive indoor positioning.

  • @vivekr4690
    @vivekr46903 ай бұрын

    This is fascinating.! Thank you so much for sharing your wisdom on youtube!

  • @kingaustin360
    @kingaustin360Ай бұрын

    Amazing piece of engineering. Good work

  • @FPVMedias
    @FPVMedias3 ай бұрын

    What an awesome project. Congrats on the work done here. I admire your dedication, ambition and passion. 👊Well Done

  • @pigglywiggly_3652
    @pigglywiggly_36525 ай бұрын

    My good sir hats off to you! Great job I’m thoroughly impressed.

  • @samdownload8192
    @samdownload81925 ай бұрын

    Just lovely!!! It was an amazing project.

  • @VR_Von_Bruegge
    @VR_Von_Bruegge5 ай бұрын

    There should be much more people like you in world. I wish I could do this stuff, very impresive

  • @Refatul_Islam
    @Refatul_Islam5 ай бұрын

    You are great bro❤ Best of luck for future projects... Love from Bangladesh 🇧🇩

  • @siegmundeurades5753
    @siegmundeurades57535 ай бұрын

    Absolutely amazing work, wait until the VR community finds out about this.

  • @andrewhofmann5453
    @andrewhofmann54536 ай бұрын

    Great video, saw your Ender 2 Pro box, I just bought mine 2 weeks ago, love it!

  • @the_python_man
    @the_python_man5 ай бұрын

    this is the kind of video that deserve my like , good work bro

  • @sanlegaYT
    @sanlegaYT6 ай бұрын

    This is very cool!!! Thanks for making it open source!!!!

  • @jonatan01i
    @jonatan01i5 ай бұрын

    OMG, I'm sold after you wrote "hello"

  • @erjonameti1066
    @erjonameti10665 ай бұрын

    Amazing Project...keep up the good work.

  • @airsoftbeast11234
    @airsoftbeast112345 ай бұрын

    A good next step could be to use your own firmware on the drones. Providing a series of inputs to the controller via the PWM signals which results in a sort of nested PID loops like you discussed can make it difficult for precise movements, but if you are able to have custom firmware work out motor speeds in order to optimize moving the drone through a path in space, you can have them zip around in patterns in the most optimal way. Could also train a neural net to do this as well, rather than needing outputs for pitch, yaw, roll and throttle which translate to flight controller, the neural net would directly control the motor speeds through a series of waypoints

  • @whatdaro
    @whatdaro5 ай бұрын

    Excellent video and work , with great results! thanks for sharing.

  • @tnat-
    @tnat-6 ай бұрын

    Nice project! Cascaded PID is a nightmare to tune! For some tips on drone pose regulators I would recommend checking out Ardupilot's Attitude control implementation. Also, I think it would be interesting to see how much an extended kalman filter would improve the pose estimation 🤔

  • @woofcaptain8212
    @woofcaptain8212Ай бұрын

    This is something I've thought of doing for some time but I've never got around to it. Cool project!

  • @strangestuff1213
    @strangestuff12135 ай бұрын

    Subbed as it's a cool project & you open sourced it - This is Top-G stuff

  • @katchupoy01
    @katchupoy0122 күн бұрын

    this is what i have been looking for... very thankful for the share...

  • @blasandresayalagarcia3472
    @blasandresayalagarcia34725 ай бұрын

    Damn man, this is amazing work. Truly inspiring!

  • @davidfeldt89
    @davidfeldt89Ай бұрын

    This is insane! Great work

  • @technoaddicted6824
    @technoaddicted68245 ай бұрын

    Kudos to your effort buddy, thanks a lot for this excellent low budget project 🔥🤝

  • @memeconnect4489
    @memeconnect44895 ай бұрын

    i knew i just had to subscribe to this hidden gem on youtube , Thanks for this upload

  • @TheSelfUnemployed
    @TheSelfUnemployed5 ай бұрын

    incredible broski! i just got aruco markers working for tracking my tabletop bot but this seems like a cool alternative to look into for future projects. thanks for sharing!

  • @oldmancomedies
    @oldmancomedies5 ай бұрын

    I thought about doing this myself at some point, but it ended there. Ideas are cheap and execution is expensive, after all. Amazing work on your end!

  • @mph8759
    @mph87595 ай бұрын

    Congrats to the amazing project, and thanks for sharing

  • @0zux45
    @0zux456 ай бұрын

    YT just recommended your vid to me! and it's great!

  • @gameofpj3286
    @gameofpj32866 ай бұрын

    This is so cool! Amazing work!

  • @GilchristMcGill
    @GilchristMcGill5 ай бұрын

    Well done! My dream is to do such things, but yeah working full-time and studying at the same time makes it tough 😂. Awesome!

  • @triton62674
    @triton626745 ай бұрын

    So cool how you combined the pid loops for a rough location of the point with non linear optimisation for converging on a minimum distance

  • @ritsel6469
    @ritsel64695 ай бұрын

    If there is a teaching of the assembly process, it is a great introduction for beginners who are interested like me.

  • @Freireg1503
    @Freireg15035 ай бұрын

    It's such an amazing and hard project! And you made it look as easy as blinking a led 😂

  • @bugrahana3395
    @bugrahana33956 ай бұрын

    this is really extraordinary. Really well for homemade.

  • @bloepje
    @bloepje5 ай бұрын

    I have had nightmares for decades about control systems and polar diagrams I got at school. But just like digital signal processing, that was only maths at my school, it is nice to finally have a real world example.

  • @meqdaddev4341
    @meqdaddev43415 ай бұрын

    Wow... So cool and knowledge-rich project. Keep Making

  • @pawanv86
    @pawanv865 ай бұрын

    Wow this is a fantastic project! Good stuff 🍻

  • @DOSputin
    @DOSputin5 ай бұрын

    This is fabulous work! Thanks😊 for sharing