Deep Learning for VR/AR: Body tracking with Intel RealSense Technology
Philip Krejov, CTO Office Intel RealSense, speaks about "Deep Learning for VR/AR: Body tracking" using Intel RealSense depth cameras
realsense.intel.com/deep-lear...
Skeletal tracking first became viable as a product around 2011, since then, many different additional approaches to the problem have been explored, but almost all of them suffer from high compute usage - which absorbs a lot of GPU or CPU cycles, as well as being designed for general pose estimation. By focusing on VR/AR, it allows us to use the position of the HMD to train with a much smaller machine learning model. Philip will share some of his work on this convolutional neural network with integrated head and controller information approach, as well as talking about some of his prior work on hand tracking in real time using only a CPU.
Пікірлер: 16
This is freaking amazing. I can't wait to see VR apps and games using tech like this to really enhance the experience
@krejov100
5 жыл бұрын
Thank you very much for the kind comment!
@123456wei
5 жыл бұрын
@@krejov100 great talk! this deserves way more views!
I hope to live to see a day when I can enjoy using this in VRChat. Or something equally as good as VRChat.
@LhunVideo
5 жыл бұрын
this.
@UnrebornMortuus
5 жыл бұрын
@@LhunVideo lavender when
Nice!
This looks great! Excellent work! I was wrapping christmas presents while watching this so maybe I missed it, but do you mention how much cpu/gpu resource you use for this? And do you see much room for optimizing the code?
@krejov100
5 жыл бұрын
So our network ran at 200fps on CPU, i7 and we used a small portion of a GPU (~15%) for solving point to capsule constraints, I believe it was a 1070gtx
Very interesting. I would like to calibrate the multiple sensors position like the way you use in the video with a big pattern board with motion. What is the library you use? The one start at 15:13 Thanks!
@mattizzle81
4 жыл бұрын
Yeah this is a real pain point for me. I am very interested in tracking and sometimes would want to use multiple cameras, but can't yet understand how to transform the 3D worldspace position of one camera to another camera.
Considering MSFT will ship a body tracking SDK with the new Azure Kinect, can we expect to have this included on the RealSense SDK anytime soon?
@IntelRealSense
4 жыл бұрын
Skeleton Tracking SDK is now available here www.intelrealsense.com/skeleton-tracking/
@Hanning_0226
27 күн бұрын
@@IntelRealSense The website showing: "Sorry! That page doesn't seem to exist.". Is there any ready to use body tracking SDK?
why only 190 subscribers on this channel.
Could this code be released to the RS customers pls.