Last year, I saw your augmented reality lecture and found out that you only make very useful videos! I'm currently a KZreadr who makes Android classes, and I'm still interested in your videos! Thank you for always making good videos.

Thanks for reminding me why I dropped ML and focused 100% on gamedev (:

@MatthewHallberg

4 жыл бұрын

omg its the worst haha

sick man ! i was wondering about this process for a very long time. You just created the gate of the metaverse !

Amazing Amazing project. With outstanding informative video, I can't hardly thank you enough but still. Thank you very much for the helpful video. Would love to try this.

Amazing!!! Never seen your tutorials before and I usually do not Subscribe on KZread but you just got a new Subscribe (and a fan)

Very amazing work. You can also rig or animate the character to generate a larger dataset. For instance, while animating change the camera views and collect the data

Impressed. Keep up the good work. Appreciate your insight.

Lol I had this exact problem with the bounds! I used basically the exact* same approach you did with the mesh sweep. Also, I think that's sweet how you tuck your laptop in.

Was recently asked to estimate the work need to make a mobile app that could detect specific (known) objects. I remembered watching this video a while back. Can some of this be combined with your previous ML video? Would more computer power be a partial solution to this? Would more lighting variations enrich your data set? Thanks so much for this video. You're my go-to guy for all those off the wall projects.

Great video ! What apps did you use to scan object to 3d models?!

You can improve using enhance image technique, and transfer learning to improve accuracy. The EIT will just stretch, rotate, drift and flip the images, and the transfer learning will help you to go at 95% accuracy

@robosergTV

2 жыл бұрын

EIT?

Fact: there is not another KZreadr out there as innovative as Hallberg.

@MatthewHallberg

4 жыл бұрын

Love you bro

Always loved watching your videos❤

@MatthewHallberg

4 жыл бұрын

thanks!!

That is commitment. A+ for effort.

Hi Matthew, I have been following your tutorials for quite a while. I’m trying to do object detection of real objects with an iPhone. I know iPhones, Unity and window don’t always work well but still want to give it a try. I can use photogrammetry for real objects but I’m not sure on how to do machine training once I have these images. Any suggestions would be appreciated!

Literally loved this .....😍😍

Well done bro! Great work!

@MatthewHallberg

4 жыл бұрын

Thanks man!! It finally worked...somewhat lol.

Have you heard of fractional factorial testing? Might allow you to change multiple variables at once and run far fewer tests.

great stuff bro, i was searching to make something like wanna kicks app. Do you have any idea how to track the feet and put the shoes on legs like that? i know it's AR but i don't have any insights about how to do it :(

nice video with details.. thanks for showing.. whats ur laptop specs?

Idea: maybe try the 3d scan in a completely diffuse lighting setup. So you do not have hard shadows on the textures of your 3d scans. I think that could mess up the recognition of the shapes. Or did you try that already? Anyway , apart from the bulbosaur(?) (not a pokemon expert here) recognition seems quite good already.

@MatthewHallberg

4 жыл бұрын

I did try that I just didn’t film that part I still did get some shadows though it might have helped if I had a box with the correct lighting setup and then I could spin the object inside.

Kudos to ur efforts man! 💯💯

@MatthewHallberg

4 жыл бұрын

Thank you I tried lol, almost didn't make this video for sure.

Great video as usual! I'm also planning to start ML sooner :)

@MatthewHallberg

4 жыл бұрын

thank you! Yeah do it so you can teach me lol.

@hamzzashaffi

4 жыл бұрын

@@MatthewHallberg guess what? I haven't started it yet lol.

@MatthewHallberg

4 жыл бұрын

Hamza Shafi haha me either

@hamzzashaffi

4 жыл бұрын

@@MatthewHallberg ayy bro

@tharaniv6267

2 жыл бұрын

@@MatthewHallberg Hey what app u used for photogammetry(scanning 3d model)

hard fan from sri lanka you AI man

Your hard drive must be massive to hold all that data!

@MatthewHallberg

4 жыл бұрын

Haha MASSIVE. And you know what they say about guys with big hard drives...

You can use colab which has a GPU better than 2080Ti, just put all files in google drive. Colab comes with TensorFlow installed by default. Synthetic data on machine learning is definitely doable, there are some papers using blender to generate these training datasets. To address the poor results, by default all convolution-based networks tend to learn the texture of the image instead of the shape, maybe unity's texture is not that photo-realistic. It's very difficult to train a model from scratch, the common approach is to do a transfer learning, where you take a pre-trained model of a large dataset like imagenet, then unfreeze the top and bottom layer, tuning on your data. With 5000 images to train from scratch, you won't get good results unless it's a super simple classification, like 28x28 handwritten digits. Since it's synthetic data, might be better to just do a pixel-wise label instead of a bounding box label.

" i have no idea what that is, And i don't even want to know " - MatthewHallberg 2:30 . . . . Much love and support for you bro , such a great content

@MathewHallberg Hey nice project.. what app did you use for photogrammetry? To make the model...please reply.

@tharaniv6267

2 жыл бұрын

I also have this doubt

@abhishekgoyaldev

2 жыл бұрын

@@tharaniv6267 Have you even watched the full video? 11:16

@tharaniv6267

2 жыл бұрын

@@abhishekgoyaldev thanks

you should join the Two Minute Papers Discord!

Great video

You need to remove the green bbox outline from images and then it will work.

You could have used Google colab, they give 12gb ram and GPU/TPU for free. You can keep it running even if you close the browser or shut down the computer

Hey Matthew I had a question is there any possibility for hand detection in unity ? (without that paid plugin of OpenCV) I had a project in my mind where you could interact with models shown by vuforia's image target using bounding box coordinates and 3d model coordinates ... i have done some workaround by scanning the hand through openCv in an external script and then sending the coordinates as packets over UDP to unity every frame .... but the main problem is it is not feasible for mobile phones :(

great video

Super!!

i use vuforia app to scan the 3d object and use it to create augmented reality app just like this one... i think it provide better experience and faster workflow..

@MatthewHallberg

4 жыл бұрын

It definitely will but my goal here was to create a generalized tracker that recognized more than one style of the same object... I just failed lol.

Good job!

Awesome!!

First like and first comment broo ❤️😘

@MatthewHallberg

4 жыл бұрын

MY DUDE!

very cool vid

that kickflip

Would reccomend to just use some cloud instance instead of a laptop GPU as it will usually start to throttle after a while. Also, most of those frameworks works better on Linux due to the fact that virtually all the server do run on whatever linux distro was chosen by the user(usually Ubuntu or Debian like distros, sometimes Red-Hat based etc)

Hi Mattehw, How can I do to enter a augmented reality house but first put the house in a specific place and then scale it?

sooo coool

Did you have the green borders in all your rendered training images (7:57)?

@MatthewHallberg

4 жыл бұрын

That was just for the video I didn’t actually train like that lol. I am dumb so that’s a valid question but not that dumb haha.

@maknien

4 жыл бұрын

@@MatthewHallberg Good :D I thought for a while should I even ask... But for the geometry randomizing, you should definitely check Houdini. I've been using that for all kinds of synthetic ML data stuff.

@MatthewHallberg

4 жыл бұрын

@@maknien INTERESTING, checking that out now thanks.

How do you draw the bounding box around the object it is detecting in-house object? I tried using renderer.bound but it does not work. Do you have samples?

you are so great

should have used a bunch of different pikachu and bulbasaur images instead of the same ones, then your algorithm would learn that they come in different forms, shapes and sizes

is it possible to feed ReferenceImageLibrary remotely ? like download the images from remote server with the 3D prefabs to place

Wondering how hard this would be using 2D Data instead of 3D Data

you should use for loop inside for loop inside for loop... instead of random :D idk anything about ml too

What is the name of mobile 3d images app

How to place 3d model on detected object in unity3d with tensorflow.

Just FYI, Google has a new API named AutoML. It is much easier than their old Vision API which is hard to use. But it cost $$$$.

prototype with 2k pictures... make sure the item you want to identify is 100% in your frame in your training pics. Don't put two items in your picture - the AI will potentially try to think there's supposed to be two; like lenses in glasses. check the the type of tensor flow model is right for what your trying to do.. some are very quick but have a lower success rate.

Am so confused when can I start learning about ML ... Do I need like probability theory and statistics class first after that intro to AI and read few books after that I can jump to ML What introductory courses would you recommend Side note : I’m coming from 2+ years with java and about few months with python and am learning flask now (just for the heck of it ) Do you think am ready to start doing stuff with tenser-flow or scikit maybe ??? And which one to start with ??

@MatthewHallberg

4 жыл бұрын

I literally have no idea on that one I never took any courses just started playing with tensorflow and following tutorials which is probably why I have no idea what I am doing in this video.

@annabelgroenenberg9448

4 жыл бұрын

You have a very good machine learning course on youube by Andrew Ng. It's the ultimate starting point. You don't have to know a lot of probability stuff but know the basics like false positive, false negative etc...

one like just for ur Efforts! I can feel the pain.

I am new to the package of perception from Unity and my experience it's just for game designing, please if anyone has experience in such thing like this, I need help

I need help with the coronavirus ar app

Can someone please explain what type of deep learning method this process uses? is it CNN?

@neverninetofive

Жыл бұрын

Yes

I wanna do keypoints annotations, anyone who could help me on that please???

someone knows of any tutorial

dude, use Unity Perception package..........

Add jarvis to the play store!!!!!

Your model was overfit because you only had a few thousand images. Next time, print the summary from your model and take notice of the total number of parameters in the model. Then try to get 2 to 3 times that many image samples. You should get decent results then.

I think you overfitted to your training data which may be why the first model didn't work. Cant be sure though.

@MatthewHallberg

4 жыл бұрын

I was definitely getting that vibe but I don’t know how to tell for sure.

@annabelgroenenberg9448

4 жыл бұрын

@@MatthewHallberg Look at the loss function via tensorboard. If your training data loss curve jumps to a high accuracy while your validation curve is way behind, you're probably overfitting

@samgrogan8815

4 жыл бұрын

@@MatthewHallberg Really depends on your setup. But I think with tensorflow there is a callback in the keras library for early stopping and you just tell it to stop if its not seeing inprovement after so many steps and that can help avoid over fitting.

two words: transfer learning :0

@MatthewHallberg

3 жыл бұрын

Yeah this is transfer learning lol did you watch the video?

@camdenparsons5114

3 жыл бұрын

@@MatthewHallberg yeah haha I watched it. I was sugesting that you use a pretrain model and retrain the last or last few layers to save many hours on training. thats how the watson and google object detection APIs work

Great vid, too bad you have no real computer :-)

@MatthewHallberg

4 жыл бұрын

Haha yeah

You don't neet 20000 epochs to train the model with such less data,.. it's over fits the model and the model can not predict all the labels with equally accuracy!!!, You have to train untill the loss is stable,.. thats it!! When your detecting every pichachu in the world,.. you need different types of pichachu with different background images,..... Don't worry about the huge data.., all you need is 1000 images each!!,.. and epochs not more than 500

Hope some loaded dude gives u a quantum computer... You might even find a way to prevent future AI to wipe out humanity...

Lol "It was so hard to figure out how to convert bounding boxes from world space to screen space...". Really?

@MatthewHallberg

4 жыл бұрын

Joseph Moulton I wish it was that simple lol Unity has a function for that.

@joemoulton1823

4 жыл бұрын

@@MatthewHallberg Instead why don't you manipulate the camera to look at the model and then window the frame? Then take the snapshot

@MatthewHallberg

4 жыл бұрын

That’s interesting never thought of that, I guess the only problem would be getting multiple objects in the same image cause that helped the model a lot.

Why... 😑

@MatthewHallberg

4 жыл бұрын

Not what you wanted??

@sahilshaikh3454

4 жыл бұрын

@@MatthewHallberg I hate machine learning 😅

@MatthewHallberg

4 жыл бұрын

Hahah same

You have to use a GPU to speed it up. Try training on Google Colab. Also, tensorflow sucks, use PyTorch :)