OpenCV Object Detection in Games Python Tutorial #1
Learn how to use OpenCV for object detection in video games. This intro tutorial will show you how to install OpenCV for Python and get started with simple image template matching. This will serve as our foundation as we explore many different computer vision techniques in future videos.
Full tutorial playlist: • OpenCV Object Detectio...
GitHub Repo: github.com/learncodebygaming/...
OpenCV documentation: docs.opencv.org/4.2.0/
Official template matching tutorial: docs.opencv.org/4.2.0/d4/dc6/...
0:24 How to install OpenCV
1:45 How OpenCV relates to PyAutoGUI
2:52 How to navigate the OpenCV documentation
4:53 Comparison methods visualized
5:39 Writing code for cv.matchTemplate()
OpenCV is an open source computer vision library with hundreds of functions for processing and understanding images. In this tutorial, I'm going to show you how to get started with OpenCV in Python by using it to find an image inside another image. This simple form of object detection will be a good starting point before we move on to more advanced image recognition techniques.
The quickest way to get started with OpenCV is: pip install opencv-python
Once installed, you can use the library by importing cv2. Numpy is used extensively when working with OpenCV data, so the top of your Python files will look like this:
import cv2 as cv
import numpy as np
That's all there is for setup. Now let's grab an image we want to process. I'm going to be using this screenshot from Albion Online, but any screenshot will do.
What we're going to do is crop out a small section from our screenshot, save that as a separate image file, and then we're going to use OpenCV to find the position of the smaller image inside our entire screenshot. From my screenshot, I'll crop out one of the cabbages.
The OpenCV function we'll be focusing on is called matchTemplate(). In the documentation, we can see we're going to give this function an image to search over, an image to search for, and a method type for doing the comparison. And we'll end up with a result array. You'll want to experiment with the different comparison methods to see what works best for your use-case.
Alright, let's write some code. The first thing we want to do is load our image files.
haystack_img = cv.imread('albion_farm.jpg', cv.IMREAD_UNCHANGED)
needle_img = cv.imread('albion_cabbage.jpg', cv.IMREAD_UNCHANGED)
The "haystack" image is our screenshot, and we'll be search that for the "needle" image we cropped out. In imread() the first parameter is the image file path, and the second parameter is a flag that allows us to do some pre-processing when loading the images. In this case, we're loading them in unchanged.
Now that we have our images loaded, we can go ahead and call matchTemplate(). I've had good luck using the TM_CCOEFF_NORMED comparison algorithm.
result = cv.matchTemplate(haystack_img, needle_img, cv.TM_CCOEFF_NORMED)
We can quickly see the results from matchTemplate() by displaying that data with imshow().
cv.imshow('Result', result)
cv.waitKey()
In imshow(), the first parameter is the window name and the second is the image we want to show. I've also called waitKey() to pause our script while we review the image. Without this, our script would quickly close before we could see the image. Pressing any key on the keyboard will trigger waitKey() to stop waiting, thus ending our script.
In this result image, the bright white pixels represent the positions that best match the cropped image. The black pixels are the worst matches. Note that these best match positions correspond with the upper left corner of where you'd place the needle image.
Now that we've visualized the results of matchTemplate(), let's get those best match coordinates. We can do that using minMaxLoc().
The minMaxLoc() function returns four values. First are the confidence values for the worst and best matches, on a scale from 0 to 1. These are how black or how white the darkest/brightest pixels are in our result image, where 0 would be perfect black and 1 would be perfect white. The last two values minMaxLoc() returns are the positions of those worst/best match pixels in the form of an (X,Y) tuple.
For every needle image that we give matchTemplate(), we will always get back some values from minMaxLoc(), even if that cropped image appears nowhere in the haystack. We can tell when we didn't find a good match because the max confidence value will be low. How low is too low depends on the images you're working with and what you're trying to achieve.
Now that we've found a good match, let's outline where we found it in the haystack image. We can do that using OpenCV's rectangle() function.
Continue with the written tutorial here: learncodebygaming.com/blog/op...
Join me on Discord: / discord
Пікірлер: 323
I just found out your video and even as a senior python and opencv developer I enjoyed watching all of it. Learning image processing through games is an excellent idea. Your presentation is clear and precise, I disagree with comments which states it is a bit boring. Your explanation level is just right for the beginners, not overexplaining things and exploding beginner's minds :D I wanted to take my time and display my support with a comment because you clearly dedicate your time and enthusiasm to create a helpful content for the community. You just earned a subscriber, keep up the good work my friend. p.s. About the not so perfect matching score, it is probably due to using jpeg format's lossy compression. After you saved those images as jpeg, they will have very small differences invisible to eye but clearly visible to code as in this example. You might prefer using PNG for lossless image saving or even BMP to get faster image write/read times (high fps).
@josh9761
2 жыл бұрын
second this by far the best explanation i have came across, zoomed in code, well explained step by step
@E30Andy
9 ай бұрын
I went through this tutorial but deviated by using png files instead. ultimately this makes the tutorial not work as written because the data was no longer in the expected format of CV_8U or CV_32F for the matchTemplate function. Just a warning for those that like to experiment lol. My understanding is that you can easily convert to the proper structure, but me, being a noob, did not find success with this. I saved my files to .jpg and moved on with the tutorial instead of getting hung up and everything worked.
It took two years but I am very happy that I managed to find your channel! Thank you for the great tutorials and videos
Honestly, the quality of your videos and tutorials is simply astounding. I'm constantly amazed that you don't have hundreds of thousands of subscribers....yet!
I've just started getting into object detection in opencv along with tensorflow, yolov4 etc. Whilst I'm not using it for gaming, these tutorials have been great for real dynamic and practical applications and for that i want to say thank you :)
This was something I was wondering about, Thank you. I don't think there are much interactive content like this out there. Glad to have found this channel. Thank you once again.
@LearnCodeByGaming
4 жыл бұрын
Glad you like it!
I'm on part #3 of your series so far but wanted to revisit this to say - you've been helping me get out of tutorial hell. Thank you so much Ben! You have an amazing teaching style; The documentation and comments along the way in both video & blog posts provides more clarity than other tutorials. Your channel is one of the best out there, please don't stop. Peace.
@LearnCodeByGaming
4 жыл бұрын
Thanks man!
Here I was wondering how i was going to pull of object detection, then my man presents the answer to me just like that! Always appreciate you, never fail to expedite my thought process!
Your OpenCV videos inspired me so much, been learning constantly new things and doing some low profile botting too. Best channel I've stumbled upon in a long time :).
Thanks for the video, I'm from Brazil and here we don't have many good videos about OpenCv and Python , the simple and objective way you speak is perfect for me who doesn't is a native and thanks for not using so many slangs
Nice to see you taking it to the next level
I love your videos. they make it more fun to learn to code. thank you very much for taking the time to make these, especially into easily digestible chunks in a playlist. youre awesome!
This channel is God sent. Keep going and you'll grow very fast
(Español) Muy buen video, conciso pero detallado. Atiende a problemas pequeños de forma que permite aprender poco a poco y construir el código lentamente y de una forma muy natural. Mis felicitaciones sinceras, espero continúes con este trabajo de divulgación tan pulido. Solo me queda felicitarte de nuevo y seguir tu canal y aportes. Mucha suerte para el futuro. (English by google translate) Very good video, concise but detailed. It attends to small problems in a way that allows you to learn little by little and build the code slowly and in a very natural way. My sincere congratulations, I hope you continue with this polished disclosure work. I can only congratulate you again and follow your channel and contributions. Good luck for the future.
Wow dude been watching a lot of videos on open cv and yours is far the best I have seen. Just the way you teach is awesome.
I was JUST looking up info on this. Who knew it would be from the guy I subbed to from a few days ago. Keep up the good work your putting out unique and great content
@LearnCodeByGaming
4 жыл бұрын
Thanks means a lot!
@diegorivera_cr
4 жыл бұрын
Thanks alot for this video! Do you think you can cover how to find an object and also detect when it is gone in a video game?
@jamiestwrt
3 жыл бұрын
Trueeee
@mangojango3745
3 жыл бұрын
@Carroll Horsely stop scamming people
@Tradie1
2 жыл бұрын
Exact same here! What an underrated channel
Hi there, this is the best channel about this content. I have watched your Videos about Python at least 20 times each! I am a beginner and was able to follow your explanation and even to costumize the code for my needs! Really amazing!
I have started learning computer vision not long ago and basically you are the best. Thank you very much for your tutorials
Just stumbled upon your channel! Love the style and format! Can't wait to check out more videos!
I'm from Latin America, and I understand your tutorials better than the ones in my language. You're great, thanks
this channel is so much underrated.Very good content bro,keep it up
God I love you and this series, the quality content we all need but do not deserve
hey i JUST found this channel today, MapleStory is my childhood favorite game so this has motivated me to learn to code and use maplestory as my game, my younger self is screaming right now lol love your content
Thanks Ben :). I really like your teaching style. Please do more if you’re available!
Yooooo Ben i hope you are still Alive and coding! I honestly cant thank you enough for all your content! It has finally gotten me started with programming and coding, im studying to become a python developer and ive truly found my passion and calling in life now. The way you teach and explain things is amazing, wish i would have gotten help like this sooner in life! Much love for you
You did a great job at making this easy to understand, I followed it all the way through and it worked, Thank you
Brilliant channel. Enjoyed every bit of it. Love the short length. Subbed!
YES !! This is going to be very interesting ! =D
Wicked video! thank you for the tutorial. After fumbling around with a few other guides, it was yours that really got me into it.
@LearnCodeByGaming
4 жыл бұрын
Awesome!
Thank you, I love how you guide us through the documentation! This will help me start creating AI's to play games and automate tasks.
Whoever disliked this needs their head checking, this was one of the best programming tutorials I have seen yet.
Underrated KZreadr. Great content!
why this guy don't have above 100k subs.. he explained it so good. +1
Hi there Ben! I stumbled upon your channel because I wanna learn how to code. I watched this video to see if it interests me though I don't have any knowledge yet. But boy! You explain things so well that I get the idea of what you are doing. I am glad I've found your channel. +1 sub!
@LearnCodeByGaming
3 жыл бұрын
Awesome, always happy to see new people giving code a try! Thanks for subscribing.
Thank u very much for this video. It was Informative and engaging. Makes me want to learn more about OpenCV!
Thank you so much. Your explanations are so clear. I learn a lot from you. You are the best!
GOLD CONTENT thanks , i just needed something like this to complete my project you earned a new subscriber :D
@LearnCodeByGaming
3 жыл бұрын
Glad you like it! Good luck on your project.
Open CV is great no denying but the library is just huge , it's full of functions to do all sorts of thing , this is a great advantage(keeps ur code short and neat) and disadvantage ( I really find it difficult to remember these lol) like think about it for a sec, there is function for like literally everything and parameters which makes no sense for newbies (like me myself), but nevertheless great video mate , really liked how u walked through the documentation (encouraging viewers to explore) and explained as much as possible while writing the code and made the video as short as it possibly could have been u certainly deserve more subs and recognition mate :)
This is my new favorite channel, thanks for good content ^_^
Thank you for this. Keep em coming :)
Thx, People like you make the world a better place
You're an EXCELLENT teacher. Thank you!
Hey man just want to let you know that I'm so happy right now as a gamer and python programmer :) You just earned a new subscriber!
@LearnCodeByGaming
4 жыл бұрын
Glad other people like this stuff too!
this guy should get more subscribers! thank you for the great contents
@LearnCodeByGaming
4 жыл бұрын
Thanks!
great stuff. no idea why the views are not in millions. kudos
YOUR TUTORIALS ARE THE BEST IVE SEEN, THENK YOU SO MUUUCH!!!!!!!
LOVED IT! Please, please, please! I would love more tutorials like this! Can we also make basic games like tower defense, basic shooter, or something similar?
Thanks for the video my man, very useful and well explained.
Thanks a lot man , This Video really helped me start using openCV
love you so much, I love this course, you just got a Chinese Subscriber!!!
You are great bruh I'd literally crown u as king.
thanks i will watch ur vids and learn this library with u keep up the good work
Thank you for sharing your knowledge. Greetings from Brazil.
absolutely amazing content !
thank you so much Ben ! I viewed your video, red your web site, and i have done my new project in few minutes !
Amazing videos!thankyou so much sir!! Much praise for your work
This me first time watching your videos and loved it!
Thanks for the tutorials, I like all of them, it's much more fun to learn coding/new libraries if the examples are interesting topics too in this case, games! keep up the good work, looking forward to future tutorials, OpenCV looks awesome.
@LearnCodeByGaming
4 жыл бұрын
Yes, that's the idea of my channel! Thanks!
Thank you for the great tutorial!
Thanks dude, I want to make a very simple bot to automate a PIA element for a really old game and this is a good excuse to learn Python a bit more
Thank you Ben , very interesting content !
Bruh ty so much! You save so much time for as !
@LearnCodeByGaming
3 жыл бұрын
You're welcome!
Thank you for your tutorials!Really appreciate it :)
@LearnCodeByGaming
4 жыл бұрын
You're welcome, glad you like them!
good video and a writeup! You are doing a great job!
Thanks for the video, I now learned what I needed. Cheers!
Great content as always!
@LearnCodeByGaming
4 жыл бұрын
Thanks Shane!
Your videos are super clear. Thankyou :)
Amazing explanation!
Nice video! Good for starting programming a mmorpg bot!
thanks for these 9 masterpiece videos
Thank you for the video :) That s very interesting
Thank you so much for these videos, this is gold
Keep making videos. This is awesome!
Can't wait to see where this goes
@LearnCodeByGaming
4 жыл бұрын
Same! I need to slow down on coding and catch up with making the videos.
Very good, thx for your videos
you are such a great teacher
Great video!
thanks for the clear explanation!
Thank you for your videos, you are very good, I have received a lot of inspiration
Wonderful video!
Great Video!
Great playlist
Nice explanation
It has been two years already, where is the continuation? Why did you abandon your channel? You explain better than all my teachers! You're the best! It has been 2 years already, perhaps you have improved object detection, bot performance, and their movement between points, or maybe there's something else interesting to see!
ความรู้ทั้งนั้นขอบคุณมากครับ
Very nice Video!
Nice this verry useful for me 👍
hey this was insanely helpful, thank you very much
print("Hello from Brazil") Thx bro
Nice Need More Like This
ur the goat @code by gaming
Your a legend mate thank you so much!!!!!!!
just found good stuff on yo channel, Thanks!
good explanation, ty
Hi, love the vids! Hope you're doing well. Keep up the content. Cheers.
@LearnCodeByGaming
4 жыл бұрын
Thanks Ron!
great explanation !!!
Hey man nice video. I was hoping if a make video on playing gta 5 using hand recognition and open cv. Thanks 👍
Great tutorial, thanks!
I just did the sentdex opencv tutorial so this is perfect
@LearnCodeByGaming
4 жыл бұрын
Awesome hope it's helpful for you
@Joshua-kr5fq
4 жыл бұрын
@@LearnCodeByGaming Stardew valley is about to get more interesting.
great tutorial
I am wondering why the needle isn´t found anymore when I am using a perfect cut out needl image with alpha channel. with just a rectangle cutout it works.