Pyautogui image detection
Automating any kind of task is pretty tricky, and the more complex the task, the harder it is to automate. At some point or the other, the need for Image Recognition with Pyautogui arises, so that we can locate elements on the screen.
Website Link:
coderslegacy.com/python/image...
Mouse Automation: • PyAutoGUI Tutorial - A...
Keyboard Automation: • PyAutoGUI Keyboard Aut...
Image Detection: • Pyautogui image detection
Failsafe Method: • PyAutoGUI Fail Safe - ...
Пікірлер: 60
Wow, I did not realize how easy GUI automation is. Thank you, great tutorial :) Not sure where I might use this but it was interesting to follow
Rất tuyệt vời. Cảm ơn bạn Nhờ có video chia sẻ của bạn mà tôi đã cài đặt được pyautogui tìm kiếm màu sắc trên màn hình ..tôi đã bế tắc suốt 5 ngày mà không tìm được trợ giúp. Chúc bạn sức khoẻ và tạo thêm nhiều video liên quan đến pyautogui❤
This is great for mouse moving or clicking on parts of the UI that do not have any other way to control. In this case you can script a sequence that activates calculator to foreground and then type number 5 as a key command. No need to make screenshots for everything that can be keyboard commanded/automated. Or even exists as shortcuts. It will only slow your system down. All you need to automate is the proper app on the foreground and keyboard numbers as input.
@ErickGarcia-qs2yh
9 ай бұрын
It's just an example, dude
very very helpful, thanks!
Just putting this out there for anyone new to this: I had to reinstall Windows and python today due to a drive failure. (It happened whilst making a backup of my C: drive lol) All of my python scripts that used locateOnScreen() or locateAllOnScreen() crashed when it couldn't find the .png I was looking for, whereas it usually returned 'None'. With the latest version of pygame you need to do this: try: found = pg.locateAllOnScreen(fName,grayscale = True,confidence = .899) for f in found: [do what you want here] except: return False OR try: found = pg.locateOnScreen(fName,grayscale = True,confidence = .899) x = found[0] y = found[1] [do what you want here] except: return False
this is very useful, thanks!
great tutorial
Thank You!
legit lifesaver ❤❤
AMAZING!!!
it is showing no such file or directory exist any solution?
thank you very much I understand how to do this, I am trying to automate the Terraria game
@coderslegacy5661
Жыл бұрын
Sounds interesting. Good luck 🤞
If the image we are seeking in code is hidden (program minimized or covered by other program window), the code crashes. How could I prevent that? I had idea to check in a while loop for active window name and if its not desired program then just pause the code, but I am a begginer and cant quite put that to work together
@coderslegacy5661
Жыл бұрын
Wrap the locate function in a try and catch statement. The code crashes because of an exception being thrown when it can't find an image. You just need to handle it. Look up "exception handling" in python
Hello I'm trying to find an image on screen based on its shape, without caring about the color matching, can you help? thank you.
'm from Brazil, it's a pleasure to meet your channel. I'm running PyautoGUI and even with other open screens or programs, the screenshot or locateCenterOnScreen, it always takes the MAC home page, the opening screen without icons, just the background screen and the top menu. Do you know why and how to solve it?
@marcossacramento5957
5 ай бұрын
meu mano, vc conseguiu resolver? eu to passando uns perrengues pra mexer eh no linux com essa lib
Hi, I have a question. How can I make my program take one action if the detection fails, and a different action if the detection succeeds? Thanks!
@coderslegacy5661
Жыл бұрын
Depends on the function. Some of them might return nothing in the output if they cannot find the image (like a return -1 or return 0). In this case we can use an if statement to first check its value and then take action accordingly. Likewise there are some functions which raise exceptions when they cannot find an image. In this case we can wrap them inside a try-except block. Hope that answered your question.
@the-real-zpero
Жыл бұрын
@@coderslegacy5661 In your example program, how would we make the program print out "could not find target" if it couldn't find the number 5, for example?
@coderslegacy5661
Жыл бұрын
This is what you are looking for. if pyautogui.locateCenterOnScreen("5.jpg") != None: print("Found!") else: print("Not Found")
@the-real-zpero
Жыл бұрын
@@coderslegacy5661 thank you!
Hey, why command confidence not working?
Confidence call not working? can you help me?
I have a question. Let's say for the sake of argument there are two '5' image on the screen. Then which image the software will click? Let's say I want the software to click on both the image on screen one by one. How can it be done? Please make a video on it. Thanks.
@coderslegacy5661
10 ай бұрын
No need for a separate video actually. Everything you need to know is within this video. Let's discuss some possible solutions. Assuming both "5" images are exactly identical, then maybe you can use region parameter to differentiate between them. (e.g the images are on different sides of the screen). Alternatively, another approach would be to find unique elements near each of the "5" images. Example, if one "5" is next to the digit 1, and the other "5" is next to the digit 9. So you can search for these unique elements, then search in the region around this unique elements. (e.g. find the image "1", use its coordinates in the region parameter to search in the near by area). There are other approaches that I can think off, but they really depend on the scenario at hand.
I have an error and this comes up. TypeError: cannot unpack non-iterable NoneType object \ and heres my code: import pyautogui x, y = pyautogui.locateCenterOnScreen("C:\\Users\\user\\PycharmProjects\\Pycharm here\\.vscode\\pyauto\\locateonscreen\\5.png") print(x, y) Can anyone help me?
@coderslegacy5661
Жыл бұрын
This is because the locateCenterOnScreen() function was not able to find the object on the screen, and return None, instead of the usual x and y coordinates. Simply change your code a little. coords = locateCenterOnScreen() if coords != None: print(coords[0], coords[1]) This will handle the error, and prevent any crashes. Now you just need to figure out why the object can't be found.
Where should be those png files be placed/located
@coderslegacy5661
Жыл бұрын
Can be anywhere. I usually keep them in same folder as my Python file, hence I just specify the filename. But if its in a different directory, then you need to use the full file path instead.
@laxify4091
Жыл бұрын
@@coderslegacy5661 how can i write full file path? can you explain with an example pls
@laxify4091
Жыл бұрын
@@coderslegacy5661 make a video on win32api and win32con
@xlobo9822
11 ай бұрын
folder_path = os.getcwd() Next = pyautogui.screenshot(region=(1836, 1033, 50, 40)) Next.save(f'{folder_path}\\Next.png') location = pyautogui.locateOnScreen(f'{folder_path}\\Next.png') pyautogui.click(location) """ IT is my exercise code. You may find the answer ! folder_path = os.getcwd() This sentence will give where your python program working""".
I know this is 2months old video but i hope you see my comment. I downloaded pyautogui and opencv but i cannot use confidence and region feature. They just dont appear when i type it. I posted my problem on my youtube channel. Could you plez look what ive done wrong thx
@coderslegacy5661
Жыл бұрын
Have you actually tried running the code? I notice you didn't do that in your videos. What error are you getting?
@FreestyleSnowman
Жыл бұрын
@@coderslegacy5661 confidence doesn't work... if the image is slightly different pyautogui doesn't recognize it. It has to be 100% same image lol
@coderslegacy5661
Жыл бұрын
@@FreestyleSnowman I imagine something is wrong with your opencv-python installation. Try using regular opencv-python functions that normally come with it. That way you can narrow down the problem. Another tip I have is to update your libraries. Maybe try uninstalling opencv-python and installing it again.
@Fares2012FA
Жыл бұрын
@@coderslegacy5661 hey, how did you use x, y = as a variable ? when I name my first line command after "x, y =" it doesn't work because "TypeError: cannot unpack non-iterable NoneType object" how can I solve that
@coderslegacy5661
Жыл бұрын
@@Fares2012FA My guess is that whatever is on the other side of the = operator is returning "None". Normally we should get a pair of two values, which can be unpacked into two variables (x and y). But "None" is a single value, hence the problem. In short, the actual problem is within the function/value on the right-hand side of the = operator.
W tutorial but if u dont mind me asking where u from
Is there a way to make the program look for the an image and if it doesn't find it, cancel any actions after the image/object search? I am learning Python and decided to build a small game bot to auto loot enemies that are killed. Sometimes, enemies will not drop loot but if there is loot present, I can automate a key press to open the loot container. If there is NO loot, the key press does nothing and a new enemy will spawn and the cycle repeats. If there is loot, it produces a static positioned, new top layer window with a button to take the loot. So sometimes the button will be on screen and sometimes it wont. Regardless, I want the program to look for the button and if it finds it, to perform a left click, but if it doesn't find it, then I don't want it to do anything. QUESTION Can you help with that IF statement? I am having a hard time figuring that out.
@coderslegacy5661
Жыл бұрын
If the image is not found, the locate function would return None. So all you need to do is setup a if-condition that checks the result to see if it's None or not. E.g result = locate() If result != None: # code
how to find pixel and click it
@coderslegacy5661
Жыл бұрын
Do you want to find a specific pixel coordinate on the screen on click it? I suggest you watch this video. It covers how to automate the mouse in PyAutoGUI. E.g: Moving mouse to specific coordinate, clicking it, double click, drag, etc. kzread.info/dash/bejne/ZYyLpsevhde6eJc.html
It worked prwviously,but not now
These videos feel dumb, but they work lol
click() got an unexpected keyword argument 'confidence'
ValueError: too many values to unpack (expected 2)
@Ggaimer2006
3 ай бұрын
u wrote something other than the 2 things u should've wrote
@quark5099
3 ай бұрын
@@Ggaimer2006 I was new back then now I have no issues
hi thanks the video why I get this error: in = pyautogui.locateOnScreen("fullscreen.png") ^^ SyntaxError: invalid syntax (Expected expressionPylance)
@coderslegacy5661
Жыл бұрын
Doesn't seem to be any issue in that particular line. The problem is most likely in the surrounding code. Maybe an indentation problem or something.
@danielkovacs3664
Жыл бұрын
@@coderslegacy5661 this is the all script import pyautogui in = pyautogui.locateOnScreen("fullscreen.png")
@joyousgaming4935
Жыл бұрын
Do you mean to be using locate on screen or screenshot?
@danielkovacs3664
Жыл бұрын
@@joyousgaming4935 locate on a screen
@xlobo9822
11 ай бұрын
"in" is a reserved name in Python and is not available as variable name. Change the name of the variable.