Pyautogui image detection

Automating any kind of task is pretty tricky, and the more complex the task, the harder it is to automate. At some point or the other, the need for Image Recognition with Pyautogui arises, so that we can locate elements on the screen.
Website Link:
coderslegacy.com/python/image...
Mouse Automation: • PyAutoGUI Tutorial - A...
Keyboard Automation: • PyAutoGUI Keyboard Aut...
Image Detection: • Pyautogui image detection
Failsafe Method: • PyAutoGUI Fail Safe - ...

Пікірлер: 60

  • @kimolsson1772
    @kimolsson17722 жыл бұрын

    Wow, I did not realize how easy GUI automation is. Thank you, great tutorial :) Not sure where I might use this but it was interesting to follow

  • @dmina7785
    @dmina778511 ай бұрын

    Rất tuyệt vời. Cảm ơn bạn Nhờ có video chia sẻ của bạn mà tôi đã cài đặt được pyautogui tìm kiếm màu sắc trên màn hình ..tôi đã bế tắc suốt 5 ngày mà không tìm được trợ giúp. Chúc bạn sức khoẻ và tạo thêm nhiều video liên quan đến pyautogui❤

  • @MultiMam12345
    @MultiMam12345 Жыл бұрын

    This is great for mouse moving or clicking on parts of the UI that do not have any other way to control. In this case you can script a sequence that activates calculator to foreground and then type number 5 as a key command. No need to make screenshots for everything that can be keyboard commanded/automated. Or even exists as shortcuts. It will only slow your system down. All you need to automate is the proper app on the foreground and keyboard numbers as input.

  • @ErickGarcia-qs2yh

    @ErickGarcia-qs2yh

    9 ай бұрын

    It's just an example, dude

  • @mumk
    @mumk Жыл бұрын

    very very helpful, thanks!

  • @sickie1961
    @sickie196125 күн бұрын

    Just putting this out there for anyone new to this: I had to reinstall Windows and python today due to a drive failure. (It happened whilst making a backup of my C: drive lol) All of my python scripts that used locateOnScreen() or locateAllOnScreen() crashed when it couldn't find the .png I was looking for, whereas it usually returned 'None'. With the latest version of pygame you need to do this: try: found = pg.locateAllOnScreen(fName,grayscale = True,confidence = .899) for f in found: [do what you want here] except: return False OR try: found = pg.locateOnScreen(fName,grayscale = True,confidence = .899) x = found[0] y = found[1] [do what you want here] except: return False

  • @xilllllix
    @xilllllix Жыл бұрын

    this is very useful, thanks!

  • @csd4ni3l
    @csd4ni3l2 жыл бұрын

    great tutorial

  • @wojtekstaniszewski
    @wojtekstaniszewski5 ай бұрын

    Thank You!

  • @DiNozzo_
    @DiNozzo_8 ай бұрын

    legit lifesaver ❤❤

  • @Howmygamerun
    @Howmygamerun5 ай бұрын

    AMAZING!!!

  • @yes.....
    @yes..... Жыл бұрын

    it is showing no such file or directory exist any solution?

  • @abnerluis9086
    @abnerluis9086 Жыл бұрын

    thank you very much I understand how to do this, I am trying to automate the Terraria game

  • @coderslegacy5661

    @coderslegacy5661

    Жыл бұрын

    Sounds interesting. Good luck 🤞

  • @wilk2727
    @wilk2727 Жыл бұрын

    If the image we are seeking in code is hidden (program minimized or covered by other program window), the code crashes. How could I prevent that? I had idea to check in a while loop for active window name and if its not desired program then just pause the code, but I am a begginer and cant quite put that to work together

  • @coderslegacy5661

    @coderslegacy5661

    Жыл бұрын

    Wrap the locate function in a try and catch statement. The code crashes because of an exception being thrown when it can't find an image. You just need to handle it. Look up "exception handling" in python

  • @makenso9009
    @makenso90094 ай бұрын

    Hello I'm trying to find an image on screen based on its shape, without caring about the color matching, can you help? thank you.

  • @leodmx10
    @leodmx10 Жыл бұрын

    'm from Brazil, it's a pleasure to meet your channel. I'm running PyautoGUI and even with other open screens or programs, the screenshot or locateCenterOnScreen, it always takes the MAC home page, the opening screen without icons, just the background screen and the top menu. Do you know why and how to solve it?

  • @marcossacramento5957

    @marcossacramento5957

    5 ай бұрын

    meu mano, vc conseguiu resolver? eu to passando uns perrengues pra mexer eh no linux com essa lib

  • @the-real-zpero
    @the-real-zpero Жыл бұрын

    Hi, I have a question. How can I make my program take one action if the detection fails, and a different action if the detection succeeds? Thanks!

  • @coderslegacy5661

    @coderslegacy5661

    Жыл бұрын

    Depends on the function. Some of them might return nothing in the output if they cannot find the image (like a return -1 or return 0). In this case we can use an if statement to first check its value and then take action accordingly. Likewise there are some functions which raise exceptions when they cannot find an image. In this case we can wrap them inside a try-except block. Hope that answered your question.

  • @the-real-zpero

    @the-real-zpero

    Жыл бұрын

    @@coderslegacy5661 In your example program, how would we make the program print out "could not find target" if it couldn't find the number 5, for example?

  • @coderslegacy5661

    @coderslegacy5661

    Жыл бұрын

    This is what you are looking for. if pyautogui.locateCenterOnScreen("5.jpg") != None: print("Found!") else: print("Not Found")

  • @the-real-zpero

    @the-real-zpero

    Жыл бұрын

    @@coderslegacy5661 thank you!

  • @myelico
    @myelico Жыл бұрын

    Hey, why command confidence not working?

  • @Zjerox
    @Zjerox Жыл бұрын

    Confidence call not working? can you help me?

  • @prakashrathod4877
    @prakashrathod487710 ай бұрын

    I have a question. Let's say for the sake of argument there are two '5' image on the screen. Then which image the software will click? Let's say I want the software to click on both the image on screen one by one. How can it be done? Please make a video on it. Thanks.

  • @coderslegacy5661

    @coderslegacy5661

    10 ай бұрын

    No need for a separate video actually. Everything you need to know is within this video. Let's discuss some possible solutions. Assuming both "5" images are exactly identical, then maybe you can use region parameter to differentiate between them. (e.g the images are on different sides of the screen). Alternatively, another approach would be to find unique elements near each of the "5" images. Example, if one "5" is next to the digit 1, and the other "5" is next to the digit 9. So you can search for these unique elements, then search in the region around this unique elements. (e.g. find the image "1", use its coordinates in the region parameter to search in the near by area). There are other approaches that I can think off, but they really depend on the scenario at hand.

  • @omarsplaytimechannel
    @omarsplaytimechannel Жыл бұрын

    I have an error and this comes up. TypeError: cannot unpack non-iterable NoneType object \ and heres my code: import pyautogui x, y = pyautogui.locateCenterOnScreen("C:\\Users\\user\\PycharmProjects\\Pycharm here\\.vscode\\pyauto\\locateonscreen\\5.png") print(x, y) Can anyone help me?

  • @coderslegacy5661

    @coderslegacy5661

    Жыл бұрын

    This is because the locateCenterOnScreen() function was not able to find the object on the screen, and return None, instead of the usual x and y coordinates. Simply change your code a little. coords = locateCenterOnScreen() if coords != None: print(coords[0], coords[1]) This will handle the error, and prevent any crashes. Now you just need to figure out why the object can't be found.

  • @laxify4091
    @laxify4091 Жыл бұрын

    Where should be those png files be placed/located

  • @coderslegacy5661

    @coderslegacy5661

    Жыл бұрын

    Can be anywhere. I usually keep them in same folder as my Python file, hence I just specify the filename. But if its in a different directory, then you need to use the full file path instead.

  • @laxify4091

    @laxify4091

    Жыл бұрын

    @@coderslegacy5661 how can i write full file path? can you explain with an example pls

  • @laxify4091

    @laxify4091

    Жыл бұрын

    @@coderslegacy5661 make a video on win32api and win32con

  • @xlobo9822

    @xlobo9822

    11 ай бұрын

    folder_path = os.getcwd() Next = pyautogui.screenshot(region=(1836, 1033, 50, 40)) Next.save(f'{folder_path}\\Next.png') location = pyautogui.locateOnScreen(f'{folder_path}\\Next.png') pyautogui.click(location) """ IT is my exercise code. You may find the answer ! folder_path = os.getcwd() This sentence will give where your python program working""".

  • @FreestyleSnowman
    @FreestyleSnowman Жыл бұрын

    I know this is 2months old video but i hope you see my comment. I downloaded pyautogui and opencv but i cannot use confidence and region feature. They just dont appear when i type it. I posted my problem on my youtube channel. Could you plez look what ive done wrong thx

  • @coderslegacy5661

    @coderslegacy5661

    Жыл бұрын

    Have you actually tried running the code? I notice you didn't do that in your videos. What error are you getting?

  • @FreestyleSnowman

    @FreestyleSnowman

    Жыл бұрын

    @@coderslegacy5661 confidence doesn't work... if the image is slightly different pyautogui doesn't recognize it. It has to be 100% same image lol

  • @coderslegacy5661

    @coderslegacy5661

    Жыл бұрын

    @@FreestyleSnowman I imagine something is wrong with your opencv-python installation. Try using regular opencv-python functions that normally come with it. That way you can narrow down the problem. Another tip I have is to update your libraries. Maybe try uninstalling opencv-python and installing it again.

  • @Fares2012FA

    @Fares2012FA

    Жыл бұрын

    @@coderslegacy5661 hey, how did you use x, y = as a variable ? when I name my first line command after "x, y =" it doesn't work because "TypeError: cannot unpack non-iterable NoneType object" how can I solve that

  • @coderslegacy5661

    @coderslegacy5661

    Жыл бұрын

    @@Fares2012FA My guess is that whatever is on the other side of the = operator is returning "None". Normally we should get a pair of two values, which can be unpacked into two variables (x and y). But "None" is a single value, hence the problem. In short, the actual problem is within the function/value on the right-hand side of the = operator.

  • @xspam_4137
    @xspam_41374 ай бұрын

    W tutorial but if u dont mind me asking where u from

  • @johnd7423
    @johnd7423 Жыл бұрын

    Is there a way to make the program look for the an image and if it doesn't find it, cancel any actions after the image/object search? I am learning Python and decided to build a small game bot to auto loot enemies that are killed. Sometimes, enemies will not drop loot but if there is loot present, I can automate a key press to open the loot container. If there is NO loot, the key press does nothing and a new enemy will spawn and the cycle repeats. If there is loot, it produces a static positioned, new top layer window with a button to take the loot. So sometimes the button will be on screen and sometimes it wont. Regardless, I want the program to look for the button and if it finds it, to perform a left click, but if it doesn't find it, then I don't want it to do anything. QUESTION Can you help with that IF statement? I am having a hard time figuring that out.

  • @coderslegacy5661

    @coderslegacy5661

    Жыл бұрын

    If the image is not found, the locate function would return None. So all you need to do is setup a if-condition that checks the result to see if it's None or not. E.g result = locate() If result != None: # code

  • @TheAlibaadshah
    @TheAlibaadshah Жыл бұрын

    how to find pixel and click it

  • @coderslegacy5661

    @coderslegacy5661

    Жыл бұрын

    Do you want to find a specific pixel coordinate on the screen on click it? I suggest you watch this video. It covers how to automate the mouse in PyAutoGUI. E.g: Moving mouse to specific coordinate, clicking it, double click, drag, etc. kzread.info/dash/bejne/ZYyLpsevhde6eJc.html

  • @ebooo7368
    @ebooo7368 Жыл бұрын

    It worked prwviously,but not now

  • @deadpopcorn8523
    @deadpopcorn8523 Жыл бұрын

    These videos feel dumb, but they work lol

  • @Zjerox
    @Zjerox Жыл бұрын

    click() got an unexpected keyword argument 'confidence'

  • @quark5099
    @quark50999 ай бұрын

    ValueError: too many values to unpack (expected 2)

  • @Ggaimer2006

    @Ggaimer2006

    3 ай бұрын

    u wrote something other than the 2 things u should've wrote

  • @quark5099

    @quark5099

    3 ай бұрын

    ​@@Ggaimer2006 I was new back then now I have no issues

  • @danielkovacs3664
    @danielkovacs3664 Жыл бұрын

    hi thanks the video why I get this error: in = pyautogui.locateOnScreen("fullscreen.png") ^^ SyntaxError: invalid syntax (Expected expressionPylance)

  • @coderslegacy5661

    @coderslegacy5661

    Жыл бұрын

    Doesn't seem to be any issue in that particular line. The problem is most likely in the surrounding code. Maybe an indentation problem or something.

  • @danielkovacs3664

    @danielkovacs3664

    Жыл бұрын

    @@coderslegacy5661 this is the all script import pyautogui in = pyautogui.locateOnScreen("fullscreen.png")

  • @joyousgaming4935

    @joyousgaming4935

    Жыл бұрын

    Do you mean to be using locate on screen or screenshot?

  • @danielkovacs3664

    @danielkovacs3664

    Жыл бұрын

    @@joyousgaming4935 locate on a screen

  • @xlobo9822

    @xlobo9822

    11 ай бұрын

    "in" is a reserved name in Python and is not available as variable name. Change the name of the variable.