Real Time Sign Language Recognition using Python | Computer Vision

Тәжірибелік нұсқаулар және стиль

#python #programming #tensorflow this video I show how to build a real time sign language recognition system using Google's Mediapipe framework and Tensorflow in python. We'll be using an open-source github repo which allows us to easily train a Keras neural network to detect our own custom hand gestures, and it runs remarkably well on a CPU.
Github repo (translated to English) :github.com/kinivi/hand-gestur...
Github repo (original version in Japanese) : github.com/Kazuhito00/hand-ge...
Chapters:
0:00 Video structure
0:30 High level overview of the project
2:18 Github repo
3:28 Requirements
6:28 Collecting data
8:25 Training
9:43 Testing
Requirements:
mediapipe
scikit-learn
tensorflow
numpy
pandas
matplotlib
opencv-python
seaborn

Пікірлер: 81

  • @themindawakener
    @themindawakener Жыл бұрын

    Allahuma barik, amazing stuff bro👍💯

  • @abdul.rehman.ikram1458

    @abdul.rehman.ikram1458

    Жыл бұрын

    Thanks a lot man !! 💯💯

  • @lumacangjessica9099
    @lumacangjessica909911 ай бұрын

    Thankyou❤️

  • @karthikgowdasn2270
    @karthikgowdasn227017 күн бұрын

    Which application are you using?

  • @singlegamer8707
    @singlegamer87073 ай бұрын

    hiii im getting error that file ValueError: The filepath provided must end in `.keras` (Keras model format). Received: filepath=model/keypoint_classifier/keypoint_classifier.hdf5

  • @abdul.rehman.ikram1458

    @abdul.rehman.ikram1458

    2 ай бұрын

    Seems like a file extension name issue, did you make an edit somewhere?

  • @user-ng9nn3kk6i

    @user-ng9nn3kk6i

    2 ай бұрын

    Bro, with the help of chatgpt i found the solution # Define the model save path with the .keras extension model_save_path = "model/keypoint_classifier/keypoint_classifier.keras" # Model checkpoint callback cp_callback = tf.keras.callbacks.ModelCheckpoint( model_save_path, verbose=1, save_weights_only=False) # Early stopping callback es_callback = tf.keras.callbacks.EarlyStopping(patience=20, verbose=1) add this in a new cell before the error cell

  • @clarence3735
    @clarence37356 ай бұрын

    Is it possible to implement this in a web app? Like use the webcam in the browser

  • @abdul.rehman.ikram1458

    @abdul.rehman.ikram1458

    5 ай бұрын

    Yes, it would be possible. I have never tried it myself but I know that react allows you to us ml/dl libraries

  • @mhmdaboalhwda
    @mhmdaboalhwda2 ай бұрын

    If you knew how much you made me happy 🥰🥰😘😘😘

  • @Alyssa-ib9hc
    @Alyssa-ib9hc8 ай бұрын

    Hi, is there any possible way to contact you? I just wanted to ask some questions regarding the number of keyframe. Thank you!

  • @abdul.rehman.ikram1458

    @abdul.rehman.ikram1458

    8 ай бұрын

    Sure, my email is abdulrehmanikram9710@gmail.com , but i don't check it often, insta abdul.rehman.ikram

  • @Alyssa-ib9hc

    @Alyssa-ib9hc

    8 ай бұрын

    Alright, I'll message you real quick. Thank you sm! @@abdul.rehman.ikram1458

  • @Alyssa-ib9hc

    @Alyssa-ib9hc

    8 ай бұрын

    @@abdul.rehman.ikram1458 Hi, I already messaged you on insta 😊

  • @user-vd7ee8ob3b
    @user-vd7ee8ob3b4 ай бұрын

    Is it possible to make it work with more than 10 classifiers? I want to build the alphabet of mine sign language.

  • @abdul.rehman.ikram1458

    @abdul.rehman.ikram1458

    2 ай бұрын

    Hey, you can still use media pipe for identifying the hand landmarks but you have to train a network to identify as many signs you want. For simplicity purposes this video used an existing repo already setup to do everything however you can use it for reference to build your own as you like

  • @arf_atelier1819
    @arf_atelier18195 ай бұрын

    Bro if the screen size of pc changes will it affect the cordinates and prediction

  • @abdul.rehman.ikram1458

    @abdul.rehman.ikram1458

    5 ай бұрын

    I Dont think that will affect the outcome too much as long as the volume of data is good and you have a lot variations

  • @raghad5932
    @raghad59326 ай бұрын

    if i wanted to convert the sign language to speech what should i work on? and is it possible to edit on this code ?

  • @abdul.rehman.ikram1458

    @abdul.rehman.ikram1458

    6 ай бұрын

    Yes that would be doable , for that you'd have to go with the same approach. It would still be a classification problem so there would be a label associated with each sign, train a neural net to classify the sign. Separately you need maybe a table or a dictionary to map each (any sort of mapping ) to any desired text of your choice. Finally use a Text To Speech library to convert that piece of text to speech. Hope that helped

  • @user-tx7nz7tz8y
    @user-tx7nz7tz8y Жыл бұрын

    hi, i did what u said. But ive a problem. It recognizes only 3 signs and i registeres 10. I changed these numbers from 3 to 10, but still it doesnt work

  • @abdul.rehman.ikram1458

    @abdul.rehman.ikram1458

    Жыл бұрын

    hello did you follow all the steps i mentioned? Did you place the labels in the correct order and keep in mind they start with index 0, when you are collecting data you have to press the key in your keyboard according to the index of the label in the file. Make sure you have collected enough data. Lastly make sure to follow step in 9:22 , you have to change num_classes to number of labels you had in your file

  • @scottlee38
    @scottlee386 ай бұрын

    Would you know where in your code I could gather the "percentage" of each hand gesture? Maybe it is in the classifier?

  • @scottlee38

    @scottlee38

    6 ай бұрын

    scratch that, I figured it out!

  • @mahechowdury

    @mahechowdury

    5 ай бұрын

    @@scottlee38 how? tell me

  • @Hemparikh-sm7tf
    @Hemparikh-sm7tf10 ай бұрын

    Hi,can anybody tell me ,if i want to add more than 10 gestures than what are the possible solutions that i can follow??

  • @raulostonaljr.6653

    @raulostonaljr.6653

    9 ай бұрын

    up

  • @asishpanigrahi1569

    @asishpanigrahi1569

    8 ай бұрын

    @@raulostonaljr.6653 what does it means

  • @ruszjea

    @ruszjea

    7 ай бұрын

    up

  • @asishpanigrahi1569
    @asishpanigrahi15698 ай бұрын

    at 6.42 login key point is not showing nor it is saving the csv values ,can you help

  • @abdul.rehman.ikram1458

    @abdul.rehman.ikram1458

    8 ай бұрын

    Did you install all of the required modules ?

  • @asishpanigrahi1569

    @asishpanigrahi1569

    8 ай бұрын

    @@abdul.rehman.ikram1458 hey brother problem solved.

  • @asishpanigrahi1569

    @asishpanigrahi1569

    8 ай бұрын

    Brother I have followed you on Instagram can we contact??

  • @adamairil8848
    @adamairil88488 ай бұрын

    hi, what are the solution if i want register more than 10 gesture

  • @asishpanigrahi1569

    @asishpanigrahi1569

    8 ай бұрын

    I also have the same doubt

  • @amar-un4bg
    @amar-un4bg7 ай бұрын

    How can i download the required things like tensow flow in my vs visual studio code

  • @abdul.rehman.ikram1458

    @abdul.rehman.ikram1458

    7 ай бұрын

    There are a lot of tutorials available on KZread , try searching how to download tensorflow on KZread . For rest of the ones it should be simple just go to terminal type pip install then the package names , tip you can install multiple packages in a single command by typing pip Install followed by package names separated by spaces

  • @amar-un4bg

    @amar-un4bg

    7 ай бұрын

    But they are saying error:could not find a version that satisfies the requirement tensorflow

  • @evilmeerkat29
    @evilmeerkat293 ай бұрын

    i have this error Traceback (most recent call last): File "/home/vit/Escritorio/Encuentro/Lenguaje_vocales/hand-gesture-recognition-mediapipe/app.py", line 543, in main() File "/home/vit/Escritorio/Encuentro/Lenguaje_vocales/hand-gesture-recognition-mediapipe/app.py", line 168, in main keypoint_classifier_labels[hand_sign_id], IndexError: list index out of range any ideas?

  • @abdul.rehman.ikram1458

    @abdul.rehman.ikram1458

    2 ай бұрын

    How many sign labels did you add?

  • @raghavank1447

    @raghavank1447

    Ай бұрын

    Sir ​@@abdul.rehman.ikram1458i add 2 labels but shows index out range

  • @SumitYadav-yz4de
    @SumitYadav-yz4de2 ай бұрын

    While converting to tflite, the kernel always dies and restarts. How to encounter this problem? and also instead of jupyter can we use google collab, if yes then how? Please help.

  • @revasingh3732

    @revasingh3732

    2 ай бұрын

    if youre facing this problem on jupyter then try making an environment

  • @Luxulight

    @Luxulight

    Ай бұрын

    Hello, I am also facing the same issue. May I ask if you have resolved it?

  • @revasingh3732

    @revasingh3732

    Ай бұрын

    @@Luxulight yes

  • @shifu6994
    @shifu69948 ай бұрын

    Hi um how can install mediapipe each time i try to install it theres an error is there a specifc python version that works on it im currently using the 3.12

  • @abdul.rehman.ikram1458

    @abdul.rehman.ikram1458

    8 ай бұрын

    Try to use a different version of python , it should work .

  • @sandhyarajesh7931
    @sandhyarajesh793110 ай бұрын

    I'm unable to find the package sci-kit? Should I be installing scikit-learn?

  • @abdul.rehman.ikram1458

    @abdul.rehman.ikram1458

    10 ай бұрын

    Yes my bad it is scimitar-learn, thanks a lot for pointing out. I will update the description

  • @debarghyakundu908
    @debarghyakundu9085 ай бұрын

    sir how can we convert the single letter to WORDS..

  • @abdul.rehman.ikram1458

    @abdul.rehman.ikram1458

    5 ай бұрын

    I did show exactly that in this video, the video shows how to classify hand sign as words

  • @gigmo1235
    @gigmo12354 ай бұрын

    When I run the app.py, there's an error saying that it can't open the 'keypoint_classifier.tflite' folder. Please help

  • @abdul.rehman.ikram1458

    @abdul.rehman.ikram1458

    4 ай бұрын

    What does the error say exactly

  • @gigmo1235

    @gigmo1235

    3 ай бұрын

    @@abdul.rehman.ikram1458 Hi thanks for the reply, I've downloaded the code and when i run the app.py the error says: Exception has occurred: ValueError Could not open 'model/keypoint_classifier/keypoint_classifier.tflite'.

  • @gigmo1235

    @gigmo1235

    3 ай бұрын

    Also I downloaded it to visual studio code

  • @abdul.rehman.ikram1458

    @abdul.rehman.ikram1458

    3 ай бұрын

    @@gigmo1235 hello , have you downloaded the key point classifier file ?

  • @gigmo1235

    @gigmo1235

    3 ай бұрын

    @@abdul.rehman.ikram1458 Yes, the text file 'keypoint_classifier.tflite' says that the file cannot be displayed in the text editor because it is either in binary or uses an unsupported text encoding

  • @adizZz..
    @adizZz..3 ай бұрын

    how should i add more than 10 signs?

  • @abdul.rehman.ikram1458

    @abdul.rehman.ikram1458

    2 ай бұрын

    Hey, you can still use media pipe for identifying the hand landmarks but you have to train a network to identify as many signs you want. For simplicity purposes this video used an existing repo already setup to do everything however you can use it for reference to build your own as you like

  • @navasdeen634
    @navasdeen6344 ай бұрын

    We need web camera for this?

  • @abdul.rehman.ikram1458

    @abdul.rehman.ikram1458

    4 ай бұрын

    You can use the one built into your computer , if there isn’t one then yes you might have to buy it separately

  • @lumacangjessica9099
    @lumacangjessica9099 Жыл бұрын

    What algorithms did u use

  • @abdul.rehman.ikram1458

    @abdul.rehman.ikram1458

    Жыл бұрын

    Hey as I mentioned in the beginning of the video at 0:32 there are two parts to this , initially I made use of the google media pipe framework and cnn which identifies the hand part of the frame/image and then identifies the locations of joints in real time, then that information is used to train a standard neural network to identify different signs, so as for your answer this video used supervised learning algorithms

  • @adharshreghu3270
    @adharshreghu32705 ай бұрын

    Bro iam a final year student.My project is about sign language trabslator can you please tell how to train all alphabet and number i can't do more than 9.please tell me its urgent

  • @abdul.rehman.ikram1458

    @abdul.rehman.ikram1458

    5 ай бұрын

    Try building a feed forward neural network and just train it to classify whatever data you want, it should be simple

  • @adharshreghu3270

    @adharshreghu3270

    5 ай бұрын

    @@abdul.rehman.ikram1458 sorry bro i didn't understand 😅. Will i be able to do it by changing the code above

  • @dannieycool4252
    @dannieycool42522 ай бұрын

    I got error in jupyter notebook which saya cp_callback is not defined. I follow all your step. Can you please help 🙏🏻

  • @user-ng9nn3kk6i

    @user-ng9nn3kk6i

    2 ай бұрын

    +1

  • @madhesh16_
    @madhesh16_5 ай бұрын

    Is possible to turn this project into a website?

  • @abdul.rehman.ikram1458

    @abdul.rehman.ikram1458

    5 ай бұрын

    Yes it is possible, look into react. It does allow you to use ml/dl libraries

  • @madhesh16_

    @madhesh16_

    5 ай бұрын

    @@abdul.rehman.ikram1458 thank you man👍🏼

  • @7fk-_q622
    @7fk-_q6228 ай бұрын

    nice video but in command model .fit it shows an error

  • @7fk-_q622

    @7fk-_q622

    8 ай бұрын

    Jupyter notebook. I have an error in "model.fit( X_train, y_train, epochs=1000, batch_size=128, validation_data=(X_test, y_test), callbacks=[cp_callback, es_callback] )" It said "InvalidArgumentError Traceback (most recent call last) Cell In[32], line 1" I don't understand why and how to fix this problem, can you help me with this error? Thank you very much!

  • @abdul.rehman.ikram1458

    @abdul.rehman.ikram1458

    8 ай бұрын

    Did you make any changes to file other than the ones specified in the video ?

  • @MarkEugeneDulay-oi5iu
    @MarkEugeneDulay-oi5iu5 ай бұрын

    any way to train more than 10 gestures

  • @abdul.rehman.ikram1458

    @abdul.rehman.ikram1458

    5 ай бұрын

    Yes, you'd have to build a simple feed forward network that would be more than enough you'd have to train it to learn any number of gestures you want, you can use the existing repo for reference

  • @alonneumann8738

    @alonneumann8738

    5 ай бұрын

    how can I do that? cant I just change the num of max_gestures and train more than 10?@@abdul.rehman.ikram1458

  • @sm-datascience
    @sm-datascience7 ай бұрын

    is not work properly. I can't take input

  • @abdul.rehman.ikram1458

    @abdul.rehman.ikram1458

    7 ай бұрын

    Did you follow all the steps properly? You have to press k to enter key logging mode

Келесі