Real Time Sign Language Recognition using Python | Computer Vision
Тәжірибелік нұсқаулар және стиль
#python #programming #tensorflow this video I show how to build a real time sign language recognition system using Google's Mediapipe framework and Tensorflow in python. We'll be using an open-source github repo which allows us to easily train a Keras neural network to detect our own custom hand gestures, and it runs remarkably well on a CPU.
Github repo (translated to English) :github.com/kinivi/hand-gestur...
Github repo (original version in Japanese) : github.com/Kazuhito00/hand-ge...
Chapters:
0:00 Video structure
0:30 High level overview of the project
2:18 Github repo
3:28 Requirements
6:28 Collecting data
8:25 Training
9:43 Testing
Requirements:
mediapipe
scikit-learn
tensorflow
numpy
pandas
matplotlib
opencv-python
seaborn
Пікірлер: 81
Allahuma barik, amazing stuff bro👍💯
@abdul.rehman.ikram1458
Жыл бұрын
Thanks a lot man !! 💯💯
Thankyou❤️
Which application are you using?
hiii im getting error that file ValueError: The filepath provided must end in `.keras` (Keras model format). Received: filepath=model/keypoint_classifier/keypoint_classifier.hdf5
@abdul.rehman.ikram1458
2 ай бұрын
Seems like a file extension name issue, did you make an edit somewhere?
@user-ng9nn3kk6i
2 ай бұрын
Bro, with the help of chatgpt i found the solution # Define the model save path with the .keras extension model_save_path = "model/keypoint_classifier/keypoint_classifier.keras" # Model checkpoint callback cp_callback = tf.keras.callbacks.ModelCheckpoint( model_save_path, verbose=1, save_weights_only=False) # Early stopping callback es_callback = tf.keras.callbacks.EarlyStopping(patience=20, verbose=1) add this in a new cell before the error cell
Is it possible to implement this in a web app? Like use the webcam in the browser
@abdul.rehman.ikram1458
5 ай бұрын
Yes, it would be possible. I have never tried it myself but I know that react allows you to us ml/dl libraries
If you knew how much you made me happy 🥰🥰😘😘😘
Hi, is there any possible way to contact you? I just wanted to ask some questions regarding the number of keyframe. Thank you!
@abdul.rehman.ikram1458
8 ай бұрын
Sure, my email is abdulrehmanikram9710@gmail.com , but i don't check it often, insta abdul.rehman.ikram
@Alyssa-ib9hc
8 ай бұрын
Alright, I'll message you real quick. Thank you sm! @@abdul.rehman.ikram1458
@Alyssa-ib9hc
8 ай бұрын
@@abdul.rehman.ikram1458 Hi, I already messaged you on insta 😊
Is it possible to make it work with more than 10 classifiers? I want to build the alphabet of mine sign language.
@abdul.rehman.ikram1458
2 ай бұрын
Hey, you can still use media pipe for identifying the hand landmarks but you have to train a network to identify as many signs you want. For simplicity purposes this video used an existing repo already setup to do everything however you can use it for reference to build your own as you like
Bro if the screen size of pc changes will it affect the cordinates and prediction
@abdul.rehman.ikram1458
5 ай бұрын
I Dont think that will affect the outcome too much as long as the volume of data is good and you have a lot variations
if i wanted to convert the sign language to speech what should i work on? and is it possible to edit on this code ?
@abdul.rehman.ikram1458
6 ай бұрын
Yes that would be doable , for that you'd have to go with the same approach. It would still be a classification problem so there would be a label associated with each sign, train a neural net to classify the sign. Separately you need maybe a table or a dictionary to map each (any sort of mapping ) to any desired text of your choice. Finally use a Text To Speech library to convert that piece of text to speech. Hope that helped
hi, i did what u said. But ive a problem. It recognizes only 3 signs and i registeres 10. I changed these numbers from 3 to 10, but still it doesnt work
@abdul.rehman.ikram1458
Жыл бұрын
hello did you follow all the steps i mentioned? Did you place the labels in the correct order and keep in mind they start with index 0, when you are collecting data you have to press the key in your keyboard according to the index of the label in the file. Make sure you have collected enough data. Lastly make sure to follow step in 9:22 , you have to change num_classes to number of labels you had in your file
Would you know where in your code I could gather the "percentage" of each hand gesture? Maybe it is in the classifier?
@scottlee38
6 ай бұрын
scratch that, I figured it out!
@mahechowdury
5 ай бұрын
@@scottlee38 how? tell me
Hi,can anybody tell me ,if i want to add more than 10 gestures than what are the possible solutions that i can follow??
@raulostonaljr.6653
9 ай бұрын
up
@asishpanigrahi1569
8 ай бұрын
@@raulostonaljr.6653 what does it means
@ruszjea
7 ай бұрын
up
at 6.42 login key point is not showing nor it is saving the csv values ,can you help
@abdul.rehman.ikram1458
8 ай бұрын
Did you install all of the required modules ?
@asishpanigrahi1569
8 ай бұрын
@@abdul.rehman.ikram1458 hey brother problem solved.
@asishpanigrahi1569
8 ай бұрын
Brother I have followed you on Instagram can we contact??
hi, what are the solution if i want register more than 10 gesture
@asishpanigrahi1569
8 ай бұрын
I also have the same doubt
How can i download the required things like tensow flow in my vs visual studio code
@abdul.rehman.ikram1458
7 ай бұрын
There are a lot of tutorials available on KZread , try searching how to download tensorflow on KZread . For rest of the ones it should be simple just go to terminal type pip install then the package names , tip you can install multiple packages in a single command by typing pip Install followed by package names separated by spaces
@amar-un4bg
7 ай бұрын
But they are saying error:could not find a version that satisfies the requirement tensorflow
i have this error Traceback (most recent call last): File "/home/vit/Escritorio/Encuentro/Lenguaje_vocales/hand-gesture-recognition-mediapipe/app.py", line 543, in main() File "/home/vit/Escritorio/Encuentro/Lenguaje_vocales/hand-gesture-recognition-mediapipe/app.py", line 168, in main keypoint_classifier_labels[hand_sign_id], IndexError: list index out of range any ideas?
@abdul.rehman.ikram1458
2 ай бұрын
How many sign labels did you add?
@raghavank1447
Ай бұрын
Sir @@abdul.rehman.ikram1458i add 2 labels but shows index out range
While converting to tflite, the kernel always dies and restarts. How to encounter this problem? and also instead of jupyter can we use google collab, if yes then how? Please help.
@revasingh3732
2 ай бұрын
if youre facing this problem on jupyter then try making an environment
@Luxulight
Ай бұрын
Hello, I am also facing the same issue. May I ask if you have resolved it?
@revasingh3732
Ай бұрын
@@Luxulight yes
Hi um how can install mediapipe each time i try to install it theres an error is there a specifc python version that works on it im currently using the 3.12
@abdul.rehman.ikram1458
8 ай бұрын
Try to use a different version of python , it should work .
I'm unable to find the package sci-kit? Should I be installing scikit-learn?
@abdul.rehman.ikram1458
10 ай бұрын
Yes my bad it is scimitar-learn, thanks a lot for pointing out. I will update the description
sir how can we convert the single letter to WORDS..
@abdul.rehman.ikram1458
5 ай бұрын
I did show exactly that in this video, the video shows how to classify hand sign as words
When I run the app.py, there's an error saying that it can't open the 'keypoint_classifier.tflite' folder. Please help
@abdul.rehman.ikram1458
4 ай бұрын
What does the error say exactly
@gigmo1235
3 ай бұрын
@@abdul.rehman.ikram1458 Hi thanks for the reply, I've downloaded the code and when i run the app.py the error says: Exception has occurred: ValueError Could not open 'model/keypoint_classifier/keypoint_classifier.tflite'.
@gigmo1235
3 ай бұрын
Also I downloaded it to visual studio code
@abdul.rehman.ikram1458
3 ай бұрын
@@gigmo1235 hello , have you downloaded the key point classifier file ?
@gigmo1235
3 ай бұрын
@@abdul.rehman.ikram1458 Yes, the text file 'keypoint_classifier.tflite' says that the file cannot be displayed in the text editor because it is either in binary or uses an unsupported text encoding
how should i add more than 10 signs?
@abdul.rehman.ikram1458
2 ай бұрын
Hey, you can still use media pipe for identifying the hand landmarks but you have to train a network to identify as many signs you want. For simplicity purposes this video used an existing repo already setup to do everything however you can use it for reference to build your own as you like
We need web camera for this?
@abdul.rehman.ikram1458
4 ай бұрын
You can use the one built into your computer , if there isn’t one then yes you might have to buy it separately
What algorithms did u use
@abdul.rehman.ikram1458
Жыл бұрын
Hey as I mentioned in the beginning of the video at 0:32 there are two parts to this , initially I made use of the google media pipe framework and cnn which identifies the hand part of the frame/image and then identifies the locations of joints in real time, then that information is used to train a standard neural network to identify different signs, so as for your answer this video used supervised learning algorithms
Bro iam a final year student.My project is about sign language trabslator can you please tell how to train all alphabet and number i can't do more than 9.please tell me its urgent
@abdul.rehman.ikram1458
5 ай бұрын
Try building a feed forward neural network and just train it to classify whatever data you want, it should be simple
@adharshreghu3270
5 ай бұрын
@@abdul.rehman.ikram1458 sorry bro i didn't understand 😅. Will i be able to do it by changing the code above
I got error in jupyter notebook which saya cp_callback is not defined. I follow all your step. Can you please help 🙏🏻
@user-ng9nn3kk6i
2 ай бұрын
+1
Is possible to turn this project into a website?
@abdul.rehman.ikram1458
5 ай бұрын
Yes it is possible, look into react. It does allow you to use ml/dl libraries
@madhesh16_
5 ай бұрын
@@abdul.rehman.ikram1458 thank you man👍🏼
nice video but in command model .fit it shows an error
@7fk-_q622
8 ай бұрын
Jupyter notebook. I have an error in "model.fit( X_train, y_train, epochs=1000, batch_size=128, validation_data=(X_test, y_test), callbacks=[cp_callback, es_callback] )" It said "InvalidArgumentError Traceback (most recent call last) Cell In[32], line 1" I don't understand why and how to fix this problem, can you help me with this error? Thank you very much!
@abdul.rehman.ikram1458
8 ай бұрын
Did you make any changes to file other than the ones specified in the video ?
any way to train more than 10 gestures
@abdul.rehman.ikram1458
5 ай бұрын
Yes, you'd have to build a simple feed forward network that would be more than enough you'd have to train it to learn any number of gestures you want, you can use the existing repo for reference
@alonneumann8738
5 ай бұрын
how can I do that? cant I just change the num of max_gestures and train more than 10?@@abdul.rehman.ikram1458
is not work properly. I can't take input
@abdul.rehman.ikram1458
7 ай бұрын
Did you follow all the steps properly? You have to press k to enter key logging mode