very useful tutorial with potential bug explainations, thank you!

This is the only video that explained me pytorch to onnx export and deployment on web clearly. Thanks for your contribution brother. Big fan of your work. Please do make more contents like this brother. May God bless you❤.

@elliotwaite

2 жыл бұрын

Thank you, Santosh. It means a lot to me to read this.

@santoshmohanram536

2 жыл бұрын

@@elliotwaite 🙏🙏

Excellent and concise, thank you!

I'm shocked that this video only has 5k views. Truly one of the better CS tutorials I've seen, especially considering it's a topic as dense as ML. It's also funny as I just had a an assignment on the MNIST dataset and was going to make a video trying to add it to my website, but it looks like you beat me to it :)

@elliotwaite

3 жыл бұрын

Thanks, Max!

Really Interesting. Thank you for sharing :)

love it! your content is so original

Excellent video. Thanks!

What a great tutorial. Clear, concise and handles some common pitfalls. Plus yo ulook like Matt Damon. Awesome :D

@elliotwaite

4 жыл бұрын

Haha, some people say I'm the abandoned love child of Matt Damon and Mark Wahlberg.

Dope background g! 💪

@elliotwaite

4 жыл бұрын

Yeee, thanks g! Gotta have that futuristic neon glow.

Wow this is great. I would love to see running a pytorch model in Android video because there's not a lot on that out there atm.

@elliotwaite

4 жыл бұрын

Thanks for the feedback. I'll add that to my list of potential future videos.

watched first 30 seconds, said f*ckin cool...good sign...ill continue to watch now :)

Hi! Thanks for your nice tutorial, Is it possible to save the ONNX model in the webbrowser indexeddb while each time we refresh the page or open it again we no need to wait for a model to be downloaded.

Awesome bro...!!!!!!

@elliotwaite

4 жыл бұрын

Thanks, Bharath!

Great video bro , Could you make a video on quantization and deploying large models to the web

@elliotwaite

4 жыл бұрын

Thanks for the suggestion! I'll add that to the list of good potential videos to make. I haven't learned much about those topics yet, but I'll probably explore them more in the future.

Thank you for making this video, it is very helpful for somebody new. I am trying to load my image segmentation model created using fastAi. I ran into the problem where the data was not structured properly for my model. You fixed it by modifying your forward function to repackage the data. Currently I do not have a forward function in my code and am trying to learn how to add one. If I did not have one when I trained my model the first time, will I have to retrain it? I wonder if I can simply tack on a forward function in the onnx conversion script where I load my pretrained model from pkl file.

@elliotwaite

2 жыл бұрын

Yeah, you should be able to wrap your trained model in a new PyTorch module that just calls your trained model's forward method from within that new module's forward method, and then you can add any reshaping operations that you need to perform on your input into that new module's forward method before calling your trained model's forward method. Let me know you need any more help with this.

awesome dude

@elliotwaite

4 жыл бұрын

Thanks, Codacus!

Great Tutorial, would be awesome if you can do somenting similar with image regognition :D

@elliotwaite

3 жыл бұрын

Thanks for the suggestion. I haven't been in video making mode lately, but I've added the idea to my list of potential future video ideas.

Thanks!

Thank you for posting this! Can you do a tutorial on using onnx.js for real-time object detection in the browser?

@elliotwaite

4 жыл бұрын

Thanks for the suggestion. I have some other things I'm working on currently, but I'll add it to my list of ideas for potential future videos.

Thanks

Can we use this technique in chrome extensions that make use of Pytorch models

@elliotwaite

3 жыл бұрын

I think so, but I haven't tried it.

how to accept images of different shapes? Usually we write a function to convert all the shapes into one particular shape accepted by the model; what should we do here?

@elliotwaite

2 жыл бұрын

There might be some JavaScript libraries that you could use to crop or transform your image into the desired shape before passing it into your model, but I'm not familiar with the JS image processing ecosystem.

Thanks for that! But I have an error: opening the html-file I only get a blank browser. It´s the same in Firefox, Chrome or Edge. Do you know why?

@elliotwaite

4 жыл бұрын

Ah, I can see how I made that confusing. I just updated the GitHub repo to hopefully make things more clear. I had originally only put the full demo code on CodeSandbox, but I should have added it to the GitHub repo as well (an oversite on my part), so I just added it. I think the HTML file you were looking at was the debug demo, which only outputs information to the JavaScript console which is why the page was blank. To view the JavaScript console, you can open your browser's developer tools window and look under the "Console" tab. The distinction between the full demo code and debug demo code should be more clear now. Thanks for pointing that out.

@danieladrose6254

4 жыл бұрын

@@elliotwaite Great! Thank you!

That was an epic video! I would have fixed my canvas input to format the data into the expected Python format rather than change the model. That's just personal preference though. Has Onnx.js added training or does it still just do interference mode?

@elliotwaite

Жыл бұрын

Thanks. Yeah, that's another way to do it, to format the data before inputting it into the model. From looking at the ONNX website (onnxruntime.ai), it looks like the ONNX runtime can do training, but only on Linux (no web support yet).

There is something to think about

I have one problem while running a py script it's say, "Using Backend: pytorch" why this, any help

@elliotwaite

2 жыл бұрын

I am not familiar with this issue, perhaps you could look into the source code of the libraries you are using to see where it originates from.

Since react native is a JS framework, does that mean ONNX.js can work in a react native app?

@elliotwaite

4 жыл бұрын

Yep, React Native works by running a JavaScript thread that then communicates with native components. So you can run the ONNX.js code just as if you were using a React web app, except you may want to include the ONNX.js library as an asset with your app instead of loading it through a CDN so that your app can work offline.

@jaden2478

4 жыл бұрын

@@elliotwaite hi once again, thank you for replying to my previous question. Currently onnx.js supports opset 7, may i know if they are going to upgrade to opset 11? if so, when?

@elliotwaite

4 жыл бұрын

@@jaden2478, I'm not sure, but from looking at the ONNX.js GitHub repo, it hasn't had a new commit in over a year, so it looks like development of the project has stalled.

When trying to load onnx model I'm getting "Uncaught (in promise) Error: graph attribute is not supported yet". Anyone had this problem yet?

@elliotwaite

2 жыл бұрын

I'm not familiar with this error message.

to anyone trying it rn, you will get a lot of error. instead try to run the export model command given in the official docs works the best

@elliotwaite

5 ай бұрын

Thanks for the suggestion. This video is quite old, so parts of it may be outdated.

this is such an awesome tutorial! Hope you keep great work. :D .....but, is app not working?

@elliotwaite

3 жыл бұрын

Thanks! The app may have seemed to not be working initially while the model was being downloaded. I've updated the demo to display a "Loading..." message while the ONNX model is being downloaded so that users can know when the model is ready to be used. Let me know if this resolves the issue, or if it still isn't working for you: vgzep.csb.app

@anphanpro2

3 жыл бұрын

​@@elliotwaite It works. I confused about implementing the model on JS, It helped me a lot. Thank you very much for your help.

@elliotwaite

3 жыл бұрын

Glad it helped.

I only have a model.pt file how can I convert that to ONNX.js?

@elliotwaite

3 жыл бұрын

You convert it into an onnx file format using torch.onnx.export(...), as explained around 6:18.

@prateeksaxena7808

3 жыл бұрын

​@@elliotwaite But what should i initialize my model with, cannot find any class to initialize my model

@elliotwaite

3 жыл бұрын

​@@prateeksaxena7808, you need the original PyTorch code for the model class to initialize the model. You'd call something like `model = Net()` to initialize the model, but replace "Net" with whatever the name of your model class is. Then once you've initialized the model, you call `model.load_state_dict(torch.load('model.pt'))` to load the state dictionary information from your "model.pt" file into the model.

how to solve this errorTypeError: cannot resolve operator 'Pad' with opsets: ai.onnx please help thank you

@elliotwaite

Жыл бұрын

That means that ONNX.js doesn't support one of the operators used in your neural network. You can read more about which operators it supports here: github.com/microsoft/onnxjs#operators TensorFlow.js nowadays has better op support, so you might be better off trying to convert your model to a TF model and using TensorFlow.js instead.

@user-ut6vd7ho3u

Жыл бұрын

@@elliotwaite Thanks I will give it a try

@user-ut6vd7ho3u

Жыл бұрын

@@elliotwaiteONNX.js doesn't support one of the operators used in my neural network then I reduced the opset version now it works successfully!!!Thanks very much!!!

@elliotwaite

Жыл бұрын

@@user-ut6vd7ho3u glad to hear you got it to work.

How do we load the image?

@elliotwaite

3 жыл бұрын

I assume you are talking about the step around 16:30 (please correct me if I am wrong). I should have shown that you can also use a pre-forward hook to accomplish the same thing without having to edit the forward method directly (which is harder to do with pre-trained models). You would register the hook with "model.register_forward_pre_hook(my_hook_fn)" and then you would do the normalizing steps in your "my_hook_fn" function. Then you can load the pre-trained checkpoint as usual, but when you run the dummy input through the model, it will also record the normalization operations in the pre-forward hook function. Let me know if this answers your question.

@elliotwaite

3 жыл бұрын

@@jonathansum9084 I haven't tried out the ndarray JS library myself, but yeah, it looks like you should be able to do something like that, though I'm not sure about the details.

@jonathansum9084

3 жыл бұрын

@@elliotwaite Hi, I didn't start it. I was thinking that ONNXJS does not support resize, normalize with std and mean. For the resize part, torch takes a PIL image to resize. I don't think ONNX will capture resize for javascript like canvas to resize

@elliotwaite

3 жыл бұрын

I'm not sure I understand which part isn't working for you. Maybe if you post some demo code somewhere I could help debug further.

By changing shapes you killed convolutions, thats why it stopped working reliably

@elliotwaite

5 ай бұрын

Can you explain?

Excellent work but it would more helpful if you could provide a general code to apply on any model you like. Thanks in advance.

@elliotwaite

2 жыл бұрын

Thanks for the suggestion. Since ONNX.js currently doesn't support all of PyTorch's operations, it doesn't seem possible to make a converter that would work with any PyTorch model, but I tried to explain strategy you can use to try to adapt your model so that it will work with ONNX.js. Another option which seems to currently support more of PyTorch's operations is TensorFlow.js. Maybe I'll make a video about how to convert PyTorch models into the TensorFlow.js format in the future.

@aayush212

2 жыл бұрын

@@elliotwaite Great 👍. That would be wonderful. I'll be eagerly waiting for that video and I'll subscribe your channel to get the notification of the video. Thanks for the quick reply.

@aayush212

2 жыл бұрын

@@elliotwaite I tried to run your model on my browser but it says fetch API error. File not supported. I have also tried installing node.js but still it doesn't work.

@elliotwaite

2 жыл бұрын

​@@aayush212, to fix this, try running your code using a local server instead of opening your HTML file directly in the browser. If you use VS Code, you can use this: marketplace.visualstudio.com/items?itemName=ritwickdey.LiveServer If you use a JetBrains IDE, it's built in: www.jetbrains.com/help/pycharm/viewing-pages-with-web-contents.html#preview To explain, when you call something like: sess.loadModel('./onnx_model.onnx') ONNX will internally try to load the model by passing that path to the JS fetch API. However, the fetch API does not supports loading file URLs (ones that start with file:///...). In the example above, I am passing in a relative path ('./onnx_model.onnx'), so it will use the first part of whatever the current URL is of the browser tab running the code. When you open a file directly in a browser, the URL will be something like, "file:///.../index.html" and so the relative path would be "file:///.../onnx_model.html", which won't work. But if you open your index file using a local server, the URL will be something like, ".../index.html", so the relative path from there will work. Hope that helps.

@aayush212

2 жыл бұрын

@@elliotwaite Thank you very much for explaining things in detail.

Thanks for the awesome video! I'm running into an issue. When I try to import the class (in the video "from train_mmist_model import net") for my own implementation for Pix2PixHD, a style transfer network (see GitHub/nvidia/pix2pixHD) and when I import the class I get an exception ModuleNotFoundError bc the class has a custom dependency (from util.image_pool import ImgePool). How can I let python know how to find the dependency? I tried importing in the line above, (from pix2pixHD.util.image_pool imort ImagePool), and it actually imports, but alas, it still throws the error on the class import

@elliotwaite

3 жыл бұрын

I'm not sure, I don't know much about Pix2PixHD. Maybe if you opened an issue on their GitHub repo about this they'd be able to help.

how old are you?

@elliotwaite

4 жыл бұрын

35