How to pre-process your spectra for research (SNV, MSC, Derivatives, etc.)
Жүктеу.....
Пікірлер: 19
@nurulainina32662 күн бұрын
thank you for your very clear explanation. It helps me a lot
@ccscisac56072 жыл бұрын
Thank you for your attention! Comments? Suggestions? Recommendations? All options are welcomed!
@MIZRAIM1984Ай бұрын
I also mention Orange. It is a Python-based free software and it has a Spectroscopy add-on, which is excellent for the spectra visualization & preprocessing. The respective tutorials are encountered in KZread too.
@mainathuku9360 Жыл бұрын
Man. I just found out that I need to preprocess my data for MSC, this channel has really helped me. Thank you for the good job!!
@marineduperat52 Жыл бұрын
Thank you so much ! I've been looking for these types of information for weeks. This is so far the mos informative stuff I watched. I just need to get how to apply it with R now hahaha. (Forest scientist trying to work with spectral data here).
@ccscisac5607
Жыл бұрын
Always makes me happy when it helps someone! In the following link I started writing R scripts for SNV, MSC, and normalization. May not be optimal but they will give you a good start. www.uprm.edu/ccs-cicsa/files-info-for-research/r-language-resources/ Have a great day! - Edwin
@rishabhjain6285
Жыл бұрын
you dont need R that much.There's a Git link in which the codes for all these are already given
@pedramporbaha Жыл бұрын
Wow that is awesome!!! The only point is the voice quality but your content and your presentation was awesome. Thanks a lot
@DarshanaGopal4 ай бұрын
Good quality content
@MrEcoscience Жыл бұрын
Hello! Thank you! The best video in the net on data pre-processing I found so far! In Unscrambler, when you run the PCA based on the original data, you have the choice to select mean centering, which I think is meant to get rid off scattering. Would you in all,cases preprocess data with SNV and maximum centering?
@ccscisac5607
Жыл бұрын
Hello! Thank you for your kind words. The choice depends on what I (Edwin) want to study with my spectra. For presentation I would use baseline correction and SNV to maintain as much as the original shape of the spectra as possible. For creating models, I usually use 12 different combinations of DP methods. Use SNV, MSC, SG1, and SG2 separately and then combine them in different orders (SNV+SG1, SG1+SNV, MSC+SG1, etc.). Once I have a matrix with each different DP method, I create a model for each different matrix. This way I can determine which combination gave the most optimal results. However if the artifact can be seen clear as day on the data you can simply use the DP method that best reduces the variation. Hope this helps! We are welcomed to any suggestions and/or corrections. Hope you have a great day!
@monaallam1303 ай бұрын
Hello , thank you for this nice video , for the scattering , you mentioned that one of reason for scattering is the molecule not at the same distance , so how can i make the molecule at the same distance if i prepared as example solution contain dye dissolved in water , i prepared different concentrations then i measured spectra for them and i got scattered for the data, thank you.
@oscarjmaytzuc2818 Жыл бұрын
Hello, an excellent video. The best explanation of spectral preprocessing techniques I've seen in years. Taking advantage of the occasion, I have a question that I would appreciate if you would help me by clarifying it. All these current techniques are on the spectra individually (rows), but in the literature, I find other techniques, such as mean centering, auto-scale, and variance scale, among others, that act on the variables (columns). In some manuals, I found the latter necessary because several multivariate algorithms compute results driven by variance patterns in the independent variables. Specifically, my question is: When to use tools such as mean centering, auto-scale, and variance scale, or are they already integrated into techniques such as PLS?
@ccscisac5607
Жыл бұрын
Thank you for your kind words. Hope you are doing well. Mean centering is used to remove the common information on your data. Chemometrics assumes that variation implies information, hence why mean centering is so useful. When using PCA or PLS, mean centering the data allows the average variation between samples to be placed on the origin (0,0) of the scores plot. Auto scaling is usually used to leave the mean at zero and the standard deviation at one. From what I've understood, you scale variance when the variables that are being analyzed have very different magnitudes. This make some variables overshadow others, hence dividing the std allows them to play on an even field. Hope this gave some insight. Regards! - Edwin
@fabianenriquequinterotorde6038 Жыл бұрын
Buenas tardes seria bueno que esots videos tambien los compartieras en español
@iam_Simbiat11 ай бұрын
Thank you so much for this informative and well-explained session. I have been trying to write some code in MATLAB to perform MSC, could you please help with a script to execute that? Thanks
@ashenafibelihu11236 ай бұрын
am ok with data normalization (i.e., scaling), however I have a doubt concerning to the importance of DP methods for modeling!
@vivasjimmy7 ай бұрын
can anyone point for me places to download spectral data for my research?? i am lokking for bearings spectrum data
@muhammadharsanto20247 ай бұрын
Professor, which one from the programs you've mentioned on the video that can "mass" preprocess spectra? like can do multiple spectra in one touch Thank you
Пікірлер: 19
thank you for your very clear explanation. It helps me a lot
Thank you for your attention! Comments? Suggestions? Recommendations? All options are welcomed!
I also mention Orange. It is a Python-based free software and it has a Spectroscopy add-on, which is excellent for the spectra visualization & preprocessing. The respective tutorials are encountered in KZread too.
Man. I just found out that I need to preprocess my data for MSC, this channel has really helped me. Thank you for the good job!!
Thank you so much ! I've been looking for these types of information for weeks. This is so far the mos informative stuff I watched. I just need to get how to apply it with R now hahaha. (Forest scientist trying to work with spectral data here).
@ccscisac5607
Жыл бұрын
Always makes me happy when it helps someone! In the following link I started writing R scripts for SNV, MSC, and normalization. May not be optimal but they will give you a good start. www.uprm.edu/ccs-cicsa/files-info-for-research/r-language-resources/ Have a great day! - Edwin
@rishabhjain6285
Жыл бұрын
you dont need R that much.There's a Git link in which the codes for all these are already given
Wow that is awesome!!! The only point is the voice quality but your content and your presentation was awesome. Thanks a lot
Good quality content
Hello! Thank you! The best video in the net on data pre-processing I found so far! In Unscrambler, when you run the PCA based on the original data, you have the choice to select mean centering, which I think is meant to get rid off scattering. Would you in all,cases preprocess data with SNV and maximum centering?
@ccscisac5607
Жыл бұрын
Hello! Thank you for your kind words. The choice depends on what I (Edwin) want to study with my spectra. For presentation I would use baseline correction and SNV to maintain as much as the original shape of the spectra as possible. For creating models, I usually use 12 different combinations of DP methods. Use SNV, MSC, SG1, and SG2 separately and then combine them in different orders (SNV+SG1, SG1+SNV, MSC+SG1, etc.). Once I have a matrix with each different DP method, I create a model for each different matrix. This way I can determine which combination gave the most optimal results. However if the artifact can be seen clear as day on the data you can simply use the DP method that best reduces the variation. Hope this helps! We are welcomed to any suggestions and/or corrections. Hope you have a great day!
Hello , thank you for this nice video , for the scattering , you mentioned that one of reason for scattering is the molecule not at the same distance , so how can i make the molecule at the same distance if i prepared as example solution contain dye dissolved in water , i prepared different concentrations then i measured spectra for them and i got scattered for the data, thank you.
Hello, an excellent video. The best explanation of spectral preprocessing techniques I've seen in years. Taking advantage of the occasion, I have a question that I would appreciate if you would help me by clarifying it. All these current techniques are on the spectra individually (rows), but in the literature, I find other techniques, such as mean centering, auto-scale, and variance scale, among others, that act on the variables (columns). In some manuals, I found the latter necessary because several multivariate algorithms compute results driven by variance patterns in the independent variables. Specifically, my question is: When to use tools such as mean centering, auto-scale, and variance scale, or are they already integrated into techniques such as PLS?
@ccscisac5607
Жыл бұрын
Thank you for your kind words. Hope you are doing well. Mean centering is used to remove the common information on your data. Chemometrics assumes that variation implies information, hence why mean centering is so useful. When using PCA or PLS, mean centering the data allows the average variation between samples to be placed on the origin (0,0) of the scores plot. Auto scaling is usually used to leave the mean at zero and the standard deviation at one. From what I've understood, you scale variance when the variables that are being analyzed have very different magnitudes. This make some variables overshadow others, hence dividing the std allows them to play on an even field. Hope this gave some insight. Regards! - Edwin
Buenas tardes seria bueno que esots videos tambien los compartieras en español
Thank you so much for this informative and well-explained session. I have been trying to write some code in MATLAB to perform MSC, could you please help with a script to execute that? Thanks
am ok with data normalization (i.e., scaling), however I have a doubt concerning to the importance of DP methods for modeling!
can anyone point for me places to download spectral data for my research?? i am lokking for bearings spectrum data
Professor, which one from the programs you've mentioned on the video that can "mass" preprocess spectra? like can do multiple spectra in one touch Thank you