Using GNU parallel painlessly -- from basics to bioinformatics job orchestration

Ғылым және технология

Basics of GNU parallel, how to use it to run scripts and functions, all the way to using parallel to orchestrate launching jobs with different parameters like we often do in bioinformatics.
You can find all the code from this video here:
omgenomics.com/parallel

Пікірлер: 16

  • @theoriginyt4869
    @theoriginyt48697 ай бұрын

    Looks pretty promising, it seems GNU parallel is able to easily call a single command over a list of arguments, with the added bonus of parallelization. I need to try that command out tomorrow. Thank you for the video!

  • @MinhTran-ep4ln
    @MinhTran-ep4ln Жыл бұрын

    Very good instruction. As a beginner in the field of bioinformatics, this helps me a great deal! Thank you so much!

  • @oletange4373
    @oletange4373 Жыл бұрын

    To test for the correct version of GNU Parallel in a script, use `parallel --minversion 20220622 && echo GNU Parallel version 20220622 or newer is installed`. This is supported use, which is guaranteed to work in the future. The `grep` trick @01:10 is not guaranteed to work in the future.

  • @tuskiomisham
    @tuskiomisham6 ай бұрын

    Thank you, my colleagues have been telling me to use parallel for months now, and I'm finally able to start scripting with it.

  • @FVdm-zo8ui
    @FVdm-zo8ui Жыл бұрын

    Thanks for this great video! 👌 If I may add several other Parallel features I find very useful : - instead of '--verbose', I use '--dry-run' to see what I run - option '--keep-order' can be useful sometimes, to tell parallel to keep output order the same as input (but arguments are not actually processed in order, it affects only output order) - all it's built-in substitution variables like '{.}' (path+filename without extension) ; '{/}' (equivalent to 'basename /path/to/file.ext') ; '{//}' (equivalent to dirname) ; '{/.}' (equivalent to 'basename /path/to/file.ext .ext')

  • @amiramiri9245
    @amiramiri9245 Жыл бұрын

    Love You Dr. Nattestad🍀

  • @EthanLR
    @EthanLR Жыл бұрын

    Really good video, thanks!

  • @Nabeelfrance
    @Nabeelfrance Жыл бұрын

    Thanks! That was really helpful.

  • @sasonbraha4599
    @sasonbraha459911 ай бұрын

    Great video!

  • @victorcorte6759
    @victorcorte67598 ай бұрын

    Very useful video!

  • @brucekang6474
    @brucekang6474 Жыл бұрын

    thanks!!

  • @oletange4373
    @oletange4373 Жыл бұрын

    Do an advanced video where you take the most cryptic GNU Parallel command you have ever used in real life, and decipher what it does. Do the same for the most 'clever' use. Maybe you replaced a 3 pages script with a single line? Maybe GNU Parallel made it possible to do something that was infeasible to do otherwise?

  • @EvanBurnetteMusic

    @EvanBurnetteMusic

    Жыл бұрын

    I used it to create 'dark mode' PDFs with a random java program I found online by splitting books or long papers into individual pages and then parallelizing the dark page creation. I am not a java programmer, and I was able to parallelize a serial program without rewriting it. (Now I just use dark reader in the browser). Other things I would use GNU parallel for is cropping/resizing/recompressing images and normalizing/converting audio files with sox or FFMPEG.

  • @shehryarqureshi8500
    @shehryarqureshi8500 Жыл бұрын

    How good at genetics do I need to be if i want to study Bioinformatics at masters level? i am currently studying biomedical sciences at undergraduate level and am looking at what to do next, i have struggled with genetics my whole degree. bioinformatics interests me a lot, even tho im terrible at genetics, mainly cus of the computing and statistical/analytical aspect of it. any advice would be greatly appreciated!

  • @dainfoma

    @dainfoma

    Жыл бұрын

    I am undertaking a graduate program now in bioinformatics, from a Biotechnology undergraduate background. We have more students from computer science than biology background. I have noticed that even when you are not good at genetics, you can still lean towards other non-genetic work like machine learning, big data analysis (data science), and disease modelling among others. However, I have noticed that you are also having worries about statistical aspect of it, which makes it tricky. But you will get the basics very fast and you may find it easy at the end

  • @pilotashish
    @pilotashish Жыл бұрын

    if you prefer reading man prallel_tutorial

Келесі