the narrative that lead to the tutorial is so cute as it never happened.

@Bjawu

3 ай бұрын

I sure hope so, or someone needs to learn about spreadsheets.

@elieobeid77

3 ай бұрын

it never happened indeed and a maximum of 4 lines of python can do this, that's why I avoid learning these tools

@jazzochannel

3 ай бұрын

spreadsheet? is that like lasagna or vegemite or nutella,@@Bjawu ?

@Bjawu

3 ай бұрын

@@jazzochannel There *are* some similarities with vegemite, now that I thing about it. You don't want to use too much.

@ChimeraBlockhead

Ай бұрын

I feel like people just don’t get this guy’s sense of humor

I enjoy using GREP and SED and I've been really wanting to learn how to use AWK. This video motivated me to do so, I'm gonna!! Also LOL, the end of the video made me laugh a lot. I love this!

I like how there was a narrative style to the video. That's really engaging. You've earned a sub!

@mjscpr

Жыл бұрын

Great pace too.

Thank you for describing the pattern-matching and default assumptions so thoroughly and illustrating each point with example code. Until today have never seen any awk tutorial that could get me up to speed so quickly as your video. Kudos! Showing how awk can replace both the grep and sed commands was icing on your cake. And your last sentence was a big flaming torch sitting atop the cake!

@BytebroUK

2 ай бұрын

Get real- once you learn regex, and in my own world, specifically Perl regex, that stuff becomes fairly trivial. "Analysing log files from umpty-hundred jobs" used to be a chore but these days I can script a working solution in an hour or so. In Perl. Because that's the tool I'm familiar with.

Many moons ago I helped maintain what we thought might be the largest awk script ever created - it converted between the proprietary FORTRAN code for compilers by our competitors into our own proprietary FORTRAN code. Of course, monthly, if not weekly, we'd find something it did not convert properly, and it would have to be changed. Not only was it possibly the largest awk script, but it was the most modified awk script...

@artemiasalina1860

3 ай бұрын

Fascinating! Do you remember how many lines (roughly) it was?

@nufosmatic

3 ай бұрын

@@artemiasalina1860 It was well over a thousand lines.

@bide7603

3 ай бұрын

Sounds awk ward to work with…. I’m sorry

@nufosmatic

3 ай бұрын

@@bide7603It's not funny, it's a way of life...

@somebody-anonymous

2 ай бұрын

I once took a little boy under my wing who thought that text manipulation equated to social skills. He was a little awk ward

This is video is great and demystified awk in a very easy to understand way

This is an amazingly instructive video. Thank you.

I used to use awk every day about 30 years ago. A nice walk down the memory lane with this video.

@jazzochannel

3 ай бұрын

memory lane? why did you stop?

@cthyau

3 ай бұрын

@@jazzochannelmy work now doesn’t involve coding and data manipulation any more.

Amazing presentation! I've been avoiding Awk for close to a decade, so much so I default to Sed for this kind of tasks! After watching your video, I feel more confident to try Awk next time :) Thank you!

I always thought awk was super hard to learn. 😭 Thanks for explaining it so well! 🙏

so much content in just 10mins. amazing!

Finally and after so many years, I understand how 'awk' works. Thank you, Legend

This was an exceptionally good tutorial. Thank you.

This is such a great explainer of a tool i rarely used, but saw the potential, although i never dove in deep enough to learn it. Thank you!

Dude........ Your content is brilliant. You must become more known! We desperately need more Unix content on youtube.

Nice tips and examples, man. Greetings from Brazil.

I get that awk is a useful tool overall, but with your given example, why exactly would writing a program be time consuming?

@Temulgeh

4 ай бұрын

i think it was just a poor attempt at a motivating example and a joke at the same time

@TheKuhtaMusic

4 ай бұрын

the whole premise was quite funny

@jeanhaley3051

4 ай бұрын

It would take as long to write in python as it would take learning awk. At least pre AI, now AI could generate the command in moments.

@Temulgeh

4 ай бұрын

@@jeanhaley3051 i just tried writing that python script and measured how much time it took me (without trying to rush it), it took 1 minute. it might be faster with awk in the long run? but 1 minute is good enough for me

@utvikler-no

4 ай бұрын

Even less code with Perl. And this example is not recommended at all. What happens when the data is changed. It all fails.

Nice Video, this is a great explanation of awk, thank you for sharing.

Wow.. I know awk well, but what a presentation. This is an underrated video. Keep it up dude.

@RobertElderSoftware

2 жыл бұрын

Thanks, will do!

Thanks for sharing. Good luck with your 1% raise!

@RobertElderSoftware

3 жыл бұрын

Thank you!

@NastyWicked

3 ай бұрын

@@RobertElderSoftware so... did you get the raise?

The awk command is now my favorite Linux command

@catsupchutney

3 ай бұрын

I avoided Awk for 15 years, but then I saw the light.

@cinderwolf32

3 ай бұрын

​@@catsupchutney I have a year left of avoiding it like the plague, maybe. Then I'll give it a shot.

Awesome explanation. Thank you

Utilities like awk are one of the reasons I love Unix. Everything is in a text file and the OS gives you the tools to parse and manipulate the contents.

The awk command is my favorite linux command. 😂😂😂 Great content! Keep going!

Hello, thanks for sharing this knowledge. Interesting way of presentation and very concise and informative.

Great video! So, it's 3 years later, I hope you got your juicy 1% raise!

'awk', 'sed', and 'Perl'. All I ever need for scraping from log-files!

@GreyDeathVaccine

3 ай бұрын

head and tail is also useful for a redneck like me :D

@obiwankenobe3962

3 ай бұрын

If you know Perl, do you still have a use for Awk? (I'm wondering if I should learn it, I already know Perl. From the looks of this video, I can do the same with Perl in maybe 20-30% more characters - not enough more to trigger learning it)

@BytebroUK

2 ай бұрын

@@obiwankenobe3962 Sure. Back in the old Unix days you needed sed and awk to to text munging. Now I just use Perl for all of it. Perhaps more characters to type in, but much simpler syntax and easier to learn, IMO.

@obiwankenobe3962

2 ай бұрын

@@BytebroUK thx for confirming my suspicion

Dude, this video was sick! Looovely stuff. You're awesome: 👑

Excellent presentation of how AWK works and how it could benefit you. I haven't used it personally (I would use Python: slightly more complex to use, but MUCH more powerful), but I know people who use it often, e.g. to scan log files for certain situations. It is great for iteratively improving the filters, browsing through the data.

Excellent explanation.

@4:10 "As you see, awk is not quite this smart. And here you can see how awk isn't quite as good as a CSV parser. " In GNU awk set the built-in FPAT variable. In your example: FPAT = "([^,]+)|(\"[^\"]+\")"

Bro went from a 0x developer to a 0.5x dev

@Dante-420

3 ай бұрын

10x0 is still 0

@cleverhippo

3 ай бұрын

what a hater, the video was great

@aimpizza6823

3 ай бұрын

It is but the title is wayy over the top

The solution to the original problem was a bit of overkill, but this was a good example for some teaching of base awk functionality. This is a more direct and shorter solution, only changing the F lines and printing all lines: $ awk '$2=="F" {$1=sprintf("%.1f",($1-32)/1.8); $2="C";} 1' temps.csv (the single "1" at the end is the condition, which evaluates as "true", evoking the default action: print -- nice shortcut if you mean only to change some lines but print all)

@ppokorny99

4 күн бұрын

Excellent. Glad to see someone else recognized and posted the solution I thought was more elegant And taught me something new with that “1” trick!

At my past job where we used Linux exclusively, I used awk extensively along with bash scripts, sed, tr, od, etc. There are so many handy Linux commands.

I've never done more with awk than printing columns from a text file, like the 'cut' command. And for other things, use a perl one-liner. Loved the average calculation example in awk. Good to have a new tool in the belt!

I like your shorts "this is my favorite command", but the longer are more explanatory and better pauseable aswell. Thanx

BEST INTRO TO AWK. PERIOD!

agreed - awk is awesome and i use it all the time - the most extreme example was a ~400 line script to analyze network files extracted from big integrated circuits but it’s great (and fast) at processing big text files - and it’s highly portable too since it’s installed on pretty much every unix/linux system dating back to the 1960s

Great presentation style. I just subscribed, hoping to hear if you got that 1% raise.

@RobertElderSoftware

3 ай бұрын

I didn't get any raise this year, but I've got a good feeling that if I work overtime for just a couple more years, someone will finally notice my hard work :)

Super Useful, and entertaining

You are the great teacher

awk is amazing , I love it !

I used awk and sed to convert entire flight manuals into our dataformat that won us the deal with pilatus as we could convert it to our software suit.

At a time when Excel had only 65000 lines, I used awk to condense big text data files to a manageable size used the associative array feature of awk. If you’re working with same files over again, I suggest to include one sample line for each pattern because you may not recognise the line from the pattern after 2 months. The Fahrenheit example I’d done in libreOffice. Import and so on. There was a time where you had difficulties getting an awk that works on windows.

I love this video. Awk is one of the basic UNIX tools I always skipped over. Bad call, lol.

nice video haha. hope you make more

very nice video .thanks!

My man ! Thank you

Thanks -- You have a new Sub!! Take care

super helpful

"I was so unproductive!" → Proceeds to open the Gnome terminal

@GreyDeathVaccine

3 ай бұрын

Alacritty FTW 🙂

@lanternofthegreen

3 ай бұрын

This is why people hate the Linux community.

@-aexc-

3 ай бұрын

using a better terminal does t make you more productive, just pipe the massive prints so you dont have to worry about print speed on compilation or something. (i say this as someone who uses a fast terminal (foot))

Great channel

yep, awk changed my habits either, thanks for sharing!

The awk command is my favorite command.

I really think we need better parsing libraries and at some point the most effective way to dev will be an editor that is easy to add features to which essentially will be coding from the command line. Even between awk and sed there are editing cases where I cannot cleanly implement my operation/macro.

@wojciech-kulik

3 ай бұрын

Neovim?

It's been 3 years since you posted the video. I hope you got that raise. Gotta offset inflation somehow.

1% raise 😂 Thanks for the tutorial!

Humorous but informative, I'm no longer afraid of awk, nice video Don't even ask how many python scripts i wrote to do this kind of stuff, grappling with pandas csv reader and such

The temperature data format is a good one for exercising awk. This is a data driven world.

Thank you for saving my life!

@RobertElderSoftware

2 жыл бұрын

;)

Awk is a very powerful program indeed. The only thing that keeps it from completely replacing grep for me is the lack of a Perl mode for regex.

Very cool. I hope you don't mind, but I've just placed a link to this video in the awk chapter of the new shell scripting book that I'm currently writing. (It will be out sometime in the spring.) Many thanks!

@RobertElderSoftware

5 ай бұрын

Thanks for the link, I feel honored :)

I use awk over grep daily because of it's usefulness in creating conversions or finding ways of pattern matching smartly. It's grep and cut in one. The only time i use cut is for specific columns due field separators being a pain sometimes.

I remember when I used to torture myself with arcane bash commands and construct a towering awk command for complex file parsing. The cryptic awk errors were chef's kiss. Honestly I just prefer any other language at that point.

It's 3 years later, did you get that raise bro?

I love how simple programs developed 50 years ago are still extremely useful. That is because the core of Computing hasn't changed much at all over that time, and UNIX did an excellent job of capturing that core. I pity the fools who know nothing outside of a GUI. Because with a GUI, you get a single tool that can do only what its designers allow you to do. But the UNIX concept is to have a load of little tools that one can combine to do exactly that what you want it to do. Perhaps I should write a GUI overlay for the shell, allowing the old UNIX tools to be configured and linked together graphically. That would be fun, and might help people discover the tools that are available at the CLI.

nice video

Real world examples IME look something like this: Some poorly documented program just spat out a noisy status message and produced a bunch of poorly named files. You want to parse the message to associate metadata with the output files, then rename the files with metadata as part of the name. First pipe message into awk to separate into fields, then do simple transform in awk or pipe into sed or grep, then sort file names for association with the metadata output. Then pipe both into awk again to generate a shell script to rename the files.

Hmm, first time I watched this video, sound was behind the image, so you looked like Bruce Lee speaking in movies 😁 I thought it was intentional at first, then realized it was a problem in my browser. AWKward.

Before awk I was living on the street and had no significant other. With awk I'm now living on a mansion and have two wives 🎉

Awk was pretty much the first scripting language; it was the ancestor of Perl. It's largely taken over shell scripting on Stack Overflow these days; the kids there love it because they're so paranoid about only calling a single process, and doing everything in a single line. I think its' syntax is a bit tricky, personally; but it does make it extremely easy to create mathematical filters that are hard to do with virtually anything else.

I challenged a guy who believes in awk, I let him define the problem. He could use awk, sed, … and I would use cpp, yacc, and lex. The challenge was on two fronts, dev time - first to start, and run time. He insisted that he may run slower than me, but he would clearly win dev time. Well, I ended up not using yacc, just cpp and lex. Beat him in dev time by nearly an hour. Mine ran in less than a second and his took a few minutes. To me awk is awkful with a silent K. It is a very useful tool, but it is just so easy to create your own specific, and much better tool quicker than using awkful.

Subscribed.

thank u

I'll never use IDEs and Vscode anymore to process texts! Thanks ❤

Its now 3 years later, he better have gotten that 1% raise

Great vid thanks! @10:20 the "C" doesn't appear in the output. Should it be inside the double quotes? ie ` C" = "` be `" C = "`?

@RobertElderSoftware

Жыл бұрын

Good catch, you're exactly right!

I enjoyed your presentation on awk and I subscribed to this channel. I have a question: Can awk to the following job -- send the first line of a file1, delimited by default space, to another file2, file2 that would take its name from the first few fields of the line 1 just transferred from the file1? Thank you.

@RobertElderSoftware

2 жыл бұрын

Given a file 'a.csv' with contents: field1 field2 field3 a b c d e f g h i You can do awk -F' ' 'NR==1{print $0 > $1"_"$2".csv";}' a.csv which produces this file: field1_field2.csv with contents: field1 field2 field3 OR what you maybe also wanted to do: is this: awk -F' ' 'NR==1{fname=$1"_"$2".csv";}NR>1{print $0 > fname;}' a.csv which produces a file named: field1_field2.csv with these contents: a b c d e f g h i I just did this on my machine using GNU awk. Not sure if that'll work the same in all POSIX awk implementations though.

@dumitrus.6925

2 жыл бұрын

@@RobertElderSoftware Super! Thank you. I used based on your awk: awk -F' ' '{fname=$1" "$2;}NR>0{print $0 > fname;}' file

Writing a dedicated program would have taken about 7 lines of code and 5 min, not forever. But, I digress. Us Nix folks live by grep, sed, awk, cut, xargs, etc.

Awk is great. Been using it for 20+ years

Awk is great, but not the only hammer for every nail. Sed, cut, bc, (node maybe), vim macro, bash, lots of ways to solve this with one line or so

There is also jq for json manipulation, but I've never used it in practice.

When I look at the command you ended up with I think if it was me it would have taken me less time to write it in python that I already know than to learn all that in awk.

the awk command is my favorite linux command.

writing a dedicated program takes forever? I agree awk does faster, but writing a python script will take at most half a minute...

cool command for pasing filies

Ah the classic 10x engineer, Making 10x the work for everyone else. I too have earned this little.

The Awk command is my favorite Linux command

@RobertElderSoftware

3 ай бұрын

This is my favorite comment.

OMG, its so coooooool

someone has earned themselves a pizza party

If you're working in a Windows environment, you can use PowerShell in much the same way for data processing tasks like this. That said, I really need to start using awk in Linux instead of just contorting sed into doing things it wasn't made to do.

Awk command is my favourite linux command.

awk is great! So many simple tools like grep, awk, and even more advanced-ish tools like jq are so often overlooked! Though for me, once I know that I'm going to need something like an average temperature, I'd probably end up just running the python command and using the python repl which is already quite fast to work with. Better yet, if they ever ask me to get the median or standard deviation, I can do so without the hassle of writing the equivalent awk script.

@GreyDeathVaccine

3 ай бұрын

jq is insanely powerful 🙂

@XerosOfficial

3 ай бұрын

@@GreyDeathVaccine for real

I started to use perl for most of those kinda things years ago. But indeed AWK is a usefull tool. Beside stuff like AWK, SED or PERL differs you from the dotNot kiddies.

1.5% raise for the gentleman please.

@10:20, A Joshua Fluke like pun there? :D

I was hoping there was a "but" to the "doesn't parse quotes" problem. it feels like I would always be running into the issue that I couldn't quite parse a file with 100% accuracy. (some line or other would be inconsistent somewhere.)

0:10 before I read about the emacs command I was a very productive employee. My boss would assign me a task and I would just do it. Then I read about emacs and now my boss assigns me a task and I spent half a day configuring emacs and another half to write something in emacs that gets the job done

in 2024 it's counter-productive to rely on awk as it's missing libraries to work with common formats: multiline rows, binary (like parquet), csv with quotes, headers, footers, excel i.e. pandas.read_csv in jupyter/ipython would do the same (could have a one-liner if you wish) but it's compatible with binary formats and anything you could possibly think of. another option is excel if there is no need to repeat the transformation.

0:30 dedicated Python code for that task is 6 lines. ``python with open('temps.txt', 'r') as input_file: with open('temps_all_c.txt', 'w') as output_file: temp, scale = line.strip().split() if scale == 'F': temp = (float(temp) - 32) * 5/9 out_file.write(f'{temp} C ') ```