I LOVE the way you and DJ Ware cover these kinds of topics- a great blend of more advanced topics that are useful for home desktop users as well as server and enterprise admins. I'm a compulsive tinkerer and videos like these help me learn ways to satisfy my tinkering/ optimization addiction.

Can't thank you enough for explaining the memory swap subject! Over 3 years using Linux as my daily driver I still found myself closing browsers, shutting down my virtual machines, even rebooting to address it (even with 32gb of ram). I now have the confidence to tackle this and really make use of my rig!

Hi Jay! Your videos are a godsend. As a career changer and a sort of a late-starter in IT, your content is pure gold to me. This video helped me fit some pieces better in place and I feel more confident with this topic now. Thank you so much!

@hector9586

2 жыл бұрын

it's never late to learn! keep on learning.

I like this format where you go through tools and features individually with their own videos. It works better for me than longer more wide-ranging videos. Thanks for the helpful content!

i've always wanted to learn how this works. thanks so much for uploading!

This is the best vid I've seen on the science of memory and use of swap!

Thanks. Nice clear explanation. Simple enough to understand.

hey Jay, nice episode as ever - you've amassed a great body of work already. What are the plaques with Japanese (I'm assuming) characters behind you?

Once again you have helped me to better understand a Linux topic, esp swap. Thank you so much!

Was looking for this. Thanks 👍🏼

Great intro to the topic, but would love a deep dive into memory.

I've always included a swap file on distributions I've installed. Now, there is a different type called a page file. In the Windows world, the difference is a page file is dynamicly allocated as needed up to the limit you set, whereas a swap file is a static size. Page files are more disk management efficient because it only takes up as much space as it needs. But, contiguous space is not assured, which can really hurt performance. My first experience with paging and swapping was in 1985 on a VAX 785, where idle procecess would be swapped out to allow higher priority and active interactive processes access to memory (all 2 megabytes of it). We were running VMS 3.4, and the system administrator made the swapper a lower priority than he should have. The VAX (which serviced the 28,000 undergrad and graduate student population - even if you weren't a student that had a reason for an account, you had an account) was freezing. After about 3 days of troubleshooting, it was found that the swapper was swapping itself out to the page file. When something is paged out, it is put in a sleep state until the swapper swaps it back into memory. Well, if the swapper is put to sleep, how is the swapper going to swap itself back in? The system admin pit the swapper back at the hiest priority so it would never be swapped out. Thank you for reading my war stories of my leading a time sharing system.

All about data handling. The whole IT field rest on that. Thanks Jay for readjusting this in my brain. Your content is a brain saver.

Any content you put out for System 76 is valuable to me, I own one & it's my first Linux machine.

Keep doing what you're doing. Much appreciated

Thanks as always for the great content. Much appreciated.

@LearnLinuxTV

2 жыл бұрын

My pleasure!

I always benefit from LLTV. My knowledge has expanded ❤️

Very helpful video. Thank You

Linux Crash Course helped me in improving Linux knowledge! Great thanks for your contribution and outstanding tutorials! On the other hand, I wanted to ask your opinion on WSL2. Did you have a chance to interact with it? Any thoughts from professional community on it's usage in day-to-day tasks?

I use swappiness on my machines to limit the swapping to my SSD. I'd love to hear your take on swappiness, and recommendations.

Excellent content, thanks. In regard to allocating storage for swap-space, maybe you can elaborate on this: When a laptop is put to sleep/suspended I thought that the OS writes the contents of RAM to swap-space and back the other way when the laptop is awakened. So, in that use-case wouldn't a sufficient amount of swap be necessary? Sleep/suspend isn't relevant to servers/workstations but maybe it's worth mentioning.

@SteveMacSticky

Жыл бұрын

@Reptilicus Rex yes that's true. if you wish to put your machine to sleep or suspend, you need swap enabled, generally 2gb bigger than your ram.

Great work 🥳🥳🥳 Thanks 💜💜💜

Tnx , this is very helpfull and informative for people migrating to linux like me :)

Wow thank so much. I just set up a Raspberry pi 4 8GB RAM, booting off a 1TB SSD for Ubuntu Server 22.04 I was worried about having enough RAM

Excellent explanation of the /proc/meminfo Cached parameter. My new slogan is "Unused RAM is wasted RAM". Now, where's the T-Shirt?

Thanks my dude

In the scenario of hosts behind a load balancer, I would rather avoid swap slowness all together, and quickly remove that host as it stops taking requests reaching memory limitations, that I would see request latency increase, especially if your application is set to backoff/retry behind the scenes. This is the best argument against it imo, its not just about disk space being cheap.

@tourdesource

10 ай бұрын

That sounds like a good strategy, thanks for sharing

Thanks for the video. Unfortunutely, my server uses swap because of the analysis dependencies. I have more than enough space, ram and storage but the analysis sometimes oversteps to swap.

@RobBCactive

7 ай бұрын

Then you have enough memory demand to write dirty anonymous pages to the swap file, there's no harm in having idle unused daemons end up with saved state in swap rather than retain memory wasting it. It's when you have a high rate, due to insufficient memory causing thrashing and have the OOM killer terminate processes, possibly due to lack of swap allocation that you have a real issue. This swap is bad mantra is over simplistic, it's there to use memory better.

I have an old original Raspberry Pi with 256 MB ram and 64 MB allocated to GPU. It runs headless and has Apache server installed along with ssh, dlna server, network scanner server and a vnc server. A desktop is installed. I very rarely need the desktop so both it and vnc server are disabled and only started if required. I did enable 512MB of swap as I think this may be needed when I do run the desktop and vnc server. I do have more modern Pi devices but old faithful has server me well and continues to do so.

I come back to this video every time I need a swap refresher. One thing I will say is that by the end of the video, I hate the word "swap".

1y old, but still be a great video.

Thank you

What's your opinion on compcache / zram like it is used by default on fedora instead of a swap partition/file.

Please make a detailed overview of the pros and cons of your system76 laptop and not a big comparison with the MacBook Air m1 13 in terms of work?

Can you make a video on audit daemon since its running in your htop?

Thanks for this Video and what is the shell used to get this kind of arrow prompt ?

@LearnLinuxTV

2 жыл бұрын

It's just bash with a custom config, nothing special.

This feeling when you understand how it works😊

How do you calculate the total memory from the available memory, free and used, etc? It used to be you could do that in the past, but in this example there seems to be megabytes of missing ram.

Even though it may cause slowing down the server, any swap is better than no swap, especially running a database aplication. 30+ years of experience suggests that. Cheers,.

@LearnLinuxTV

Жыл бұрын

Agreed!

good video

This guy is awesome 😎

@LearnLinuxTV

2 жыл бұрын

I know! He's just so amazing. How does he do it? He's just soooo cool 😃

A m.2 at a good clock only moves 5000 mb/s at best and ram does 35500 mb/s. A sata ssd is around 500 mb/s. It's better to use the ram for video than hard drive. That's my theory is try not to use swap at all. But you need swap if you freeze your computer. The swap will slowly untangle it. So that is why the older thinking of twice the ram.

Now days I use zram swap that uses ram block instead of drive and decrease swapiness to 10 to desktop and 30 to a server.

The issue with swap memory is also about how much to spare. Now is 1 GB good enough or should it 50% of the RAM ? like I have a 16 GM RAM and partitioned 4 GB for swap in my SSD

@LearnLinuxTV

2 жыл бұрын

It all depends on whether or not you plan to use hibernate, or "suspend then hibernate". If you do, swap should be slightly larger than how much RAM you have. If you're not using hibernate (and never plan to) then I usually just standardize on 2GB (4GB at the most).

So now I will learn whether reducing swap to 10 makes Linux faster or not.

@matthiasbendewald1803

Жыл бұрын

It is a compromise: It will make your system more "snappy", applications that didn't run will stay in RAM for longer and therefore be available more quickly. On the other hand, data of running applications will not be cached as well, so accessing data from disc will take longer and therefore your running applications will be slower. If you want to know more learn to use vmstat and how to interpret the values. This gives you the tool to fine-tune your applications.

Is there a way, driver or something that would allow one to use storage as system memory? Say use a flash drive as part of system memory and a hard drive for swap. If I have 2 GB of ram and need 32+gb, setting a 64gb swap isn't going to help. How about a distributed shared ram?

I would love to see also tutorials that touch this harder topics (why its show less mem than actually physical mem, what are shared and cache plus free values differ from available, etc.). This concerns also other your videos/topics. As an Linux admin/user for years I already know basics, please consider to make videos for more advanced Linux user.

Nice video,, but the swap space is used even I have free and available memory,, what would be the cause?

Can you do a Video on fio command

I’ve got 15.5 GB RAM and the max thats being used is 3.8GB ( 2.1GB AVG). Swap is always used up to 3 GB ( 1.7 AVG). The performance of the applications I am hosting is not very good and I feel that is because it is not using enought memory even thou I’ve set 2.5GB for each ( total 7.5GB). For example right now in bpytop it shows the ram as 19% used ( 2.94 GB) ,81% free (12.6GB) , 79% cached (12.3GB) , 1% Free ( 163 MB) and the swap 328 MB used. So at this point I don’t understand swap since I am not running out of memory.

I use the -h on things like free or df, h is for human readable...

@Siamhie

2 жыл бұрын

Same for me.

Agreed, I have said myself "why argue about putting aside a 'tiny scratch' on a huge disk", I think some individuals just attach to ideas and 'get comfortable'. Swapping may be slow but it could be the best place to be when needed.

Excellent video, easy to understand explanation of the essentials. However I only use desktops, I trust the defaults chosen by the kernel designers, so I never touch swapinness anymore :) I moved my applications to VMs and my Host OS is a minimal install of Ubuntu 22.04 LTS, my VMs run off OpenZFS 2.1.2. My Host OS has 2 swap partitions; first priority a 2GB Swap on a nvme-SSD (3400/2300MB/s) and second priority a 16GB Swap on HDD as an overflow swap, but intended more for future hibernation. I often see swap usages between 100MB up to 1500MB and I don't care, since I don't notice any negative effect on the responsiveness. Remember many programs have large parts, that are never used or only once during e.g. initialization, closing the program or in case of errors or exceptions. I see it as "unused and wasted" RAM too :) Those parts of the program will be swapped first (least frequently used), so you will not see any effect with respect to responsiveness. I limit the amount of RAM I assign to VMs, because I want to be able to run 3 to 4 VMs in the 16GB of my Ryzen 3 2200G. The VMs share the RAM with a 0.5GB to 3GB ZFS memory cache (L1ARC). So for Windows 11 Pro I use 4GB and for Linux VMs I use 1.5GB to 2.5GB. I have no complaints about responsiveness. Also there I see the SWAP is used extensively, but that swap storage is part of the virtual disk and that virtual disk is cached by the ZFS memory cache (L1ARC) of the Host OS. The L1ARC is lz4 compressed, so currently it stores 6.12GB of compressed stuff in 3GB physical space. Looking at the size of Linux after booting say 0.6GB to 1GB of uncompressed storage, the Host OS stores most of the used disk content of the 4 VMs in the L1ARC, including any used swap space. Consequently my VMs run from memory :) :) and even on the 14nm Ryzen 3 2200G (4C4T, OC 3.7GHz) it is fast. The boot time and initial program load times depend on the speed of the storage, so Xubuntu 22.04 LTS boots from nvme-SSD in 9 seconds and Xubuntu 20.04 LTS boots from two striped HDDs supported by a 90GB sata-SSD Cache (L2ARC) in 30 seconds. Booting Xubuntu 20.04 LTS for the second time takes 11 seconds, since now it largely boots from the L1ARC memory cache. Note that e.g. ext4 installations also extensively uses caching, but by default the cache is uncompressed and the algorithm is LRU. ZFS uses a more sophisticated method of caching, mixing LRU and MFU, while dynamically adapting the boundary between the two based on the dynamic results of both methods. My VMs use ext4, so I have some caching there too, but the available memory is relatively small, so caching is relatively often deferred to the Host level and the superior L1ARC.

When I installed a 2TB SSD 16G mem in an old Toshiba laptop running Ubuntu I got 15sec boot time I set up 16G swap file, everything starts in seconds I can run multiple applications and I never ever need to use Htop or other system monitors.

Hii please reply. I need help. I kinda set my swapsize to 2092 and now i can't remotely desktop connect to my Jetson Nano with Linux system. And I checked in it's SSH that the "system has been minimized". Is it because i set up it high? Did i broke my device???

I'll just give you this real example: you might have an app which is memory leaking, it will first consume all the ram, then it will make other processes memory to be pushed into swap. And then it's a matter of chance, what kind of error handler for memory allocation exceptions that buggy app will have. In vast majority of cases it will have something like "okay, we failed to malloc, we wait for a while and try again". Which in practical term means your system will become very unresponsive, because whatever memory will be freed, it will be immediately occupied by that single buster. And your chances to quickly ssh to that machine and kill the nasty app will be getting thinner and thinner, as now SSH daemon, and login/sh processess it spawns will be 100 times slower, as they'll be swapped to. So swap is bad not because of religious holywar. And to be 100% safe, have some resource manager set up on your system. And test how your system behaves, when some shitty process eats all the ram. It can be coded with just few lines of C code. And it will open your own eyes what a disaster it can be, if any of apps your server might be running will get some memory leak.

Someone please tell if it's possible to install GCC 12.1 stable version on Ubuntu 22.04 or not. I've been struggling to update mine from 12.0.1 which is experimental. I just can't find a way to install 12.1 which is the latest version.

-m is show output in mebibytes; while --mega will show output in megabytes

"free -h" is the best readable for noobies like me :)

How about making a video about open source BIOS. If I want to install an open source BIOS in my computer replacing previous closed source BIOS what kind of challenges do I meet? Or, is that even possible?

@LearnLinuxTV

2 жыл бұрын

It’s very possible to do, but depends on what model you have. Unfortunately, that particular topic wouldn’t translate well into a video because a small number of people would have the tools and appropriate hardware.

@armajroy

2 жыл бұрын

@@LearnLinuxTV Yup, that's true. Well, in near future, if it is possible explain these concepts with your amazing video.

How to like twice a video? 😉 great content

@LearnLinuxTV

2 жыл бұрын

Thanks!

Why not make the swap file very large ( for ex 20G) to have a lot of ram

@aleksandartyankov1454

Жыл бұрын

test it out - assign 20GB of your disk for swap space, set the swappiness to 100 (use swap aggressively) and you will get the answer to your question very quickly :) basically you would think you have a machine with a lot of memory, while in fact everything will be extremely slow

"How do we monitor memory usage without having to install htop?" … use top instead. It shows you mem & swap stats as numbers sans the fancy graphs.

Why doesn't "disks" set up the linux file system when you are partitioning a external HDD

I love you

I am naughty, I haven't used swap for years and it goes fine. :) 32 GB DDR4,

For me free shows 32087 MB total RAM, just over 32 GB of what I have installed.

I f I reboot the system . How I am always keep the swap memory on

Could you make the explanation of free and available more simplified than it is on the video

swap drinking game confirmed! 😂

I wish every distro, whether it be for desktop or server, came with vim and htop. These 2 programs are my first downloads for any distro I install. Great content as always, keep up the great work!

Ig im the first XD, btw love your vids!

@LearnLinuxTV

2 жыл бұрын

Yay! Thank you!

OK, I'm laughing my butt off. It looked to me like you used your mouse to quit htop, so I went to a terminal and opened it and clicked quit and it quit. I've used htop for I don't even know how many years and didn't know you could use a pointing device. I don't know that I'll do it again, but it sure made me laugh.

I use zram so my swap is in my ram

i've set swap = 0 but my proxmox dont care abt it ((((

I really like your videos but this one is so complicated when you are trying to explain differences between free and available

This is probably the same amount of water in your book.

No swap is bad. You want the kernel to kill process that eats memory, not swapping.

Hey friend I'm Jiejie from KAMRUI. We sell all kinds of MINI PC on Amazon. We are looking forward to presenting it to your audience through your professional presentation. Can I have a chance to work with you?

You should upload your videos to odysee.

Too streched out to cover very little information. With 20 min, you could've went a lot deeper. The video has way too many filler sentences

You waste a lot of time saying the obvious and when it comes down to understanding the differences you just say "I'm going to stop my explanation right here". What's the point of making this video? Engagement only I guess

learn to focus your thoughts, stop jumping around, one thought at a time. unwatchable

No program is made to use swap. You are wrong. And you are wrong about swapiness