For anyone else struggling to determine which GPU is which, run `ls -l /dev/dri/by-path`, and cross reference the addresses in that output with the output of `lspci`, which will also list the full GPU name.

@massivebull

2 ай бұрын

I've been rewatching the video twice trying to figure this out - your comment saved me a lot of headaches - thanks a lot !

This is quickly becoming my favorite channel to watch :D Great stuff! Can't wait to see what you have for us next!

@Jims-Garage

5 ай бұрын

Haha, thanks for the feedback. Next step is network shares on LXC. Then onto clusters on LXC with GPU shared.

@darthkielbasa

5 ай бұрын

The “eat like an American…” wall hanging got me. The content is secondary.

Spent the last couple of weeks trying to achieve this myself and couldn't - had to stick with a privileged container. This worked perfectly first time, thank you Jim!

@Jims-Garage

5 ай бұрын

Nice work! Enjoy the added security :)

Thank you. Your tutorials are some of the best, very well explained and functional.

@Jims-Garage

5 ай бұрын

You're very welcome!

I just used this guide to get hardware encoding working in an unprivileged Immich lxc container, through docker compose. ( After a lot of work) Thank you so much for your great and comprehensive guides.

@Jims-Garage

4 ай бұрын

Great stuff, well done ✅

Finally Subscribed to your channel! Thank YOU!

@Jims-Garage

4 ай бұрын

Thanks for subbing!

Came from the Selfhosted Newsletter a few days ago and I am loving it. Great video, and I will definetly try that as soon as I have time

@Jims-Garage

5 ай бұрын

Awesome! Thank you!

Great video of Proxmox outside.Thank you Sir!

@Jims-Garage

4 ай бұрын

You're welcome 😁

This is great stuff! Thanks Jim :-)

@Jims-Garage

5 ай бұрын

Thanks. It's a really good feature of LXCs.

My weekend project right here. I run unraid in a VM with some docker containers running in it. I want to move all containers outside the unraid VM. Now I can test this and also sharing the iGPU!!! Not straight put to a single VM. NICE!

@Jims-Garage

5 ай бұрын

Absolutely, it's pretty huge being able to share the iGPU between LXCs

I really love your channel Jim. I learn(ed) a lot from you !! I would love to see how to get the low power encoding working 🙂

@Jims-Garage

5 ай бұрын

Coming soon!

Nice Jim 😀. You keep making great content👌🤌

@Jims-Garage

5 ай бұрын

Thanks 👍

Your github is a pot of gold. TY sir

@Jims-Garage

5 ай бұрын

Thanks 👍

Great video. I did a similar thing ages ago to passthrough a couple of printers to an lxc unprivileged cups printer server! Was a headache to figure everything out at the time hehehe

@Jims-Garage

5 ай бұрын

Ooh, that's a great use case. I like it.

Great video. I am going to play with this this week so both Jellyfin and Plex have access to the GPU. Maybe other stuff eventually.

@Jims-Garage

2 ай бұрын

Great stuff 👍

I struggled with an AMD igpu pass through with Jellyfin and you were very kind to help , in my case it did not work with a regular VM , but with this it was a breeze to setup Jellyfin with HW transcoding , the only hiccup was the lxc image of Debian 12 did not work , but Ubuntu did , latest proxmox fully updated , thanks again your walkthroughs are really helpful thanks !

@Jims-Garage

Ай бұрын

That's great, good job 👍

@codexclusiveNL

Ай бұрын

Could you help me out?

@jafandarcia

Ай бұрын

@@codexclusiveNLfollow the steps that Jim laid out, some films depending on the setup will work depending on the browser or device also, for me even the groups that Jim has in the video are the same , y setup is a beeliknk mini pc with an AMD 5650

thank you for this tutorial. It works. just want to let you know, it works also with handbrake docker container, just in the compose file add GROUP_ID=107 in the environment section. intel qsv is detected!

@Jims-Garage

Ай бұрын

Awesome, thanks for letting me know

Congrats, good stuff. You may want to check out how to run docker images as LXC containers, since they are OCI compliant. It would remove an abstraction layer, but instead of compose it would be set up with ansible.

@Jims-Garage

5 ай бұрын

Good suggestion, something I can check out later. Thanks

Great tutorial.

@Jims-Garage

Ай бұрын

Thanks 👍

Great Content! Would be nice if you elaborated more on the low power encoder in one of your next videos.

@Jims-Garage

5 ай бұрын

Noted!

I also run my Kubernetes test env. in LXC on my laptop, makes a lot of sense.

@Jims-Garage

5 ай бұрын

That's great. I'm hoping to do similar for GPU sharing.

@autohmae

5 ай бұрын

@@Jims-Garage You've already figured out the hard part. 13:34 in practice by the way it doesn't matter. As long as the host is newer or the same and you load any kernel modules you might need. Linux mostly adds new functionality, as Linus always says: "don't break user space". I was able to run Debian 2/Hamm LXC container on a modern Linux kernel aka Debian 12. Not like I've never done this before. I was running Linux containers before LXC existed, before I ever touched VMs. On Debian Woody with Linux-VServer.

@Jims-Garage

5 ай бұрын

@@autohmae wow, that's impressive. Thanks for sharing

@autohmae

5 ай бұрын

@@Jims-Garage well, it's supposed to work 🙂

Do you make your own thumbnails? Yours are top tier!!!

I just discovered your series and it is amazing. I am Trying to do something similar on my homelab since a year ago but still failed. I already had some id maps in place for my mounts (more in my next comment on that video) but you essentially solved for me what I was struggeling and nearly given up. Now Jellyfin is HW rranscoding on my NUC Lab host and I am so happy with it :D One more thing that I am currently struggeling with - and you might have an idea / solution / future video: Docker swarm seems not to work inside an lxc container. Containers get deployed but are not accessible via the ingress network. Anyways thanks again I am looking forward to the new videos while watching the back catalog.

@Jims-Garage

5 ай бұрын

Great work 👍 Firstly, don't use the KVM image, use a standard cloud image (there's an issue). Let me know if that solves it.

@alexanderos8209

5 ай бұрын

@@Jims-Garage Thank you - I got it working on a debian 12 lxc container. Some of the IDs needed to be different but now it is merged with my lxc mounts and everything is working. If i now only could get docker swarm to work. (but this a known problem in LXC - works fine in VM).

I dont belive I see this, you are the best

Hardware passthrough to LXC is definitely something I want to explore. I have a few services running in an Alpine QEMU and the footprint is small but I would prefer to have one LXC per service

@robertyboberty

Ай бұрын

I started down the hardware passthrough rabbithole with CUPS. Network printing is another use case

What is the command to get the gid or uid when you mention LXC namspace or host namespace? Greate video i hope this help me to solve the HWA.

this is way cool ! LXC all the way

@Jims-Garage

3 ай бұрын

Thanks, it's a great tool to have.

2:40 actually it's for some intel gpus possible to split between vms. but didn't do any benchmark on it and had no use, so i went for priviledged lxc at the time i was setting up mine. but now i'm considering redoing it unpriviledged, thanks for the video!

@Jims-Garage

5 ай бұрын

It was. Unfortunately it's now discontinued...

@sku2007

5 ай бұрын

@@Jims-Garage right, there are lots of tiny differences on intel gpus. had it running with an 7700k about a year ago, i think this still would work today if the hw supports it (?) also played around with a DVA xpenology vm, unfortunately the 7700 igpu is too new for that

@Jims-Garage

5 ай бұрын

@@sku2007 my understanding is that you have to use sr-iov now.

@vitoswat

5 ай бұрын

​​@@Jims-Garage as long as you have older GPU it works but it is quite limited. On mini PC with i5-10500T I was able to split iGpu into 2 GVT devices. Interesting part is that even if you assign vGPU to VMs you can still use real iGPU in LXCs. Of course the performance will suffer this way but in case of load like transcoding it is perfectly fine. I suggest you give it a try.

@BoraHorzaGobuchul

5 ай бұрын

There is a video where a passthrough nvidia GPU is split between vms.

amazing. use for my iGPU. are there any other devices apart of the GPU in addition to video and render? can i not pass all the functions to the LXC or virtual machine? On my system it says the iGPU is the same IOMMU group as the USB controllers and such. So i can't pass it through the the VM, would it be possible the share the iPU among VMs?

Great video! How does this work with nvidia drivers with a GPU? Does the driver need to be installed on the host and then in each LXC?

@Jims-Garage

5 ай бұрын

Yes it does

Excellent, thanks! Was able to get it working with the iGPU. Is the process the same for a NVIDIA GPU?

@Jims-Garage

15 күн бұрын

It should be, just change the mappings to suit. Thanks a lot for the donation!

It's possible do the same thind with an external disk drive and an unprivileged LXC? I try to do a Proxmox Backup Server in this scenario and backup on a USB external disk drive, I managed to install with no problems PBS, but failed multiple time the USB hard drive passthrough... If enyone has some useful hints it will be nice...

Ty for sharing your knowledge Two questions if you may know the answer? 1. Can Proxmox install Nvidia linux drivers over Nouveau and still share the video card? 2. If one adds a newer headless GPU like the Nvidia L4, can you use this as a secondary or even primary video card in a VM or CT?

@Jims-Garage

5 ай бұрын

Yes to both. Follow the same procedure and mount the additional GPU.

Is there a way to do the opposite? As in consolidate multiple GPUs, RAM etc. into one server? I have 2 laptops and an external GPU I want to connect together to combine their compute to then be able to redistribute it out to multiple devices similar to this video. Is it possible?

@Jims-Garage

5 ай бұрын

I don't think so. The closest I could imagine is pooling the resources into a Kubernetes cluster or docker swarm.

I tested this out and Jellyfin worked great in a Proxmox LXC container also with Intel A380 passthrough. Can you please make a guide on how to get it running on Plex? I could not get Plex working with Hardware Acceleration for the life of me.

Has anyone gotten this working without running JF in Docker? Or is there anyone who can point me to documentation for commands/configurations for JF for the "group add" and "devices" variables from the yaml for docker compose?

Hello, it's possible on the same PVE node to have a split gpu for LXC and for VM ? Thanks for this good video.

@Jims-Garage

4 ай бұрын

Not possible with the same GPU as VM requires the GPU is not loaded by the host. Dual GPU would work.

@sebgln

4 ай бұрын

@@Jims-Garage that was what it seemed to me, thanks. (I am French and you are easy to understand)

Jim in your video why is it after you edit the conf file and boot up the 104 container that when you run ls -l /dev/dri the render is showing group ssh 226, 129, shouldnt it be render 226, 129

@Alkaiser88

5 ай бұрын

on my CT the render group is 106 but when I try to edit the conf file and use lxc.idmap: u 0 100000 65536 lxc.idmap: g 0 100000 44 lxc.idmap: g 44 44 1 lxc.idmap: g 45 100045 62 lxc.idmap: g 106 104 1 lxc.idmap: g 107 100107 65428 it fails to boot. it only works if I use lxc.idmap: u 0 100000 65536 lxc.idmap: g 0 100000 44 lxc.idmap: g 44 44 1 lxc.idmap: g 45 100045 62 lxc.idmap: g 107 104 1 lxc.idmap: g 108 100108 65428 but again its showing the /dev/dri is in group _ssh for me instead of render on my CT do we need to edit the conf file before the first boot to have render grouped to 107?

@rotesblut9904

2 ай бұрын

hello, have you figure it out? how to change the group of renderd128 to render?

how can we do this for an NFS share on an Unprivileged LXC?

How does it work with dedicated GPUs? Do I need to install the driver on the Proxmox Host or in the LXC? Do I need to specify the card in the docker compose or is the ID enough? Do I need the Container Toolkit for Docker? I really like your content, one of the best channels right now about selfhosting, but haven't found a solution to this.

@Jims-Garage

3 ай бұрын

The video is using a dedicated intel arc a380 GPU. For Nvidia you should be able to follow the same process. I believe most modern OS will have drivers but you might need to add them.

@copytoothpaste

3 ай бұрын

@@Jims-Garage Thank you for the answer. I'll try it.

How exactly do i get this running with a different user other than root? You said that you could do this through somewhere that you mentioned in the start of the tutorial, but i cant seem to figure it out. Pls help hahaha

What you say 6 minutes into the video about the /etc/subgid file is wrong. These entries are not mappings but ranges of gid's. It's a start gid and a count. I'm still trying to get my head dialled in on the lxc.idmap entries in the .conf file. Getting closer. Thanks for the video.

@Jims-Garage

5 ай бұрын

The subguid is a moot point if you're running as root and can be skipped

@PODLine

5 ай бұрын

@@Jims-Garage, what about adding root to the video and render groups on the host (@12:30)...is that necessary? This is a weird step to me.

Really awesome! But how is this working on the technical level without GPU virtualization at all?

@Jims-Garage

5 ай бұрын

The LXC is sharing access with the host's GPU

I did not catch this quite right -so is this a way that works only with many LXC+Docker inside or many LXC+ anything inside. That is - can i run, say, 4 LXC debian containers and in each one of them, one Windows 10 VM? If so - it is interesting and great! Otherwise (LXC+Docker)... isn't it already possible to share GPU with every docker container after installing nvidia cuda docker, and pass -gpu all

@Jims-Garage

5 ай бұрын

Unfortunately you cannot have a windows LXC. You could use this for a Linux desktop though with GPU acceleration. E.g., you could have a Linux gaming remote client

@theunsortedfolder6082

5 ай бұрын

@@Jims-Garageso, you are saying: yes, it is not exclusive for LXC+Docker, but anything running in LXC can access gpu? If so, what would one get just for sake of having it: proxmox > LXC (debian with gpu) > cockpit > windows VM > gpu intensive app like game or cad software?

Great guide. I just got this working for two LXC and Jellyfin. I am trying to use Plex in a Docker container but can't get the hardware transcoding to work. Can anyone help?

@Jims-Garage

3 ай бұрын

Check the docs here, it's what I use. Almost identical: github.com/linuxserver/docker-plex

@narkelo

2 ай бұрын

@@Jims-Garage great video! I got it working with Jellyfin just like in your video, but under Plex(using the link you provided) I get "No VA display found for device /dev/dri/renderD128" in the Plex transcoder settings it recognizes the iGPU, "lshw" in the container also sees the iGPU. any ideas you can share would be a big help. thanks!

@Jims-Garage

2 ай бұрын

@@narkelo It's likely t o be permissions with the Plex user. Try running as root then dial it back if that works.

Hey Jim, thanks for your thorough tutorials. Most of my home lab setup is done with the help of your videos. Following this specific tutorial a question arises: since the Jellyfin CT is using the host's hardware, should I enable the GPU passthrough prior to sharing my iGPU to the Jellyfin CT, or doing the groups mapping trick is enough? By 'sharing' the iGPU does it mean I can still use it for the proxmox host (if I have to connect a display straight to my server and access the proxmox CLI)?

@Jims-Garage

20 күн бұрын

By following this video the host and the CTs can use the iGPU. If you do a passthrough then only that VM can use it (the host cannot).

@stanislavtodorov8705

20 күн бұрын

That's awesome! And thanks for the swift response. In fact, thank you for all the awesome work you do!

@stanislavtodorov8705

19 күн бұрын

@@Jims-Garage mapped the uid and gid values as you did; 1. I am able to see my renderD128 device in the LXC: crw-rw---- 1 nobody _ssh 226, 128 Jul 4 07:53 renderD128; 2. lspc icommand in the LXC shows: 00:02.0 VGA compatible controller: Intel Corporation CometLake-S GT2 [UHD Graphics 630] (rev 03) 3. Installed intel gpu tools and ran intel_gpu_top - Failed to initialize PMU! (Permission denied) 4. Installed Jellyfin and ffmpeg6, then proceeded to enabling transcoding, following the official Jellyfin guide. when trying to check the supported QSV / VA-API codecs gives me the following output: root@Jellyfin:~# /usr/lib/jellyfin-ffmpeg/vainfo --display drm --device /dev/dri/renderD128 Trying display: drm Failed to open the given device! I will appreciate any help.

@stanislavtodorov8705

18 күн бұрын

@@Jims-Garage nevermind. I had mistake in my gid mappings. In your video you map gid 107 (LXC) to 104 (Host). In my case i had to map 106 to 104. Slightly changed my mappings and everything works as a charm now! Thank you once again for the sublime tutorial

@Jims-Garage

18 күн бұрын

@@stanislavtodorov8705 you're most welcome. Perhaps the difference between iGPU and dGPU

Impressive, I wonder if its as simple with an AMD igpu, with an xcp-ng hypervisor, probably not. But it is amazing to share an igpu like this, multiple graphic cards is rediculous. Seems like this solves gpu sharing in general 🤔

@Jims-Garage

5 ай бұрын

It should work on Proxmox with an iGPU in almost exactly the same way, I've no experience with xcp-ng though... SR-IOV is also another way to do it but consumer devices don't typically support it.

Is this possible with LXD?

Can you shed some light on why the container would start but then not show anything on the console but a black screen? In my case I'm running an unmanic container on Debian 12 and followed the guide and while I don't get any errors, my Console is just a black screen (but the container shows up on my network - it's unreachable though)

@Jims-Garage

8 күн бұрын

Can you SSH?

@brick4667

6 күн бұрын

@@Jims-Garage ​ Alright so strangely enough it must just take a very long time to startup because going back to the container console after a while does present a prompt. However, now I cannot access the app interface (in this case - unmanic) via the IP:PORT but the IP does show up on the network

I lost SSH and terminal login access after updating my container. How do I get it back?

@zapatista8784

4 ай бұрын

me too. how did you solve it?

If I am using an nvidia card do i need to install he drivers for it on the Proxmox host first?

@Jims-Garage

20 күн бұрын

You'll need the same drivers on the host and LXC.

@sohail579

20 күн бұрын

@@Jims-Garage I have 2 GPUs a 4090 for my gaming VM and a Quadro P2000 for my LXCs for Plex, Frigate etc.. will the driver on the host cause me issues with my 4090 and VM passthrough?

@Jims-Garage

20 күн бұрын

@@sohail579 you should be fine, drivers are device specific. I imagine the same driver will be used for both of those cards. A 4090 is wild for a gaming VM. Jealous!

@sohail579

20 күн бұрын

@@Jims-Garage Thanks it used to be a 3090ti but found a deal I couldn't refuse so sold the 3090 and took the plunge on the 4090 and I have plenty of cores with my TR Pro 5975wx its prob not the best thing but I have 1 box which does it all as im only home-lab-ing - and let me tell you that you have been a god send I have really been ramping up what my server does since i came accross your channel you explain so well keep up the good work.. now to go figure out the NVIDIA drivers

You do have an error in your github notes. After carefully following the directions and c/p from your notes I thought it odd when no /etc/subguid could be found. Still I proceeded but the container wouldn't start. After looking around a bit I noticed that /etc/subguid should have been /etc/subgid. After fixing the issue the container started just fine. Regardless, great video and you gained a new sub. Thanks..

@mnejmantowicz

5 ай бұрын

OMG! Thank you for that! I've been pulling my hair out.

@Jims-Garage

5 ай бұрын

Thanks! I will fix this now.

Three questions: 1) Have you tried gaming with this, simultaneously? 2) Have you tested this method using either an AMD GPU and/or a NVIDIA GPU? 3) Do you ever run into a situation where the first container "hangs on" to the Intel Arc A380 and wouldn't let go of it such that the other containers aren't able to access said Intel Arc A380 anymore? I am asking because I am running into this problem right now with my NVIDIA RTX A2000 where the first container sees it and even WITHOUT the container being started and in a running state -- my second container (Plex) -- when I try to run "nvidia-smi", it says: "Failed to initialize NVML: Unknown Error". But if I remove my first container, than the second container is able to "get" the RTX A2000 passed through to it without any issues.

@Jims-Garage

5 ай бұрын

1. No, not sure how I'd test it. Would have to be Linux desktop environment I assume. 2. No, but the process should be identical, it's not intel specific. 3. No, haven't seen that issue. As per the video I created 4 and all had access and survived reboots etc

@ewenchan1239

5 ай бұрын

@@Jims-Garage 1. I would think that if you ran "apt install -y xfce4 xfce4-goodies xorg dbus-x11 x11-xserver-utils xfce4-terminal xrdp", you should be able to at least install the desktop environment that you can then remote into and install Steam (for example) and then test it with like League of Legends or something like that -- something that wouldn't be too graphically demanding for the Arc A380, no? 2. The numbers for the cgroup2 stuff that you have to add to the .conf changes depending on whether it's an Intel (i)GPU (or dGPU) vs. NVIDIA. i.e. with my Nvidia RTX A2000, I don't have that RenderD128 option or whatever it is that it corresponds to. 3. Are you able to test passing the same GPU between from a CT to a VM and back? This is the issue that I am running into right now with my A2000 where my VM won't release the GPU, even after the VM has been stopped. The CT will report back (when I try to run "nvidia-smi") "Failed to initialize NVML: Unknown Error". However, prior to shutting down my LXC container and starting the VM, the CT is able to "see" and use said A2000 (as reported by "nvidia-smi") when I am running a GPU accelerated CFD application. Shut down the CT, start the VM, run the same GPU accelerated CFD application, shut down the VM, and start the CT again -- that same GPU accelerated CFD application now won't load/utilize said A2000 and "nvidia-smi" will give me that error. So I am curious if you're running into the same thing, if you were to try and pass the GPU back and forth between VM CT.

@Jims-Garage

5 ай бұрын

@@ewenchan1239 I could do that by installing a desktop or game I think. I think the issue you're facing is that because you're using a VM for passthrough you're likely blacklisting devices and drivers. This would stop the host being able to share the GPU with the LXC

@ewenchan1239

5 ай бұрын

​@@Jims-Garage "I think the issue you're facing is that because you're using a VM for passthrough you're likely blacklisting devices and drivers. This would stop the host being able to share the GPU with the LXC" But you would think that when the VM is stopped, it would release the GPU back to the host, so that you can use it for something else, e.g. a LXC.

Is there a chance this setup can be broken with a future update? That being said is safer to pass through gpu and hdd to a vm so you won’t have to worry about your pass through hardware from not being pass through.

@Jims-Garage

Ай бұрын

Yes, kernel updates can break this without following specific procedures. VMs don't have that problem.

@mg3299

Ай бұрын

⁠@@Jims-Garagedo you have the specific procedures so it won’t break when there’s a kernel update?

@Jims-Garage

Ай бұрын

@@mg3299 there's a handy script here, but do take time to understand it. github.com/tteck/Proxmox

@mg3299

Ай бұрын

@@Jims-Garage are you referring to the hardware acceleration script? If yes I am reading the script and correct if I am wrong but I believe the script requires the container to be a privileged container which is not a good thing.

Did I miss a previous step? I have no /dri folder under /dev "ls: cannot access '/dev/dri': No such file or directory"

@zabu1458

3 ай бұрын

Not sure if I should just edit my comment, but... I'm just dumber than I thought. I had a gpu passthrough to a vm. I just removed the gpu from the hardware of that vm and shut it down. But since it's been a while I forgot that I actually had to edit GRUB so proxmox won't load/use the GPU itself. i just removed the extra stuff from this line from /etc/default/grub: GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt pcie_acs_override=downstream,multifunction nofb nomodeset video=vesafb:off,efifb:off" so it would be back at GRUB_CMDLINE_LINUX_DEFAULT="quiet"

@hiteshhere

3 ай бұрын

@@zabu1458 Thanks for taking your time and shareing this. It helped me revolve mine :)

@Bruno-vz8vk

Ай бұрын

@@zabu1458 Very Interesting, I have the same problem. When I change my grub, and restart, it won't add the /dri folder in /dev, but my frigate lxc won't start... I effectively tried multiple tutos to do gpu passthrougt... may i have to do another action to see again the /dri?

@Bruno-vz8vk

Ай бұрын

Found ! had to edit /etc/modules and remove : vfio vfio_iommu_type1 vfio_pci vfio_virqfd

After days of struggling between guides on the internet I was able to install the NVIDIA drivers on the host. I have tried to install the drivers in the lxc without success. How did you get yours to work? Thank you for the answer, and thank you for the awesome guide.

@Jims-Garage

3 ай бұрын

I'm using an intel arc a380 GPU. The drivers are baked into the OS. It's definitely possible with Nvidia though, I'll try to find some instructions.

can i share my gtx 1650 between couple of vms or not ?

@Jims-Garage

5 ай бұрын

Yes, there is a hack for it using vGPU. For an LXC you can follow this video (but it's Linux only).

@systemmodmen2157

5 ай бұрын

i forget a an important one of the vms is a windows vm and this pc is under my tv can i accses the gpu with hdmi and play directly from it or not and thanks for the respond @@Jims-Garage

What if I want to add multiple GPUs?

@Jims-Garage

Ай бұрын

That should be possible, you'd need to follow the same process and add the other devices. I haven't ever done it though (perhaps in future).

Would there be a use case for a higher end card like a spare RYX 3070?

@Jims-Garage

5 ай бұрын

This solution is GPU agnostic, you can use whatever you want.

Is this simplified if one were to go with a privileged container?

@Jims-Garage

5 ай бұрын

A privilegeled LXC doesn't require the idmap, you can simply mount

@ckthmpson

5 ай бұрын

@@Jims-Garage Thanks. Might try the unprivileged method...just seems like a rather complicated process which would be simplified in the privileged scenario. Do realize the security implications.

@Jims-Garage

5 ай бұрын

@@ckthmpson if it's simply for internal applications you're probably okay

Can i ask why you're running docker inside of a lxc container ?

@Jims-Garage

5 ай бұрын

Why not? Simplifies deployment as I have all of the compose files ready. You could do it manually.

@basdfgwe

5 ай бұрын

@@Jims-Garage Does it provide any advantage of containerising insider of a container ? Don't get me wrong I have docker containers running on unraid, which is running on proxmox....But my reason is: I made a mistake putting my storage on unraid and shifting from unraid is going to cost 000s.

@Jims-Garage

5 ай бұрын

@@basdfgwe think of the LXC as a virtual machine. It's the same as running a standard docker instance.

@texasermd1

5 ай бұрын

What would this look like with a high end GPU like a GTX 3070?

@PODLine

5 ай бұрын

I do the same as Jim and it makes perfectly sense (to me). As a starting point, you could see docker as app containers and lxc as OS containers.

I think this might need updating. I followed this exactly but didn't work.

@Jims-Garage

29 күн бұрын

Worth hopping onto Discord, this still works for me.

@yoshidis4

29 күн бұрын

@@Jims-Garage Thanks but no thanks, that app needs my phone number for some reason, I don't want to get robocalls from them. Do you have anything better set up, like Slack?

Is Jim ai generated?

@Jims-Garage

5 ай бұрын

"No, he is real" - JimBotv2.0

So I've been playing around with this some more, and found that if I deleted the VM, and was ONLY running LXC containers (right now, I am using all privileged containers -- haven't tested with unprivileged containers yet) -- I am able to have multiple LXC containers do different things with my RTX A2000. Going to be testing with gaming next, so we'll see. But yeah - it would appear that I can't have both VMs and CTs on the same host, sharing a GPU. I can either have ONE VM using the GPU at a time, or I can have NO VMs (at all, on the host, that uses the GPU), and at least a few LXC containers, sharing the one GPU.

@Jims-Garage

5 ай бұрын

Yes, makes sense as a VM requires isolation of the hardware, a LXC doesn't.

@ewenchan1239

5 ай бұрын

@@Jims-Garage But the crazy thing is that you would think that when the VM ISN'T running, that the LXC should be or ought to be able to use the "free" GPU that isn't being used/tied to a VM anymore. That doesn't appear to be the case. It wasn't until I removed said VM, did it "release" the GPU back over to the LXC containers.

@Jims-Garage

5 ай бұрын

@@ewenchan1239 I could be wrong but it sounds like you aren't blacklisting the drivers and device completely. To my knowledge the LXC wouldn't work with hardware passthrough if you were as the host won't be loading drivers

@ewenchan1239

5 ай бұрын

@@Jims-Garage "I could be wrong but it sounds like you aren't blacklisting the drivers and device completely." I'm at work right now, so I'll have to pull my config files later, when I get back home. *edit* Here are the config files: /etc/modprobe.d/nvidia.conf blacklist nvidia blacklist nouveau blacklist vfio-pci /etc/default/grub GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt pcie_acs_override=downstream nofb nomodeset initcall_blacklist=sysfb_init video=vesafbff,efifbff vfio-pci.ids=10de:2531,10de:228e disable_vga=1" /etc/modprobe.d/vfio.conf options vfio-pci ids=10de:2531,10de:228e disable_vga=1 /etc/modprobe.d/kvm.conf options kvm ignore_msrs=1 /etc/modprobe.d/iommu_unsafe_interrupts.conf options vfio_iommu_type1 allow_unsafe_interrupts=1 /etc/modprobe.d/pve-blacklist.conf blackllist nvidiafb blacklist nvidia blacklist nouveau blacklist radeon /etc/modules vfio vfio_iommu_type1 vfio_pci vfio_virqfd nvidia nvidia-modeset nvidia_uvm Yeah...so that's what I have, in my config files. As far as I can tell, it's complete (because it works for both VMs and CTs, just not being able to pass the GPU back and forth between said VM(s) and CT(s)). But between CTs, not a problem.

@ewenchan1239

5 ай бұрын

@@Jims-Garage "To my knowledge the LXC wouldn't work with hardware passthrough if you were as the host won't be loading drivers" Updated my previous comment. With the config information that I just shared, it works for both VMs and CTs - just not when they exist on the same host, at the same time.

does not work for me FFmpeg gives this error [AVHWDeviceContext @ 0x642ff9562240] No VA display found for device /dev/dri/renderD128. Device creation failed: -22. [h264 @ 0x642ff954c540] No device available for decoder: device type vaapi needed for codec h264. Stream mapping: Stream #0:0 -> #0:0 (h264 (native) -> h264 (h264_vaapi)) Stream #0:2 -> #0:1 (aac (native) -> aac (native)) Device setup failed for decoder on input stream #0:0 : Invalid argument

@Jims-Garage

4 ай бұрын

What are you trying to pass through?

@ziozzot

4 ай бұрын

@@Jims-Garage I tried passing through the iGPU without success. I then attempted it with a privileged container, and it works. I installed Jellyfin directly in the LXC without Docker. Probably there is an issue with the permissions.

@ziozzot

4 ай бұрын

with the help of ChatGPT i figured out the config that works for me lxc.cgroup2.devices.allow: c 226:0 rwm lxc.cgroup2.devices.allow: c 226:128 rwm lxc.mount.entry: /dev/dri/renderD128 dev/dri/renderD128 none bind,optional,create=file lxc.idmap: u 0 100000 65536 lxc.idmap: g 0 100000 44 lxc.idmap: g 44 44 1 lxc.idmap: g 45 100045 59 lxc.idmap: g 104 104 1 lxc.idmap: g 105 100105 65431

You have solved just one of my little problems , I've moved jellyfin form one server to another and frigate VA worked , but jellyfin was giving me a error . Stream mapping: Stream #0:0 -> #0:0 (h264 (native) -> h264 (h264_amf)) Stream #0:1 -> #0:1 (aac (native) -> aac (libfdk_aac)) Press [q] to stop, [?] for help [h264_amf @ 0x557e719b81c0] DLL libamfrt64.so.1 failed to open double free or corruption (fasttop) Could not work it out it was, it was from a backup so the same configs etc , look at your notes and there was a OOPs forgot to the. usermod -aG render,video root Now all working again .

@Jims-Garage

5 ай бұрын

Awesome, glad it's fixed