One of the only backup tutorial videos I’ve seen that actually covers restoring as well. One of the biggest short comings of most peoples back up routines is they don’t know how to restore and don’t test the restore pretty much ever Love the videos 👍

@Jims-Garage

10 ай бұрын

Thanks, I totally agree. Just as important, if not more so, to test you can restore!

Brilliant. Thanks Jim!

thanks for the demo and info, have a great day

@Jims-Garage

Жыл бұрын

You're welcome, Chris.

Great video! So concise and to-the-point

@Jims-Garage

Ай бұрын

Thanks 👍

Suggestion for anyone watching this video: look up autorestic, it's basically a wrapper for restic that makes it easier to configure and allows to stop containers before making a backup and restart them after. I make a backup everyday at 4am to an nfs share from my truenas which pushes daily to backblaze b2 storage.

@ScottLeeAllen

4 ай бұрын

Also, check out backrest, stupid name so google doesn't help much unless you search "backrest git" it's a webui for restic

Looking forward to this one!

@Jims-Garage

Жыл бұрын

Hope you like it!

I think Duplicati does all three in one, can be docker'd and is available on win,mac,unix and accessible by browser. I only got into this yesterday, but some other channel just recovered his whole promox docker using duplicati from online mega storage. Because of this and the ease of use, I gave it a try. I was also interested in Restic because of your other backup video. But this seems less intuitive, thank though.

Great video and explanation, I was able to get it running very easily. Is there a way to also get notifications in case it fails to work? Don’t think it was covered yet it’s pretty important for security

@Jims-Garage

5 ай бұрын

Unfortunately there is no native notification system that I am aware of. You could create one with a script though, something I hope to do in the near future.

How can I backup multiple directories with different tag for each one?

Great demonstration, looking forward to implementing the solution. However, I do have a question about the containers where databases are running like postgres, mariabd, etc. I hear that we should be shutting down the containers before the backups are being taken. Does restic support this? or maybe you have a better way of doing it.

@Jims-Garage

Жыл бұрын

Thanks, I appreciate the feedback. If you're using sqlite you should be fine (as it's a file). However, using mariadb mysql etc, it's best to use the in-built backup method for that solution (typically you can exec into the container and do it - possibly via Cron). Otherwise, stop the container and run restic (again can be done as a Cron).

Great, thanks! Where would I get the backup id for restoring when restic container itself is lost? Any chance of restoring backups with a fresh restic instance?

@Jims-Garage

5 ай бұрын

Yes, redeploy and keep your keys the same. Should be fine.

so helpful! thank you. can this be used to create a restic server of sorts? not sure how to get my other linux devices to backup to this without putting a docker compose on each one...

@Jims-Garage

5 ай бұрын

This is designed for docker containers. I recommend Proxmox backup server for other Linux VMs (I have a video)

@evidentlynot6291

5 ай бұрын

@@Jims-GarageI actually used your video for a non-proxmox setup. I am so grateful for this. I just know some folks set up a restic server in a docker on the main host and then send backups to it from other devices. I am not sure how to do that, but honestly, this is good enough and is working amazingly well on both my devices.. so I don't think I care lol. Thanks again!

Great video! Do you have any experience with borg backup and how it compares to Restic?

@Jims-Garage

9 ай бұрын

Sorry, never used it. One to add to the list... I did use Restic for a few years and it was flawless.

@georgebobolas6363

9 ай бұрын

@@Jims-Garage Thanks for the prompt reply. From some reading I did on Reddit, seems Restic is indeed a better, more efficient and reliable solution, with the only downsides being the lack of a good GUI and occasional spikes in RAM usage. But I guess none of the those apply to server backups.

@Jims-Garage

9 ай бұрын

@@georgebobolas6363 agreed, set it and forget it. You don't really need a gui.

Very nice tutorial, was able to set it up, I have my portainer_data in a docker volume which is in a different location so i was able to map both my docker folders and the docker volume separately into the volumes path following your tutorial. I did a test restore and everything is working well. I do have a question, recovery makes sense when original vm is up and running (and I AM making backups of that vm). However, I'm having trouble thinking through how to use restic if I am restoring to a fresh machine without the original docker running. Will I be able to do this directly from command line with my encryption password without the docker layer?

@Jims-Garage

Жыл бұрын

Thanks. Yes, restic can be run as a local application. You don't need Docker, but you will need your encryption keys.

@victor2410

Жыл бұрын

@@Jims-Garage thanks! I'll experiment with this from a separate computer that has that path mapped to do a test restore just so I understand my options a bit better. Cool stuff!

@Jims-Garage

Жыл бұрын

@@victor2410 I love restic, awesome program 😎

Nice video. Do you have any experience doing this with sql db's without corruption?

@Jims-Garage

11 ай бұрын

I would recommend using SQL's inbuilt backup tool, and outside of that, use restic when the container is shutdown. Using it whilst in use can result in failed backups or data loss. You could script to shutdown prior to backup.

@50_Pence

11 ай бұрын

@@Jims-Garage jesus- that was quick. Thanks - I'll try this week with your video. I appreciate your help

@Jims-Garage

11 ай бұрын

@@50_Pence no worries, let me know how you get on.

Hey James! Are there any benefits for installing restic using docker as opposed to a regular binary install?

@Jims-Garage

4 ай бұрын

IMO yes, the configuration is saved as code. Every time you deploy it it's the same (and quicker).

@sapumalkalutota

4 ай бұрын

@@Jims-Garage Ah I see, Thank you!

How would you backup to sftp? I have tried but can't get it to work

@Jims-Garage

2 ай бұрын

I've not tried but I suspect you mount an sftp to the host and then map it to the container locally.

If you have Mutiple instances of Portainers do you need to run rustic on all or can you just pull from all of them

@Jims-Garage

4 ай бұрын

You need restic per docker instance.

@GeekyDrones

4 ай бұрын

And do you point the back up from to where your /config are? Because it keeps failing to find anything when I do...

@Jims-Garage

4 ай бұрын

@@GeekyDrones you need to backup to somewhere that exists

@GeekyDrones

4 ай бұрын

Sorry correct thats my synology nas share, but, where am I pulling from.

Is the docker-compose file correct? The restic main job has a repository env_var of "/restic", then later you bind a local folder on the host to the docker container volumes section but in the "check" and "prune" containers there is no binding of the host folder to the container (there is no volumes section at all), and the logs of both services gives me "0 snapshots" found.

@Jims-Garage

7 ай бұрын

Yes, it's correct. You can validate against the official repository.

@RamiKattan

7 ай бұрын

@@Jims-Garage is it optional? if I want to check and prune it "must" be done on the same repository of the restic backup, checking a different repository has no sense. am I correct?

@Jims-Garage

7 ай бұрын

@@RamiKattan the prune and check containers are optional. You don't need to use them. They do need to use the same folder though.

@altgr391

6 ай бұрын

@@Jims-Garage I have checked the official repos. Its seem not using local storage as a place to backup. I think we still need to mount the folder to prune and check as well

@squalazzo

2 ай бұрын

@@altgr391 i thought the same... how can they work without the volumes mounted?

you sure the docker-compose file is correct? in the other 2 containers, there's a restic repository = /restic but this path is not mapped on an actual volume like in the 1st one, so what they're supposed to check? it seems like the other 2 are using something that in kubernetes world is called "ephemeral storage"... which will be lost on k8s pod restart...

@Jims-Garage

2 ай бұрын

Yes, as demonstrated. The /restic is an internal folder within restic container that you map to your backups on your host.

@squalazzo

2 ай бұрын

@@Jims-Garage exactly, but it's mounted only to the 1st container in the docker compose... how do the other 2 know about that if not explicitely added as a volume to those 2?

@Jims-Garage

2 ай бұрын

@@squalazzo ok, looks like it has changed in more recent versions. Previously it would default to that location. Check here for the latest version github.com/djmaze/resticker/blob/master/docker-compose.example.yml

This is great and working. I got all my containers going just as you have outlined above. There is one thing though, I like to get a status emailed to me after a backup regardless of success or failure. I'm looing for a way to do that with this docker implementation. Is there an easier way?

@Jims-Garage

8 ай бұрын

There's no native support but you could integrate it into a Cron job etc. there are some user scripts on the restic forums.

@rickhernandez2114

8 ай бұрын

@@Jims-Garage That is exactly what I'm going to do. Looks pretty straight forward to get accomplished. The gem I got from this video though, was restic. So slick and better than just running an rsync. Thanks again!

@Jims-Garage

8 ай бұрын

@@rickhernandez2114 you're welcome, it's an awesome tool 🔥

Anyone have any idea why my snapshots are 200 bytes for a docker data over 9gb? It keeps happily making backups, but they're clearly not the correct size. It says "processed 208179 files, 9.270 GiB in 0:12" and then makes a snapshot thats tiny

@Jims-Garage

Жыл бұрын

Have the files changed much since the original backup? Remember it's differential backups, not full.

@damo_c

Жыл бұрын

@@Jims-Garage that was the first backup

@Jims-Garage

Жыл бұрын

@@damo_c hmm, that doesn't look right, possibly a permissions/access issue. Can you jump onto Discord and paste the logs?

@damo_c

Жыл бұрын

@@Jims-Garage away for the weekend with no access, will do when I get back. Thanks!

Isn't part of the backup process the backing up of the docker-compose/stack configurations from Portainer also important?

@Jims-Garage

6 ай бұрын

Absolutely, I cover that in the next video using rClone to Google Drive

I'm obviously missing something, but it seems to me that: If the original backup captures ALL the data available to be backed up, and all subsequent backups only capture the incremental changes, and all backups are eventually pruned (when they pass their 'use by date' in other words) eventually you will only have a lot of incremental pieces of data still saved, so, when you want to restore and Restic Restore runs back through all the available backups that exist, it may find less than the total amount of data required, because the original 'Full' backup was Pruned long ago. Where am I going wrong with this logic?🙄

@Jims-Garage

11 ай бұрын

You specify a backup schedule, those are full backups. Between those dates you specify an increment backup schedule. So I do weekly backups, daily incremental backup (aka snapshot).

@Dreamwoodinternational

11 ай бұрын

Thank you for the instant response again. Aha, I misheard that all subsequent backups were incremental - will watch again.

Hi there, I seem to be getting an error with the backup, below is the error log..... /etc/dockervolumebackup/conf.d was found, using configuration files from this directory. /root/entrypoint.sh: source: line 18: can't open '/etc/dockervolumebackup/conf.d/*': No such file or directory