PostgreSQL Backup & Point-In-Time Recovery

To get the commands used in this tutorial, visit: www.scalingpostgres.com/tutor....
Learn how to take a physical backup of an entire PostgreSQL database cluster and then restore it to a specific point in time. Also known as Point-In-Time Recovery or PITR. We not only show you the backup process but the restore process as well. We also highlight some differences between PostgreSQL 10 and PostgreSQL 9.
Want to learn more about Postgres performance?
Join my FREE mini-course called the PostgreSQL Performance Starter Kit here: www.scalingpostgres.com/cours...

Пікірлер: 71

  • @RU-qv3jl
    @RU-qv3jl4 күн бұрын

    Thank you for your all your videos. As a SQL Server DBA with an interest in PostgreSQL it’s extremely helpful that you share so much information in a nice and easy to understand way.

  • @badaldudy9624
    @badaldudy96243 жыл бұрын

    Thanks for tutorial and series, Great job

  • @unmatal
    @unmatal Жыл бұрын

    Thank you very much for a detail and informative tutorial

  • @balvanthreddy
    @balvanthreddy3 жыл бұрын

    Hi , Im new to PostgreSQL.. but it's very easy to understanding concepts. thanks for your video.

  • @ScalingPostgres

    @ScalingPostgres

    3 жыл бұрын

    Welcome!

  • @muhanadtur8785
    @muhanadtur87854 жыл бұрын

    Thank you for the great explanation.

  • @frankchen9264
    @frankchen92643 жыл бұрын

    Thanks for your video. It's very helpful for me.

  • @abhishekyadava
    @abhishekyadava3 жыл бұрын

    Thanks for sharing. Very nice presentation.

  • @YogeshKushwahaindia
    @YogeshKushwahaindia5 жыл бұрын

    A nice and clear explanation, if you could share the document in text format. It would be even more useful. Thanks for the tutorail.

  • @samilekesiz1354
    @samilekesiz13544 жыл бұрын

    Thanks, great job and free. Awesome :)

  • @vikramgodara8468
    @vikramgodara84683 жыл бұрын

    Hi, Thanks for sharing the wonderful information it's helping me to compare the operation in Postgres as I am oracle dba.

  • @ScalingPostgres

    @ScalingPostgres

    3 жыл бұрын

    You are welcome!

  • @chiragkarkera1
    @chiragkarkera1 Жыл бұрын

    Best explanation. Thank You

  • @calebmbugua745
    @calebmbugua7453 жыл бұрын

    Thanks so much...i really have enjoye this

  • @vedraj7247
    @vedraj72474 жыл бұрын

    Thanks for this tutorial, i have a query. While you were doing point-in-time recovery up to some specific time, you didn't use all recent Wal files from your achieve directory. You have only used the base backup files that only contains all the when you have taken the base backup and it did not contains other two row which you have added later.

  • @amardeepkumar9301
    @amardeepkumar9301 Жыл бұрын

    Great explanation .thank you

  • @violetsania7314
    @violetsania73144 жыл бұрын

    thank you for the tutorial

  • @code7bit301
    @code7bit3013 жыл бұрын

    v12 PostgreSQL the move has been made, and the recovery.conf parameters are now part of postgresql.conf , recovery.conf (/etc/postgresql/12/main/postgresql.conf) triggered recovery mode. Since the file is gone new file have taken its place: recovery.signal: tells PostgreSQL to enter normal archive recovery it can be empty. just in case someone stumbles like i did :)

  • @uqroztrk

    @uqroztrk

    2 жыл бұрын

    I added these parameters in postgresql.cfg but it didn't work. restore_command = '/usr/sbin/nsroapprecover -o pg_p_opt="%p" -o pg_f_opt="%f" -z /nsr/apps/config/postgres.cfg' recovery_target_time = '2021-11-04 14:18:00 +03' recovery_target_inclusive = false

  • @f5edy117
    @f5edy11711 ай бұрын

    man you helped me a lot

  • @gusjoagomez
    @gusjoagomez2 жыл бұрын

    Great video!! Thanks!!!!

  • @shiwanginishishulkar4744
    @shiwanginishishulkar47444 жыл бұрын

    nice explanation

  • @bhanukonthala6223
    @bhanukonthala62232 жыл бұрын

    Thanks for the tutorial and i want know how we can add the recovery.conf to particular directory?

  • @voffknur
    @voffknur4 жыл бұрын

    Thank you. 👍

  • @ranvijaymehta
    @ranvijaymehta2 жыл бұрын

    Thank you so much

  • @srilinux2490
    @srilinux24904 жыл бұрын

    Thanks for the procedure. I am trying to do PITR for individual PostgreSQL. Please provide me the reference document to do. Thanks

  • @ab5sr
    @ab5sr4 жыл бұрын

    Thanks for the tutorial and series. Great job. SQL Server guy, here, so there is some pain :\ Do you answer questions posted here - I see a few with no response...

  • @mahesh3267
    @mahesh32672 ай бұрын

    This really helps. can this be done in AWS Aurora postgres serverless ?

  • @Draz-
    @Draz-2 жыл бұрын

    If you do a PITR, doesn't all the data inserted after the restore point still get saved to the WAL files? So after the restore procedure, those WAL files still exist, right? Wouldn't that cause issues for future restores? You essentially wanted to get rid of that data, but if you were to do a new restore without specifying a restore point, the data would return.

  • @suend
    @suend5 жыл бұрын

    Thanks for your tutorial. When using pg_basebackup, it will not overwrite files if exists. How would you schedule recurring daily backup?

  • @ScalingPostgres

    @ScalingPostgres

    5 жыл бұрын

    I use a shell script that grabs the current date and appends it to the file name that pg_basebackup writes to.

  • @AndeerStar
    @AndeerStar2 жыл бұрын

    Hello. In the archive_command parameter where you put 'test ! -f /home/...' this 'test' is the name of the data base I have or leave it as you added it to the command line with the test itself?

  • @MichaelJendryke
    @MichaelJendryke4 жыл бұрын

    Currently I use a pg_dump -> pg_restore strategy for a 4TB db, no fun. This tutorial convinced me to try continuous backup, thanks @Scaling Postgres. Once I do pg_basebackup it also saves my indices and materialized views, correct? Also, what should I do with the recovery.conf file after the db has been recovered? If I leave the file where it is, will it recover the db everytime I restart postgres? .. would not make sense...

  • @Piyushmishrabanaras

    @Piyushmishrabanaras

    3 жыл бұрын

    Hi Michael- When recovery completed, recovery.conf file gets renamed and cluster gets opened for read write operation. Recovery.conf -----> recovery.done It happens automatically, you don't need to rename manually

  • @konireddylavanya5486
    @konireddylavanya54864 жыл бұрын

    hi, nice and clear explanation , but i have a small doubt because am new to this... what is systemctl in this command "sudo systemctl restart postgresql@10-main"

  • @MichaelJendryke

    @MichaelJendryke

    4 жыл бұрын

    It's a Linux specific command for systemd to manage services.

  • @vishwas22khare
    @vishwas22khare3 жыл бұрын

    Sir did you install posters on Ubuntu . Bcoz I have installed postegres in Windows and Ubuntu using virtual box

  • @shantanushekharsjunerft9783
    @shantanushekharsjunerft97833 жыл бұрын

    I am wondering why you are doing "psql -c select pg_switch_wal();"? Is it because you are trying to tell the Postgres to generate the WAL file even though it may not have reached 16MB?

  • @JustinPierce01

    @JustinPierce01

    3 жыл бұрын

    that's my guess too -- write partial WAL files to disk

  • @ikmaljohari1367
    @ikmaljohari13672 ай бұрын

    Hi, I'm new to postgres. Have one question regarding your tutorial. If I created new user aside from postgres user, how can I run the wal archive for the new user?

  • @mohitchauhan4154
    @mohitchauhan41546 жыл бұрын

    m doing same thing according to your video but my log file not saving in db_file_backup.can u help me please.what is the reason behind this .

  • @leon6295
    @leon62952 жыл бұрын

    Can you please make a video to do postgres 13 incremental backup and replication in windows version

  • @lalithadurga9677
    @lalithadurga9677 Жыл бұрын

    Hi , default archive log file size is 16mb and can we reduce the size of archive log file.(i use the compressed version and it will reduce 16mb to 20k).but with out compression i need other solution .can u advise in this ?

  • @skizzi86
    @skizzi862 жыл бұрын

    Is it really necessary to use the restore_command if not doing a point in time recovery? The basebackup generates a full valid backup and so it should not be necessary to use the restore_command to use wal files from the archive folder, right?

  • @ellispercival6177

    @ellispercival6177

    Жыл бұрын

    In this case, he rotated his WAL, then made a basebackup, so it wouldn't actually need to replay any of the archived WAL files. If he'd have done a basebackup, then inserted some rows, then rotated the WAL, then deleted and restored the basebackup, then it would have actually used the archived WAL to insert the rows that were missing from the basebackup.

  • @sostenesapollo
    @sostenesapollo11 ай бұрын

    Do managed databases in aws and gcp have this built-in ?

  • @amandhaka20002
    @amandhaka200024 жыл бұрын

    Is it possible to make oracle RAC like solution in postgresql ?

  • @mohitchauhan4154
    @mohitchauhan41546 жыл бұрын

    tar backup is working .

  • @ScalingPostgres

    @ScalingPostgres

    6 жыл бұрын

    Great!

  • @md.imrulhasan8757
    @md.imrulhasan87572 жыл бұрын

    After running the command in my pg-13 sudo service postgresql restart It could not restart After the command when I use ~$ psql It shows "error: could not connect to server: No such file or directory Is the server running locally and accepting connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?"

  • @tmsa771
    @tmsa7715 жыл бұрын

    I’m trying to do incremental backup, For example when i do the backup for 3 rows and keep the backup inside folder as backup1, then the data inserted anther 3 rows (so i have now 6 rows inside table). After that i need to do another backup BUT i want only the LAST 3 rows inserted to be backed up as backup2 NOT the all 6 rows included inside the same table. can you explain how to do this

  • @ScalingPostgres

    @ScalingPostgres

    5 жыл бұрын

    Essentially, taking a physical backup of the files and incrementally applying WAL files is an incremental backup. So as this video describes, take a backup and then keep all the WAL files produced. Then you can replay them on a restored database. The WAL files essentially are your incremental backup.

  • @joakimolsson5210

    @joakimolsson5210

    5 жыл бұрын

    @@ScalingPostgres Does this mean you basically just make a cronjob with the select pg_switch_wal(); with the interval you prefer throughout the day? How does rotation and cleanup work?

  • @ScalingPostgres

    @ScalingPostgres

    5 жыл бұрын

    @@joakimolsson5210 You don't have to use pg_switch_wal(). It will do it automatically. However, that lets you archive a WAL file manually. There is no rotation of the archive. It produces a constant stream of WAL files. It is up to you to clean them up periodically. What I tend to do is zip them up send them to a longer term storage like Amazon S3 and then delete them after a few days.

  • @zr7927
    @zr79275 жыл бұрын

    Do I have to place an entry in the pg_hba.conf? After running the pg_basebackup command, it says fatal: no entry in the pg_hba.conf....

  • @sagar9762

    @sagar9762

    5 жыл бұрын

    @Zenina, update your pg_hba.conf with following lines(append the lines at the end of file), # Allow replication connections from localhost, by a user with the # replication privilege. local replication psql trust host replication psql 127.0.0.1/32 trust host replication psql ::1/128 trust After this please reload the process, pg_ctl stop -D /usr/local/pgsql/data/ pg_ctl start -D /usr/local/pgsql/data/ systemctl restart postgresql

  • @varuntaneja4654

    @varuntaneja4654

    Ай бұрын

    @@sagar9762 Still getting the same error

  • @varuntaneja4654

    @varuntaneja4654

    Ай бұрын

    It worked, as my user as postgres not psql

  • @mohammadrezaghofrani6914
    @mohammadrezaghofrani69144 жыл бұрын

    Can Anybody explain why he uses this part " psql -c "select pg_switch_wal();" "? Is it necessary?

  • @venichka556

    @venichka556

    4 жыл бұрын

    He switch the log to force wal archiving (change log file), it's just for the example.

  • @mohammadrezaghofrani6914

    @mohammadrezaghofrani6914

    4 жыл бұрын

    @@venichka556 So you mean that it is not necessary?

  • @rahulkarmakar1736
    @rahulkarmakar17364 жыл бұрын

    I did the same. But like ur video I did not get last 2 rows...like u didn't get..

  • @rodrigomendoza1060
    @rodrigomendoza10603 жыл бұрын

    hello, one question, is it possible to recover the database after applying a drop to the entire database?

  • @ScalingPostgres

    @ScalingPostgres

    3 жыл бұрын

    Only if you have a backup.

  • @malymohsem285
    @malymohsem2852 жыл бұрын

    Hey, I Have been stuck after the "pg_basebackup -Ft -X none -D - | gzip > /var/lib/postgresql/db_file_backup.tar.gz" command line. it shows "pg_stop_backup still waiting for all required WAL segments to be archived (3840 seconds elapsed)" which means it has already taken 1 hour and continuing. Is it normal or just happening to me?

  • @connorstange9170

    @connorstange9170

    8 ай бұрын

    Did you find a solution to this?

  • @connorstange9170

    @connorstange9170

    8 ай бұрын

    This is a simple fix actually, just remove all files in the pg_log_archive