xAPPnow Low Code Platform - Operations (Backup-Restore)
Backup and Recovery of the xAPPnow Low Code Platform is an important part of the DevOps strategy. To provide a backup mechanism tailored to the client’s needs, xAPPnow offers different ways to back up the databases. These are:
-
Offline Backup
-
Online Backup with Log Shipping
-
Streaming Replication
Offline Backups are the standard way of backup in case of an update or server restart. All xAPPnow services need to be stopped before starting this backup.
During the usual operations, the xAPPnow services are not shutdown. But regularly backups are equally important as continuous operations. For this, xAPPnow provide Online Backups, without shutdown, using Log Shipping. Each database transaction is written inside a log file, called write ahead log (WAL), before it was executed. The archiving time, when these logs are written into the log file, can be specified. It is important to find a good trade-off between data integrity and performance, so the interval should be monitored and adjusted. After each interval, the logs should be packed and moved to an extra folder.
If no logs are written, no archiving is done. It is recommended to do a full backup regularly and incremental backups in between.
Streaming Replication will stream/duplicate all database changes (external or internal) to another postgreSQL server instance. This is called synchronous replication, which leads to a high availability. The disadvantage is that all errors in the master are directly duplicated to all slaves. The advantage is that, if one database server is not working, a direct switch to another instance can be done.
The recommended configuration is Streaming Replication with Log Shipping. It leads to high availability with low risk of data loss and better performance. The data replication here is asynchronous, which means the backup server may fall back in time behind its master.
It is recommended to store all backups on an external physical storage for all backup mechanism.
Offline Backup
It is important to stop xAPPnow services before execute offline backup process, and start xAPPnow services after backup is finished.
-
Stop xAPPnow services./appollo-stop.sh <INSTANCE_NAME>
-
Execute backup command inside directory APPOLLO_HOME
./appollo-backup-create.sh <CONTAINER_NAME> <APPOLLO_BACKUP_DIR>
Parameter Explanation:
-
CONTAINER_NAME is the name of the database container. For xAPPnow instance should be stack<INSTANCE_NAME>dbhost_13. Start xAPPnow services
./appollo-start.sh <INSTANCE_NAME>
As an advice we propose to store backups on external media. The mechanism is designed to not interfere with custom backup policies.
Verify Offline Backup
If backup task finished successfully there should be created a backup file at APPOLLO_BACKUP_DIR
<APPOLLO_BACKUP_DIR>/<INSTANCE_NAME>_db-data.tar
Restore Offline Backup
On an unexpected situation like data loss or server crash the system can be restored using a previously created backup. It is important to stop xAPPnow services before execute backup restore process, and start xAPPnow services after backup restore process finishes.
-
Stop xAPPnow services./appollo-stop.sh <INSTANCE_NAME>
-
Execute backup command inside directory APPOLLO_HOME./appollo-backup-restore.sh <CONTAINER_NAME> <APPOLLO_BACKUP_DIR>/<BACKUP_DATE>/stack<INSTANCE_NAME>_db-data.tar
Parameter Explanation:
-
DATE is the date the backup was done3. Start xAPPnow services
./appollo-start.sh <INSTANCE_NAME>
Online Backup
-
Configure database for log shipping
Proceed to configure log shipping executing the following command
./appollo-configure-log-shipping.sh <CONTAINER_NAME> <SECONDS>
Parameter Explanation:
-
SECONDS is the number of seconds between backup update with the last transactions on the database
Example:
./appollo-configure-log-shipping.sh stackdevelop_dbhost_1 300
2. Configure ssh
To save a backup on a remote server as requirement it is needed on the local and remote server ssh keys configured for the user used to access via ssh. This is needed for rsync command to work as automatic task.
If the folder /home/REMOTE_USER/.ssh does not exists, the keys can be generated using the command:
ssh-keygen -t rsa
3. Accept the questions by previous command pressing enter key. Another requirement is to allow host public ssh key on the remote server. This can be achieved with the command:
ssh-copy-id -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -o GSSAPIAuthentication=no REMOTE_USER@REMOTE_IP
4. Sync archive backup directory volume to remote server
./appollo-sync-backup.sh <MOUNT_VOLUME_PATH> <REMOTE_USER> <REMOTE_IP> <REMOTE_VOLUME_PATH>
Example:
./appollo-sync-backup.sh /home/appollo/archive-develop appollo 192.168.1.240 /home/appollo/archive-develop
5. Create backup directory on remote server
In the following step a backup directory is created including backup date and saving old backups is allowed using NUMBER_BACKUPS parameter.
./appollo-backup-archive.sh <MOUNT_VOLUME_PATH> <BACKUP_DIRECTORY> <NUMBER_BACKUPS>
Parameter Explanation:
-
MOUNT_VOLUME_PATH is the instance mounted volume for backups.
-
NUMBER_BACKUPS is the number of backups to preserve.
-
BACKUP_DIRECTORY is directory to archive backups
Example:
./appollo-backup-archive.sh /home/appollo/archive-develop appollo 192.168.1.240 /home/appollo/backup-archive-develop 2
6. Compress backup files on remote server
Backup directory must be compresses on a backup file to be restored, executing following command:
./appollo-compress-backup-directory.sh <BACKUP_DIRECTORY> <BACKUP_FILE>
Example:
./appollo-compress-backup-directory.sh /home/appollo/backup-archive-ams/2019-08-19_22\:30\:15 /home/appollo/online-backup.tar
Create Base Backup
In this step postgresql data directory is compressed and saved, transactions during this process are saved also into the backup file. This step is a precondition to execute incremental backups.
Execute backup command
./appollo-backup-online-create-base.sh <CONTAINER_NAME> <APPOLLO_BACKUP_DIR>
Verify Base Backup
If backup task finished successfully there should be created a backup file at APPOLLO_BACKUP_DIR
<APPOLLO_BACKUP_DIR>/base/<DATE>/<BACKUP_FILE>
Create Incremental Backup
In this step a backup file is created using a previously created base backup (See Create Base Backup). It saves all transactions on the database, allowing to restore the state of the database given a moment in the past, also known as point in time recovery. It also can restore the latest state of the database when backup was created.
Execute backup command:
./appollo-backup-online-create-incremental.sh <CONTAINER_NAME> <APPOLLO_BACKUP_DIR>
Verify Incremental Backup
If backup task finished successfully there should be created a backup file at APPOLLO_BACKUP_DIR
<APPOLLO_BACKUP_DIR>/incremental/DATE_<BASE_DATE>/ <BACKUP_FILE>
Restore Online Backup
-
Stop xAPPnow services
./appollo-stop.sh <INSTANCE_NAME>
2. Make offline backup of current database information
./appollo-backup-create.sh <CONTAINER_NAME> <APPOLLO_BACKUP_DIR>
3. Restore online backup
./appollo-backup-online-restore.sh <CONTAINER_NAME> <APPOLLO_BACKUP_DIR/DATE/incremental/DATE_base-DATE/BACKUP_FILE>
4. Restore online backup using point in time recovery
./appollo-backup-online-restore.sh <CONTAINER_NAME> <APPOLLO_BACKUP_DIR/incremental/DATE/BACKUP_FILE “TIME”>
5. Start xAPPnow services
./appollo-start.sh <INSTANCE_NAME>
