How to backup and restore informix db using Onbar tool with ONPSM cloud storage device?

Question

I am having a hard time using the Onbar tool (which relies on Onpsm primary storage manager for Informix db (server 12.1) using the cloud device (AWS s3).

https://www.ibm.com/support/knowledgecenter/SSGU8G_12.1.0/com.ibm.bar.doc/ids_bar_526.htm

On the backup server, I have created a a DBSPOOL and LOGPOOL devices using commands similar to the following

onpsm -D add AMAZON1 \
-g DBSPOOL \
-p HIGHEST \
-t CLOUD \
--url https://ifmx-s3-dev.s3.amazonaws.com \
--provider S3 \
--user AKIAIT1111155555X4PA \ 
--password A2nB21111155555nvTI0X9ZxGzUJNJivoBQY9MrD \
--container ifmx-s3-dev \
--max_part_size 25600

I then setup a cron job to perform a standard, level-0 backup of all online storage spaces and used logical logs.

onbar -b -L 0

onbar -b -l

Reading the following IBM publication which advises to backup the storage catalog tables since are not backed up with the database instance, I set up a cron job to backup the $INFORMIXDIR/etc/psm which is supposed to contain the storage catalog tables

https://www.ibm.com/support/knowledgecenter/SSGU8G_12.1.0/com.ibm.bar.doc/ids_bar_526.htm

Copied $INFORMIXDIR/etc/psm directory and $INFORMIXDIR/etc/ixbar.? from the source server to the target and renamed the ixbar.? file replacing the IBM Informix server name that is used on the source computer with the IBM Informix server name of the target computer.

3- Changed ownership of the copied /psm directory and the ixbar file to user informix

4- Shutdown the db to perform a cold restore

onmode -yuk

5- Checked that the backup objects are there

onpsm -O list

6- Attempted to restore data and the critical files during the cold restore

onbar -r -cf yes

Got this weird error:

the environment variable BAR_BSALIB_PATH must be set in order to use the -cf option!

I checked the BAR_BSALIB_PATH in the ONCONFIG file and it is set to $INFORMIXDIR/lib/libbsapsm.so Even done this BAR_BSALIB_PATH="/opt/informix-12.10.fc10/lib/libbsapsm.so" :/home/informix echo $BAR_BSALIB_PATH /opt/informix-12.10.fc10/lib/libbsapsm.so

Still the same error!

7- Regenerate the emergency boot file - If the ixbar file is not empty and contains object data, onsmsync -b updates the sysutils database and the ixbar file so that they are in sync.

onsmsync -b

bar_act.log

2018-07-20 00:19:07 25656 25370 (-43207) Unable to open connection to database server: . 2018-07-20 00:19:07 25656 25370 onsmsync complete, returning 155 (0x9b) 2018-07-20 00:21:24 25933 25931 /opt/informix-12.10.fc10/bin/onbar_d -r

8- Restore the data

onbar -r

Warning: Parameter's user-configured value was adjusted. (ALARMPROGRAM) gzip: stdin: unexpected end of file

gzip: stdin: unexpected end of file

9- The server came back in fast recovery mode

10 - bar_dbug.log

GET /awwdst13a/rootdbs/0/rootdbs.377.1 HTTP/1.1

then failure messages like * Failed writing body (0 != 16347) * Closing connection 0 2018-07-20 00:21:25 25937 25933 smtranid.c:155 Object Transaction List was empty. 2018-07-20 00:21:27 25933 25931 bar_ixbar_list: enter 2018-07-20 00:21:27 25933 25931 bar_ixbar_list: return 0 (0x00) 2018-07-20 00:21:27 25933 25931 bar_build_timeline: enter 2018-07-20 00:21:27 25933 25931 bar_build_timeline: return 0 (0x00) 2018-07-20 00:21:27 25933 25931 smcatalog.c:521 nsmOpenDatabase: Count: opened = 9, missing = 0. 2018-07-20 00:21:27 25933 25931 smtranid.c:84 Object Transaction List was empty. 2018-07-20 00:21:27 25941 25933 smcatalog.c:521 nsmOpenDatabase: Count: opened = 9, missing = 0. 2018-07-20 00:21:27 25941 filter write_to_filter: Write failed on parent's output pipe. errno = 32.. 2018-07-20 00:21:27 25941 filter
write_to_filter: Error writing to handle/fd 17 located at 0x1e05930.

11- bar_act.log

2018-07-20 00:21:27 25945 25933 Successfully connected to Storage Manager. 2018-07-20 00:21:27 25933 25931 Begin cold level 0 restore rootdbs (Storage Manager copy ID: 0 377). 2018-07-20 00:21:40 25945 25933 Informix Primary Storage Manager session 867 closed 2018-07-20 00:21:40 25945 25933 The child process for the backup and restore filter is terminating with exit code 0. 2018-07-20 00:21:41 25933 25931 Completed cold level 0 restore rootdbs.

2018-07-20 00:21:41 25969 25933 Begin cold level 0 restore api7dbs (Storage Manager copy ID: 0 381). 2018-07-20 00:21:42 25968 25933 Informix Primary Storage Manager session 869 opened. 2018-07-20 00:21:42 25968 25933 Successfully connected to Storage Manager. 2018-07-20 00:21:42 25968 25933 Starting Filter /usr/bin/gunzip. 2018-07-20 00:21:42 25968 25933 Informix Primary Storage Manager session 869 closed

2018-07-20 00:21:44 25971 25933 Begin cold level 0 restore dat2dbs (Storage Manager copy ID: 0 380). 2018-07-20 00:21:51 25983 25971 There are no more bytes to read. ISAM Error = 0. OS Error = 10 (No child processes). 2018-07-20 00:21:51 25983 25971 XBSA Error: (BSAGetData) A system error occurred. Aborting XBSA session. 2018-07-20 00:21:51 25983 25971 Informix Primary Storage Manager session 875 closed 2018-07-20 00:21:51 25983 25971 The child process for the backup and restore filter is terminating with exit code 0. 2018-07-20 00:21:53 25971 25933 Unable to close the storage space restore: The physical restore was not completed.. 2018-07-20 00:21:53 25971 25933 Process 25971 25933 completed. 2018-07-20 00:21:53 25933 25931 (-43246) The ON-Bar process 25971 exited with a problem (exit code 131 (0x83), signal 0).

Luís Marques · Answer 1 · 2018-07-30T10:25:58.053

Finally had the time to make a test with psm and onbar to make an Informix backup to Amazon S3.

The test setup is 2 CentOS 7 virtual machines (machine1 and machine2) with an Informix server on each. The machines can talk to each other and with Amazon S3.
machine1 will have Informix server ifx_a and machine2will have Informix server ifx_b.
Informix 12.10.FC12 was installed on each machine, using the default location (/opt/IBM/informix/12.10/).
It was a root installation, but everything else is run under user informix.

On machine1 I setup and initialize Informix server ifx_a.

I have the following environment variables:

PATH=/opt/IBM/informix/12.10/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin
INFORMIXDIR=/opt/IBM/informix/12.10
INFORMIXSERVER=ifx_a
INFORMIXSQLHOSTS=/opt/IBM/informix/12.10/etc/sqlhosts
ONCONFIG=onconfig.ifx_a

I have the following parameters in the onconfig.ifx_a file:

ROOTNAME rootdbs
ROOTPATH $INFORMIXDIR/tmp/demo_on.rootdbs

SERVERNUM 1
DBSERVERNAME ifx_a

NETTYPE soctcp,1,50,NET

BAR_ACT_LOG $INFORMIXDIR/tmp/bar_act.log
BAR_DEBUG_LOG $INFORMIXDIR/tmp/bar_dbug.log
BAR_DEBUG 5
BAR_MAX_BACKUP 3
BAR_RETRY 1
BAR_PROGRESS_FREQ 10
BAR_BSALIB_PATH $INFORMIXDIR/lib/libbsapsm.so
BACKUP_FILTER  '/usr/bin/gzip -c'
RESTORE_FILTER '/usr/bin/gzip -d -c'
BAR_PERFORMANCE 3
BAR_CKPTSEC_TIMEOUT 15
BAR_HISTORY 0

PSM_ACT_LOG      $INFORMIXDIR/tmp/psm_act.log
PSM_DEBUG_LOG    $INFORMIXDIR/tmp/psm_dbug.log
PSM_DEBUG        5
PSM_CATALOG_PATH $INFORMIXDIR/etc/psm
PSM_DBS_POOL     DBSPOOL
PSM_LOG_POOL     LOGPOOL

ALARMPROGRAM $INFORMIXDIR/etc/alarmprogram.sh

The sqlhosts file:

ifx_a    onsoctcp    machine1.local    15010    k=1
ifx_b    onsoctcp    machine2.local    15020    k=1

Create the root dbspace file and set up permissions:

$ touch /opt/IBM/informix/12.10/tmp/demo_on.rootdbs
$ chown informix:informix /opt/IBM/informix/12.10/tmp/demo_on.rootdbs
$ chmod 660 /opt/IBM/informix/12.10/tmp/demo_on.rootdbs

Initialize the Informix server ifx_a:

$ oninit -vi

$ onstat -d

IBM Informix Dynamic Server Version 12.10.FC12 -- On-Line -- Up 00:15:50 -- 164468 Kbytes

Dbspaces
address          number   flags      fchunk   nchunks  pgsize   flags    owner    name
44c23028         1        0x1        1        1        2048     N  B     informix rootdbs
 1 active, 2047 maximum

Chunks
address          chunk/dbs     offset     size       free       bpages     flags pathname
44c23268         1      1      0          150000     85073                 PO-B-- /opt/IBM/informix/12.10/tmp/demo_on.rootdbs
 1 active, 32766 maximum

NOTE: The values in the "size" and "free" columns for DBspace chunks are
      displayed in terms of "pgsize" of the DBspace to which they belong.


Expanded chunk capacity mode: always

Create a new database test_db on the new Informix server:

$ printf "CREATE DATABASE test_db WITH LOG;" | dbaccess sysmaster

Follow Back up to Amazon Simple Storage Service using ON-Bar and the PSM and setup the Amazon S3 bucket and credentials. The following steps are based on the mentioned documentation:

Initialize the psm catalog by just using an onpsm command:

$ onpsm -C detail

The Informix Primary Storage Manager catalog is not present in the '/opt/IBM/informix/12.10/etc/psm/'
directory.
The system will try to create it.
Informix Primary Storage Manager catalog creation in the '/opt/IBM/informix/12.10/etc/psm/' directory
succeeded.

Informix Primary Storage Manager State:

    PSM Unique ID    : 1532732188
    Catalog Location : /opt/IBM/informix/12.10/etc/psm/
    Catalog State    : Unlocked
    Catalog Owner    : 0
    Catalog Lock Mode: Regular


Sessions:
    Session ID       Process ID
Empty SESSION table.

Add the Amazon S3 bucket to the storage pools for dbspaces (DBSPOOL) and logical logs (LOGPOOL):

$ onpsm -D add AMAZON1 \
-g DBSPOOL \
-p HIGHEST \
-t CLOUD \
--url https://ifx-a-cloud-bucket.s3-eu-west-1.amazonaws.com \
--provider S3 \
--user UUUUUUUUUUUUUUUUUUUU \
--password XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX \
--container ifx-a-cloud-bucket \
--max_part_size 25600

$ onpsm -D add AMAZON1 \
-g LOGPOOL \
-p HIGHEST \
-t CLOUD \
--url https://ifx-a-cloud-bucket.s3-eu-west-1.amazonaws.com \
--provider S3 \
--user UUUUUUUUUUUUUUUUUUUU \
--password XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX \
--container ifx-a-cloud-bucket \
--max_part_size 25600

$ onpsm -D list

Informix Primary Storage Manager Device List

Type   Prio     Block/Size (MB)  Pool Name     Device Name
FILE   LOW          --/--        DBSPOOL       /opt/IBM/informix/12.10/backups
CLOUD  HIGHEST      --/--        DBSPOOL       AMAZON1

FILE   LOW          --/--        LOGPOOL       /opt/IBM/informix/12.10/backups
CLOUD  HIGHEST      --/--        LOGPOOL       AMAZON1

With a recently created Amazon S3 bucket, the url should point to the region of the bucket, else Amazon S3 will return a temporary redirect payload and onpsm does not deal nicely with it (it seems to crash).

Make a backup of Informix server ifx_a (in this case I am making a whole system backup, option "-w"):

- $ onbar -b -w -L 0

Check onbar log file at /opt/IBM/informix/12.10/tmp/bar_act.log:

 2018-07-28 00:45:26.596713 7539  7537 /opt/IBM/informix/12.10/bin/onbar_d -b -L 0 -w
 2018-07-28 00:45:26.757667 7539  7537 Using IBM's PSM version 12.10.FC12 as the Storage Manager. XBSA API version is 1.0.3.
 2018-07-28 00:45:26.933961 7539  7537 Archive started on rootdbs (Requested Level 0).
 2018-07-28 00:45:26.966597 7539  7537 Begin level 0 backup rootdbs.
 2018-07-28 00:45:26.966861 7539  7537 Starting Filter /usr/bin/gzip -c.
 2018-07-28 00:45:26.971701 7542  7539 Successfully connected to Storage Manager.
 2018-07-28 00:45:34.500992 7542  7539 The child process for the backup and restore filter is terminating
with exit code 0.
 2018-07-28 00:45:34.507614 7539  7537 Successfully connected to Storage Manager.
 2018-07-28 00:45:34.511597 7539  7537 Completed level 0 backup rootdbs (Storage Manager copy ID: 0 1).
 2018-07-28 00:45:34.545924 7539  7537 Archive on rootdbs Completed (Requested Level 0).
 2018-07-28 00:45:34.549264 7539  7537 Begin backup logical log 1.
 2018-07-28 00:45:34.549524 7539  7537 Starting Filter /usr/bin/gzip -c.
 2018-07-28 00:45:34.570435 7561  7539 Successfully connected to Storage Manager.
 2018-07-28 00:45:38.455047 7561  7539 The child process for the backup and restore filter is terminating
with exit code 0.
 2018-07-28 00:45:38.461007 7539  7537 Successfully connected to Storage Manager.
 2018-07-28 00:45:38.555035 7539  7537 Completed backup logical log 1 (Storage Manager copy ID: 0 2).
 2018-07-28 00:45:38.572049 7539  7537 Begin backup logical log 2.
 2018-07-28 00:45:38.572251 7539  7537 Starting Filter /usr/bin/gzip -c.
 2018-07-28 00:45:38.577040 7576  7539 Successfully connected to Storage Manager.
 2018-07-28 00:45:43.152739 7576  7539 The child process for the backup and restore filter is terminating
with exit code 0.
 2018-07-28 00:45:43.159746 7539  7537 Successfully connected to Storage Manager.
 2018-07-28 00:45:43.214322 7539  7537 Completed backup logical log 2 (Storage Manager copy ID: 0 3).
 2018-07-28 00:45:43.227226 7539  7537 Begin backup logical log 3.
 2018-07-28 00:45:43.227451 7539  7537 Starting Filter /usr/bin/gzip -c.
 2018-07-28 00:45:43.231582 7592  7539 Successfully connected to Storage Manager.
 2018-07-28 00:45:46.529906 7592  7539 The child process for the backup and restore filter is terminating
with exit code 0.
 2018-07-28 00:45:46.536171 7539  7537 Successfully connected to Storage Manager.
 2018-07-28 00:45:46.561509 7539  7537 Completed backup logical log 3 (Storage Manager copy ID: 0 4).
 2018-07-28 00:45:46.586104 7539  7537 Successfully connected to Storage Manager.
 2018-07-28 00:45:46.659232 7539  7537 Begin backup of critical file '/opt/IBM/informix/12.10/etc/ixbar.1'.
 2018-07-28 00:45:46.662166 7539  7537 Successfully connected to Storage Manager.
 2018-07-28 00:45:46.665362 7539  7537 Completed backup of critical file '/opt/IBM/informix/12.10/etc/ixbar.1' (Storage Manager copy ID: 0 5).
 2018-07-28 00:45:47.106933 7539  7537 Begin backup of critical file '/opt/IBM/informix/12.10/etc/oncfg_ifx_a.1'.
 2018-07-28 00:45:47.111936 7539  7537 Successfully connected to Storage Manager.
 2018-07-28 00:45:47.117141 7539  7537 Completed backup of critical file '/opt/IBM/informix/12.10/etc/oncfg_ifx_a.1' (Storage Manager copy ID: 0 6).
 2018-07-28 00:45:47.566010 7539  7537 Begin backup of critical file '/opt/IBM/informix/12.10/etc/onconfig.ifx_a'.
 2018-07-28 00:45:47.571084 7539  7537 Successfully connected to Storage Manager.
 2018-07-28 00:45:47.577958 7539  7537 Completed backup of critical file '/opt/IBM/informix/12.10/etc/onconfig.ifx_a' (Storage Manager copy ID: 0 7).
 2018-07-28 00:45:48.216676 7539  7537 Begin backup of critical file '/opt/IBM/informix/12.10/etc/sqlhosts'.
 2018-07-28 00:45:48.221868 7539  7537 Successfully connected to Storage Manager.
 2018-07-28 00:45:48.227252 7539  7537 Completed backup of critical file '/opt/IBM/informix/12.10/etc/sqlhosts' (Storage Manager copy ID: 0 8).
 2018-07-28 00:45:48.649965 7539  7537 /opt/IBM/informix/12.10/bin/onbar_d complete, returning 0 (0x00)

Setup automatic backup on completion for logical logs.

Edit the ALARMPROGRAMscript and change BACKUPLOGS to Y:
```
BACKUPLOGS=Y
```
Force a logical log completion/change and it will backup all logical logs that need.
```
$ onmode -l
```

Check onbar log file at /opt/IBM/informix/12.10/tmp/bar_act.log:

07/28/18 01:11:02  Logical Log 4 Complete, timestamp: 0x4c867.
07/28/18 01:11:03  Logical Log 4 - Backup Started
07/28/18 01:11:05  Logical Log 4 - Backup Completed
07/28/18 01:11:09  Logical Log 5 Complete, timestamp: 0x55db7.
07/28/18 01:11:10  Logical Log 5 - Backup Started
07/28/18 01:11:12  Logical Log 5 - Backup Completed
07/28/18 01:11:36  Logical Log 6 Complete, timestamp: 0x57a93.
07/28/18 01:11:37  Logical Log 6 - Backup Started
07/28/18 01:11:38  Logical Log 6 - Backup Completed

Verify that the backup objects are listed in the psm catalog:

$ onpsm -O list

Informix Primary Storage Manager Object List

Object ID            Date Created         Size (MB) Logical path (name.version)
1                    2018-07-28 00:45:26        2.5 /ifx_a/rootdbs/0/ifx_a.1
2                    2018-07-28 00:45:34        1.2 /ifx_a/1/1/ifx_a.1
3                    2018-07-28 00:45:38        1.4 /ifx_a/1/2/ifx_a.1
4                    2018-07-28 00:45:43        0.9 /ifx_a/1/3/ifx_a.1
5                    2018-07-28 00:45:46        0.0 /ifx_a/critical_files/ixbar/ifx_a.1
6                    2018-07-28 00:45:47        0.0 /ifx_a/critical_files/oncfg/ifx_a.1
7                    2018-07-28 00:45:47        0.1 /ifx_a/critical_files/onconfig/ifx_a.1
8                    2018-07-28 00:45:48        0.0 /ifx_a/critical_files/sqlhosts/ifx_a.1
9                    2018-07-28 01:11:03        0.6 /ifx_a/1/4/ifx_a.1
10                   2018-07-28 01:11:10        0.6 /ifx_a/1/5/ifx_a.1
11                   2018-07-28 01:11:37        0.1 /ifx_a/1/6/ifx_a.1

Export the psm catalog so it can be moved to machine2.
- This will create a psm_catalog.exp in the current directory. It will contain text files to be imported into a new catalog.
```
$ onpsm -C export
```
- Another way is just to just copy the psm catalog to machine2.
  - On Informix version 12.10.FC10 the export option is broken for CLOUD type devices, so the catalog must be copied instead.
- The psm catalog is not part of the backups. It should itself be backup up by an external tool, as per the documentation HCL Informix Primary Storage Manager

On machine2 I setup Informix server ifx_b, but I do not initialize it.

I have the following environment variables:

PATH=/opt/IBM/informix/12.10/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin
INFORMIXDIR=/opt/IBM/informix/12.10
INFORMIXSERVER=ifx_b
INFORMIXSQLHOSTS=/opt/IBM/informix/12.10/etc/sqlhosts
ONCONFIG=onconfig.ifx_b

I have the following parameters in the onconfig.ifx_b file which are mostly equal to Informix server ifx_a, but I prefer to give it a different SERVERNUM and DBSERVERNAME (it is not required, since they are in separate machines):

ROOTNAME rootdbs
ROOTPATH $INFORMIXDIR/tmp/demo_on.rootdbs

SERVERNUM 2
DBSERVERNAME ifx_b

NETTYPE soctcp,1,50,NET

BAR_ACT_LOG $INFORMIXDIR/tmp/bar_act.log
BAR_DEBUG_LOG $INFORMIXDIR/tmp/bar_dbug.log
BAR_DEBUG 5
BAR_MAX_BACKUP 3
BAR_RETRY 1
BAR_PROGRESS_FREQ 10
BAR_BSALIB_PATH $INFORMIXDIR/lib/libbsapsm.so
BACKUP_FILTER  '/usr/bin/gzip -c'
RESTORE_FILTER '/usr/bin/gzip -d -c'
BAR_PERFORMANCE 3
BAR_CKPTSEC_TIMEOUT 15
BAR_HISTORY 0

PSM_ACT_LOG      $INFORMIXDIR/tmp/psm_act.log
PSM_DEBUG_LOG    $INFORMIXDIR/tmp/psm_dbug.log
PSM_DEBUG        5
PSM_CATALOG_PATH $INFORMIXDIR/etc/psm
PSM_DBS_POOL     DBSPOOL
PSM_LOG_POOL     LOGPOOL

ALARMPROGRAM $INFORMIXDIR/etc/alarmprogram.sh

The sqlhosts file:

ifx_a    onsoctcp    machine1.local    15010    k=1
ifx_b    onsoctcp    machine2.local    15020    k=1

Create the root dbspace file and set up permissions:

$ touch /opt/IBM/informix/12.10/tmp/demo_on.rootdbs
$ chown informix:informix /opt/IBM/informix/12.10/tmp/demo_on.rootdbs
$ chmod 660 /opt/IBM/informix/12.10/tmp/demo_on.rootdbs

Copy into machine2 the Informix server ifx_a psm catalog export.

Import the Informix server ifx_a psm catalog. The imported catalog will assume all the same absolute paths from Informix server ifx_a. But both Informix servers were setup to have the same absolute paths.

In the directory where the psm_catalog.exp is, I run:

$ onpsm -C import

 The Informix Primary Storage Manager catalog is not present in the '/opt/IBM/informix/12.10/etc/psm/'
 directory.
The system will try to create it.
 Informix Primary Storage Manager catalog creation in the '/opt/IBM/informix/12.10/etc/psm/' directory
 succeeded.

This option replaces the Informix Primary Storage Manager catalog using the
 contents of the
'psm_catalog.exp' directory.
Are you sure you want to continue? (y/n) y

Check the psm catalog:

$ onpsm -C detail

Informix Primary Storage Manager State:

    PSM Unique ID    : 1532739005
    Catalog Location : /opt/IBM/informix/12.10/etc/psm/
    Catalog State    : Unlocked
    Catalog Owner    : 0
    Catalog Lock Mode: Regular


Sessions:
    Session ID       Process ID
Empty SESSION table.


$ onpsm -O list

Informix Primary Storage Manager Object List

Object ID            Date Created         Size (MB) Logical path (name.version)
1                    2018-07-28 00:45:26        2.5 /ifx_a/rootdbs/0/ifx_a.1
2                    2018-07-28 00:45:34        1.2 /ifx_a/1/1/ifx_a.1
3                    2018-07-28 00:45:38        1.4 /ifx_a/1/2/ifx_a.1
4                    2018-07-28 00:45:43        0.9 /ifx_a/1/3/ifx_a.1
5                    2018-07-28 00:45:46        0.0 /ifx_a/critical_files/ixbar/ifx_a.1
6                    2018-07-28 00:45:47        0.0 /ifx_a/critical_files/oncfg/ifx_a.1
7                    2018-07-28 00:45:47        0.1 /ifx_a/critical_files/onconfig/ifx_a.1
8                    2018-07-28 00:45:48        0.0 /ifx_a/critical_files/sqlhosts/ifx_a.1
9                    2018-07-28 01:11:03        0.6 /ifx_a/1/4/ifx_a.1
10                   2018-07-28 01:11:10        0.6 /ifx_a/1/5/ifx_a.1
11                   2018-07-28 01:11:37        0.1 /ifx_a/1/6/ifx_a.1
12                   2018-07-28 01:32:40        2.5 /ifx_a/rootdbs/0/ifx_a.2
13                   2018-07-28 01:32:47        0.5 /ifx_a/1/7/ifx_a.1
14                   2018-07-28 01:32:49        0.0 /ifx_a/critical_files/ixbar/ifx_a.2
15                   2018-07-28 01:32:49        0.0 /ifx_a/critical_files/oncfg/ifx_a.2
16                   2018-07-28 01:32:50        0.1 /ifx_a/critical_files/onconfig/ifx_a.2
17                   2018-07-28 01:32:50        0.0 /ifx_a/critical_files/sqlhosts/ifx_a.2

For onbar to work, it needs an ixbar file (it´s own offline catalog). onsmsync command can recreate the ixbar file, but it can only do it from the sysutils database, which does not exists in Informix server ifx_b, since it has not been initialized (and even then, it would be empty). So we need to get it from the psm storage devices.
- Get object 14 because it is the most recent backup of the ixbar file.
- Check the object catalog metadata:
```
$ onpsm -O detail -o 14

Informix Primary Storage Manager Object Detail List

    Object ID       : 14
    XBSA PATH       : /ifx_a/critical_files/ixbar
    XBSA CopyId     : 0:14
    Object Type     : C
    Object version  : 2
    Physical pieces :
        https://ifx-a-cloud-bucket.s3.amazonaws.com//ifx_a/critical_files/ixbar/ixbar.14.1
    Number of pieces: 1
```
- Get ("dump") the ixbar file from the storage device:
```
onpsm -O dump -o 14
```
- It creates a file ifx_a_critical_files_ixbar.14 on the current directory. Copy and rename the file to the become the ixbar file for the Informix server ifx_b (SERVERNUM 2):
```
$ cp ifx_a_critical_files_ixbar.14 /opt/IBM/informix/12.10/etc/ixbar.2
```

Launch the onbar restore command.

$ onbar -r -w

Check onbar log file at /opt/IBM/informix/12.10/tmp/bar_act.log:

 2018-07-28 01:57:45.727270 11009  11007 /opt/IBM/informix/12.10/bin/onbar_d -r -w
 2018-07-28 01:57:45.794570 11009  11007 Using IBM's PSM version 12.10.FC12 as the Storage Manager. XBSA API version is 1.0.3.
 2018-07-28 01:57:45.799655 11009  11007 Successfully connected to Storage Manager.
 2018-07-28 01:57:45.799874 11009  11007 Starting Filter /usr/bin/gzip -d -c.
 2018-07-28 01:57:45.831695 11011  11009 Successfully connected to Storage Manager.
 2018-07-28 01:57:46.435703 11009  11007 Begin reserved pages read (level 0 of rootdbs, Storage Manager copy ID: 0 12).
 2018-07-28 01:57:46.486687* 11011  11009 Filter closed.
 2018-07-28 01:57:46.531063* 11011  11009 Filter closed.
 2018-07-28 01:57:46.540890 11009  11007 Completed reserved pages read (level 0 of rootdbs).
 2018-07-28 01:57:46.555287 11009  11007 Unable to start logical log salvage: Unable to read boot strap config file - '/opt/IBM/informix/12.10/etc/oncfg_ifx_b.2' .
 2018-07-28 01:57:51.563908 11009  11007 Successfully connected to Storage Manager.
 2018-07-28 01:57:51.564216 11009  11007 Starting Filter /usr/bin/gzip -d -c.
 2018-07-28 01:57:51.575587 11019  11009 Successfully connected to Storage Manager.
 2018-07-28 01:57:52.096184 11009  11007 Begin reserved pages read (level 0 of rootdbs, Storage Manager copy ID: 0 12).
 2018-07-28 01:57:52.153351* 11019  11009 Filter closed.
 2018-07-28 01:57:52.198363* 11019  11009 Filter closed.
 2018-07-28 01:57:52.209302 11009  11007 Completed reserved pages read (level 0 of rootdbs).
 2018-07-28 01:57:52.222134 11009  11007 Successfully connected to Storage Manager.
 2018-07-28 01:57:52.222251 11009  11007 Starting Filter /usr/bin/gzip -d -c.
 2018-07-28 01:57:52.227977 11024  11009 Successfully connected to Storage Manager.
 2018-07-28 01:57:52.753398 11009  11007 Begin cold level 0 restore rootdbs (Storage Manager copy ID: 0 12).
 2018-07-28 01:58:05.396340 11024  11009 The child process for the backup and restore filter is terminating
with exit code 0.
 2018-07-28 01:58:05.448496 11009  11007 Completed cold level 0 restore rootdbs.
 2018-07-28 01:58:05.790012 11009  11007 Completed whole system restore.
 2018-07-28 01:58:06.812501 11009  11007 Successfully connected to Storage Manager.
 2018-07-28 01:58:06.812747 11009  11007 Starting Filter /usr/bin/gzip -d -c.
 2018-07-28 01:58:06.827447 11049  11009 Successfully connected to Storage Manager.
 2018-07-28 01:58:07.413841 11009  11007 Begin restore logical log 7 (Storage Manager copy ID: 0 13).
 2018-07-28 01:58:09.251399 11049  11009 The child process for the backup and restore filter is terminating
with exit code 0.
 2018-07-28 01:58:09.382900 11009  11007 Completed restore logical log 7.
 2018-07-28 01:58:09.388014 11009  11007 Successfully connected to Storage Manager.
 2018-07-28 01:58:09.831410 11009  11007 Completed logical restore.
 2018-07-28 01:58:10.046783 11009  11007 Successfully connected to Storage Manager.
 2018-07-28 01:58:10.104630 11009  11007 /opt/IBM/informix/12.10/bin/onbar_d complete, returning 0 (0x00)

Check online log file at /opt/IBM/informix/12.10/tmp/online.log:

01:57:53  IBM Informix Dynamic Server Started.
01:57:53  Requested shared memory segment size rounded from 4308KB to 4796KB

Sat Jul 28 01:57:55 2018

07/28/18 01:57:55  Requested shared memory segment size rounded from 110629KB to 110632KB
07/28/18 01:57:55  Successfully added a bufferpool of page size 2K.

07/28/18 01:58:00  Restartable Restore has been ENABLED
07/28/18 01:58:00  Recovery Mode
07/28/18 01:58:01  Listen thread init SUCCESS
07/28/18 01:58:01  Physical Restore of rootdbs started.

07/28/18 01:58:02  Chunk 1 '/opt/IBM/informix/12.10/tmp/demo_on.rootdbs' is sparse - inflating...
07/28/18 01:58:09  Logical Recovery has reached the transaction cleanup phase.
07/28/18 01:58:09  Logical Recovery Complete.
          8 Committed, 0 Rolled Back, 0 Open, 0 Bad Locks

07/28/18 01:58:09  Logical Recovery Complete.
07/28/18 01:58:09  Quiescent Mode
07/28/18 01:58:09  Checkpoint Completed:  duration was 0 seconds.
07/28/18 01:58:09  Sat Jul 28 - loguniq 8, logpos 0x18, timestamp: 0x5f8dd Interval: 25

07/28/18 01:58:09  Maximum server connections 0
07/28/18 01:58:09  Checkpoint Statistics - Avg. Txn Block Time 0.000, # Txns blocked 0, Plog used 19, Llog used 1

07/28/18 01:58:10  Booting Language <spl> from module <>
07/28/18 01:58:10  Loading Module <SPLNULL>
07/28/18 01:58:10  Defragmenter cleaner thread now running
07/28/18 01:58:10  Defragmenter cleaner thread cleaned:0 partitions

The Informix Server ifx_b is now in Quiescent mode (memory is initialized, it can perform some administrative tasks). Change the Informix server ifx_b to online mode.

```
 $ onmode -m
```

Check that the database test_db was restored.

printf "select name from sysdatabases;" | dbaccess sysmaster

Database selected.

name  sysmaster
name  sysutils
name  sysuser
name  sysadmin
name  test_db

5 row(s) retrieved.

Database closed.

Thanks for the detailed explanation Luis. I got stuck here: onpsm -O dump -o 457. I have edited your answer to include details of the error due to lack of space in comment — gbaz, Jul 27 '18 at 01:59
@gbaz I did some testing and posted my step by step. Maybe it can help. — Luís Marques, Jul 28 '18 at 04:39
Thank you for the detailed, and prompt, response; I really appreciate it. Looking at the setup I have and yours the one main difference I see is using the ALARMPROGRAM script for logical backups. Is it possible to just setup a cron jobs to perform full level 0 backups hourly with `onbar -b -L0 -w` followed by `onbar -b -l` every 5 minutes for logical logs without relying on the **alarmprogram.sh** script? If so, how would the recovery process look like? — gbaz, Jul 30 '18 at 05:57
Yes, it is possible. You should also do an `onmode -l` to switch to the next logical before the `onbar -b -l`, because `onbar` will not backup logical logs that are still `CURRENT`. The restore, with the Informix server stopped (a cold restore) is simply using `onbar -r`. It will restore the most recent whole system backup (`onbar -b -w -L 0`) and then restore all logical logs up to the most recent. — Luís Marques, Jul 31 '18 at 20:30
I looked at the ontape tool and it seems easier to do `ontape -s -L 0 -d` then `ontape -c` for continuous backups and do a s3 sync for the TAPEDEV and LTAPEDEV directories without having to worry about backing up sysutils database and all. — gbaz, Aug 03 '18 at 05:33
Yes, it will also work. However `ontape` is a `serial` type of backup, as only 1 process is streaming the dbspaces, so it will take longer than `onbar`, that can use multiple parallel streams. Usually when the Informix instance reaches a certain dimension, `onbar` is used to keep backup times at reasonable levels. You can also use `ontape` on the `alarmprgram.sh` script, instead of `ontape -c` as is described here: [Starting a continuous logical-log file backup](https://www.ibm.com/support/knowledgecenter/en/SSGU8G_11.70.0/com.ibm.bar.doc/ids_bar_311.htm). — Luís Marques, Aug 03 '18 at 07:08
thanks for the feedback. I have one last question regarding the ontape tool: when using ontape to backup to a directory, is there a native way to automatically remove older backup files (be it L0, L1, or logical logs)? I am doing an S3 sync from TAPEDEV and LTAPEDEV directories and would like to delete older backups from the local backup directory to avoid it filling up the drive. While I could do it with a shell script and a cron job I was hoping the tool had a native option of doing so — gbaz, Aug 06 '18 at 07:27
Not that I am aware. The only thing `ontape` does is renaming the previous backup file so it is not overwritten. You will have to script a cron job to remove older `LX` backups and the logical logs backups. Also, the documentation advises to keep a copy of the `onconfig` file, which `ontape` does not backup [Keep a copy of your configuration file](https://www.ibm.com/support/knowledgecenter/en/SSGU8G_12.1.0/com.ibm.bar.doc/ids_bar_467.htm) . — Luís Marques, Aug 06 '18 at 07:49

How to backup and restore informix db using Onbar tool with ONPSM cloud storage device?

1 Answers1