0.00%
Search · Index

Weblog Page

Filtered by date 2017-06-04, 1 - 2 of 2 Postings (all, summary)

AOLserver keepalive with inittab

Created by Gustaf Neumann, last modified by Gustaf Neumann 04 Jun 2017, at 10:42 AM

This is an alternative method for keeping the AOLserver process running. The recommended method is to run AOLserver supervised.

This step should be completed as root. This can break every service on your machine, so proceed with caution.

  • There are 2 general steps to getting this working.

    1. Install a script called restart-aolserver. This script doesn't actually restart AOLserver - it just kills it.

    2. Ask the OS to restart our service whenever it's not running. We do this by adding a line to /etc/inittab.

    Calling restart-aolserver kills our service. The OS notices that our service is not running, so it automatically restarts it. Thus, calling restart-aolserver effectively restarts our service.

  • Copy this file into /var/tmp/restart-aolserver.txt.

  • This script needs to be SUID-root, which means that the script will run as root. This is necessary to ensure that the AOLserver processes are killed regardless of who owns them. However the script should be executable by the web group to ensure that the users updating the web page can use the script, but that general system users cannot run the script. You also need to have Perl installed and also a symbolic link to it in /usr/local/bin. 

    [joeuser ~]$ su -
Password: ***********
[root ~]# cp /var/tmp/restart-aolserver.txt /usr/local/bin/restart-aolserver
[root ~]# chown root.web /usr/local/bin/restart-aolserver
[root ~]# chmod 4750 /usr/local/bin/restart-aolserver
[root ~]# ln -s /usr/bin/perl /usr/local/bin/perl
[root ~]# exit

  • Test the restart-aolserver script. We'll first kill all running servers to clean the slate. Then, we'll start one server and use restart-aolserver to kill it. If it works, then there should be no more servers running. You should see the following lines. 

    [joeuser ~]$ killall nsd
nsd: no process killed
[joeuser ~]$ /usr/local/aolserver/bin/nsd-postgres -t ~/var/lib/aolserver/birdnotes/nsd.tcl
[joeuser ~]$ restart-aolserver birdnotes
Killing 23727
[joeuser ~]$ killall nsd
nsd: no process killed

    The number 23727 indicates the process id(s) (PIDs) of the processes being killed. It is important that no processes are killed by the second call to killall. If there are processes being killed, it means that the script is not working.

  • Assuming that the restart-aolserver script worked, login as root and open /etc/inittab for editing. 

    [joeuser ~]$ su -
Password: ************
[root ~]# emacs -nw /etc/inittab

  • Copy this line into the bottom of the file as a template, making sure that the first field nss1 is unique. 

    nss1:345:respawn:/usr/local/aolserver/bin/nsd-postgres -i -u nobody -g web -t /home/joeuser/var/lib/aolserver/birdnotes/nsd.tcl

  • Important: Make sure there is a newline at the end of the file. If there is not a newline at the end of the file, the system may suffer catastrophic failures.

  • Still as root, enter the following command to re-initialize /etc/inittab. 

    [root ~]# killall nsd
nsd: no process killed
[root ~]# /sbin/init q

  • See if it worked by running the restart-aolserver script again. 

    [root ~]# restart-aolserver birdnotes
Killing 23750

If processes were killed, congratulations, your server is now automated for startup and shutdown.

Manual backup and recovery

Created by Gustaf Neumann, last modified by Gustaf Neumann 04 Jun 2017, at 10:38 AM

This section describes how to make a one-time backup and restore of the files and database. This is useful for rolling back to known-good versions of a service, such as at initial installation and just before an upgrade. First, you back up the database to a file within the file tree. Then, you back up the file tree. All of the information needed to rebuild the site, including the AOLserver config files, is then in tree for regular file system backup.

  1. Back up the database to a file.

    • Oracle.

      • Download the backup script. Save the file export-oracle.txt as /var/tmp/export-oracle.txt

      • Login as root. The following commands will install the export script: 

        [joeuser ~]$ su -
[root ~]# cp /var/tmp/export-oracle.txt /usr/sbin/export-oracle
[root ~]# chmod 700 /usr/sbin/export-oracle

      • Setup the export directory; this is the directory where backups will be stored. We recommend the directory /ora8/m02/oracle-exports. 

        [root ~]# mkdir /ora8/m02/oracle-exports
[root ~]# chown oracle:dba /ora8/m02/oracle-exports
[root ~]# chmod 770 /ora8/m02/oracle-exports

      • Now edit /usr/sbin/export-oracle and change the SERVICE_NAME and DATABASE_PASSWORD fields to their correct values. If you want to use a directory other than /ora8/m02/oracle-exports, you also need to change the exportdir setting.

        Test the export procedure by running the command: 

        [root ~]# /usr/sbin/export-oracle
mv: /ora8/m02/oracle-exports/oraexport-service_name.dmp.gz: No such file or directory

Export: Release 8.1.6.1.0 - Production on Sun Jun 11 18:07:45 2000

(c) Copyright 1999 Oracle Corporation.  All rights reserved.

Connected to: Oracle8i Enterprise Edition Release 8.1.6.1.0 - Production
With the Partitioning option
JServer Release 8.1.6.0.0 - Production
Export done in US7ASCII character set and US7ASCII NCHAR character set
  . exporting pre-schema procedural objects and actions
  . exporting foreign function library names for user SERVICE_NAME
  . exporting object type definitions for user SERVICE_NAME
  About to export SERVICE_NAME's objects ...
  . exporting database links
  . exporting sequence numbers
  . exporting cluster definitions
  . about to export SERVICE_NAME's tables via Conventional Path ...
  . exporting synonyms
  . exporting views
  . exporting stored procedures
  . exporting operators
  . exporting referential integrity constraints
  . exporting triggers
  . exporting indextypes
  . exporting bitmap, functional and extensible indexes
  . exporting posttables actions
  . exporting snapshots
  . exporting snapshot logs
  . exporting job queues
  . exporting refresh groups and children
  . exporting dimensions
  . exporting post-schema procedural objects and actions
  . exporting statistics
Export terminated successfully without warnings.

    • PostgreSQL.Create a backup file and verify that it was created and has a reasonable size (several megabytes). 

      [root root]# su - $OPENACS_SERVICE_NAME
[$OPENACS_SERVICE_NAME $OPENACS_SERVICE_NAME]$ pg_dump -f /var/lib/aolserver/$OPENACS_SERVICE_NAME/database-backup/before_upgrade_to_4.6.dmp $OPENACS_SERVICE_NAME

[$OPENACS_SERVICE_NAME $OPENACS_SERVICE_NAME]$ ls -al /var/lib/aolserver/$OPENACS_SERVICE_NAME/database-backup/before_upgrade_to_4.6.dmp 
-rw-rw-r-x    1 $OPENACS_SERVICE_NAME  $OPENACS_SERVICE_NAME   4005995 Feb 21 18:28 /var/lib/aolserver/$OPENACS_SERVICE_NAME/database-backup/before_upgrade_to_4.6.dmp
[$OPENACS_SERVICE_NAME $OPENACS_SERVICE_NAME]$ exit
[root root]#
su - $OPENACS_SERVICE_NAME
pg_dump -f /var/lib/aolserver/$OPENACS_SERVICE_NAME/database-backup/before_upgrade_to_4.6.dmp openacs-dev
ls -al /var/lib/aolserver/$OPENACS_SERVICE_NAME/database-backup/before_upgrade_to_4.6.dmp
exit

  2. Back up the file system.Back up all of the files in the service, including the database backup file but excluding the auto-generated supervise directory, which is unneccesary and has complicated permissions.

    In the tar command,

    • c create a new tar archive

    • p preserves permissions.

    • s preserves file sort order

    • z compresses the output with gzip.

    • The --exclude clauses skips some daemontools files that are owned by root and thus cannot be backed up by the service owner. These files are autogenerated and we don't break anything by omitting them.

    • The --file clause specifies the name of the output file to be generated; we manually add the correct extensions.

    • The last clause, /var/lib/aolserver/$OPENACS_SERVICE_NAME/, specifies the starting point for backup. Tar defaults to recursive backup. 

    [root root]# su - $OPENACS_SERVICE_NAME

[$OPENACS_SERVICE_NAME $OPENACS_SERVICE_NAME]$ tar -cpsz --exclude /var/lib/aolserver/$OPENACS_SERVICE_NAME/etc/daemontools/supervise  --file /var/tmp/$OPENACS_SERVICE_NAME-backup.tar.gz /var/lib/aolserver/$OPENACS_SERVICE_NAME/
tar: Removing leading `/' from member names
[$OPENACS_SERVICE_NAME $OPENACS_SERVICE_NAME]$

  3. Suffer a catastrophic failure on your production system.(We'll simulate this step) 

    [root root]# svc -d /service/$OPENACS_SERVICE_NAME
[root root]# mv /var/lib/aolserver/$OPENACS_SERVICE_NAME/ /var/lib/aolserver/$OPENACS_SERVICE_NAME.lost
[root root]# rm /service/$OPENACS_SERVICE_NAME
rm: remove symbolic link `/service/$OPENACS_SERVICE_NAME'? y
[root root]# ps -auxw | grep $OPENACS_SERVICE_NAME
root      1496  0.0  0.0  1312  252 ?        S    16:58   0:00 supervise $OPENACS_SERVICE_NAME
[root root]# kill 1496

[root root]# ps -auxw | grep $OPENACS_SERVICE_NAME
[root root]# su - postgres
[postgres pgsql]$ dropdb $OPENACS_SERVICE_NAME
DROP DATABASE
[postgres pgsql]$ dropuser $OPENACS_SERVICE_NAME
DROP USER
[postgres pgsql]$ exit
logout
[root root]#

  4. Recovery.

    1. Restore the operating system and required software. You can do this with standard backup processes or by keeping copies of the install material (OS CDs, OpenACS tarball and supporting software) and repeating the install guide. Recreate the service user ($OPENACS_SERVICE_NAME).

    2. Restore the OpenACS files and database backup file. 

      [root root]# su - $OPENACS_SERVICE_NAME

[$OPENACS_SERVICE_NAME $OPENACS_SERVICE_NAME]$ cd /var/lib/aolserver
[$OPENACS_SERVICE_NAME aolserver]$ tar xzf /var/tmp/$OPENACS_SERVICE_NAME-backup.tar.gz
[$OPENACS_SERVICE_NAME aolserver]$ chmod -R 775 $OPENACS_SERVICE_NAME

[$OPENACS_SERVICE_NAME aolserver]$ chown -R $OPENACS_SERVICE_NAME.web$OPENACS_SERVICE_NAME

    3. Restore the database

      • Oracle.

        1. Set up a clean Oracle database user and tablespace with the same names as the ones exported from (more information).

        2. Invoke the import command 

          imp $OPENACS_SERVICE_NAME/$OPENACS_SERVICE_NAME FILE=/var/lib/aolserver/$OPENACS_SERVICE_NAME/database-backup/nighty_backup.dmp FULL=Y

      • Postgres.If the database user does not already exist, create it. 

        [root root]# su - postgres
[postgres ~]$ createuser $OPENACS_SERVICE_NAME

Shall the new user be allowed to create databases? (y/n) y
Shall the new user be allowed to create more new users? (y/n) y
CREATE USER
[postgres ~]$ exit

        Because of a bug in Postgres backup-recovery, database objects are not guaranteed to be created in the right order. In practice, running the OpenACS initialization script is always sufficient to create any out-of-order database objects. Next, restore the database from the dump file. The restoration will show some error messages at the beginning for objects that were pre-created from the OpenACS initialization script, which can be ignored. 

        [root root]# su - $OPENACS_SERVICE_NAME

[$OPENACS_SERVICE_NAME ~]$ createdb $OPENACS_SERVICE_NAME

CREATE DATABASE
[$OPENACS_SERVICE_NAME ~]$ psql -f /var/lib/aolserver/$OPENACS_SERVICE_NAME/packages/acs-kernel/sql/postgresql/postgresql.sql $OPENACS_SERVICE_NAME
(many lines omitted)
[$OPENACS_SERVICE_NAME ~]$ psql $OPENACS_SERVICE_NAME < /var/lib/aolserver/$OPENACS_SERVICE_NAME/database-backup/database-backup.dmp
(many lines omitted)
[$OPENACS_SERVICE_NAME ~]$ exit
[postgres ~]$ exit
logout

    4. Activate the service 

      [root root]# ln -s /var/lib/aolserver/$OPENACS_SERVICE_NAME/etc/daemontools /service/$OPENACS_SERVICE_NAME

[root root]# sleep 10
[root root]# svgroup web /service/$OPENACS_SERVICE_NAME