This is a test suite for Gfarm named 'systest'.  It does the following
tests which are not examined by the regression tests of Gfarm.

    * disable a network interface suddenly
    * gfmd failover
    * stop PosgreSQL
    * stress tests
    * perform 'config-gfarm'
    * perform 'config-gfsd'

Preparation
===========
Most of all testcases in 'systest' assume that a Gfarm system is already
running on your hosts.  The minimum configuration of the system is:

    * two gfmd servers (gfmd1, gfmd2)
    * four gfsd servers (gfsd1, gfsd2, gfsd3, gfsd4)
    * one client host (client1)

Since 'systest' uses 'gfservice' command internally, you need to
prepare a configuration file of 'gfservice' (the default path is
"$HOME/.gfservice") for those hosts, like as follows:

    gfmd1=mds1-ctl.example.com
    gfmd1_CONFIG_GFARM_OPTIONS="-r -h mds1.example.com -A $LOGNAME"
    gfmd1_IFUP_CMD="/sbin/ifup eth2"
    gfmd1_IFDOWN_CMD="/sbin/ifdown eth2"
    gfmd1_LOG_FILE_PATHS=/var/log/syslog

    gfmd2=mds2-ctl.example.com
    gfmd2_CONFIG_GFARM_OPTIONS="-r -h mds2.example.com -A $LOGNAME"
    gfmd2_REPLICATION_MODE=async
    gfmd2_IFUP_CMD="/sbin/ifup eth2"
    gfmd2_IFDOWN_CMD="/sbin/ifdown eth2"
    gfmd2_LOG_FILE_PATHS=/var/log/syslog

    gfsd1=fsn1-ctl.example.com
    gfsd1_CONFIG_GFSD_OPTIONS="-h fsn1.example.com"
    gfsd1_IFUP_CMD="/sbin/ifup eth1"
    gfsd1_IFDOWN_CMD="/sbin/ifdown eth1"
    gfsd1_LOG_FILE_PATHS=/var/log/syslog

    gfsd2=fsn2-ctl.example.com
    gfsd2_CONFIG_GFSD_OPTIONS="-h fsn2.example.com"
    gfsd2_IFUP_CMD="/sbin/ifup eth1"
    gfsd2_IFDOWN_CMD="/sbin/ifdown eth1"
    gfsd2_LOG_FILE_PATHS=/var/log/syslog

    gfsd3=fsn3-ctl.example.com
    gfsd3_CONFIG_GFSD_OPTIONS="-h fsn3.example.com"
    gfsd3_IFUP_CMD="/sbin/ifup eth1"
    gfsd3_IFDOWN_CMD="/sbin/ifdown eth1"
    gfsd3_LOG_FILE_PATHS=/var/log/syslog

    gfsd4=fsn4-ctl.example.com
    gfsd4_CONFIG_GFSD_OPTIONS="-h fsn4.example.com"
    gfsd4_IFUP_CMD="/sbin/ifup eth1"
    gfsd4_IFDOWN_CMD="/sbin/ifdown eth1"
    gfsd4_LOG_FILE_PATHS=/var/log/syslog

    client1=pc1.example.com
    MOUNT_TOPDIR=/work/gfarm-systest
    MOUNT_OPTIONS="-syslog=local7"

Note that the "-A" options has to be specified in gfmd*_CONFIG_GFARM_OPTIONS,
because the config-gfarm command invoked by gfservice cannot get
appropriate default value, when it's invoked from ssh, because it doesn't
have a tty and the "who am i" command fails in that case.

The example above declares variables not explained in gfservice.conf(5).
They are not interpreted by 'gfservice' but 'systest':

    gfmd<n>_IFDOWN_CMD
    gfsd<n>_IFDOWN_CMD
    client<n>_IFDOWN_CMD
        Those variables specify how to disable a network interface
        used by gfmd, gfsd or a client to communicate with other
        servers and clients.  The default value is "/sbin/ifdown eth0".

        It is recommended that hostnames used by a Gfarm system (shown
        by 'gfmdhost -l' and 'gfhost -l') and used by 'gfservice' ($gfmd<n>
        and $gfsd<n>) differ.  For example,

            gfmd1=mds1-ctl.example.com
            gfmd1_CONFIG_GFARM_OPTIONS="-h mds1.example.com"

            gfsd1=fsn1-ctl.example.com
            gfsd1_CONFIG_GFSD_OPTIONS="-h fsn1.example.com"

        In this case, specify an interface name associated with
        'mds1.example.com' for 'gfmd1_IFDOWN_CMD' and an interface
        name associated with 'fsn1.example.com' for 'gfsd1_IFDOWN_CMD'.

    gfmd<n>_IFUP_CMD
    gfsd<n>_IFUP_CMD
    client<n>_IFUP_CMD
        Those variables specify how to enable a network interface.
        The default value is "/sbin/ifup eth0".

    gfmd<n>_LOG_FILE_PATHS
    gfsd<n>_LOG_FILE_PATHS
        The absolute paths to log files on which log messages of gfmd
        (or gfsd) are recorded.  If you'd like to specify two or more
        files, list paths separated by a whitespace:

            gfmd1_LOG_FILE_PATH="/var/log/messages /var/log/gfarm"

        The default value is "/var/log/messages".

    MOUNT_TOPDIR
	The absolute paths to mount potions.  'systest' mounts gfarm2fs
	on sub-directories under $MOUNT_DIR.  The default value is $HOME
	(home directory).  If a home directory of the client host is
	hosted by NFS, specify a path to a directory on a local disk.
	Note that the directory must be writable by the test user.

    MOUNT_OPTIONS
	Additional options to 'gfarm2fs'.

Read gfservice.conf(5) for more details about the configuration file.

Some testcases require 'gfarmroot' privilege.  The user who executes
the testcaes needs to belong to 'gfarmroot' group.  Otherwise the
testcases result in UNSUPPORTED.

How to run a test case
======================
To run a test case, execute 'gfruntest' command:

    gfruntest <test case>

The example below runs the test case 'make_dir':

    gfruntest gfarm2fs::make_dir

You can run multiple test cases using a scenario file:

    gfruntest -s <scenario file>

The following scenario files are currently available:

    * gfarm2fs/failover
      gfmd failover
    * gfarm2fs/file_dir_operations
      generic operations about file and directory.
    * gfarm2fs/ifdown
      disable a network interface
    * gfarm2fs/pgsql_down
      stop PostgreSQL
    * gfarm2fs/read_files
      read files (stress tests)
    * gfarm2fs/write_files
      write files (stress tests)
    * gfarm2fs/write_files_each_mntpnts
      write files from multiple mount points in parallel
      (stress tests)
    
Details about the test cases are described in 'testcases.xls' (written
in Japanese).
