...\" @OPENGROUP_COPYRIGHT@
...\" COPYRIGHT NOTICE
...\" Copyright (c) 1990, 1991, 1992, 1993 Open Software Foundation, Inc.
...\" Copyright (c) 1996, 1997, 1998, 1999, 2000 The Open Group
...\" ALL RIGHTS RESERVED (MOTIF). See the file named COPYRIGHT.MOTIF for
...\" the full copyright text.
...\" 
...\" This software is subject to an open license. It may only be
...\" used on, with or for operating systems which are themselves open
...\" source systems. You must contact The Open Group for a license
...\" allowing distribution and sublicensing of this software on, with,
...\" or for operating systems which are not Open Source programs.
...\" 
...\" See http://www.opengroup.org/openmotif/license for full
...\" details of the license agreement. Any use, reproduction, or
...\" distribution of the program constitutes recipient's acceptance of
...\" this agreement.
...\" 
...\" EXCEPT AS EXPRESSLY SET FORTH IN THIS AGREEMENT, THE PROGRAM IS
...\" PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
...\" KIND, EITHER EXPRESS OR IMPLIED INCLUDING, WITHOUT LIMITATION, ANY
...\" WARRANTIES OR CONDITIONS OF TITLE, NON-INFRINGEMENT, MERCHANTABILITY
...\" OR FITNESS FOR A PARTICULAR PURPOSE
...\" 
...\" EXCEPT AS EXPRESSLY SET FORTH IN THIS AGREEMENT, NEITHER RECIPIENT
...\" NOR ANY CONTRIBUTORS SHALL HAVE ANY LIABILITY FOR ANY DIRECT,
...\" INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
...\" DAMAGES (INCLUDING WITHOUT LIMITATION LOST PROFITS), HOWEVER CAUSED
...\" AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
...\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN
...\" ANY WAY OUT OF THE USE OR DISTRIBUTION OF THE PROGRAM OR THE
...\" EXERCISE OF ANY RIGHTS GRANTED HEREUNDER, EVEN IF ADVISED OF THE
...\" POSSIBILITY OF SUCH DAMAGES.
...\" 
...\" 
...\" HISTORY
# $XConsortium: ch02.sm /main/3 1995/07/13 20:08:39 drk $
...\"  (c) Copyright 1992 OPEN SOFTWARE FOUNDATION, INC.
.H 1 "Developing Supporting Files"
.P
Automated tests need two types of input: scripts and instructions. These
supporting files and the syntax used to write them, are described in the
following sections.
.H 2 "Writing Instructions"
.P
Instructions are presented to the user in a series of InformationDialogBoxes, called "panels," which contain a list of steps for the user to perform, and three buttons. (For manual test directories, a four-button mode with \*LPass\*L and \*LFail\*O buttons is supported. See Chapter 5 for details.)
The first two buttons will always be labeled \*LContinue\*O and \*LExit\*O. The third button may, at the discretion of the test writer, be used for other test activities. For example, the third button might be labeled \*LShowSize\*O and when pressed, print out the \*LXmNheight\*O and \*LXmNwidth\*O of the widget under test. By default, the third button is insensitive.
.P
A new panel of instructions is posted in a message dialog each time a call to 
\*LCommonPause\*O is made in the test code. The instruction files include
a formatting syntax, which enables the writer to control the appearance of
the instructions in the panel.
.P
.H 3 "Instruction File Syntax"
The following is a list
of the special formatting characters that should be used in the instruction
file:
.VL .6i
.LI "\*L#)\*O"
Formats the data between this symbol and the next symbol as one single line of instructions. Each instruction will be sequentially numbered. The message string will be 60 characters long unless another width was specified using the \*L-w\*O option on the command line when invoking the test.
.LI "\*L@)\*O"
Extended format. Places a newline into the data stream first, then formats the text which follows this symbol by indenting 5 spaces from the left margin. The width of the strings will be 60 characters or the length specified by the user with \*L-w\*O. No instruction numbers will be placed in this block of text. This option allows the instruction writer to "set off" a block of informational, rather than instructional, text.
.LI "\*L!)\*O"
No Format. The data will not be formatted. This symbol should be used sparingly. Be sure the width of the unformatted area does not exceed the width of
the InstructionBox.
.LI "\*LC)\*O"
End of Panel. Specifies the end of one panel of instructions. This symbol indicates to the parsing routine that it has reached the end of a panel of instructions. An instruction to "Press the Continue button for more testing" will be automatically appended.
.LI "\*LE)\*O"
End of instructions. This symbol indicates the end of all instructions. A message "Exit Please" is automatically produced.
.LI "\*L\e<char>\*O"
Escape the next character. This allows any character with symbolic meaning to appear as normal in an instruction.
.LE
.P
Newlines, tabs, and extra
spaces will be stripped from the instruction text, unless they appear in
a "no format" area. When the special formatting characters are read, 
the appropriate characters are inserted into the data stream, in order
to produce the desired format.
The \*L.dat\*O file is read in two passes, first to reconcile the special characters, then to format the text to the proper length.
.H 3 "Sample Instruction File"
.P
The instruction file created by the test writer a test might look
something like this.
.sS
.iS
#) Move the pointer to the scrollbar's slider area and press Mouse
button1.\n\n 
#) Drag the slider to the top of the scrollbar (with key \\#).
@) Note the slider will not take any input because it is insensitive
C)
#) The Scrollbar will change orientation
to\t\t\n\n
XmVERTICAL.
!)
.sp
Try the following translations:
      1) BSelect Press:		Select()
      2) BDrag Press  :		Select()
.sp
E)
.iE
.sE
.P
On screen, the instructions will appear formatted as follows.
.P
Screen 1:
.sS
.iS
	1) Move the pointer to the scrollbar's slider area
	   and press Mouse button1.
	2) Drag the slider to the top of the scrollbar (with
	   key #).
.sp
	   Note the slider will not take any input because 
	   it is insensitive.
.sp
        Press the Continue Button for more testing.
.sp
.iE
.sE
.P
Screen 2:
.sS
.iS
	1) The Scrollbar will change orientation to XmVERTICAL.
.sp
	Try the following translations:
		1) BSelect Press:         Select()
		2) BDrag Press  :         Select()
.sp
	Test Finished -- Exit Please.
.sp
.iE
.sE
.H 2 "Writing Scripts"
.P
A script is a series of instructions. The instructions describe various mouse
and keyboard events, and are parsed into synthetic events. These synthetic
events are sent to the client under test. The language, parser and event-synthesis mechanism, together, form the automation technology.
.P
Scripts differ from instructions, since scripts are used by the automated technology to synthesize events, and instructions are used by a human user to operate the test. However, the test writer should find that the information to be
included in a script should flow quite naturally from the existing instructions. In fact, OSF testers frequently begin a new script by copying the \f(CW.dat\fP file into a new \f(CW.scr\fP file, and using the existing instructions as
comments for the new script.
.P
Our goal in designing the language was to allow the automation of user
actions, and \fBonly\fP user actions. There are no commands, for example,
that allow a tester to check and print out the value of resources. There
are no looping constructs, although you can specify multiple button clicks
on the same line. 
The scripts simply capture a description of how a human would interact
with a test in order to verify some aspects of the code under test.
.P
The automation technology, however, lets us synthesize a user with a photographic memory of the appearance of widgets in previous versions. An automated test cannot judge correctness of appearance the first time it sees a new widget; human intelligence is sill required for that. 
However, automated visual comparisons
are particularly efficient at capturing minute differences between two versions of the same thing: namely, regression testing.
.H 3 "The Scripting Language"
.P
The Chapter 6 of this book contains the reference 
pages for all the  commands in
the automation scripting language.
Refer to these pages for details on the syntax of the commands. 
The following subsections will describe briefly the \*Epurpose\*O 
of each command and give
examples of how a script writer might use it.
.H 4 "Locating the Pointer"
.P
Perhaps without realizing it, a user must explicitly choose a widget with
which he wants to interact, and move the pointer over that widget before
he can do so. The automation command \*LLocatePointer \*Vwidget_name\*O
moves the pointer over the midpoint of the named widget. 
This command usually precedes
any other mouse button command. Optionally, \*LLocatePointer\*O takes
arguments that let the scripter specify a move to a particular \*Vcomponent\*O
of a widget, such as a ScrollBar's slider, and a \*Vdestination\*O, which
for example, allows the scripter to move first \*Eto\*O the widget and then
slightly \*Eoff\*O the widget. This might be useful to test pressing a
button and then moving off it before releasing the button:
.oS
LocatePointer button2
PressMB Btn1
LocatePointer button2 off
ReleaseMB Btn1
.oE
.P
Most composite widgets, and even a few simpler ones, have components, or
regions, that one might need to specify. See the \*LComponentList\*O 
reference page
for a complete list of supported components.
.P
Sometimes, the desired location is not a widget. In this case, the command
\*LLocatePointerXYAbs\*O can be used to specify an explicit x,y location 
relative to the root window.
.H 4 "Mouse Button Commands"
.P
Perhaps the most frequent user interaction with Motif takes place via the
buttons on the mouse. Automation supports commands to simulate the
user clicking, pressing, or releasing any of the five mouse buttons.
The three commands are \*LClickMB\*O, \*LPressMB\*O, and \*LReleaseMB\*O,
where \*LMB\*O stands for \*LMouse Button\*O. Each command takes an argument 
which specifies which button - 1, 2, 3, 4 or 5 - should be manipulated.
The scripter may also specify multiple clicks with an optional 
final integer argument.
.H 4 "Dragging Commands"
.P
Some widgets are manipulated by pressing down a mouse button over some
part of widget, then moving the mouse to a desired location with the button
still held down, and finally releasing the mouse button in the desired location. The shorthand for all this activity is \*Edragging\*O. The automation scripting language supports four kinds of drag operations.
.P
\*LDragComponent\*O allows the scripter to drag between two components of the same widget: for instance, to do a drag-select in a list by dragging from the first to the third list item. The sequence of commands to accomplish this goal would be:
.oS
/* 0 is the first item in a list */
LocatePointer List1 RListItem 0  
DragComponent List1 RListItem 3
.oE
.P
\*LDragSliderValue\*O is valid only for the Scale and the ScrollBar
widgets, both of which have an inherent notion of "value",
expressed by the position of the slider. This command provides a shorthand
for dragging the slider so that it is set at a particular value.
.P
\*LDragXYAbs\*O can be used after any call to a \*LLocatePointer...\*O
command.
It is used to drag from the current location to any x,y location 
relative to the root window. 
\*LDragXYRelative\*O 
allows you to move the pointer by an x,y
\*Vdelta\*O, relative to the current pointer location, 
in the specified direction.
.H 4 "Keyboard Commands"
.P
Three command synthesize user actions from the keyboard: \*LClickKey\*O,
\*LPressKey\*O, and \*LReleaseKey\*O. Even though X supports both
PressKey and ReleaseKey events, users will generally think of their
action as a key click: pressing and releasing the key as one movement.
Scripters will probably find few situations where the separate \*LPress\*O
and \*LRelease\*O construct is necessary. In addition, any key can
be accompanied by one or more modifier keys, 
such as \*L<Shift>\*O or \*L<Ctrl>\*O.
.P
Please examine the reference page called \*LKeyList\*O.  Notice that most
keys are specified as \fIvirtual bindings\*O rather than as explicit
keys on the user's keyboard. Virtual keys are used so that the actual
configuration of an individual's keyboard does not matter. For example,
a script might specify the following sequence:
.oS
LocatePointer ScrollBar1
ClickKey KPageDown
.oE
.P
It does not matter which actual key or combination of keys a user might
need to press on any given keyboard: the virtual binding \*LKPageDown\*O
will always produce the same action. For more information on virtual
bindings, read the \*LVirtual Bindings\*O reference page in the \*LOSF/Motif
Programmer's Reference\*O and the \*LTranslations\*O section of 
the reference page for the widget with which you are working.
.P
A shortcut is provided for typing long strings, which one might use for
entering selections in a FileSelectionDialog, for instance. The command
\*LInputString \*Vstring\*O will simulate typing \*Vstring\*O into
the widget with the focus. It is equivalent to a series of \*LClickKey\*O
commands which enter each character separately.
.H 4 "Visual Comparisons"
.P
Visual comparisons are central to automated regression testing. The command
\*LCompareVisual \*Vwidget\*O instructs the automation code to take a
"photograph" of the widget \*Vwidget\*O 
as it appears on the screen at that time. The
\*LCompareVisual\*O command can be used at any time, so for example one
can record an image of a PushButton both while a mouse button is held down and
after it is released:
.oS
LocatePointer Pushbutton1
PressMB Btn1
CompareVisual Pushbutton1
ReleaseMB Btn1
CompareVisual Pushbutton1
.oE
.P
Any widget from \*LShell1\*O down may be recorded, but it is advisable
to record the smallest possible unit that captures the information the
tester is interested in. This helps keep the size of the stored visual files
from becoming too large, and simplifies debugging. However, tests for
overall layout of an application should include at least one visual
comparison of the whole hierarchy, in order to assure that the size and
relative positioning of widgets is the same.
It is useful to note
that the image of any widget includes that area which would contain
the focus (traversal) highlight.
.P
Visual images are saved in a \*L.vis\*O file when the test is recorded.
When the test is rerun, differences noted between saved and current images
are flagged in the output (\*L.prt\*O) file produced during the run.
Later sections of this book describe strategies for analyzing flagged
visual differences.
.P
Images can be recorded on one color monitor and replayed on any other.
You may not, however, record images on a monochrome or gray-scale monitor,
and play them back on a color monitor, or vice versa. If it is necessary
to run regression tests on both color and mono devices, it will also
be necessary to store separate visual output.
.P
There is an important point to be noted about the way colors are used
and therefore tested with \*LVisualCompare\*O. When an image is recorded,
the automation code forces each distinctly-colored region within a widget
to a color of the automation code's choosing. (This is the reason for
the strange appearance of tests being recorded.) On subsequent comparisons,
the automation code \*Edoes\*O verify that the new widget has the same
distinct regions as the recorded widget. It \*Edoes not\*O, however,
guarantee that the exact same color is used for any region between
record and playback. Only the size and shape of the region, as distinct
from the regions that surround it, is verified.
.P
As an example, consider a BulletinBoard widget with no children. The
BulletinBoard has the following distinct regions: top and left shadow;
bottom and right shadow; and background. In a normal client, these might
be rendered in subtly different shades of blue. During the record stage,
the top and left shadow region might be colored blue, the bottom and right
shadow colored pink, and the background colored yellow. A
subsequent comparison
will pass if the new image contains three regions, each of a different color,
and each matching the size and shape of the corresponding region at record
time. However, if the shadow contrast algorithm has been changed slightly,
the visual difference between the new and old shadows will not be detected
unless the shadows have also changed size or shape, or if fewer than three
distinct colors are used.
.H 4 "Window Manager Commands"
.P
A large subset of commands exists to allow testers to manipulate and control
the various areas in the window border, that region which is drawn and
controlled by the Motif Window Manager (\*Lmwm\*O). As stated earlier, the 
\*Lmwm\*O
used during all phases of testing must be specially compiled to support
automated testing. Also note that \*Lmwm\*O must be run with a specific
configuration: for example, \*Lmwm\*O must be run with a \*LkeyboardFocusPolicy\*O of \*LPOINTER\*O.
The \*L.Xdefaults\*O, \*L.motifbind\*O,
and \*L.mwmrc\*O 
files are provided in the sample environment, located on the tape
in \*L$TOP/tests/environment\*O.
.P
A summary of the Window Manager commands follows. See the Reference section
for syntax and detailed descriptions of each of these commands.
.sp
.in +3
.VL 1.75i
.LI "\*LIconMenuPost\*O"
Posts the system menu for an iconified window.
.LI "\*LIconMenuSelect\*O"
Selects a specific item on the system menu of an iconified window.
.LI "\*LIconMenuUnpost\*O"
Unposts the system menu for an iconified window.
.LI "\*LIconMove\*O"
Moves an icon to a specific x,y location relative to the root window.
.LI "\*LWindowDeiconify\*O"
Returns an iconified window to the normalized (uniconified) state.
.LI "\*LWindowFocus\*O"
Gives a window focus by clicking on the title area of the window frame.
.LI "\*LWindowIconify\*O"
Iconifies a window.
.LI "\*LWindowItemCheck\*O"
Verifies the existence of a particular window decoration; for example, 
the minimize button.
.LI "\*LWindowMaximize\*O"
Resize a window to the largest possible size.
.LI "\*LWindowMenuPost\*O"
Posts the system menu for the specified window.
.LI "\*LWindowMenuSelect\*O"
Selects a specific item on the system menu of a specified window.
.LI "\*LWindowMenuUnpost\*O"
Unposts the system menu for an specified window.
.LI "\*LWindowMove\*O"
Moves a window to a specific x,y location relative to the root window.
.LI "\*LWindowNormalize\*O"
Resizes a window to its original size.
.LI "\*LWindowRaise\*O"
Raises a window above other windows on the screen.
.LI "\*LWindowResize\*O"
Resizes a window in a specific direction, to a desired size. Eight directions
are supported, corresponding to the four sides and four resize handles in the
corners of a window.
.LE
.in -3
.P
Note that Window and Icon commands are supported only for the OSF/Motif Window
Manager (\*Lmwm\*O).
.H 4 "Other Commands"
.P
The \*LWait\*O command is most useful for tests run on slow servers or
heavily loaded workstations. \*LWait\*O instructs the automation code
to pause before executing the next instruction, usually to give the server
time to "catch up" with refreshing the display. It is often difficult to
know in advance when to use \*LWait\*O, but OSF testers have found that
slow servers often need time to catch up when a large scrolled text or
scrolled list widget has been scrolled by several screens, and
before doing a visual comparison. Most likely,
the need for the \*LWait\*O command will become apparent when spurious
visual comparison errors surface during runs.
.P
The \*LContinue\*O and \*LExit\*O commands simply instruct the automation
code to press the corresponding buttons on the Instruction Panel message
boxes which are used in the OSF/Motif QATS tests. If a test or application
does not use Instruction Panels (via calls to \*LCommonPause\*O), these
commands will not be needed.
.P
The \*LSystem\*O command allows for extensibility of the testing language
without making actual modifications to the automation code. Any shell script,
or executable can be invoked via the \*LSystem\*O command. For example,
one might write scripts aimed at performance testing, and use the \*LSystem\*O command to call certain profiling tools after certain sequences of events.
.H 4 "Comments"
.P
Both C-style (\*L/* \*Vcomment text \*L*/\*O) and shell script-style comments 
(\*L# \*Vcomment text\*O) are supported.