AT&T SRE Toolkit
SOFTWARE RELIABILITY ENGINEERING
Reliability Estimation Tools
SRE TOOLKIT
_____________________________________________________
Introduction
The reliability estimation tools described in this
guide are particularly useful during system test
and field trial This version of the SRE TOOLKIT
contains the standard release of the reliability
estimation tool EST, the graphics support tool PLOT
and a number of prototype tools used in conjunction
with exercises in course SN9110 (Software Reliabil-
ity Engineering - Application) provided through
Kelly Education and Training Center. This guide is
a reference guide for using the standard tools EST
and PLOT. Programmer Notes on using the prototype
tools are provided in Appendix B. As such, they
are not intended as a training or tutorial guide on
Software Reliability Engineering and should be used
along with training from course SN9110.
The reliability estimation tool EST described in
this guide is particularly useful during system
test and field trial phases of software product
development. During these phases, failure events
are encountered and the underlying faults that
cause the failures are being removed. This results
in "reliability growth" during product test or tri-
al. The tools implement techniques discussed in
reference [MIO] to estimate the current level of
software product reliability and to predict the
remaining time to attain a specified reliability
objective.
The tool EST discussed in this guide can be used to
"fit" one of two reliability models described in
reference [MIO] to failure data. In turn, EST uses
the "fitted" model to estimate several useful reli-
ability measures such as present failure intensity,
remaining time to reach a specified failure inten-
sity objective and a calendar date when a failure
intensity objective will be met. EST produces its
Page 1. Iss. 1D1
Reliability Estimation Tools
SRE TOOLKIT
output as a tabular report and a file of "plot"
commands. The tool PLOT in turn takes the file of
plot commands and produces a set of plots on a
graphics media. Graphic media currently supported
set of plots on a graphics medium. Graphic media
currently supported include postscript printers
accessible through UNIX8r9 systems or graphics moni-
tors on PC's running under MS-DOS8r9.
There are two versions of the tools, one version
runs under the UNIX There are two versions of the
tools: one version runs under the UNIX operating
environment while the other version runs under the
MS-DOS operating environment. Both versions of the
tools have been carefully engineered so that the
format of input and output data, plots, and general
use of the tools is the same under either UNIX or
MS-DOS. This provides considerable flexibility to
the user. A user can distribute work between a
large shared UNIX system to take advantage of its
support facilities and a small PC workstation to do
quick turn-around "what if" analysis of collected
data. The user can use terminal emulator programs
(such as "ctrm") to up-load and down-load input and
output files between a UNIX system and a PC works-
tation to take advantage of particular facilities
available on each system. A user can distribute
work between a large shared UNIX system taking
advantage of its support facilities and a small PC
work-station obtaining a quick turn-around "what
if" analysis of collected data by using the
screen-graphics capabilities of the MS-DOS version.
Then, the user might use a terminal emulator pro-
gram (such as "ctrm") to up-load input data files
to a UNIX system and take advantage of documenter
and printer facilities to produce an output report
including graphical output from the tools.
This guide is a reference guide for using the tools
and is not intended as a training or tutorial guide
on Software Reliability Engineering. This guide
should be used in conjunction with training from
This guide should be used with training from course
SN9110 (Software Reliability Engineering - Practi-
cal Applications) Engineering - Application) pro-
vided through Kelly Education and Training Center
_____________________________________________________
Organization
of this
Guide
The remainder of this guide is divided into two
parts. The remainder of this guide is divided into
three parts. The first part contains information
on required hardware and operating environments for
running the tools, instructions on installing the
tools, instructions on getting started and tips on
using the tools. At the end of the first part is
information on what is available in the way of
training, project support, tool support and refer-
ences.
The second part (in the Appendix) is a set of manu-
al pages providing a detail The second part is a
set of manual pages in Appendix A providing a de-
tail reference on using the tools themselves. The
manual pages provide examples of inputs to and
resulting outputs from the tools. The manual pages
also provide pointers back into reference [MIO] for
further information on Software Reliability itself,
on the input data that is needed by the tools and
on interpreting output of the tools. on Software
Reliability Engineering itself, on the input data
that is needed by the tools and on interpreting
output of the tools.
The third part is a set of Programmer Notes in Ap-
pendix B on running a set of prototype tools
developed for SN9110. There are also UNIX and MS-
DOS versions these tools. Included with the Pro-
grammer Notes are a set of Manual Pages for using
these tools. Again, the manual pages provide exam-
ples of inputs to and resulting outputs from the
tools.
_____________________________________________________
Hardware and
Software
Requirements
The UNIX version of the tools runs under any ver-
sion of UNIX System V and on any version of
hardware processor that supports UNIX System V.
Care was exercised to use a restricted set of UNIX
system library calls to maintain as much portabili-
ty as possible between systems.
Iss. 4D4 Page 4.
SOFTWARE RELIABILITY ENGINEERING
The MS-DOS version should work with MS-DOS release
3.3 or greater running on any AT&T compatible PC.
To use graphics, the PC should be equipped with a
CGA, EGA, VGA or Hercules compatible graphics
board.
If extensive plotting is to be done or large
(greater than 150 failure events) failure data sets
will be analyzed, floating point hardware will sig-
nificantly reduce processing time (from minutes to
seconds). For running the tools on a PC, this would
mean investing in a numeric coprocessor (sometimes
referred to as a math coprocessor) chip. this
would mean investing in a numeric co-processor
(sometimes referred to as a math co-processor)
chip. These chips usually have a model designation
of 8xx87 or 8x87 depending on the type of processor
in your PC.
_____________________________________________________
Installation
UNIX Version - The UNIX version of the tools is
distributed using UNIX electronic mail (email) fa-
cilities. To obtain a copy, contact
Michael R. Lyu
giving your name and email address. You will
receive email back confirming the receipt of your
request and indicating when and how the tools will
be sent to you. The tools will be sent to you via
the UNIX uuto(1) command (see reference [ATT]).
command (see reference [ATTa]). You will receive email
indicating the tools have arrived on your system.
At that time, you should execute the UNIX uupick(1)
command to retrieve them. First change directories
to whichever directory you want the tools directory
SRE_TOOLS installed into, then execute uupick. At
the prompt
from system !whuxr: directory SRE_TOOLS ?
from system !mtfmi: directory SRE_TOOLS ?
type "m ." followed by a carriage return. At this
point the SRE_TOOLS directory will be installed in
the current directory you are in. Then, change
directory to the SRE_TOOLS directory and read the
file READ.ME which provides further information on
Iss. 6D6 Page 6.
SOFTWARE RELIABILITY ENGINEERING
installing the tools. Complete installation
requires approximately 5 to 15 minutes depending on
the processing speed and processing load levels of
your system.
MS-DOS Version - The MS-DOS version is distributed
either on 5-1/4 inch or 3-1/2 inch floppies. Each
floppy contains "executable" program files The MS-
DOS version is distributed on a 5-1/4 inch, 360
Kbyte double-sided, double-density floppy. The
floppy contains "executable" program files est.exe
and plot.exe and test data files tst.fp, tst.ft,
tst.pc, tst_stg.fp, tst_stg.ft and tst_stg.pc. The
floppy diskette can be inserted in the appropriate
drive, the drive selected by typing a: or b:. If
your PC has a hard disk, we recommend copying the
program and data files into a directory on your
hard disk and running the tools from your hard
disk. Otherwise, you can run the tools directly
from the floppy diskette (of course, making a
backup copy of the diskette first). Alternatively,
with the UNIX version of the delivered tools, there
is a directory named dos that contains copies of
the "executable" program and data files. These
files can be up-loaded to your PC using the file-
transfer capabilities of your favorite terminal
emulator package (such as ctrm).
___________________________________________________
__
Page 7. Iss. 7D7
Reliability Estimation Tools
SRE TOOLKIT
_____________________________________________________
Getting
Started,
Using the
Manual Pages
You might want to first review the manual pages in
the appendix. You might want to first review the
manual pages in Appendix A. Manual page EST(1)
describes the est program. First, quickly browse
the "Description" section of the manual page and
then read the "Example" section to follow the exe-
cution of a particular example. After reading the
manual page, you may want to proceed to the next
section of this guide to execute the example in
your UNIX or MS-DOS environment. If you want to
learn more about the input data files for est, then
you'll want to read the .FP(5) manual page that
describes the contents of the "failure parameter"
file and .FT(5) manual page that describes the con-
tents of the "failure time" file. Again, browse
the "Description" section of the manual page, then
read the "Example" section. Refer back to the
"Description" section read the "Example" section.
Refer to the "Description" section whenever you
need more detail in following the example. In gen-
eral, you don't need to be familiar with the
PLOT(1) manual page. However, if you reach a point
where you want to tailor some of the plots, then
you can read this manual page to see how to change
the plot commands in the associated .pc files pro-
duced by est.
The same applies to getting started with the proto-
type tools. First, review the Programmer notes to
understand the caveats in running the tools. Then
browse the "Description" section of the manual page
and concentrate on the "Example" section of the
manual page.
_____________________________________________________
Getting
Started,
Running the
Examples
UNIX Version - After installing the UNIX version of
the tools, change directories to the directory your
tools were installed in and then to the testdir
9110exer subdirectory under this directory. There
are two sets of project data provided in this
directory. The first is associated with project
tst. This data is in files tst.fp and tst.ft.
First run the est program against this project data
by typing est tst. Note the tabular output report
produced. Manual page EST(1) in the appendix can be
used to interpret the contents of this report.
Note the tabular output report produced. Manual
page EST(1) in Appendix A can be used to interpret
the contents of this report. The program creates
the plot commands in file tst.pc. Now, you can
type plot tst to generate the plots. In the UNIX
version, a file tst.po containing pic(1) and
troff(1) commands is produced (see the UNIX User
Reference Manual for more information on pic and
troff). You can now run your favorite command for
formatting "troff" text You can now run your favor-
ite command for formatting troff text files819 and
routing to output to postscript printers or other
printers with graphics capabilities with the tst.po
file. Don't forget to either first run the UNIX
pic(1) command against the file or to include the
appropriate option on the command line of the troff
text formatter to preprocess the file using the
"pic(1)" command. using the pic(1) command. The
__________
1. Examples of such commands are mmx(1), mmt(1), xroff(1).
An example invocation of such a command with the tst
data would be mmt -p tst.po where the -p option on the
command line indicates the file should first be
processed by "pic(1)". indicates the file should first
be processed by "pic(1)." indicates the file should
first be processed by pic(1). Check with your UNIX
system administrator to find out what commands are
available on your UNIX system. Reference [GE] provides
further information on document formating commands
Reference [GE] provides further information on document
formatting commands under UNIX.
Page 9. Iss. 9D9
Reliability Estimation Tools
SRE TOOLKIT
second set of project data with project name
tst_stg is the same as the first except "staged
delivery" information has been added to the failure
data (see the .FT(5) manual page in the appendix
for a description of staged delivery). manual page
in Appendix A for a description of staged
delivery). You may now want to run est and plot
programs with this project data and compare the
resulting tabular report and plots with the tst
project data.
MS-DOS Version - If you have created a directory on
your hard disk with the program and data files
included with the distribution diskette, then you
should first "change directories" to this direc-
tory. As with the UNIX version, there are two sets
of project data provided. The first is associated
with project tst. This data is in files tst.fp and
tst.ft. First run the est program against this
project data by typing est tst. Note the tabular
output report produced. Manual page EST(1) in the
appendix can be used to interpret the contents of
this report. in Appendix A can be used to inter-
pret the contents of this report. In the MS-DOS
version of the data files, the genplt parameter has
been set in the failure parameter file tst.fp file
so no plot commands are produced (this was done
because PC's not having a "math coprocessor" board
will require a long time to run). not having a
"math co-processor" chip will require a long time
to run). Instead, the "plot commands" correspond-
ing to project tst has already been created and
supplied as file tst.pc with the distribution
diskette. You may now want to run plot tst to pro-
duce the plots directly on your video monitor. The
second set of project data with project name
tst_stg is the same as the first except "staged
delivery" information has been added to the failure
data (see the .FT(5) manual page in the appendix
for a description of staged delivery). manual page
in Appendix A for a description of staged
delivery). You may now want to run est and plot
programs with this project data and compare the
resulting tabular report and plots with the tst
project data. Again, genplt has been set so no
plot commands are generated. Instead, tst_stg.pc
file has been provided with your distribution
diskette.
Iss. 10D10 Page 10.
SOFTWARE RELIABILITY ENGINEERING
_____________________________________________________
When running under MS-DOS, you can get hard-copy of
screen output by using the MS-DOS mode mode and
graphics graphics commands and a locally connected
dot-matrix printer (see you MS-DOS User's Guide).
To do this you generally execute the mode mode com-
mand to define the characteristics of your printer
and then the graphics graphics command to load a
memory-resident program. To print a copy of the
display currently appearing on your terminal's
screen, you would depress the "Prt Sc" or "Print
Screen" key (the name of this key is dependent on
exactly what type of keyboard you have).
The program plot(1) plot(1) that is provided with
this tool set is a useful tool in itself for pro-
ducing displays. The novice user need not get into
learning about the plot commands that the plot(1)
plot(1) program uses in generating plots. The ex-
pert user can begin writing analysis programs that
can generate .pc .pc files to create their own
graphs. Or, better yet, you can change .pc .pc
files created by the est(1) est(1) program to add
additional lines, points, labels on particular
graphs and so on. One approach to using the tools
is to do the heavy CPU est(1) est(1) runs on a UNIX
system with processing horse-power. Then download
Iss. 12D12 Page 12.
SOFTWARE RELIABILITY ENGINEERING
the resulting .pc .pc files to a PC, edit the .pc
.pc files to do "touch-ups" and customize graphs
and run plot(1) plot(1) on the PC to immediately
see the effects of changes to the .pc .pc files.
Finally, the .pc .pc files can then be up-loaded to
a UNIX system to runoff final reports with the
graphical output on a laser printer.
For those who may be familiar with the RELTOOLS
tool set and the reltab reltab and relplt relplt
programs, this tool set provides everything in the
way of features of these tools plus more. The
structure of the input files for est(1) est(1) pro-
gram is quite similar to the input files of reltab
reltab and relplt relplt programs. The one notable
exception is the failure time file. For the
reltab/relplt reltab/relplt programs, the
corresponding file is referred to as a failure
interval file. The times in the failure interval
file are "times between failures" rather than
actual "failure times." The format of the failure
interval and failure time files are different. The
failure parameter files for both the est(1) est(1)
and reltab/relplt reltab/relplt file are almost
identical (there are a few differences in some
parameter names).
_____________________________________________________
SOFTWARE RELIABILITY ENGINEERING
Caveats
Version 3.7 is a "Beta Trial" version of the tools
that is being made available on a "friendly user"
basis. As such, the basic functionality of the
tools has been extensively tested and results com-
pared with the predecessor software RELTAB/RELPLT.
With this version of the tool, the heuristic algo-
rithm (in EST) that determines scale values for the
X and Y axes has not been fully implemented (this
algorithm selects a scale so the X and Y scale
values printed contain only a few significant di-
gits). Also, some of the less frequently used PLOT
PARAMETER options for EST (such as charht, clopt,
conlvl, dshopt, grdopt, xwinb, xwine, ywinb, ywine)
have not been fully tested.
Users of this "Beta Trial" version of the tools are
asked to communicate any problems via email to
"whuxr!wwe" or (w.w.everett@att.com).
_____________________________________________________
References
[ATT] "UNIX System V Release 3.0 User Reference
Manual", to order, "UNIX System V Release
3.0 User Reference Manual," to order,
[ATTa] UNIX System V Release 3.0 User Reference
Manual, to order, call AT&T Customer's In-
formation Center 1-800-432-6600 and order
Select Code 307-184.
[ATTb] Reliability by Design, Chapter 8, to order,
call AT&T Customer's Information Center 1-
800-432-6600.
[MIO] Musa, J. D., A. Iannino and K. Okumoto,
Software Reliability - Measurement, Predic-
tion, Application Software Reliability -
Measurement, Prediction, Application,
McGraw-Hill, 1987, ISBN 0-07-044093-X.
[MA] Musa, J and A. F. Ackerman, Quantifying
Software Validation: When to Stop Testing?,
IEEE SOFTWARE, May 1989, pg. 19-27.
[GE] Gehani, N., Document Formatting on the UNIX
System, Silicon Press, 1986, ISBN 0-
9615336-0-9.
_____________________________________________________