AT&T SRE Toolkit





                       SOFTWARE RELIABILITY ENGINEERING



                         Reliability Estimation Tools



       				SRE TOOLKIT



                   _____________________________________________________



       Introduction

                   The reliability estimation tools described in this

                   guide are particularly useful during system test

                   and field trial This version of the SRE TOOLKIT

                   contains the standard release of the reliability

                   estimation tool EST, the graphics support tool PLOT

                   and a number of prototype tools used in conjunction

                   with exercises in course SN9110 (Software Reliabil-

                   ity Engineering -  Application) provided through

                   Kelly Education and Training Center.  This guide is

                   a reference guide for using the standard tools EST

                   and PLOT.  Programmer Notes on using the prototype

                   tools are provided in Appendix B.  As such, they

                   are not intended as a training or tutorial guide on

                   Software Reliability Engineering and should be used

                   along with training from course SN9110.



                   The reliability estimation tool EST described in

                   this guide is particularly useful during system

                   test and field trial phases of software product

                   development. During these phases, failure events

                   are encountered and the underlying faults that

                   cause the failures are being removed. This results

                   in "reliability growth" during product test or tri-

                   al. The tools implement techniques discussed in

                   reference [MIO] to estimate the current level of

                   software product reliability and to predict the

                   remaining time to attain a specified reliability

                   objective.



                   The tool EST discussed in this guide can be used to

                   "fit" one of two reliability models described in

                   reference [MIO] to failure data.  In turn, EST uses

                   the "fitted" model to estimate several useful reli-

                   ability measures such as present failure intensity,

                   remaining time to reach a specified failure inten-

                   sity objective and a calendar date when a failure

                   intensity objective will be met.  EST produces its







       Page 1.                                             Iss. 1D1















       Reliability Estimation Tools

       SRE TOOLKIT









                   output as a tabular report and a file of "plot"

                   commands.  The tool PLOT in turn takes the file of

                   plot commands and produces a set of plots on a

                   graphics media. Graphic media currently supported

                   set of plots on a graphics medium. Graphic media

                   currently supported include postscript printers

                   accessible through UNIX8r9 systems or graphics moni-

                   tors on PC's running under MS-DOS8r9.



                   There are two versions of the tools, one version

                   runs under the UNIX There are two versions of the

                   tools: one version runs under the UNIX operating

                   environment while the other version runs under the

                   MS-DOS operating environment. Both versions of the

                   tools have been carefully engineered so that the

                   format of input and output data, plots, and general

                   use of the tools is the same under either UNIX or

                   MS-DOS. This provides considerable flexibility to

                   the user.  A user can distribute work between a

                   large shared UNIX system to take advantage of its

                   support facilities and a small PC workstation to do

                   quick turn-around "what if" analysis of collected

                   data.  The user can use terminal emulator programs

                   (such as "ctrm") to up-load and down-load input and

                   output files between a UNIX system and a PC works-

                   tation to take advantage of particular facilities

                   available on each system.  A user can distribute

                   work between a large shared UNIX system taking

                   advantage of its support facilities and a small PC

                   work-station obtaining a quick turn-around "what

                   if" analysis of collected data by using the

                   screen-graphics capabilities of the MS-DOS version.

                   Then, the user might use a terminal emulator pro-

                   gram (such as "ctrm") to up-load input data files

                   to a UNIX system and take advantage of documenter

                   and printer facilities to produce an output report

                   including graphical output from the tools.



                   This guide is a reference guide for using the tools

                   and is not intended as a training or tutorial guide

                   on Software Reliability Engineering.  This guide

                   should be used in conjunction with training from

                   This guide should be used with training from course

                   SN9110 (Software Reliability Engineering - Practi-

                   cal Applications) Engineering -  Application) pro-

                   vided through Kelly Education and Training Center



                   _____________________________________________________



       Organization



        of this



         Guide

                   The remainder of this guide is divided into two

                   parts.  The remainder of this guide is divided into

                   three parts.  The first part contains information

                   on required hardware and operating environments for

                   running the tools, instructions on installing the

                   tools, instructions on getting started and tips on

                   using the tools. At the end of the first part is

                   information on what is available in the way of

                   training, project support, tool support and refer-

                   ences.



                   The second part (in the Appendix) is a set of manu-

                   al pages providing a detail The second part is a

                   set of manual pages in Appendix A providing a de-

                   tail reference on using the tools themselves. The

                   manual pages provide examples of inputs to and

                   resulting outputs from the tools. The manual pages

                   also provide pointers back into reference [MIO] for

                   further information on Software Reliability itself,

                   on the input data that is needed by the tools and

                   on interpreting output of the tools.  on Software

                   Reliability Engineering itself, on the input data

                   that is needed by the tools and on interpreting

                   output of the tools.



                   The third part is a set of Programmer Notes in Ap-

                   pendix B on running a set of prototype tools

                   developed for SN9110.  There are also UNIX and MS-

                   DOS versions these tools. Included with the Pro-

                   grammer Notes are a set of Manual Pages for using

                   these tools.  Again, the manual pages provide exam-

                   ples of inputs to and resulting outputs from the

                   tools.



                   _____________________________________________________



       Hardware and



        Software



       Requirements

                   The UNIX version of the tools runs under any ver-

                   sion of UNIX System V and on any version of

                   hardware processor that supports UNIX System V.

                   Care was exercised to use a restricted set of UNIX

                   system library calls to maintain as much portabili-

                   ty as possible between systems.











       Iss. 4D4                                                  Page 4.

















                                        SOFTWARE RELIABILITY ENGINEERING









                   The MS-DOS version should work with MS-DOS release

                   3.3 or greater running on any AT&T compatible PC.

                   To use graphics, the PC should be equipped with a

                   CGA, EGA, VGA or Hercules compatible graphics

                   board.



                   If extensive plotting is to be done or large

                   (greater than 150 failure events) failure data sets

                   will be analyzed, floating point hardware will sig-

                   nificantly reduce processing time (from minutes to

                   seconds). For running the tools on a PC, this would

                   mean investing in a numeric coprocessor (sometimes

                   referred to as a math coprocessor) chip.  this

                   would mean investing in a numeric co-processor

                   (sometimes referred to as a math co-processor)

                   chip. These chips usually have a model designation

                   of 8xx87 or 8x87 depending on the type of processor

                   in your PC.



                   _____________________________________________________



       Installation

                   UNIX Version - The UNIX version of the tools is

                   distributed using UNIX electronic mail (email) fa-

                   cilities. To obtain a copy, contact



                                   Michael R. Lyu



                   giving your name and email address.  You will

                   receive email back confirming the receipt of your

                   request and indicating when and how the tools will

                   be sent to you.  The tools will be sent to you via

                   the UNIX uuto(1) command (see reference [ATT]).

                   command (see reference [ATTa]).  You will receive email

                   indicating the tools have arrived on your system.

                   At that time, you should execute the UNIX uupick(1)

                   command to retrieve them.  First change directories

                   to whichever directory you want the tools directory

                   SRE_TOOLS installed into, then execute uupick.  At

                   the prompt



                           from system !whuxr: directory SRE_TOOLS ?

                           from system !mtfmi: directory SRE_TOOLS ?



                   type "m ." followed by a carriage return.  At this

                   point the SRE_TOOLS directory will be installed in

                   the current directory you are in.  Then, change

                   directory to the SRE_TOOLS directory and read the

                   file READ.ME which provides further information on







       Iss. 6D6                                                  Page 6.

















                                        SOFTWARE RELIABILITY ENGINEERING









                   installing the tools.  Complete installation

                   requires approximately 5 to 15 minutes depending on

                   the processing speed and processing load levels of

                   your system.



                   MS-DOS Version - The MS-DOS version is distributed

                   either on 5-1/4 inch or 3-1/2 inch floppies.  Each

                   floppy contains "executable" program files The MS-

                   DOS version is distributed on a 5-1/4 inch, 360

                   Kbyte double-sided, double-density floppy.  The

                   floppy contains "executable" program files est.exe

                   and plot.exe and test data files tst.fp, tst.ft,

                   tst.pc, tst_stg.fp, tst_stg.ft and tst_stg.pc.  The

                   floppy diskette can be inserted in the appropriate

                   drive, the drive selected by typing a:  or b:.  If

                   your PC has a hard disk, we recommend copying the

                   program and data files into a directory on your

                   hard disk and running the tools from your hard

                   disk. Otherwise, you can run the tools directly

                   from the floppy diskette (of course, making a

                   backup copy of the diskette first).  Alternatively,

                   with the UNIX version of the delivered tools, there

                   is a directory named dos that contains copies of

                   the "executable" program and data files.  These

                   files can be up-loaded to your PC using the file-

                   transfer capabilities of your favorite terminal

                   emulator package (such as ctrm).



                   ___________________________________________________

                   __













































       Page 7.                                                  Iss. 7D7

















       Reliability Estimation Tools

       SRE TOOLKIT









                   _____________________________________________________



        Getting



        Started,



       Using the



       Manual Pages

                   You might want to first review the manual pages in

                   the appendix.  You might want to first review the

                   manual pages in Appendix A.  Manual page EST(1)

                   describes the est program. First, quickly browse

                   the "Description" section of the manual page and

                   then read the "Example" section to follow the exe-

                   cution of a particular example.  After reading the

                   manual page, you may want to proceed to the next

                   section of this guide to execute the example in

                   your UNIX or MS-DOS environment. If you want to

                   learn more about the input data files for est, then

                   you'll want to read the .FP(5) manual page that

                   describes the contents of the "failure parameter"

                   file and .FT(5) manual page that describes the con-

                   tents of the "failure time" file.  Again, browse

                   the "Description" section of the manual page, then

                   read the "Example" section. Refer back to the

                   "Description" section read the "Example" section.

                   Refer to the "Description" section whenever you

                   need more detail in following the example.  In gen-

                   eral, you don't need to be familiar with the

                   PLOT(1) manual page. However, if you reach a point

                   where you want to tailor some of the plots, then

                   you can read this manual page to see how to change

                   the plot commands in the associated .pc files pro-

                   duced by est.



                   The same applies to getting started with the proto-

                   type tools. First, review the Programmer notes to

                   understand the caveats in running the tools. Then

                   browse the "Description" section of the manual page

                   and concentrate on the "Example" section of the

                   manual page.



                   _____________________________________________________



        Getting



        Started,



       Running the



        Examples

                   UNIX Version - After installing the UNIX version of

                   the tools, change directories to the directory your

                   tools were installed in and then to the testdir

                   9110exer subdirectory under this directory.  There

                   are two sets of project data provided in this

                   directory. The first is associated with project

                   tst.  This data is in files tst.fp and tst.ft.

                   First run the est program against this project data

                   by typing est tst.  Note the tabular output report

                   produced. Manual page EST(1) in the appendix can be

                   used to interpret the contents of this report.

                   Note the tabular output report produced. Manual

                   page EST(1) in Appendix A can be used to interpret

                   the contents of this report.  The program creates

                   the plot commands in file tst.pc.  Now, you can

                   type plot tst to generate the plots.  In the UNIX

                   version, a file tst.po containing pic(1) and

                   troff(1) commands is produced (see the UNIX User

                   Reference Manual for more information on pic and

                   troff).  You can now run your favorite command for

                   formatting "troff" text You can now run your favor-

                   ite command for formatting troff text files819 and

                   routing to output to postscript printers or other

                   printers with graphics capabilities with the tst.po

                   file.  Don't forget to either first run the UNIX

                   pic(1) command against the file or to include the

                   appropriate option on the command line of the troff

                   text formatter to preprocess the file using the

                   "pic(1)" command.  using the pic(1) command.  The





       __________



        1. Examples of such commands are mmx(1), mmt(1), xroff(1).

           An example invocation of such a command with the tst

           data would be mmt -p tst.po where the -p option on the

           command line indicates the file should first be

           processed by "pic(1)".  indicates the file should first

           be processed by "pic(1)."  indicates the file should

           first be processed by pic(1).  Check with your UNIX

           system administrator to find out what commands are

           available on your UNIX system.  Reference [GE] provides

           further information on document formating commands

           Reference [GE] provides further information on document

           formatting commands under UNIX.









       Page 9.                                                  Iss. 9D9

















       Reliability Estimation Tools

       SRE TOOLKIT









                   second set of project data with project name

                   tst_stg is the same as the first except "staged

                   delivery" information has been added to the failure

                   data (see the .FT(5) manual page in the appendix

                   for a description of staged delivery).  manual page

                   in Appendix A for a description of staged

                   delivery).  You may now want to run est and plot

                   programs with this project data and compare the

                   resulting tabular report and plots with the tst

                   project data.



                   MS-DOS Version - If you have created a directory on

                   your hard disk with the program and data files

                   included with the distribution diskette, then you

                   should first "change directories" to this direc-

                   tory.  As with the UNIX version, there are two sets

                   of project data provided. The first is associated

                   with project tst.  This data is in files tst.fp and

                   tst.ft.  First run the est program against this

                   project data by typing est tst.  Note the tabular

                   output report produced. Manual page EST(1) in the

                   appendix can be used to interpret the contents of

                   this report.  in Appendix A can be used to inter-

                   pret the contents of this report.  In the MS-DOS

                   version of the data files, the genplt parameter has

                   been set in the failure parameter file tst.fp file

                   so no plot commands are produced (this was done

                   because PC's not having a "math coprocessor" board

                   will require a long time to run).  not having a

                   "math co-processor" chip will require a long time

                   to run).  Instead, the "plot commands" correspond-

                   ing to project tst has already been created and

                   supplied as file tst.pc with the distribution

                   diskette.  You may now want to run plot tst to pro-

                   duce the plots directly on your video monitor.  The

                   second set of project data with project name

                   tst_stg is the same as the first except "staged

                   delivery" information has been added to the failure

                   data (see the .FT(5) manual page in the appendix

                   for a description of staged delivery).  manual page

                   in Appendix A for a description of staged

                   delivery).  You may now want to run est and plot

                   programs with this project data and compare the

                   resulting tabular report and plots with the tst

                   project data.  Again, genplt has been set so no

                   plot commands are generated.  Instead, tst_stg.pc

                   file has been provided with your distribution

                   diskette.







       Iss. 10D10                                               Page 10.

















                                        SOFTWARE RELIABILITY ENGINEERING









                   _____________________________________________________





                   When running under MS-DOS, you can get hard-copy of

                   screen output by using the MS-DOS mode mode and

                   graphics graphics commands and a locally connected

                   dot-matrix printer (see you MS-DOS User's Guide).

                   To do this you generally execute the mode mode com-

                   mand to define the characteristics of your printer

                   and then the graphics graphics command to load a

                   memory-resident program.  To print a copy of the

                   display currently appearing on your terminal's

                   screen, you would depress the "Prt Sc" or "Print

                   Screen" key (the name of this key is dependent on

                   exactly what type of keyboard you have).



                   The program plot(1) plot(1) that is provided with

                   this tool set is a useful tool in itself for pro-

                   ducing displays.  The novice user need not get into

                   learning about the plot commands that the plot(1)

                   plot(1) program uses in generating plots.  The ex-

                   pert user can begin writing analysis programs that

                   can generate .pc .pc files to create their own

                   graphs.  Or, better yet, you can change .pc .pc

                   files created by the est(1) est(1) program to add

                   additional lines, points, labels on particular

                   graphs and so on.  One approach to using the tools

                   is to do the heavy CPU est(1) est(1) runs on a UNIX

                   system with processing horse-power.  Then download







       Iss. 12D12                                               Page 12.

















                                        SOFTWARE RELIABILITY ENGINEERING









                   the resulting .pc .pc files to a PC, edit the .pc

                   .pc files to do "touch-ups" and customize graphs

                   and run plot(1) plot(1) on the PC to immediately

                   see the effects of changes to the .pc .pc files.

                   Finally, the .pc .pc files can then be up-loaded to

                   a UNIX system to runoff final reports with the

                   graphical output on a laser printer.



                   For those who may be familiar with the RELTOOLS

                   tool set and the reltab reltab and relplt relplt

                   programs, this tool set provides everything in the

                   way of features of these tools plus more. The

                   structure of the input files for est(1) est(1) pro-

                   gram is quite similar to the input files of reltab

                   reltab and relplt relplt programs. The one notable

                   exception is the failure time file. For the

                   reltab/relplt reltab/relplt programs, the

                   corresponding file is referred to as a failure

                   interval file. The times in the failure interval

                   file are "times between failures" rather than

                   actual "failure times."  The format of the failure

                   interval and failure time files are different.  The

                   failure parameter files for both the est(1) est(1)

                   and reltab/relplt reltab/relplt file are almost

                   identical (there are a few differences in some

                   parameter names).



                   _____________________________________________________



                                        SOFTWARE RELIABILITY ENGINEERING



        Caveats

                   Version 3.7 is a "Beta Trial" version of the tools

                   that is being made available on a "friendly user"

                   basis. As such, the basic functionality of the

                   tools has been extensively tested and results com-

                   pared with the predecessor software RELTAB/RELPLT.

                   With this version of the tool, the heuristic algo-

                   rithm (in EST) that determines scale values for the

                   X and Y axes has not been fully implemented (this

                   algorithm selects a scale so the X and Y scale

                   values printed contain only a few significant di-

                   gits). Also, some of the less frequently used PLOT

                   PARAMETER options for EST (such as charht, clopt,

                   conlvl, dshopt, grdopt, xwinb, xwine, ywinb, ywine)

                   have not been fully tested.



                   Users of this "Beta Trial" version of the tools are

                   asked to communicate any problems via email to

                   "whuxr!wwe" or (w.w.everett@att.com).



                   _____________________________________________________



       References



                   [ATT]   "UNIX System V Release 3.0 User Reference

                           Manual", to order, "UNIX System V Release

                           3.0 User Reference Manual," to order,



                   [ATTa]  UNIX System V Release 3.0 User Reference

                           Manual, to order, call AT&T Customer's In-

                           formation Center 1-800-432-6600 and order

                           Select Code 307-184.



                   [ATTb]  Reliability by Design, Chapter 8, to order,

                           call AT&T Customer's Information Center 1-

                           800-432-6600.



                   [MIO]   Musa, J. D., A. Iannino and  K. Okumoto,

                           Software Reliability - Measurement, Predic-

                           tion, Application Software Reliability -

                           Measurement, Prediction, Application,

                           McGraw-Hill, 1987, ISBN 0-07-044093-X.



                   [MA]    Musa, J and A. F. Ackerman, Quantifying

                           Software Validation: When to Stop Testing?,

                           IEEE SOFTWARE, May 1989, pg. 19-27.



                   [GE]    Gehani, N., Document Formatting on the UNIX

                           System, Silicon Press, 1986, ISBN 0-

                           9615336-0-9.



                   _____________________________________________________