Several software reliability  tools  are  currently  available  for

users to apply one or more of the known software reliability models

to a development effort. Popular tools include Statistical Modeling

and  Estimation  of  Reliability  Functions for Software  (SMERFS),  

and  Software  Reliability  Modeling Programs  (SRMP). In  addition

to  allowing users  to make reliability estimates, these tools also

allow  users  to  determine the applicability of a particular model

to a set of failure data.

A major issue in modeling software reliability,  however,  lies  in

the  ease-of-use  of currently available tools.  Nearly all current

tools have command-line interfaces, and do not  take  advantage  of

the  high-resolution  displays that would allow the construction of

menu-driven or direct-manipulation  user  interfaces.  Command-line

interfaces  can  make  it  more difficult for users to remember the

specific steps required to accomplish a task, while  a  menu-driven

interface can be set up to guide users through the necessary steps.

Finally, most currently available tools display  their  outputs  in

tabular  form.   Although some tools provide a graphical display of

results, in most cases this is  of  limited  utility  in  that  the

display  resolution  is low since character-based graphics are used

in painting the screen.  In addition, the variety of  results  that

may  be  graphically  displayed  is usually limited to interfailure

times or failure intensities.  In measuring  software  reliability,

it  is  useful to see high-resolution displays of these quantities,

as well as cumulative number  of  errors,  the  reliability  growth

curve,  and  the  results  of statistical methods used to determine

whether the model being executed is  appropriate  for  the  current


CASRE is implemented as a software reliability modeling  tool  that

addresses  the ease-of-use issue as well as other issues.  CASRE is

an extension of the public-domain tool SMERFS, and is  intended  to

execute  both  in  a  DOS  Windows environment and a UNIX X-windows

environment.  The command  interface  is  menu  driven;  users  are

guided through the selecting of a set of failure data and executing

a model by selectively enabling pull-down  menu  options.  Modeling

results  are  also  presented  in a graphical manner.  After one or

more models have been executed, the predicted  failure  intensities

or  interfailure  times  are  drawn  in a graphical display window.

Users can manipulate this window's controls to display the  results

in  a  variety of ways, including cumulative number of failures and

the reliability growth curve.  Users may also display  the  results

in a tabular fashion if they wish.

In addition, CASRE is  facilitated  with  a  useful  functionality.

Namely,  results  from  different models can be combined in various

ways to yield reliability estimates  whose  predictive  quality  is

better than the individual  models  themselves.  CASRE  is able  to

increase  prediction accuracy  by combining  the results of several

models in a linear fashion.  Moreover, CASRE allows users to define

their  own  combinations  and  record  them  as  part of the tool's

configuration.  Weights for the components of the  combination  may

be  static  or  dynamic, and may be based on statistical techniques

used to determine the applicability of a model to a set of  failure

data.   Once  combination  models  have  been  defined,  the  steps

required to execute them are no different than executing  a  simple


                           CASRE Context

There are six major functional areas of CASRE:

     1.  File Operations                ("File" menu)

     2.  Editing Operations             ("Edit" menu)

     3.  Transformation and Smoothing Operations("Filter" menu)

     4.  Model Selection and Application("Model" menu)

     5.  Program Configuration          ("Setup" or "Settings" menu)

     6.  Help System                    ("Help" menu)

One additional  functional  area  is  implemented  to  set  the

controls  associated  with  a  graphics display window in which

failure  data  and  the  results  of  modeling  activities  are

displayed.  Contents of the graphics display window can also be

shown in tabular form in another text window.

  1.  File Operations

There are six operations that can  be  performed  by  selecting

items from the File pull-down menu.  These are: "Open", "Save",

"Save As", "Setup Printer", "Print", and "Exit".   The  defini-

tion and usage of these operations follow the usual conventions

in modern user-interface designs.

The File menu appears underneath the File button  on  the  main

menu bar when the File button is selected and clicked.  As long

as the File button remains selected, users can navigate through

the File menu.  Menu item selection is accomplished by pointing

to an item in the menu and then releasing the mouse button.

  2.  Editing Operations

Users will have available to them four editing operations which

allow them to modify the currently displayed work space.  These


1.   Undo - allows users to undo the most recent CASRE operation.

2.   Convert Type - allows users to convert  failure  data  in  the

     form of interfailure times to failure counts and test interval

     lengths, and vice versa.

3.   External Editor - allows users to invoke a preferred text edi-

     tor,  word  processor, or other application from within CASRE.

     The application is selected from a user-configurable menu.

4.   Escape to DOS - allows users to temporarily escape to DOS  and

     execute DOS commands.  Users may then re-enter CASRE by typing

     "exit" at the DOS prompt.

  3.  Transformation and Smoothing Operations

Filtering operations allow users to perform global modifications to

the  failure  data displayed in the work space.  Since failure data

is frequently noisy, and may be noisy to the point that it is  dif-

ficult  or  impossible to estimate model parameters, users may wish

to apply a smoothing operation to the data to remove  some  of  the

noise.   Users  may  also wish to apply other transformations which

will allow them change the shape and position of the failure  data.

For  instance,  one  of  the transformations can change exponential

data to a straight line.

There are three types of filtering operations; these are  transfor-

mation,  smoothing,  and  data subsetting.  Multiple transformation

and smoothing filters may be applied to  the  data  in  a  pipeline

fashion.   There  is  also  a  "Undo All Filters" facility allowing

users to remove all of the filters that have been  applied  to  the

data.  The filtering capabilities are as follows:

     o  Affine Transformations

          -    Scaling and Offset - (X(i) * A) + B

          -    Power - X(i) ^ A

          -    Logarithmic - ln(A * X(i) + B)

          -    Exponential - exp(A * X(i) + B)

     o  Smoothing (to use "Hann Window")

     o  Subset Data (to select severity)

     o  Remove All Filters

  4.  Model Selection and Application

CASRE allows users to select and apply existing software  reliabil-

ity  models  to the data displayed in the work space.  These models

include Farr Survey

1.   Brooks and Motley Model (BM)

2.   Geometric Model (GM)

3.   Goel-Okumoto (GO)

4.   Jelinski-Moranda (JM)

5.   Littlewood Model (LM)

6.   Littlewood-Verrall (LV)

7.   Musa-Okumoto (MO)

8.   Generalized Poisson Model (PM)

9.   Schneidewind Model (SM)

10.  Yamada Delayed S-Shape Model (YM)

Users are also allowed to define combinations of  existing  models,

edit specifications of user-defined models, and remove user-defined

models from the menu of available models.  Users may  also  specify

the parameter estimation method, the confidence bounds that will be

reported for model parameters, and the  amount  of  time  into  the

future  for  which reliability predictions will be made.  The Model

menu items are:

1.   Model Selection - allows users to select and apply one or more

     software  reliability  models to the failure data displayed in

     the work space.

2.   Model Definition - allows users to define  combination  models

     to  supplement  models  already provided with the tool.  User-

     defined models remain available during the current and  subse-

     quent sessions.

3.   Model Editing/Model Removal - allows users to change or remove

     descriptions  of  combinations  that  were  previously created

     using the "Model Definition"  capability.   Only  user-defined

     combinations of models can be changed or removed.

4.   Parameter Estimation - allows users to select  the  method  of

     parameter  estimation that will be used.  The choices are max-

     imum likelihood (default) and least squares.

5.   Predictions - allows users to specify an interval of time over

     which  predictions  about  future reliability behavior will be


  5.  Program Configuration

Two operations, "Add External Configuration" and  "Remove  External

Configuration,"  are available from this manual.  The former allows

users to add the name of an application to  the  "External  Editor"

submenu from which external applications can be invoked.  The later

allows users to remove the name of an application from the  "Exter-

nal Editor" submenu from which applications can be invoked.

  6.  Help System

The help system provides context-sensitive  on-line  assistance  to

users  by  allowing them to search for and read descriptions of the

major CASRE functional areas.

  7.  Displays

The graphics display window is invoked when the "Open" button under

main  window's  "File"  menu  operation  is selected.  The graphics

display window plots the failure data displayed in the  work  space

and  the  results of applying models to that data.  A separate menu

bar is associated with this window, allowing users to  control  the

contents  and appearance of the display or send the contents of the

window to an output device (disk file or printer).  The menu  items

on the main menu bar are:

1.   Plot - There are 6 plot operations ("Save", "Save As", "Redraw

     Data",   "Redraw   Results",   "Draw   from   File",  "Printer

     Setup/Print") that can be selected for file  manipulation  and


2.   Model Result Selection - Multiple sets of model results can be

     displayed  in  the  graphics  display window.  This capability

     allows users to specify the models  whose  results  should  be

     plotted in that window.

3.   Display Type - Modeling results and evaluations of models  can

     be  displayed  in  a  variety of ways.  The following types of

     model results can be plotted in the graphics window:

     o    Interfailure times

     o    Number of failures per test interval

     o    Test interval lengths

     o    Cumulative number of failures

     o    Reliability growth function

  In addition to estimating reliability functions, users  may  wish

  to  determine  the applicability of the model(s) being run to the

  current set of  failure  data.   The  following  types  of  model

  evaluation results can be drawn in the graphics window:

  o    Goodness-of-fit tests - Chi-Square or Kolmogorov-Smirnov

  o    Current value(s) of the prequential likelihood

  o    Prediction noisiness

  o    u-plot

  o    y-plot

  o    Scatter-plot of u's

  o    Model rankings

4.   Scaling - allows users to scale the x and y axes of the graph-

     ics display window and to shift its origin.

5.   Table - allows users to  view  the  modeling  results  in  the

     graphics  display  window in a tabular form.  This table shows

     the detailed data on which plotted model results, evaluations,

     and rankings are based.

CASRE On-Screen Appearances Figures 1-8 in the following show a series of screen dumps for the described CASRE tool. It can be seen that the application of models to failure data is a straightforward process. Users are also given a considerable amount of choice in the models to be applied. This combination of simple operation and variety in the available models makes it easy for users to identify an appropriate model for a particular development effort or investigate a family of models. o Screen 1 - opening a failure data file The screen is shown in Figure 2 in the Appendix. To choose a set of failure data on which a reliability model will be run, users select the "File" menu with the mouse. After selecting the "Open" option in the File menu, a dialogue box for selecting a file appears on the screen. The current directory appears in the edit- able text window at the top of the dialogue box. The failure his- tory files in that directory are listed in the scrolling text win- dow. Users select a file by highlighting its name (scrolling the file name window if necessary) and then pressing the "Open" button. To change the current directory, users enter the name of the new directory in the "Current Directory" window and presses the "Change Directory" button. Pressing the "Cancel" button removes the dialo- gue box from the screen. o Screen 2 - initial failure data display The screen is shown in Figure 3 in the Appendix. After opening a failure history file from the "File" menu, the contents of the file are displayed in tabular and graphic forms. The tabular represen- tation resembles a spreadsheet, and users can perform similar types of operations (e.g. selecting a range of data, deleting one or more rows of data). All of the fields can be changed by users except for the "Interval Number" field (or "Error Number" field if the data is interfailure times). In this example, the selected data set is in the form of test interval lengths and number of failures per test interval. Users can scroll up and down through this tabu- lar representation and resize it as per the MOTIF conventions. The large graphics window displays the same data as the worksheet. If the failure data set is interfailure times, the initial graphi- cal display is interfailure times. If, as in this example, the failure data set is test interval lengths and failure counts, the initial graphical display is the number of failures per test inter- val. The display type can be changed by selecting one of the items from the "Display Type" menu associated with the graphics window. Users can move forward and backward through the data set by press- ing the right arrow or left arrow buttons at the bottom of the graphics window. Finally, the iconified window at the lower left corner of the screen lists the summary statistics for the data. To open this window, users just clicks on the icon. o Screen 3 - selecting failure data range The screen is shown in Figure 4 in the Appendix. Users will fre- quently use only a portion of the data set to estimate the current reliability of the software. This is because testing methods may change during the testing effort, or different portions of the data set may represent failures in different portions of the software. To use only a subset of the selected data set, users may simply "click and drag" on the tabular representation of the data set to highlight a specific range of observations. Users may also select previously-defined data ranges. To do this, users choose the "Select Range" option of the Edit menu. This brings up a dialogue box containing a scrolling text window in which the names of previously-defined data ranges and the points they represent are listed. To select a particular range, users highlight the name of the range in the scrolling text window and presses the "OK" button. Pressing the "Cancel" button removes the dialogue box and the Edit menu from the screen. Once a range has been selected, all future modeling operations will be only for that range. The selected data range is highlighted in the tabular representation. The graphics display will change to include only the highlighted data range. All other observations will be removed from the graphics display. o Screen 4 - data filtering The screen is shown in Figure 5 in the Appendix. After selecting a data range, users may wish to transform the file or smooth the data. Software failure data is frequently very noisy; smoothing the data or otherwise transforming it may improve the modeling results. To do this, users select one of the options in the "Filter" menu. There are five affine transformations which users may apply to the data, and six types of smoothing. Transformations and smoothing operations may be pipelined - for example, users could select the "ln(A * X(i) + B)" transformation followed by the B-spline smoothing operation. The number of filters that may be pipelined is limited only by the amount of available memory. The tabular representation of the failure is changed to reflect the filter, as is the graphical display of the data. The type of filter applied to the data is listed at the right hand edge of the graphics display window. Once a series of filters has been applied to the data, users may remove the effect of the most recent filter by selecting the "Undo" option of the Filter menu. To remove the effect of the entire series of filters, users select the "Undo All Filters" option of the Filter menu. o Screen 5 - applying software reliability models The screen is shown in Figure 6 in the Appendix. After users have opened a file, selected a data range, and done any smoothing or other transformation of the data, a software reliability model can be run on the data. In the Model menu, users have the choice of 10 individual models or a set of models which combine the results of two or more of the individual models. Users may also choose the method of parameter estimation (maximum likelihood or least squares), the confidence bounds that will be calculated for the selected model, and the interval of time over which predictions of future failure behavior will be made. o Screen 6 - prioritization of model selection criteria The screen is shown in Figure 7 in the Appendix. There are many models from which to choose in this tool. Users may not know which model is most appropriate for the data set being analyzed. Using CASRE, users can request, "display the results of the individual model which best meets the four prioritized criteria of accuracy (based on prequential likelihood), biasedness, trend, and noisiness of prediction." To do this, a user first selects the "Individual" option of the Model menu. A submenu then appears, on which 10 individual models are listed, as well as a "Choose Best" option. The user selects the "Choose Best" option, which results in a "Selection Criteria" dialogue box being displayed. The user moves the four sliders in this dialogue box back and forth to establish the relative priorities of the four criteria. Numerical values of the priorities are displayed in the text boxes on the right side of the dialogue box. Once the priorities have been established, the user presses the "OK" button. CASRE then proceeds to run all of the individual models against the data set, first warning the user that this is a time-consuming operation and allowing cancellation of the operation. If the user continues, CASRE provides the opportunity for cancellation at any time if the user decides that the operation is taking too much time. o Screen 7 - display of model results The screen is shown in Figure 8 in the Appendix. Once a model has been run on the failure data, the results are graphically displayed. Actual and predicted data points are shown, as are con- fidence bounds. The model is identified in the window's title bar; the percent confidence bounds are given at the right side of the graphics window. This concludes one round of software reliability estimation with CASRE. o Screen 8 - determination of model bias The screen in which one of the result of model evaluation can be displayed is shown in Figure 9. Statistical methods can be applied to determine the applicability of the model to the failure data set on which it was executed. To display the evaluation results, a user selects the "Evaluations" pull-down menu in the graphics display window's main menu bar. Several model evaluation methods, including u-plots and y-plots, are available to users. In this example, the user have chosen the "U-Plot" menu item. The u-plot, which indicates biases in the model, is displayed on screen. CASRE also indicates whether the model has an optimistic bias (predic- tions of time to the next failure tend to be greater than observed inter-failure times) or a pessimistic bias. To return to the display shown in the figure, the user may select the "Display Type" pull-down menu and choose the desired type of reliability-related display.