9817_Empirical evaluation of change impact predictions using a requirements management tool with form

master’s thesis
Empirical evaluation of change impact predictions using a
requirements management tool with formal relation types
A quasi-experiment
R.S.A. van Domburg
Enschede, 26 November 2009
Software Engineering Group
Faculty of Electrical Engineering, Mathematics and Computer Science
University of Twente
Final project (239997)
Business Information Technology
School of Management and Governance
University of Twente
graduation committee
dr. ir. K.G. van den Berg
EWI
dr. A.B.J.M. Wijnhoven
MB
dr. I. Kurtev
EWI
A. Goknil, MSc
EWI
Acknowledgements
First, I would like to thank my graduation committee for their invalu-
able advice and input. I have regarded our regular meetings as very enjoy-
able and beneficial to the quality of my work.
Second, I would like to thank all participants for their participation in
the experiment and Johan Koolwaaij and Martin Wibbels for their expert
insight into the WASP requirements specification. Y
our input has been very
important to attain any research results in the first place.
Third, I would like to thank Ton Augustin, Pieter van Rossum, Klaas
Sikkel and Theo Thijssen for their expert insight into requirements trace-
ability. Y
our comments have enabled me to reflect upon this research from a
practical perspective.
Last but not least, I would like to thank Kim Scholte van Mast for her
review of my final draft. Y
our comments have improved the correctness and
readability of this thesis.
iii
Abstract
Background: This research was part of a master’s thesis to evaluate the
impact of using TRIC, a software tool with formal requirements relation-
ship types, on the quality of change impact prediction in software.
Objective: To analyze the real-world impact of using a software tool with
formal requirements relationship types; for the purpose of the evaluation of
effectiveness of tools; with respect to the quality of change impact predic-
tions; in the context of software requirements management; from the view-
point of system maintenance engineers.
Method: This research features a quasi-experiment with 21 master’s de-
gree students predicting change impact for five change scenarios on a real-
world software requirements specification. The quality of change impact
predictions was measured by the F-measure and the time in seconds to
complete the prediction.
Two formal hypotheses were developed. Null hypothesis 1 stated that the
F-scores of change impact predictions of system maintenance engineers us-
ing TRIC will be equal to or less than those from system maintenance engi-
neers not using TRIC. Null hypothesis 2 stated that the time taken to com-
plete change impact predictions of system maintenance engineers using
TRIC will be equal or longer than those from system maintenance engi-
neers not using TRIC. The data were collected by a web application and
analyzed using ANOVA and !2 statistical analyses.
Results: No significant difference in F-scores between TRIC and the
other groups was detected. TRIC was found to be significantly slower for
four out of five change impact predictions. These inferences were made at
“=0,05 with a mean statistical power of 54%.
Limitations: The validity was hampered by a limited availability of usable
software requirements specifications, experts from industry and theory re-
garding the impact of change scenarios on change impact prediction. The
results cannot be generalized for other software requirements specifica-
tions, change scenarios or groups of participants. The condition to provide a
solution validation was therefore not met.
Conclusion: Empirical experiments cannot provide a solution validation to
new software tools because there are not enough experts in the new tool.
Using TRIC to perform change impact prediction on a software require-
ments specification of low complexity does not yield better quality predic-
tions but does take a longer time.
v
Table of Contents
1.
Introduction!
13
1.1.
……………………………………………………………………………………………
The QuadREAD Project!
13
1.2.
…………………………………………………………………………………………..
Requirements metamodel!
15
1.3.
…………………………………………………………………………………………………….
Problem statement!
16
1.4.
…………………………………………………………………………………………………….
Research objective!
16
1.5.
………………………………………………………………………………………………………
Research method!
18
1.6.
…………………………………………………………………………………………………………..
Contributions!
19
1.7.
…………………………………………………………………………………………………..
Document structure!
19
2.
Background and related work!
21
2.1.
……………………………………………………………………………………………………………..
Introduction!
21
2.2.
………………………………………………………………………………………………
Software requirements!
22
2.3.
…………………………………………………………………………..
Software requirements specifications!
24
2.4.
……………………………………………………………………………
Software requirements management!
25
2.5.
…………………………………………………………………………………..
System maintenance engineers!
26
2.6.
……………………………………………………………………………………………………..
Change scenarios!
26
2.7.
………………………………………………………………………………………..
Change impact predictions !
27
2.8.
…………………………………………………………………………….
Requirements models and relations!
32
2.9.
…………………………………………………………………………………………………………..
Software tools !
34
2.10.
……………………………………………………………………………………………….
Validation approaches!
44
2.11.
……………………………………………………………………………………………………………….
Conclusion!
49
3.
Experimental design!
51
3.1.
……………………………………………………………………………………………………………..
Introduction!
51
3.2.
………………………………………………………………………………………………………………………..
Goal!
51
3.3.
……………………………………………………………………………………………………………….
Hypothesis!
51
3.4.
……………………………………………………………………………………………………………………..
Design!
52
3.5.
………………………………………………………………………………………………………………
Parameters!
53
3.6.
………………………………………………………………………………………………………………….
Variables!
54
3.7.
…………………………………………………………………………………………………………………..
Planning!
56
3.8.
………………………………………………………………………………………………………………
Participants!
61
3.9.
……………………………………………………………………………………………………………………
Objects!
62
3.10.
………………………………………………………………………………………………………..
Instrumentation!
64
3.11.
………………………………………………………………………………………………………….
Data collection!
71
3.12.
……………………………………………………………………………………………………
Analysis procedure!
71
3.13.
…………………………………………………………………………………………………….
Validity evaluation!
72
3.14.
……………………………………………………………………………………………………………….
Conclusion!
74
vii
4.
Execution!
75
4.1.
……………………………………………………………………………………………………………..
Introduction!
75
4.2.
…………………………………………………………………………………………………………………….
Sample!
75
4.3.
………………………………………………………………………………………………………………
Preparation!
75
4.4.
…………………………………………………………………………………………
Data collection performed!
78
4.5.
…………………………………………………………………………………………………….
Validity procedure!
78
4.6.
……………………………………………………………………………………………………………….
Conclusion!
79
5.
Analysis!
81
5.1.
……………………………………………………………………………………………………………..
Introduction!
81
5.2.
…………………………………………………………………………….
Change scenario representativeness!
81
5.3.
…………………………………………………………………………………………
Golden standard reliability!
82
5.4.
………………………………………………………………………………
Precision-Recall and ROC graphs!
86
5.5.
……………………………………………………………………………..
One-way between-groups ANOVA!
86
5.6.
………………………………………………………………………………………………
Non-parametric testing!
91
5.7.
……………………………………………………………………………………………….
Analysis of covariance!
94
5.8.
…………………………………………………………………………………
Multivariate analysis of variance!
95
5.9.
……………………………………………………………………………………………………………….
Conclusion!
96
6.
Interpretation!
97
6.1.
……………………………………………………………………………………………………………..
Introduction!
97
6.2.
…………………………………………………………………………….
Change scenario representativeness!
97
6.3.
…………………………………………………………………………………………
Golden standard reliability!
97
6.4.
………………………………………………………………………………
Precision-Recall and ROC graphs!
99
6.5.
……………………………………………………………………………..
One-way between-groups ANOVA!
99
6.6.
………………………………………………………………………………………………
Non-parametric testing!
99
6.7.
……………………………………………………………………………………………..
Analysis of covariance!
100
6.8.
………………………………………………………………………………
Multivariate analysis of variance!
100
6.9.
……………………………………………………………………………………………………………..
Conclusion!
101
7.
Conclusions and future work!
103
7.1.
……………………………………………………………………………………………………………….
Summary!
103
7.2.
…………………………………………………………………………………………………………………..
Results !
104
7.3.
…………………………………………………………………………………………………………….
Limitations !
104
7.4.
……………………………………………………………………………………………………………
Future work!
106
8.
Glossary!
109
9.
References!
113
A.
Interviews!
119
A.1.
…………………………………………………………………………………………………………..
Introduction!
119
A.2.
………………………………………………………………………………………………………………………
Goal!
119
viii
A.3.
…………………………………………………………………………………………………………….
Preparation!
119
A.4.
………………………………………………………………………………………………………………
Execution!
120
A.5.
………………………………………………………………………………….
Information systems academic!
120
A.6.
………………………………………………………………………………….
Industry experts at Capgemini!
122
A.7.
……………………………………………………………………………………………………………
Conclusions!
125
B.
Tasks!
127
B.1.
…………………………………………………………………………………………………………..
Introduction!
127
B.2.
…………………………………………………………………………………
Warming up (REQ_BDS_007)!
127
B.3.
…………………………………………………………………………………………
Task 1 (REQ_PHN_001)!
127
B.4.
…………………………………………………………………………………………
Task 2 (REQ_SPM_004)!
127
B.5.
…………………………………………………………………………………………
Task 3 (REQ_MAP_002)!
127
B.6.
…………………………………………………………………………………………
Task 4 (REQ_NAV_003)!
128
B.7.
…………………………………………………………………………………………
Task 5 (REQ_TOR_001)!
128
C.
Group matching!
129
C.1.
…………………………………………………………………………………………………………..
Introduction!
129
C.2.
…………………………………………………………………………………………………………………..
Coding!
129
C.3.
……………………………………………………………………………………..
Pre-experiment randomized!
130
C.4.
………………………………………………………………………………………………
Pre-experiment tuned!
131
C.5.
……………………………………………………………………………………………………..
Experiment final!
132
D.
Golden standards!
133
D.1.
…………………………………………………………………………………………………………..
Introduction!
133
D.2.
…………………………………………………………………………………………
Task 1 (REQ_PHN_001)!
133
D.3.
…………………………………………………………………………………………
Task 2 (REQ_SPM_004)!
135
D.4.
…………………………………………………………………………………………
Task 3 (REQ_MAP_002)!
137
D.5.
…………………………………………………………………………………………
Task 4 (REQ_NAV_003)!
139
D.6.
…………………………………………………………………………………………
Task 5 (REQ_TOR_001)!
142
E.
Box plots!
145
E.1.
…………………………………………………………………………………………………………..
Introduction!
145
E.2.
…………………………………………………………………………………………
Task 1 (REQ_PHN_001)!
146
E.3.
…………………………………………………………………………………………
Task 2 (REQ_SPM_004)!
147
E.4.
…………………………………………………………………………………………
Task 3 (REQ_MAP_002)!
148
E.5.
…………………………………………………………………………………………
Task 4 (REQ_NAV_003)!
149
E.6.
…………………………………………………………………………………………
Task 5 (REQ_TOR_001)!
150
F.
Precision-Recall and ROC graphs!
151
F.1.
…………………………………………………………………………………………………………..
Introduction!
151
F.2.
…………………………………………………………………………………………………………………..
Legend!
151
F.3.
……………………………………………………………………………………………………………………
Task 1!
152
F.4.
……………………………………………………………………………………………………………………
Task 2!
153
ix
F.5.
……………………………………………………………………………………………………………………
Task 3!
154
F.6.
……………………………………………………………………………………………………………………
Task 4!
155
F.7.
……………………………………………………………………………………………………………………
Task 5!
156
G.
WASP requirements!
157
G.1.
…………………………………………………………………………………………………………..
Introduction!
157
x
List of abbreviations
AIS
Actual Impact Set
ANCOVA
Analysis of Covariance
ANOVA
Analysis of V
ariance
EIS
Estimated Impact Set
EWI
Faculty of Electrical Engineering, Mathematics and Computer Science
DIS
Discovered Impact Set
FPIS
False Positive Impact Set
GUI
Graphical User Interface
IEC
International Electrotechnical Commission
IEEE
Institute of Electrical and Electronics Engineers
ISO
International Organization for Standardization
MANOVA
Multivariate Analysis of V
ariance
MB
School of Management and Governance
MoSCoW
Must have, Should have, Could have, W
on’t have
NR
Non-Randomized
O
Experimental observation
OMG
Object Management Group
QuadREAD
Quality-Driven Requirements Engineering and Architecture Design
ROC
Receiver Operating Characteristic
Std
Standard
SysML
Systems Modeling Language
TBD
To Be Determined
TRIC
Tool for Requirements Inference and Consistency Checking
UML
Unified Modeling Language
URL
Uniform Resource Locator
WASP
W
eb Architectures for Services Platforms
X
Experimental treatment
xi
1. Introduction
This master’s thesis reports on the evaluation of the impact of using a software tool with for-
mal requirements relationship types on the quality of change impact predictions in software.
The tool and formal requirements relationship types were developed as part of a requirements
metamodel in a research project called QuadREAD, which will be introduced before describ-
ing the problem statement, research objective, context and further document structure.
1.1.
The QuadREAD Project
This research is conducted at the laboratory of the Software Engineering Group from March
2009 up to and including November 2009. It takes place within the context of the Quad-
READ Project, which is a joint research project of the Software Engineering and Information
Systems research groups at the Department of Computer Science in the Faculty of Electrical
Engineering, Mathematics and Computer Science at the University of Twente. The Quad-
READ Project runs from December 2006 up to and including December 2010.
The context of the QuadREAD Project is the early phases in software development processes:
the establishment of user requirements based on analysis of business goals and the application
domain and the subsequent architecture design of desired systems. The first phase concerns
requirements engineering; the second, architectural design. In practice, it appears that these
two phases are poorly integrated [50].
The project aims at a better alignment between analysts and architects. The project elaborates
on traceability research and focuses on tracing between user requirements and architectural
design decisions [50]. T
raceability is defined as the degree to which a relationship can be estab-
lished between two or more products of the development process, especially products having a
predecessor-successor or master-subordinate relationship to one another [58].
One depiction of traceability in software development is constructed by combining two spe-
cializations of traceability in the context of requirements engineering [61]. First, a distinction
can be made between pre-requirements specification traceability (forward to requirements
and backwards from requirements) and post-requirements specification traceability (forward
from requirements and backwards to requirements) [26]. Second, inter-level and intra-level
trace dependencies may be distinguished [3]. See Figure 1.
13
Business Model
Business Model
evolves to
Requirements
Requirements
evolves to
traces to
Architectural
Design
Architectural
Design
evolves to
traces to
Detailed Design
Detailed Design
evolves to
traces to
Figure 1: Traceability in sofware development [61] Figure 1 shows several types of traceability. For example, requirements elements are traced
backwards to elements in business models and forward to elements in the architectural design.
Requirements elements may have intra-level dependency relations and may evolve to a new
configuration of requirements elements. There are traceability links between artifacts and
links representing the evolution or incremental development of these artifacts [61].
In a goal-oriented approach, the QuadREAD Project is developing a framework in which the
alignment of requirements engineering and architectural design is supported with practical
guidelines and tools. The specific contribution of the project lies in the quantification of qual-
ity attributes and tradeoffs in relation to trace information [50].
The project will provide a framework for qualitative and quantitative reasoning about re-
quirements and architectural decisions to ensure selected quality properties. Thereby it will
enable decision-making in the quality-driven design of software architectures meeting user
requirements and system properties [50].
The research conducted in the QuadREAD Project is intended to have practical applicability
by the central role of case studies from participating business partners in the project, includ-
14
ing Atos Consulting, Chess Information Technology, Getronics PinkRoccade, Logica CMG,
Shell Information Technology and Kwards Consultancy [50].
This research is part of the final project of a master’s student of Business Information Tech-
nology, which is a master’s degree program that is headed by the School of Management and
Governance of the University of Twente.
The final project is worth 30 European Credits. It is supervised by two assistant professors,
one from the School of Management and Governance and one from the Faculty of Electrical
Engineering, Mathematics and Computer Science, as well as a postdoctoral scholar and Doc-
tor of Philosophy student from the latter faculty.
Biweekly meetings are held to evaluate the research progress, quality and results. Feedback
was also provided by research fellows from the Information Systems Group and business part-
ners participating in the QuadREAD Project, as well as other master’s students executing
their final project at the Software Engineering Group.
1.2.
Requirements metamodel
Research in the QuadREAD Project has contributed a requirements metamodel with formal
requirements relationship types to enable reasoning about requirements [25]. Henceforth, this
metamodel will be referred to as the requirements metamodel. It was constructed based on a re-
view of literature on requirements models. The project also contributed a prototype software
tool named TRIC that supports the requirements metamodel. TRIC was illustrated using a
single fictional case study featuring a course management system [37].
Based on the case study results, it was concluded that TRIC supports a better understanding
of mutual dependencies between requirements, but that this result could not be generalized
pending a number of industrial and academic case studies with empirical results [25].
This research on the requirements metamodel can be classified as technique-driven with a lack
of solution validation [69]. This classification does not imply that the research quality is poor:
papers presenting new technology do not necessarily need to validate the proposed solution,
though they should explain why the solution, if validated, would be useful to stakeholders.
V
alidation that a proposed solution actually satisfies the criteria from an analysis of stake-
holder goals is a research problem and does not need to be done in a technology paper [70].
15
1.3.
Problem statement
The problem that this research deals with is the lack of solution validation of the require-
ments metamodel, which can inhibit its adoption because the benefits are not clear. It should
be clear to practitioners for which problems a technique has shown to be successful in the real
world [69].
1.4.
Research objective
The research objective should formulate a means to providing a solution to the research prob-
lem. As a starting point, this paragraph compiles a set of solution requirements. A research
objective is subsequently formulated.
Solution requirements
The research objective should work towards satisfying two solution requirements:
1.
It should evaluate the requirements metamodel as a real-world solution [69] on criteria
that were defined in its original research [70].
2.
It should be aligned with the goals of the QuadREAD Project, because that is the context
in which this research takes place.
The following paragraphs construct a research objective in an iterative fashion by examining
these solution requirements more closely.
Evaluation criteria in original research
The original research has defined two evaluation criteria for the requirements metamodel:
1.
The number of inconsistent relations in requirements documents
2.
The number of inferred new relations in requirements documents
Henceforth, sofware requirements specification is used as a replacement term for requirements
documents in the context of software engineering. The term “software requirements specifica-
tion” is defined in the IEEE Standard Computer Dictionary [58]. It will prove to be useful dur-
ing upcoming discussions on quality of software requirements specifications, for which the
IEEE has well-known recommended practices [59].
Requirements modeling of the case was performed in two iterations using the TRIC software
tool, which has support for the formal relationship types from the requirements metamodel.
16
The first iteration revealed a number of inconsistencies in the software requirements specifi-
cation. This enabled the researchers to correct these issues. The second iteration reported
zero detected inconsistencies [25]. In this case, using formal requirements relationship types
led to a higher degree of consistency of the software requirements specification.
In addition to improved consistency, both iterations also reported a greater number of rela-
tions than was given initially. The additional relations were inferred by using formal require-
ments relationship types and led to greater knowledge about the specific requirements in the
software requirements specification in the context of requirements modeling [25].
However, the validity of this conclusion may be questioned. Because no tools other than
TRIC were used, it could also be concluded that requirements modeling became more effec-
tive because any software tool was used. There is no evidence that specifically the formal re-
quirements metamodel that TRIC supports increased the effectiveness of requirements mod-
eling.
Finally, engineers should study real-world problems and try to design and study solutions for
them [69]. Likewise, this research should analyze the real-world impact of the formal require-
ments metamodel by using real-world software requirements specifications and using a real-
world impact measure.
Consequently, this research should address this threat to validity by analyzing the real-world
impact of the formal requirements metamodel by analyzing TRIC alongside other require-
ments modeling tools, which support other and less formal requirements metamodels.
Alignment with QuadREAD Project goals
The requirements metamodel contributes to the QuadREAD Project by providing better
techniques for change impact analysis, which is necessary for cost-effective software develop-
ment [6]. It intends to do so by improving the precision of software requirements specifica-
tions. Current techniques are imprecise [25] which can reduce the quality of software require-
ments specifications in terms of ambiguity, modifiability and traceability [59].
Of all users of a software requirements specification, system maintenance engineers are the
most concerned with change impact analysis. System maintenance engineers use the require-
ments to understand the system and the relationships between its parts during requirements
management [55].
17
Indeed, impact is usually associated with maintenance effort [61]. By identifying potential im-
pacts before making a change, system maintenance engineers can greatly reduce the risks of
embarking on a costly change because the cost of unexpected problems generally increases
with the lateness of their discovery [12]. Having high-quality change impact predictions is thus
beneficial to system requirements management.
Goal-Question-Metric approach
Subsequent to the considerations above, a research objective can be formulated according to
the goal template of the Goal-Question-Metric approach [73]. The research objective can be
formulated as follows;
To improve the adoption of the requirements metamodel and advance the state of the art in
change impact analysis, the research should:
Analyze the real-world impact of using a software tool with formal requirements relation-
ship types; for the purpose of the evaluation of effectiveness of tools; with respect to the
quality of change impact predictions; in the context of software requirements manage-
ment; from the viewpoint of system maintenance engineers.
Operationalizations for this goal are provided in Chapter 2.
1.5.
Research method
The research method will involve performing change impact analysis on selected change sce-
narios on software requirements specifications in two ways: using classic software tools and
using the prototype TRIC software tool that supports formal requirements relationship types.
Such a research setup involves control over behavioral events during change impact analysis,
for which experimental research is the most appropriate [72].
Experimental research has several subtypes, one of them being quasi-experimental research.
By definition, quasi-experiments lack random assignment. Assignment to conditions is by
means of self-selection or administrator selection [52] such as is the case in our setup with se-
lected change scenarios and a predetermined set of software tools. Consequently, quasi-
experimentation is the most appropriate research method.
The quasi-experimental research design is described in Chapter 3.
18
1.6.
Contributions
Through a systematic design and execution of a quasi-experiment to empirically validate the
impact of the TRIC software tool on change impact predictions, this research reveals the fol-
lowing:
• Empirical experiments cannot provide a solution validation to new software tools because
there are not enough experts in the new tool. This is a phenomenon that will apply to any
research regarding new software tools.
• Approximating the experts by training a group of non-experts is difficult to do reliably and
hampers internal validity to such a point that an empirical approach to solution validation is
infeasible.
• Using TRIC to perform change impact prediction on a software requirements specification
of low complexity does not yield better quality predictions but does take a longer time than
compared to using Microsoft Excel or IBM Rational RequisitePro.
• It is hypothesized that TRIC is a more intelligent software tool and its benefits will only
materialize given a sufficiently complex software requirements specification.
• There is a lack of theory surrounding the nature of change scenarios which poses a reliability
issue to any research that deals with them.
1.7.
Document structure
This document aims to present the research in a rigorous structure. Such a structure makes it
easier to locate relevant information and lowers the risk of missing information [30]. The re-
search is presented as follows:
• Chapter 2: Background and related work clarifies how this research relates to existing
work, including a description of software requirements specifications, specific requirements,
their quality criteria, the requirements metamodel and alternative solutions.
• Chapter 3: Experimental design describes the outcome of the experiment planning
phase, including goals, hypotheses, parameters, variables, design, participants, objects, in-
strumentation, data collection procedure, analysis procedure and evaluation of the validity.
• Chapter 4: Execution describes each step in the production of the experiment, including
the sample, preparation, data collection performed and validity procedure.
19
• Chapter 5: Analysis summarizes the data collected and the treatment of the data, includ-
ing descriptive statistics, data set reductions and hypothesis testing.
• Chapter 6: Interpretation interprets the findings from the analysis including an evalua-
tion of results and implications, limitations of the study, inferences and lessons learned.
• Chapter 7: Conclusions and future work presents a summary of the study, including
impact, limitations and future work.
A glossary and list of references is presented afterwards.
20
2. Background and related work
2.1.
Introduction
This chapter describes the related work that is relevant for this research. The related areas
follow from the research objective, which is repeated here:
Analyze the real-world impact of using a software tool with formal requirements relation-
ship types for the purpose of the evaluation of the effectiveness of tools with respect to
the quality of change impact predictions in the context of software requirements man-
agement from the viewpoint of system maintenance engineers.
A conceptual framework for background and relevant work can be developed by relating the
keywords in this research objective. The nature of the relationships is discussed in the follow-
ing paragraphs. See Figure 2.
Requirements
models
Requirements
relations
Change impact
predictions
Experiment
Software tools
Software
requirements
specifications
Change
scenarios
contain
input for
empirically validated in
supported by
represented in
input for
changes expressed in
Software
requirements
management
System
maintenance
engineers
part of
performed by
supported by
Software
requirements
captured in
Figure 2: Conceptual famework for background and relevant work
The topics in Figure 2 are discussed in the following order. First, core topics to introduce the
domain are discussed. These are software requirements, software requirements specifications,
software requirements management and system maintenance engineers.
Discussed next are topics that provide specific instrumentation to this research. These are
change scenarios, change impact predictions, requirements models and relationships and
21
software tools. Finally, the topic of experiments is raised with a discussion of the investigated
approach, alternative validation methods and related experiments.
2.2.
Software requirements
The term requirement is not used in a consistent way in the software industry [55]. This re-
search uses the definition provided by the IEEE Standard Computer Dictionary [58]:
1.
A condition or capability needed by a user to solve a problem or achieve an objective;
2.
A condition or capability by a system or system component to satisfy a contract, standard,
specification, or other formally imposed documents;
3.
A documented representation of a condition or capability as in 1 or 2.
Requirements are part of a software requirements specification [59]. Knowledge about the
characteristics of requirements is thus necessary to understand software requirements specifi-
cations as a greater whole.
Requirements can differ in structure, contents and style. The following paragraphs describe
related work on these characterizations.
Requirements structure
Requirements are often written in natural language but may be written in a particular re-
quirements specification language. When expressed in specification language, they may addi-
tionally retain their description in natural language. Representation tools can describe the ex-
ternal behavior of a requirement in terms of some abstract notion [59]. Note that TRIC does
not describe external behavior of requirements but the relationships between requirements,
and thus is not a representation tool.
Requirements may be uniquely identified if they have a unique name or reference number,
which facilitates forward traceability. They may facilitate backwards traceability if they explic-
itly reference their source in earlier documents [59].
Some requirements descriptions use the phrase “to be determined” or “TBD”. In that case,
the description can state the conditions causing this status, what must be done to eliminate it,
who is responsible for the elimination and when it should be eliminated [59].
Requirements can be ranked for importance or stability. Stability can be expressed in terms of
the number of expected changes to any requirement based on experience of forthcoming
22
events [59]. Importance can refer to the level of necessity or priority [39]. One widely used
technique for ranking importance or necessity is called MoSCoW
, which defines “Must Have”,
“Should Have”, “Could Have” and “W
on’t Have requirement” rankings [7]. Any other scale
may be developed [39], one example being the “Essential”, “Conditional” and “Optional” scale
that is presented in IEEE Std 830-1998 [59]. Priorities are usually used as weighting factor and
can likewise be measured on any scale [39].
A highly common way to express requirements is using the feature requirement style [39]. Ex-
ample requirements expressed using this style are the following:
R1: The product shall be able to record that a room is occupied for repair in a specified
period.
R2: The product shall be able to show and print a suggestion for staffing during the next
two weeks based on historical room occupation. The supplier shall specify the calculation
details.
R3: The product shall be able to run in a mode where rooms are not booked by room
number, but only by room type. Actual room allocation is not done until check-in.
R4: The product shall be able to print out a sheet in which room allocation for each room
booked under one stay.
Note that the requirements are described in natural language and have a unique identifier, and
are not ranked or expressed in a specification language. Other styles for expressing require-
ments are discussed later.
Requirements contents
Requirements can be classified depending on the kind of condition or capability that they de-
scribe. The classification is not standardized, but it is generally agreed that functional re-
quirements specify a function that a system or system component must be able to perform
[59] and that non-functional requirements specify how well the system should perform its in-
tended functions [39].
Additional classes of requirements can be found in the literature. For example, Lauesen [39] also discusses the following:
• Data requirements: data that the system should input, output and store internally.
• Other deliverables: required deliverables besides hardware and software, such as docu-
mentation and specified services.
23
• Managerial requirements: when deliverables will be delivered, the price and when to pay
it, how to check that everything is working, what happens if things go wrong, etc. IEEE Std
830-1998 [59] also recognizes these, but maintains that these should not be provided as spe-
cific requirements but rather as a separate part in software requirements specifications.
Sommerville [55] also discerns domain requirements that come from the application domain
of the system and reflect characteristics and constraints of that domain. These requirements
may be either functional or non-functional and thus are not truly a separate class of require-
ments with respect to their contents. For this reason, this research disregards domain re-
quirements as a separate classification.
Requirements styles
Requirements may be expressed in a variety of styles depending on the classification of a re-
quirement. Lauesen [39] describes over 25 styles, including the previously illustrated feature
list style. Each style has its own advantages and disadvantages. Indeed, there is no best style to
express requirements. TRIC only supports the feature list style.
2.3.
Software requirements specifications
Software requirements specifications are documentation of the essential requirements of the
software and its external interfaces [12]. Documented representations of specific requirements
in various styles are but one part of it, as it typically also contains other elements [55].
The parts of software requirements specifications are not standardized, although several
guidelines exist, including IEEE Std 830-1998 [59], the V
olere template [51] and those provided
by Lauesen [39] and Sommerville [55].
This research uses IEEE Std 830-1998 as leading guideline for two reasons. First, because it
contains recognized quality criteria for software requirements specifications that may serve as
useful metrics. Second, because it is aligned with ISO/IEC 12207 [29], an industrial standard in
information technology for for software life cycle processes, which is useful in the context of
change impact analysis and the QuadREAD Project.
IEEE Std 830-1998 discusses essential parts of a software requirements specification and pro-
vides several example templates on an informative basis [59]. The essential parts are captured
in a prototype software requirements specification outline. See Figure 3.
24
Table of Contents
1.
Introduction
1.
Purpose
2.
Scope
3.
Definitions, acronyms and abbreviations
4.
References
5.
Overview
2.
Overall description
1.
Product perspective
2.
Product functions
3.
User characteristics
4.
Constraints
5.
Assumptions and dependencies
3.
Specific requirements
Appendixes
Index
Figure 3: Prototype sofware requirements specification outline [59] Other guidelines generally agree with the parts that a software requirements specification
should contain. Differences lie in the ordering and composition of parts. For example, the
V
olere template dictates to have separate parts for functional and non-functional requirements
[51] while IEEE Std 830-1998 makes no such distinction in its description of specific require-
ments [59]. In all guidelines, the parts containing requirements representations are separate
from parts containing domain and product insights.
2.4.
Software requirements management
Requirements evolution both during the requirements engineering process and after a system
has gone into service is inevitable. Software requirements management is the process of un-
derstanding and controlling changes to requirements for software products [55].
Requirements management should be done by a change control board with the authority to
decide on changes to be made or not. The basic change cycle is as follows [39]:
1.
Reporting: a requirements issue is reported to the change control board
2.
Analysis: the issue is analyzed together with other issues
3.
Decision: evaluate the issue and plan what to do with it
25