The
Importance of Requirements Definition in IT Systems
Development
by Ray Phariss
IS
6840, Systems Analysis
November
29, 2006
Numerous surveys and studies indicate that
one of the major challenges in IT system development is the determination of
system requirements. It might be
surprising that after several decades of industry experience in these
endeavors, developers would still pursue projects for which the requirements
are not abundantly clear. However, these
problems persist and the findings are validated by a long list of infamous IT
project failures ranging from the very recent to those reaching back nearly as
far as computers have existed.
While we may be inclined to associate such
failures with certain types of companies, certain areas of the world, or
certain types of business practices, Robert N. Charette
notes, “failures are universally unprejudiced:
they happen in every country; to large companies and small; in
commercial, nonprofit, and governmental organizations; and without regard to status
or reputation.” [11] This being the case, the failure of IT projects represents
a complex problem that must be thoroughly studied from many perspectives to be
fully understood. Further, the insights
gained from such studies must be carefully applied in order for future projects
to be successful. As information
technology continues to expand into nearly every form of human activity, this
is a problem that must be addressed. The
scope of the problem is not trivial and neither are the costs. One estimate is that “the yearly tab for
failed and troubled software conservatively runs somewhere from $60 billion to
$70 billion in the
In addition to direct financial costs,
failed projects can also lead to expensive litigation, and some people make a
career out of helping to sort out the mess.
One such individual is Brooks Hilliard, who “has been a consulting and
testifying expert witness, on computer fraud, for both plaintiffs and defendants,
in more than 30 cases involving computer systems that were alleged not to have
performed as promised.” [2] One should also consider that some failures are
never publicized because the parties would prefer to cover up the failure to
avoid loss of face, as well as a possible loss in their company’s value, which
can often be considerable. One notorious
example of staggering financial loss is Oxford Health Plans Inc. In 1997, its automated billing system failed
and the result was a one-day loss in its stock value from $68 to $26 per share,
a 62% drop which equaled $3.4 billion in the corporation’s overall value [11].
This paper will explore the challenges
associated with requirements definition as a major cause of failure in IT
systems development by drawing on documented examples, existing analysis and
studies, and theoretical concepts.
Additionally, it will offer suggested remedies to mitigate the risk
associated with the challenges inherent in requirements definition and
management. The ultimate purpose of the
paper is to learn from others’ mistakes as a means to reducing the rate of
failure in IT projects. [19]
While there are potentially
endless reasons why IT projects fail in general, this paper focuses on the
requirements definition stage of development.
This is a critical stage in the process – if not done right, all other
activity is irrelevant and a waste of time and money. Partly, this is due to the fact that
requirements are necessarily defined early in the project development
schedule. Unfortunately, this presents
the opportunity for the early introduction of errors that can result in
terrible consequences later in the project.
As Dave Gluch notes, “a good percentage of
errors occur at the requirements stage, and if they’re not discovered until
later on, the cost of fixing them is substantially higher.” [10] While this idea
might seem obvious, the concept is easily forgotten in practice when other
factors begin to influence the process.
Often, a failure to properly define the project’s requirements is only
discovered in the postmortem analysis, i.e. the case study. Accordingly, it is from these and other
sources that the following subjects were selected for further analysis in this
paper:
·
IT
Project Team Performance
·
Size
and Complexity of the Project
·
Inadequate
Understanding of Business Processes
·
Selecting
the Right Development Methodology
·
Organizational
Culture
Together, these factors account for a
substantial portion of mistakes made in the requirements definition stage of IT
project development. As such, they merit
attention to better understand and, hopefully, minimize their effects on future
projects.
IT
projects nearly always require teams of people for their implementation. The level of performance within these teams
contributes greatly to the overall outcome of the project. In her paper titled “Failed IT Projects: The
Human Factor,” Sheila Wilson examines several case studies and concludes that
the following factors affect team performance in the context of IT projects:
skill level of the team members, varying levels of commitment of team members,
insufficient or too many team members, turnover of team members (especially
those in key leadership positions) and lack of use of a structured methodology
[20]. If asked directly, most project
managers would agree that these factors are important points to consider when
building a team for a specific project.
However, managers may not always conduct a deliberate analysis of such
considerations in actual practice. Rather,
teams are often assembled based upon the personnel available to them at the
time, a process that might disregard some or all of the aforementioned
factors.
In his article “Maine’s Medicaid Mistakes”, Allan
Holmes describes a failed $25 million project that was built to process
Medicaid claims and comply with new requirements mandated by the Health
Insurance Portability and Accountability Act of 1996 (HIPAA). [15] A major
cause of the system’s failure was traced back to poor requirements, which in turn
were traced back to numerous problems with the implementation team. First, Dick Thompson, who was then the head
of procurement for the state of
Another challenge to
effective and successful team performance, especially in the requirements
stage, is the distributed nature of many project teams. As a result of companies outsourcing software
development, especially offshore, project teams often consist of geographically
dispersed members across multiple time zones.
This makes it difficult for team members to stay synchronized on
continuously evolving requirements documentation, often complicated by multiple
copies of the same information being exchanged between members until they lose
track of which one is the most current.
Microsoft has recognized the challenges inherent in requirements
gathering, especially in a distributed development environment. It proposes a “requirements authoring
solution” based on its nearly ubiquitous Microsoft Office application suite,
claiming that it “can enable development teams to more easily author and edit
requirements documents for complex projects.”
[7] This represents recognition by a major software company of the
difficulty in effectively capturing and managing requirements
documentation. When applicable, a keen
project manager should seek out tools and methods to help mitigate these types
of challenges.
A
challenge primarily associated with large projects is that they are often quite
complex when one considers all the intricate business rules and dependencies
that exist, as well as interconnections with other systems, some of which might
be external to the organization. This
complexity can become a serious problem as it “increases the possibility of errors,
because no one really understands all the interacting parts of the whole or has
the ability to test them.” [11] As a result, developers may build the system
around the parts they were able to
identify and accept risk in the other areas, hoping that it will all work out
in the end. However, such hope is often
in vain as “studies indicate that large-scale projects fail three to five times
more often than small ones.” [11] Larger projects
also take more time to complete.
Unfortunately, business needs change so frequently that the original
requirements, no matter how well they might have met the business needs at the
time they were identified and documented, may no longer meet the business needs
by the time the project is completed.
Therefore, large projects may require a development methodology that
allows for some flexibility in requirements during the course of the project.
Another challenge that arises
due to the large size of a project is confusion or lack of agreement about who
is in charge. As Warne and Hart point
out, “the larger the scope of a proposed information system, the more likely it
is to be attended by organizational conflict regarding who should own and
control it.” [13] In the article “Anatomy
of Failure”, Lauren Gibbons Paul presents an example of a failed data
warehouse project for a large organization. [18] Paul explains, “the challenge was to get the distinct and autonomous
business units to settle on…critical definitions.” [18] Further, internal
disagreement was created “because the managers looked at things differently…defining
the fact groups therefore became a hellish ordeal.” In the end, the project was cancelled after
spending three times the original budget.
This example illustrates how the size of a project can lead to internal
disagreement, thus contributing to the complexity of both the design and
implementation of the project.
Successful requirements
definition relies heavily on a thorough and accurate understanding of the
business processes a project is supposed to model. When access to the necessary stakeholders and
subject matter experts is limited or non-existent, as was the case with Maine’s
Medicaid billing project, “developers, having no input from the users, build what
they believe is needed, without having any real knowledge of the business.” [3]
When this occurs, as Scott Ambler points out in his writing about Agile
modeling, as a project manager you must “recognize that you are going at risk
if you do not include some direct users in your software development
efforts.” [1] In order to minimize risk
to the project, therefore, it is imperative that IT project managers ensure
they have adequate stakeholder involvement in the requirements stage, and that
the “users…know what it is they want, and be able to specify it
precisely.” [3]
Regrettably, customers
themselves may not always have the experience and/or knowledge about IT systems
necessary to adequately define their requirements. Ambler notes that “project stakeholders will
very often sense that there is a problem or opportunity that should be
addressed but they don’t know how.” [1] As a result of their inability to fully
grasp their own needs, customers might sign off on requirements they do not
understand or that do not adequately meet their business needs - they just do
it to keep the project moving along.
Later, when the project is completed and they are disappointed with the
results, they immediately point blame at the developer, who in turn falls back
on the requirements document that was approved by the customer. Such was the case when Prudential Europe “terminated
a $50 million contract with Unisys following the collapse of its Unite project,
which aimed to deliver real-time processing of policies and pensions over the
Internet. Unisys argued that some
specifications and requirements were not signed off or fully defined by
Prudential.” [6]
Sometimes, even well
understood and documented requirements change.
In the case of
Another complicating factor of
requirements changing during a project is the effect it can have on the ability
to determine whether or not the requirements have been met. Derek Sisson’s explains in an article about
requirements and specifications for web site development that “requirements
tend to change through the course of a project, with the result that the
product as delivered may not adhere to the available requirements – this is a
constant and annoying facet to the quality assurance process.” [9] Effective
project managers must ensure that changes in requirements are captured and
maintained so that this documentation can be referenced later to validate that
the requirements have indeed been met.
In order to provide the
flexibility necessary to accommodate large, complex projects and the changes in
requirements that often occur throughout their development cycle, project
managers should carefully select the development methodology that is most
appropriate for these projects. In his article
Challenges in Government Software
Development, Scott Cronenweth presents interviews
he conducted to examine effects of the recently-created Office of Electronic
Government within the
Notorious for its
bureaucratic tendencies and stove-piped information systems, the federal
government would seem like a hostile environment for anyone wishing to
integrate its information systems across departmental boundaries. Further, the sheer scale and complexity of
such an endeavor might make the task prohibitively difficult for any single
company to successfully accomplish. As
Michael Burkett, one of Cronenweth’s interviewees,
notes, “no one manufacturer can deliver an entire system anymore. Companies have consolidated their businesses
around core competencies. So a prime
contractor, such as Lockheed-Martin, must inevitably work with many
subcontractors and suppliers, each of which has critical input into the design
and manufacturing of specific system components.” [4] Since entire systems now require multiple
companies in order to deliver an end-to-end solution, the level of complexity involved
in their development has increased.
Notwithstanding the
complexities involved, the Office of Electronic Government persists in trying
to force the governmental agencies to work together in the interest of the
citizens. Cronenweth
describes an example of a system that would allow a citizen to find out what
money is owed to him by the government in a consolidated way, even though
several distinct agencies might each owe him some amount (IRS, Social Security
Administration, etc.). [4] Scott Frohman, another of Cronenweth’s
interviewees, explains that “the first challenge…concerns requirements
management: getting the right people collaborating to determine what the system
needs to look like. The goal is a
unified, agreed-upon set of requirements that defines an agency’s business
process and reflects a common vision.” [4] The agreement of the various
participants is fundamental to all subsequent activity – without it, the
project will certainly fail. However,
this does not mean that every last detail must be agreed upon from the
start. Frohman explains that “with an
iterative approach, you don’t assume that requirements are sparkling perfect
when you start; you know they’re subject to ongoing adjustment. This greatly mitigates the risk of building
the wrong thing. In addition, you start
testing early on to find and eliminate bugs that would be very costly to fix
later on. Iterative development
mitigates all forms of project risk.” [4]
Another methodology that appears to be
growing in popularity is the open-source development model. In this methodology, no single person or
entity owns the software. Rather, it is
publicly available to everyone with the stipulation that it must remain free
and that the source code must remain available, or “open” so that others may
modify it. This model relies on the
combined talents of programmers who form communities of interest based on a
common goal of developing applications for various purposes. One of the most widely known examples of this
is the Linux operating system, originally conceived as an alternative to
Microsoft Windows. While the number of
Linux users today is not nearly as large as the number of users running
Windows, Linux has penetrated niche functions such as web servers. This has led some people to suggest that the
open-source model might be well-suited for other purposes. One example of this is revealed in a post by
an individual identified only by a username (Saint) and an email address, who
suggests on a Linux-based forum for medical IT professionals that “a
closed-source medical software development model is either doomed to failure,
or doomed to be prohibitively expensive.” [8] Citing first-hand knowledge of
numerous failed medical IT projects, Saint suggests that the medical industry
does not represent a large enough market to justify the high cost of
traditional, closed-source development for what he considers to be an extremely
complex business application. He claims
that “there is no single undertaking other than writing an entire operating
system that rivals the complexity and knowledge required for a successful
clinical computing system.” [8] Although Saint does not provide any factual
evidence to support his position, he does raise interesting points of consideration,
especially the notion that we should consider alternative methodologies for
some projects that might not be suitable for the more traditional models.
Another factor that can
create challenges in determining requirements for IT projects is organizational
culture. The leap from manual processes
to an automated system requires a cultural shift in the mindset of employees,
many of whom are often incapable of grasping IT-related concepts due to their
lack of experience with them. Further,
this lack of experience may manifest itself in a culture that is not conducive
to IT system development. In fact, it
might even become a barrier to progress.
One such customer was the FBI, whose recent public failure with its
Virtual Case File (VCF) project has come under intense scrutiny and is widely
used as a case study of a large-scale IT project failure.
“The VCF was supposed to automate the FBI’s
paper-based work environment, allow agents and intelligence analysts to share
vital investigative information, and replace the obsolete Automated Case
Support (ACS) system. Instead, the FBI
claims, the VCF’s contractor, Science Applications
International Corp. (SAIC), in San Diego, delivered 700,000 lines of code so
bug-ridden and functionally off target, that this past April, the bureau had to
scrap the US $170 million project, including $105 million worth of unusable
code.” [12]
One of the major reasons the
VCF project failed was because the culture at the FBI did not place a high value
on IT and this affected its ability to work with SAIC to adequately define the
project’s requirements. Some have also
concluded that the FBI had a somewhat arrogant attitude toward requirements
definition.
“Poor benchmarking and requirements gathering…affected
the project, with FBI overseers…relying upon a ‘I’ll
know it when I see it’ attitude to determine when the project would be
ready.” [14]
The FBI’s cavalier approach
during the development process was a major factor in the VCF’s
ultimate failure. It was the
organizational culture that arguably doomed the project to failure from the
beginning by preventing developers from effectively capturing workable
requirements. In numerous investigative
findings, the lack of well-defined requirements is listed as one of the major
causes:
“In a devastating 81-page audit, released
in 2005, Glenn A. Fine, the U.S. Department of Justice’s inspector general,
described eight factors that contributed to the VCF’s
failure. Among them: poorly defined and slowly evolving design
requirements…” [12]
“It was a classic case of not getting the
requirements sufficiently defined in terms of completeness and correctness from
the beginning. And so it required a
continuous redefinition of requirements that had a cascading effect on what had
already been designed and produced.”
[12]
“The requirements and design
documentation were incomplete, imprecise, requirements and design tracings have
gaps, and the software cannot be maintained without difficulty. And it is therefore unfit for use.” [12]
This example reveals that the
challenges of determining requirements for an IT project are not just technical
in nature. Rather, these efforts can be
thwarted by the culture of the organization attempting the project, perhaps
even unknowingly and unintentionally.
While the employees themselves cannot be reasonably expected to identify
or mitigate such cultural biases, a good IT project manager should look for
indicators of such conditions and take steps to deal with them.
Requirements determination is a critical stage
in any software development process due to its fundamental importance to making
sure the project ultimately solves the right problem in the right way. As many studies have shown, errors in this
stage of development often lead to overall project failure with disastrous
consequences. Therefore, prudence
requires that IT project managers search for ways to mitigate the common
mistakes that occur in requirements definition so that they can improve their
likelihood of success in future projects.
This paper has discussed the following factors of successful
requirements gathering:
·
IT
Project Team Performance
·
Size
and Complexity of the Project
·
Inadequate
Understanding of Business Processes
·
Selecting
the Right Development Methodology
·
Organizational
Culture
Project managers must create teams of
highly skilled, committed persons who work well together, are
able to communicate with each other (even while distributed throughout the
world and across different time zones) and share a range of expertise
applicable to the success of the particular project at hand. The size of a project must also be taken into
consideration, and the complexity of a particular project based on its
large-scale size must be accounted for.
The time, resources and cooperation/communication needed to accommodate
large scale projects is often orders of magnitude greater than those needed for
small scale projects, an aspect of requirements determination that is often
overlooked but is ultimately imperative to the success of a project.
Adequate understanding of business
processes is also key throughout the design and
implementation of the project. Project
managers must be able to determine when customers may or may not have
appropriate understanding of the complexity of project development and must
make applicable adjustments to ensure the proper communication and
determination of the project’s end goals.
Understanding the many different methodologies and applying the correct
methodology for each project is also particularly important in meeting the
project’s end goals. Finally, an
organization’s culture must be taken into consideration by project managers to
ensure a project’s smooth implementation.
Understanding each organization’s culture will help managers plan and
design systems that work best for each of those organizations and will help
managers avoid cookie-cutter project designs that ultimately end in
failure.
While project failures often go
unreported or underreported due to fear of loss of reputation or financial
value, we still are able to find ample evidence that the factors discussed
throughout the paper are key elements of a project’s ultimate failure or
success. In examining these examples, we
are able to use past experiences to enhance our understanding not only of what
not to do, but also what project managers can and should do to ensure a
project’s success. Examining in
particular the requirements determination stage of project design allows managers
to avoid unnecessary pitfalls and mistakes early on in the project’s
development, thus ensuring a smoother, more successful project further later,
as it moves through its many stages toward completion.
Works Cited
Web:
1.
Ambler, Scott W. “Overcoming Requirements Modeling
Challenges.” April 29, 2006. Online. http://www.agilemodeling.com/essays/requirementsChallenges.htm
2.
BizAuto. “Web Advertisement for Computer Expert Witness,
Litigation Support & Computer Forensics.” Online. http://www.bizauto.com/expert.htm
3.
Coley Consulting. “Why Projects Fail.” 2001 – 2005. Online. http://www.coleyconsulting.co.uk/failure.htm#nav
4.
Cronenweth, Scott. “Challenges in
Government Software Development.” The Rational Edge. January 15, 2003.
Online.http://www-128.ibm.com/developerworks/rational/library/2223.html
5.
Factor, Phil. “Failed Project?” Simple-Talk.com, December 22, 2005.
Online. http://www.simple-talk.com/sql/t-sql-programming/failed-project/
6.
IT Cortex. “Failure Examples of IT Projects.” IT Cortex. Online. http://www.it-cortex.com/Examples_f.htm
7.
Microsoft. “Managing Requirements Authoring for Complex Application
Development Using the Microsoft Office System.” Online. http://www.microsoft.com/office/showcase/rapidauthoring/default.mspx
8.
Saint. “Medicine’s Dirty Software Secret.” LinuxMedNews.
Online. (ivaldes@hal-pc.org): http://www.linuxmednews.com/955063770/index_html
9.
Sisson, Derek. “A Thoughtful Approach to Web Site Quality: Requirements
and Specifications.” 1999 – 2002. Online. http://www.philosophe.com/design/requirements.html
10. Thomas, Bill. “Meeting the
Challenges of Requirements Engineering.” SEI Interactive, March, 1999.
Online. www.sei.cmu.edu/news-at-sei/features/1999/mar/Spotlight.mar99.pdf
Non-Web:
11. Charette, Robert N. “Why Software Fails.” IEEE Spectrum,
September 2005.
12. Goldstein, Harry. “Who Killed the Virtual Case File?”
IEEE Spectrum, September 2005.
13. Hart, Dennis and L. Warne. “The Impact of
Organizational Politics on Information Systems Project Failure – A Case
Study.” Conference Study: 29th
Annual
14. Hickson, Thomas B. “Failure at the Speed of Light: Project
Escalation and De-Escalation in the Software Industry.” Thesis: The
15. Holmes, Allen. “
16. Lyytinen, Kalle and D. Robey. “Learning Failure in Information Systems
Development.” Info Systems J
(1999) 9, pgs. 85-101.
17. May, Lorin J. “Major Causes
of Software Project Failures.” May, 07, 1998.
18. Paul, Lauren Gibbons. “Anatomy of Failure.” CIO
Enterprise. November 15, 1997.
19. Web, Allen. “Project Failure and How to Avoid It.”
March, 2001.
20. Wilson, Sheila. “Failed IT Projects: The Human Factor.”
University of Maryland/Bowie State University. May 16, 1998.