Reviewing Requirements Determination in Information Systems Projects
Information Systems 6840
University of Missouri-St. Louis
20 November 2015
Frederick Brooks asserts that software systems are inherently complex, and that complexity must be dealt with and managed, not abstracted away: “The complexity of software is an essential property, not an accidental one. Hence descriptions of a software entity that abstract away its complexity often abstract away its essence.” According to many researchers, a not insignificant amount of that complexity is generated early in the development lifecycle of systems projects, during requirements determination, making it a critical stage in the project lifecycle.
Requirements determination has many names and shades, including requirements elicitation, information requirements discourse, information requirements determination, requirements determination (and relatedly, service requirements), requirements engineering, and simply requirements analysis. Although each term includes its own perspective and connotations, I will use requirements analysis (RA) throughout for simplicity, following Alvarez. Alvarez defines RA as a process of “eliciting and encoding into the new system the requirements that clients verbalize to analysts.” Irrespective of the term employed, however, researchers and practitioners agree that while RA is a critical component of systems development,,,, it is also highly problematic, being an area of endemic miscommunication between stakeholders and leading to significant problems down the road,, including persistent requirements uncertainty and project management issues,. A review of some of the various RA methodologies is given next, followed by a discussion of their commonalities and differences, under which circumstances they are likely best applied, and ultimately what can be learned from them in order to better determine requirements in systems projects.
Jiang et. al. examine RA through a lens of requirements uncertainty, including the various components of that uncertainty and the perception gaps of the stakeholders involved. Key aspects of requirements uncertainty in Jiang’s model include: requirements instability (changes in requirements throughout the project) and requirements diversity (user disagreement over the requirements). Other aspects of requirements uncertainty include incomplete or ambiguous requirements and lack of user support. Via analysis of a survey of more than 150 information systems (IS) professionals, the authors confirm their most of their hypotheses, including those that requirements instability and diversity increase the perception gap and subsequent requirements uncertainty.
To mitigate these issues, Jiang’s team advocates for horizontal coordination between stakeholder groups, specifically direct communication between users and developers. This forms part of their broader recommendation that organizations must work to improve alignment between users and developers both at the outset and throughout the duration of systems development projects. Specific techniques include what they term “pre-project partnering,[a]” which starts before specific tasks are undertaken and where success factors and measurements are jointly determined. Organizational change is also recommended to align project goals with organizational ones, including concomitant changes to organizational and team cultures. Strong communication structures are also important in Jiang’s analysis, owing to the different perspectives of the involved stakeholder groups[b]. They recommend both formal and informal project reviews to facilitate knowledge sharing.
Brown and Ramesh similarly note that RA is a crucial component of systems projects, while observing that it is often pursued in an ad hoc manner and is poorly understood. However, they also observe that since it occurs early, improvements at this stage could have outsized benefits for the project overall. Brown and Ramesh focus on problems encountered during RA and group them into four classes: 4) constraints on humans as information processes; 2) variety and complexity of requirements; 3) communication problems; and 4) user unwillingness to provide requirements. A variety of factors underlie these issues, including memory limitations (and how memory interacts with learning), social and political pressures, instability of user needs/preferences, and differing perspectives due to different backgrounds.
To address these problems, Browne and Ramesh posit a number of approaches, including explaining terminology and discussing potential biases at the outset, with the goal of limiting them by making all involved parties aware of their impact. Further, they assert that improving alignment by explaining how sharing information benefits the user and organization as well as the analyst and the systems project can reduce motivational biases and political pressures, echoing earlier research. Use of what-if questions can prompt users to extend their thinking beyond what is into what could/should be, and introducing surprise, into test tasks can prompt experienced users to retrace their steps in order to complete an action, thereby rendering explicit what was otherwise unconscious knowledge. Creation of (after detailed explanation to users to aid comprehension and alignment) models, diagrams, and charts can make unclear and abstruse aspects of a system or workflow more salient., Lastly, as RA is inherently complex and humans are faced with many constraints, Browne and Ramesh caution that improving RA is an incremental process, building on prior knowledge and extending it via new projects and approaches to better address enduring problems.
Alvarez looks at RA through a dialectical lens (termed critical discourse analysis), a dialogue between analysts and clients strongly mediated by their different frames of reference. Similar to others, Alvarez observes that RA is beset by communication problems and that failures during RA often lead to failures of systems projects. Although there are many RA methods through which requirements are determined, Alvarez notes that interviews continue to be a key aspect of RA and focuses on interviews—and the core aspect of interviews, talk—as she examines the processes of RA. She goes a step further in asserting that language and talk underlie RA and systems development by declaring that “information technology and systems is not about the design of physical things, it is about the design of practices and possibilities realized through linguistic communication systems.”
A core component of those communication systems is framing, a process involving speech, tone, and expression and through which speakers impart how their messages should be deciphered. As RA stakeholders, especially users and analysts, come from different backgrounds and bring different perspectives to the table, framing is often a contentious process, one wherein power relations are revealed and sometimes contested. A central area of contention is that analysts view RA in technical terms, seeking brief answers (often yes or no ones) to questions, while users employ stories to describe their tasks and needs. Alvarez sums it up thusly: “for the analyst the encounter is a technological encounter, for the client it is about personal work experiences and dilemmas.” Power imbalances between the two groups also exist, in that the analysts generally control the narrative by asking the questions, deciding what should be discussed, when, and under what terms.
Alvarez concludes that the analysts’ technological frames predominant during RA, with continual but limited counter-framing by users. She further notes that the imbalanced framing often seen during RA runs counter to key goals of the process, namely, effective requirements gathering and ultimately effective systems (which, she observes, will be used by the [aptly named] users rather than by analysts), and that the involved groups will need to work together to more fully understand the other’s perspectives in order to create better requirements and better systems.
Similar to Alvarez, Halaweh asserts that RA efforts often fail due to overemphasis on technological frames of reference, and like other researchers cites communication shortcomings between users and analysts as a central issue. To address these problems, Halaweh proposes a grounded theory (GT) approach, wherein preconceived notions are set aside and data are continually defined and refined through an iterative coding process. (Setting aside predefined concepts is considered essential in GT, as Halaweh argues that systems often fail because analysts believe the proposed system is similar to previous ones with which they are familiar and thus the requirements are already known.) As part of GT, Halaweh notes that it is often useful to distinguish between user requirements (services or tasks the system needs to perform) and system requirements (detailed, technical descriptions of what the system does). Halaweh asserts that GT helps to identify non-technical requirements through continuous and iterative engagements with users. (A potential limitation of his research, which Halaweh explicitly admits, is that GT may skew too far toward non-technical requirements. In his case study, he notes that few technical requirements were generated using a GT approach, with the strong majority being user / task-oriented ones.)
Kirsch examines RA from the perspective of developing global systems, and the particular difficulties involved in balancing enterprise and local needs in systems development. She focuses on two aspects of the RA stage of global systems projects: knowledge acquisition, described as comparatively structured and rational (as users understand the need for a global system and the general benefits of standardization), and negotiation, wherein users vie to have their perspectives heard and argue for their requirements to become the requirements used in the new global system.
Kirsch defines a global or common system as one in which “core software modules [are] designed around common global requirements to accommodate the needs of multiple business units within a firm.” Similar to the other researchers, Kirsch points out that the RA phase is an inherently difficult one in systems development, citing lack of domain knowledge as a key issue. Kirsch defines domain knowledge as including both knowledge of business processes as well as the technical skills required to build systems, while noting the difficulties in obtaining such composite knowledge. Again similar to others (especially Alvarez) Kirsch highlights the importance of user involvement in the design of effective systems. (Further reinforcing Alvarez’s findings, Kirsch cites another study comparing traditional user/analyst interviews with a dialectical approach focusing on mutual understanding and information sharing, wherein the dialectical approach yielded superior results.) Kirsch’s work involves two cases, and the successes in each are related to the inclusion of key stakeholders who possess both business and IS experience (and were perceived to have both by their organizations, thereby increasing buy-in to and eventual compliance with the final system).
Kirsch’s RA process includes open coding practices, similar to Halaweh. One benefit of this approach is to get close to users, generating interesting user-driven benefits.[c] Another is the acknowledgement of the need to collaborate and compromise, as when various regions hammered out common terminology to be used, with each giving up some of their local usage (a sometimes contentious process). The shared focus also enabled practical prioritization, as when big or high-revenue regions were given greater sway in determining some requirements (the smaller regions being somewhat more accepting of this, not just due to recognition of size differences and business needs, but as they had been involved in terminology sessions and witnessed their larger counterparts accepting changes to some of their own preferred terms, implying a sense of fairness through shared sacrifice). The creation and use of superior shared tools and infrastructure also fostered acceptance of other tradeoffs.
Despite the recognition of the benefits of global solutions, Kirsch notes that local buy-in was often hard to achieve. Frequently users would agree in principle to global requirements, but continue with their local processes in practice; creating sufficient acceptance and compliance was difficult, and ensuring users had a clear understanding how the global requirements would benefit them was key. Allowing local groups sufficient flexibility to employ customized approaches when appropriate was also essential. Moreover, learning when to give and take on standardization versus flexibility was an ongoing effort. Balancing these fundamental tensions was especially precarious as requirements changed or new ones emerged, as when new regions were brought into the process, necessitating the reopening some dearly closed discussions. Skillful facilitators were integral to success, and were observed to pursue two goals: 1) encouraging participation to ensure all relevant sides were heard, and 2) focusing requirements and driving for consensus. This dual focus required continual rebalancing, in addition to great persistence and perseverance. Having facilitators with both business and IS experience was also perceived as crucial to achieving needed buy-in to and compliance with the final systems. Kirsch also notes that involvement in the negotiation process itself could change perceptions and even desired requirements, as participants learned new approaches and processes from their counterparts in other business units. Another takeaway was the technique of initially categorizing efforts as: new system building, system replacement, or consolidation of existing systems, aiding in shaping focus in the early stages of RA.
Duggan examines RA through the specific techniques of joint application development (JAD) and nominal group technique (NGT). Key findings include (in a laboratory setting) that while JAD was found to be relatively effective as a standalone method, inclusion of NGT led to superior results over JAD alone (specifically, the integrated approach [referred to as NJAD] was as efficient as JAD by itself, while reducing the requirement of excellent facilitation skills necessary with JAD). Similar to others, Duggan notes the criticality of RA and the communication issues often encountered at this stage. He also observes that a follow-on effect of this problem is high maintenance costs for many systems, stemming from the continuous changes and fixes required due to poor initial requirements.
Duggan describes NGT as a process wherein 1) individuals generate ideas; 2) the facilitator records all ideas in a round-robin format; 3) ideas are then discussed for clarification purposes only, without judgment; 4) ideas are then independently rated and ranked by participants; and 5) the group prioritizes ideas based on voting and mathematical assessment of the individual rankings. Duggan also notes that rather than striving for equal participation, generating participation according to knowledge and experience is more important. Through his experiments, Duggan noted that despite its more rigorous structure, NJAD did not take more time than JAD alone, thereby not increasing the time needed to complete the RA process; moreover, requirements quality was also found to be high. Similar to Alvarez, Kirsch, and others, Duggan further observed that acceptance of and compliance with a system is often as reliant on social factors as on technical ones, and the mix of individual and group processes in NJAD was found to produce superior requirements and buy-in compared to JAD alone.
Many themes and threads emerge from the foregoing discussion. First, the complexity of RA is unanimous. One aspect of that complexity that is highlighted by multiple researchers is that requirements are unstable, that they can change during RA, throughout the project, and even within individual users, as both circumstances evolve and preferences shift., Communication problems are another oft-cited issue,, with the most common culprit being different backgrounds and frames of reference. Also, the observation that RA is ongoing and iterative, formulated by Halaweh, is supported by others as well.,,
A key item is not just if various techniques or approaches are effective, but under which circumstances. While interviews were found to be useful by multiple researchers (including Alvarez and Kirsch), they do not address the potential of dominating persons, which may overpower their more reticent but perhaps no less experienced or expert peers. In those situations, employment of NGT and similar techniques to generate broader and deeper idea sets may be warranted. Moreover, Duggan asserts that JAD and NJAD are superior to interviews in terms of RA efficacy and efficiency. While likely conceding the efficiency point, Alvarez would nonetheless likely contest the efficacy one, believing that greater mutual understanding achieved through an in-depth dialectical approach (although admittedly difficult) more strongly enhances user buy-in and compliance (and thus overall system effectiveness). The key components here are time and system complexity. If a system is comparatively straightforward, JAD (or NJAD) may be sufficient, but if sufficient uncertainty exists (or buy-in is likely to be especially difficult), thorough interviews with key stakeholders may be needed as well.
While most of the researchers’ recommendations appear sound, the practical utility of some findings is questionable. Notably, a key part of Kirsch’s results involves the participation of individuals with both business and IS experience (and who are perceived to be knowledgeable about both by others). While the inclusion of such people in RA is doubtless beneficial, persons with that combination of experience and skills are likely rare, and thus relying on them may work in large endeavors where there is sufficient revenue or risk at play to warrant their input, but it would likely be difficult in small-to-medium projects, when their duties and responsibilities may pull them elsewhere.
While differences exist in the methodologies examined and the conclusions drawn from them, the similarities are the more salient overall. Notably, specific techniques or approaches aside, all researchers agree that RA is a complex component of a complex endeavor, and that communication issues inherent to RA must be dealt with in order to create effective systems and to ultimately ensure their use. Working with individual with strong self-awareness and social skills (and to the extent possible, providing training in these areas to key stakeholders who lack them) will benefit the RA process, the systems project, and the organizational overall. Nevertheless, while persons with strong business, interpersonal, and technical skills are significant assets, they are likely to be in short supply on most projects. Therefore, while creating and following a clear approach to systems projects will provide a useful framework, recognizing the uniqueness of each project is crucial to ensuring the appropriate amount of flexibility within a given framework is provided to ensure stakeholder buy-in and ultimately system success.
[a] Jiang’s description of pre-project partnering is similar to a process called early engagement in my own organization. The extent to which early engagement is effectively pursued and implemented varies considerably across project teams in my experience, indicating that more organizational and cultural change is needed to better embed early engagement / pre-project partnering into organizational processes.
[b] The criticality of effective communication is reinforced during a 16 November 2015 lecture by Sharyn Lemmons, PMP. Ms. Lemmons, who has been working in project management for nearly 25 years in the private sector, emphasizes that in her experience strong communication is the single most important component of successful projects.
[c] One such benefit occurred in a financial services firm studied by Kirsch, where users took the various types of letters of credit then in use, and wrote specifications on the letters, creating direct links between their current tasks and proposed requirements, thereby enhancing the perceived quality of the requirements and increasing organizational buy-in.
 Brooks FP. (1995). The mythical man-month. Reading Mass: Addison Wesley Longman, 183.
 Alvarez, R.
(2002). Confessions of an information worker: a critical analysis of
information requirements discourse. Information and Organization, 12, 85–107.
 Browne GJ, Ramesh V. (2002). Improving information requirements determination: a cognitive perspective. Information and Management, 39, 625-645.
 Kirsch LJ, Haney MH. (2006). Requirements determination for common systems: turning a global vision into a local reality. Journal of Strategic Information Systems, 15, 79-104.
 Estebanez LR et. al. (2015). Enhancing service requirements of technical product-service systems. Procedia CIRP, 37, 7-11.
 Halawah M. (2012). Using grounded theory as a method for systems requirements analysis. Journal of Information Systems and Technology Management, 9(1), 23-38.
 Alvarez, 86.
 Yang H, Tang J. (2005). Key user roles on web-based information systems requirements. Industrial Management & Data Systems, 105(5), 577-595.
 Business requirements analysis: clearly agreeing what you’re going to deliver. (n.d.) Retrieved from https://www.mindtools.com/pages/article/newPPM_77.htm
 Olson JV. (2010). Accurately identifying your requirements—will any computer system be right for you? Journal of Validation Technology, Winter, 29-31.
 McMurtrey M. (2013). A case study of the application of the systems development life cycle (SDLC) in 21st century health care: something old, something new? Retrieved from http://quod.lib.umich.edu/j/jsais/11880084.0001.103/--case-study-of-the-application-of-the-systems-development?rgn=main;view=fulltext
 Browne and Ramesh, 625.
 Alvarez, 86.
 Jiang J, Klein G, Wu S PJ, and Liang TP. (2009). The relation of requirements uncertainty and stakeholder perception gaps to project management performance. The Journal of Systems and Software, 82, 801-808.
 Sarigiannidis L, Chatzoglou PD. (2011). Software development project risk management: A new conceptual framework. Journal of Software Engineering and Applications, 4, 293-305.
 Jiang, 802.
 Ibid, 807.
 Browne, 625.
 Ibid, 627.
 Baer D. (2015, October 27). 4 strategies for remembering everything you learn. Retrieved from https://agenda.weforum.org/2015/10/4-strategies-for-remembering-everything-you-learn/?utm_content=buffer24222&utm_medium=social&utm_source=twitter.com&utm_campaign=buffer
 Davis FD. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, September, 319-340.
 Surprise and learning. (n.d.) Retrieved from http://changingminds.org/explanations/learning/surprise_learning.htm
 NSF funds probe of the quintessence of surprise. (2005, November 28). Retrieved from http://www.eurekalert.org/pub_releases/2005-11/uosc-nfp112805.php
 Sharma A. (2015, April 28). Data flow diagrams—are they worth it? Retrieved from http://www.batimes.com/articles/data-flow-diagrams-are-they-worth-it.html
 Maxted SJ. (2008, December 1). The business context model: as good as it gets. Retrieved from http://www.batimes.com/articles/the-business-context-model-as-good-as-it-gets.html
 Alvarez, 90.
 Alvarez, 101.
 Kirsch, 80.
 Duggan EW, Thachenkary CS. (2004). Integrating nominal group technique and joint application development for improved systems requirements determination. Information & Management, 41, 399-411.
 Jiang, 802.
 Browne and Ramesh, 631.
 Browne and Ramesh, 631.
 Alvarez, 86, 92.
 Analyzing and defining requirements. (2013, September). Retrieved from http://www.mitre.org/publications/systems-engineering-guide/se-lifecycle-building-blocks/requirements-engineering/analyzing-and-defining-requirements
 Tattersall G. (2015, October 27). Supporting iterative development through requirements management. Retrieved from http://www.ibm.com/developerworks/rational/library/2830.html
 Eliciting, collecting, and developing requirements. (2015, August). Retrieved from http://www.mitre.org/publications/systems-engineering-guide/se-lifecycle-building-blocks/requirements-engineering/eliciting-collecting-and-developing-requirements
 Duggan, 400.