Importance of Interview and Survey Questions in Systems Analysis

Jeremy Brugger
IS Analysis | Dr. Sauter
University of Missouri – St. Louis
November 15, 2010

Introduction | Question Types | Importance of Effective Questions | Summary | References

(Printer friendly verison)

Introduction


Analysis is the second phase of the Systems development life cycle (SDLC) in which system requirements are studied and structured. Requirements must drive design and development decisions throughout the product development life cycle. (1) Meticulous requirements gathering will define the current system, including the location of problems/opportunities and an explanation of how to fix/enhance/replace the system.(2)  Many times development teams are given unrealistic deadlines for completing the system they have been tasked to develop or redesign. When time is tight and deadlines are fast approaching the initial list of functionality must be reviewed and trimmed. This task can be simple if the analysis team administered effective surveys and conducted thorough follow-up interviews to provide strong requirements.

Interviewing and surveying stakeholders (3), along with review of the current system ensures that you understand the situation before introducing any improvements. (4)  With the survey and interview results the analysis team would know exactly what requirements are essential to the end-users and which requirements would be nice, but are not imperative. As depicted below in Figure 1, a misunderstanding early on can create a huge difference between what the client wants and what gets created. By performing in-depth requirements gathering, analysis teams will be able to provide the design team with with a solid requirements document. A better understanding of how to word, ask and conduct survey and interview questions will save time and increase the quality of the analysis. This will in turn save time for the design and implementation phases of the systems development life cycle (SDLC).

Figure 1



(20)

Question Types


An understanding of the types of questions and their correct uses is very important in creating an effective survey and conducting a productive interview. The first type of question is the closed-ended question (5). This question type usually consists of:

         
The closed-ended question is quick and requires little time to answer. These questions are particularly useful for demographic-related questions. However, demographics questions should be asked at the end of the questionnaire in order not to affect negatively the preparedness of respondents to answer questions due to the feeling of losing anonymity, which could happen if they were asked those questions at the beginning of the questionnaire. (6) The downside of this question type is that they can be leading, hence irritating or threatening; they also discourage disclosure since the possible answers are predefined. (7)

The next type of question and most important to systems analysis is the open-ended question (8).  This question type allows for maximum flexibility in the interviewee's response. Some examples of this question type:

Open-ended questions allow the analyst to get answers that are unanticipated and potentially helpful in avoiding issues later in the process. These questions develop trust, are perceived as less threatening, allow for an unrestrained or free response. However, they can be time-consuming (7) and it can be difficult to develop general statements or assumptions. There is also the potential for the interviewee to ramble off into an useless answer.

           While sticking to the two question types above it is also important to avoid the following question types:

(9)

Some of the types of questions above may unintentionally find their way into your list of questions. A good way to avoid this would peer review by a colleague or even test the survey out on a few colleagues to see if the responses fit the expected responses. (10) Taking a little time to refine your questions prior to the interview will increase the quality of the answers you get and decrease the amount of time spent on this task.

Questionnaire Resources

CustomInsights.com

SurveyMonkey.com

Zoomerang.com

ZapSurvey.com

FormDesk.com

Importance of Effective Questions


There are two ways a questionnaire might be terrible.

(11)
Effectively written and analyzed questions can make requirements gathering a streamlined process. If surveys are constructed in a way that allows you get the answers you truly want then there will be fewer follow-up interviews needed. This sounds simple, however because no codified rules for question asking exists, it might appear that few, if any basic principals exist to differentiate good from bad questions. (12) The same is true for interviews, analysts may know what answers they need but that does not mean that they also know how to ask the questions to get to those answers.  Just like people, no two systems are alike so analysts need to approach them with a fresh, non-stereotyped perspective. If given a chance to talk freely, people appear to know a lot about what's going on. (13) While there is no definably “right” or “wrong” way to ask questions, here are five basic characteristics of questions and answers that are fundamental to a good measurement process:

(14)


Figure 2



(21)
There is a history in survey research of thinking question design as an art, not a science. (15)

 

Every analysis project will need a different mix of open- and closed-ended questions; this will depend on many factors, including the analyst familiarity with the system and company being analyzed. Qualitative data is your greatest opportunity to get actionable information from your survey. (16)  However surveys do have potential downsides and pitfalls. Often, stakeholders are interviewed about their requirements or asked to write them down, but this approach rarely uncovers the real requirements that reflect a customer’s true interests or needs.  Thorough analysis beyond the semantics of the survey answers or interview notes will uncover areas of ambiguity that will likely need clarity down the road. (17)  To avoid the need for this extra analysis it is important to review your questions prior to interviewing or surveying the stakeholders. In event more questions arise after asking the questions, it is imperative that the analyst ask probing questions or schedule time to perform follow-up interviews to flush out any ambiguities. Making assumptions at this stage could be costly

Not everything can be uncovered with surveys or interviews. If certain requirements need further clarity or if something from the survey or interviews didn't fully answer your question then you may consider shadowing the user while they interact with the part of the system in question. This can be useful when:

(18)

 

Summary

Learning to construct quality questions is a skill that takes time and practice. Once armed with the knowledge of how to ask quality questions an analyst can make their job easier and the quality of that work much better.  Your time and the time of your customer is on the line, so it makes sense to do all you can to ask questions that provide accurate, insightful responses. (19)  Through the execution of effective surveys and interviews the analysis phase of the SDLC can be maximized and save time in the design and implementation phases as well.

 Figure 3


(22)

 

References

  1. "Requirements Engineering for Product Development." IBM - Product Lifecycle Management. N.p., n.d. Web. 7 Nov. 2010. <www-01.ibm.com/software/plm/pdif/solutions/requirements-engineering.html?cn=agus_rtnlrepd-20100514&cm=k&csr=google&cr=engineering_requirements&ct=AGRAK616&ck=engineering_requirements&mkwid=s2EGymWY3_5508017014_432dt32930&cmp=109HG>.
  2. Hoffer, Jeffrey A., Joey F. George, and Joseph S. Valacich. "The Systems Development Environment." Modern systems analysis and design . 6th ed. Upper Saddle River, N.J.: Pearson Prentice Hall, 2011. Print.
  3. "Stakeholder - Definition and More from the Free Merriam-Webster Dictionary." Merriam-Webster Online. N.p., n.d. Web. 14 Nov. 2010. <http://www.merriam-webster.com/dictionary/stakeholder>.
  4. Robertson, Suzanne, and James Robertson. Mastering the requirements process . 2nd ed. Upper Saddle River, NJ : Addison-Wesley, 2006. Print.
  5.  "closed question definition." BusinessDictionary.com - Online Business Dictionary. N.p., n.d. Web. 7 Nov. 2010. <http://www.businessdictionary.com/definition/closed-question.html>.
  6. Lietz, P. (2010). Research into questionnaire design. International Journal of Market Research, 52(2), 249-272. Retrieved from Business Source Premier database.
  7.  Richardson Jr., John. "Open versus Closed Ended Questions." UCLA Department of Information Studies. N.p., n.d. Web. 7 Nov. 2010. <http://polaris.gseis.ucla.edu/jrichardson/dis220/openclosed.htm>.
  8.  "open ended question definition." BusinessDictionary.com - Online Business Dictionary. N.p., n.d. Web. 7 Nov. 2010. <http://www.businessdictionary.com/definition/open-ended-question.html>.
  9. "SurveyMonkey Help Center - Tips for writing effective and relevant survey questions.." SurveyMonkey Help Center - Support Home Page. N.p., n.d. Web. 7 Nov. 2010. <http://help.surveymonkey.com/app/answers/detail/a_id/134/~/tips-for-writing-effective-and-relevant-survey-questions.>.
  10. Driscoll, Dana Lynn; Brizee, A. "Creating Good Interview and Survey Questions." Purdue Owl: Conducting Primary Research. N.p., 17 Apr. 2010. Web. 7 Nov. 2010. <owl.english.purdue.edu/owl/resource/559/06/>.
  11. Kevin Jones.  (2008, July). Got time to answer a few questions? B & T Weekly, 17. Retrieved November 14, 2010, from ABI/INFORM Trade & Industry. (Document ID: 1512294071).
  12. Bradburn, Norman M., Seymour Sudman, and Brian Wansink. Asking questions: the definitive guide to questionnaire design : for market research, political polls, and social and health questionnaires. Rev. ed. San Francisco: Jossey-Bass, 2004. Print.
  13. Seidman, Irving. "Why Interview?." Interviewing as qualitative research: a guide for researchers in education and the social sciences. 3rd ed. New York: Teachers College Press, 2006. 8. Print.
  14. Fowler, Floyd J.. Improving survey questions: design and evaluation. Thousands Oaks, Calif.: Sage, 1995. 3-4. Print.
  15. Payne, S. (1951). The are of asking questions. Princeton, NY: Princeton University Press.
  16. "Writing Effective Survey Questions." 360 Degree Feedback, Employee Engagement Surveys, Talent Management Solutions - CustomInsight. N.p., n.d. Web. 7 Nov. 2010. <http://www.custominsight.com/articles/effective-survey-questions.asp>.
  17. Rupp, C.; , "Requirements and psychology," Software, IEEE , vol.19, no.3, pp.16-18, May/Jun 2002. doi: 10.1109/MS.2002.1003447. URL: http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=1003447&isnumber=21654
  18. Robertson, Suzanne, and James Robertson. Mastering the requirements process . 2nd ed. Upper Saddle River, NJ : Addison-Wesley, 2006. Print.
  19. Oriola, Steve . "8 Tips for Writing Effective Survey Questions." Email Marketing Solutions from Constant Contact. N.p., n.d. Web. 7 Nov. 2010. <http://www.constantcontact.com/learning-center/hints-tips/ht-2007-11os.jsp>.
  20. Figure 1. "SDLC Image." blog.bksanders.com. N.p., 1 July 2010. Web. 7 Nov. 2010. <blog.bksanders.com/wp-content/uploads/2010/07/sdlc-big.jpg>.
  21. Figure 2. "Color Palette." survey.cvent.com. N.p., 1 June 2010. Web. 7 Nov. 2010. <http://cdn.content.compendiumblog.com/uploads/user/b8bbc9ab-67b1-4a8e-ac5c-5811daa967bd/7cb46e13-4ccd-4f4d-b53b-268da9c29b3e/Image/3c351c23db89396f015447041b51fd8a_w640.jpeg>
  22. Figure 3."Pocket Watch Image". rosettasister.files.wordpress.com. N.p., 1 Sept. 2008. Web. 7 Nov. 2010. <http://rosettasister.files.wordpress.com/2008/09/time_is_of_the_essence98142218_std.jpg>