The Capability Maturity Model and
ISO 9000 Standards: Similarities and Differences

Table of Contents

Introduction

The Capability Maturity Model (CMM) and the ISO 9000 series of standards share a common concern with quality and process management. The two were created under the notion that to create a quality product, the underlying organization must be a quality driven. The purpose of this report will be to introduce the Capability Maturity Model and ISO 9000 as each relates to software development (ISO 9001), indicating their differences and similarities.

Overview of CMM

The Capability Maturity Model (CMM) is made up of five hierarchical levels that are used to grade an organization's ability to consistently and predictably create high quality software.[1] CMM does this by accessing the extent to which an IT organization uses predictable, manageable processes for building information systems. "By using record keeping and assessment tools based on this model, an IT organization can determine how its processes compare to a theoretical ideal and can see how quickly it is moving toward that ideal."[2] The underlining premise of CMM is that an organization that does not have a defined and standardized software development process is unable to provide a consistent and reliable product. The largest proponent of the Capability Maturity Model is the U.S. Department of Defense (DoD), which uses the model as a baseline to determine whether any specific organizations is qualified to bid on a US Military contract. Incidentally, the Software Engineering Institute (SEI), the author of CMM, was established by, receives its funding and mission from the DOD [3]

The purpose of CMM is to help organizations that produce software, improve the maturity of their software processes. The improvement is conceptualized as an evolutionary path from an AD-HOC, chaotic stage, to mature, disciplined software process. Each level within the CMM framework is referred to as a 'maturity level.' Each maturity level consists of several key processes areas (KPA) that identify requirements/best practices that an organization would need to demonstrate, to be graded at any specific maturity level. Please review Table 1 for detailed information regarding the maturity level framework of CMM.


Table 1 [4]
Maturity Level
Rating Description KPAs...
5 Optimizing Continuous process improvement is enabled by quantitative feedback from the process and from piloting innovative ideas and technologies. cover the issues that both the organization and the process must address to implement continual, measurable software process improvement. The KPAs are:
  • Defect Prevention
  • Technology Change Management
  • Process Change Management.
4 Managed Detailed measures of the software process and product quality are collected. Both the software process and products are quantitatively understood and controlled. focus on establishing a quantitative understanding of both the software process and the software work products being built. The KPAs are:
  • Quantitative Process Management
  • Software Quality Management
3 Defined The software process for both management and engineering activities is documented, standardized and integrated into standard software processes for the organization. All project use an approved, tailored version of the organization's standard software process for developing and maintaining software. address both project and organizational issues, as the organization establishes an infrastructure that institutionalizes effective software engineering and management processes across all projects. The KPAs are:
  • Organization Process Focus
  • Organization Process Definition
  • Training Program
  • Integrated Software Management
  • Software Product Engineering
  • Intragroup Coordination
  • Peer Reviews
2 Repeatable Basic project management processes are established to track cost, schedule, and functionality. The necessary process discipline is in place to repeat earlier successes on project with similar applications. Focus on the software project's concerns related to establishing basic project management controls. The KPAs are:
  • Requirements Management
  • Software Project Planning
  • Software Project Tracking and Oversight
  • Software Subcontract Management
  • Software Quality Assurance
  • Software Configuration Management
1 Initial The software process is characterized as ad-hoc, and occasionally even chaotic. Few processes are defined, and success depends on individual effort and heroics. None

Common Features of KPAs

KPAs are organized into a set of five common features "that help indicate whether the implementation and institutionalization of a key process area is effective, repeatable, and lasting." [5] The five common features of all KPAs are: Commitment to Perform; Ability to Perform; Activities Performed; Measurement and Analysis; and Verify Implementation. See Table 2 for detailed information about the five common features of KPAs.


Table 2 [5]
Common Feature Description
Commitment to Perform Describes the actions the organization must take to ensure that the process is established and will endure. Includes practices on policy and leadership.
Ability to Perform Describes the preconditions that must exist in the project or organization to implement the software process competently. Includes practices on plans, procedures, work performed, tracking, and corrective action.
Activities Performed Describes the roles and procedures necessary to implement a key process area. Includes practices on plans, procedures, work performed, tracking, and corrective action.
Measurement and Analysis Describes the need to measure the process and analyze the measurements. Includes examples of measurements.
Verifying Implementation Describes the steps to ensure that the activities are performed in compliance with the process that has been established. Includes practices on management reviews and audits.

CMM Assessment Process

An organization achieves a specific maturity level by means of an assessment, initiated by the organization wishing to be assessed. The assessment can be conducted by the organization being assessed or by an independent agency (SEI or SEI-licensed assessment vendor). The assessment provides feedback to the organization regarding its current software development capabilities and trains the organization on ways to improve its capabilities.[6]


Table 3 [6]
Phase Description
Selection The Organization is identified as an assessment candidate, and the qualified assessing organization conducts an executive-level briefing.
Commitment The organization commits to the full assessment process whereby CEO signs an assessment agreement.
Preparation The organization's assessment team receives training, and the on-site assessment process is fully planned. All assessment participants are identified and briefed. The maturity questionnaire is completed by the organization.
Assessment The On-site assessment is typically conducted in a week. The assessment team then meets to formulate preliminary recommendations.
Report The entire assessment team helps prepare the final report and present it to the organizations assessment participants and senior management. The report includes team findings and recommendations for actions.
Assessment Follow-up The assessed organization's team, with guidance from the independence assessment organization, formulates an action plan. After approximately 18, months, it is recommended that the organization have a reassessment in order to assess progress and sustain the software process improvement cycle.

An assessment conducted by a SEI-certified organization logically would be viewed to be more credible and objective that of a self-assessment. However, in a paper by Goldenson and Herbsleb with SEI, found that there is "evidence that people in fact try to answer survey questions honestly." The study, "After the Appraisal: A systematic Survey of Process Improvement, its Benefits, and factors that Influence Success, found 'little difference between the appraised and reported maturity levels, " of submitted information provided to SEI by organizations that had been previously assessed by SEI from 1992 to 1993.[7] However, an additional explanation could be that organizations that show significant statistical difference between an appraised review and a self reported review might be singled out for further study.

Benefits of CMM on Software Process Improvement


Table 4 [8]
Category Range Median
Years involved in CMM 1 - 9 3.5
Yearly Cost per Engineer $490 - $2004 $1375
Productivity Gain/Year 9% - 67% 35%
Early Defect detection gain per year 6% - 25% 22%
Yearly reduction in time to market 15% - 23% 19%
Yearly Reduction in post-release defect reports 10% - 94% 39%
Business Value 4.0 - 8.8 5.0

ISO 9000 Overview

ISO 9000 is a family of standards, published by the International Organization for Standardization, is a set of five individual, but related international standards on quality management and quality assurance. They are generic and not specific to any particular product or service. ISO provides a certification process to organizations whereby stating to other organization that the certified organization has a Quality Management System in place and that the organization adheres to this system in conducting business. Please refer to Table 5 to a brief review of the ISO 9000 family of standards.


Table 5 [9]
ISO Standard Title Description
ISO 9000 Quality Management and Quality Assurance Standards--Guidlines for Selection and Use Guidelines for the selection and use of ISO 9001, 9002 and 9003.
ISO 9001* Quality Systems--Model for quality assurance in design/development, production, installation and servicing. Standard covers design, development, production, installation, and servicing, this applies to the software industry.
ISO 9002 Quality systems--Model for quality assurance in production and installation. Assesses the production and installation processes.
ISO 9003 Quality systems--Model for quality assurance in final inspection and test. Evaluation the final inspection and test phase.
ISO 9004 Quality management and quality system elements--Guidlines Defines the 20 fundamental quality system concepts included in the three models.

*Of the ISO 9000 series, ISO 9001 is the standard most pertinent to software development and maintenance.[10]

ISO-9000 Standards generally are only used by organizations when: [10]

An organization can choose to be certified against one of the three quality systems in ISO 9000: ISO 9001, ISO 9002 and ISO 9003. The organization undergoing certification chooses which standard they wish to pursue based on the organizations business processes. None of the standards are considered more important than any one of the others and the organization that would undergo certification would need to identify which standard before pursuing certification. Once an organization chooses to undergo certification, a certified register audits the company to determine compliance with the applicable standard.

Please refer to Table 5, for a typical timetable for an organization preparing for certification.


Table 6 [11]
Time Period Action(s)
Months 1-3 Organization commits at a board meeting to be certified within 14 months. Employees are instructed to familiarize themselves with the ISO 9000 concepts.
Month 4 Organization hires a certified quality auditor as a consultant to help prepare the company.
Month 5 Consultant spends one week talking to groups of 17 to 40 people about how they do their jobs and about existing company quality programs. Consultant submits 26-page document of findings and recommendations.
Month 6-8 Organization records descriptions of quality procedures and adds other procedures as recommended by the auditor.
Month 9 Organization completes all written materials and mails them to the auditing firm as a manual.
Month 10 Auditor reviews the manual, makes suggestions, and schedule an on-site visit in two to four months.
Month 11 Organization makes more changes as recommended by the auditor.
Month 12 Auditor makes an on-site visit for up to three days, concluding with a conference to discuss concerns and to listen to the organizations explanations. Auditor either grants or denies ISO 900 certification.

Only a certified register can award ISO certification. To be a certified register, an organization must be accredited and its auditors must be certified. In the United States, the sole accrediting organization is the Register Accreditation Board.

Similarities and Differences

The Similarities of the Capability Maturity Model to ISO-9000 Standards [12]

The Differences Between the Capability Maturity Model to ISO-9000 Standards [12]


Table 7 [13]
ISO CMM
Minimum requirements with implied continuous improvement Explicit Continuous Quality Improvement
Not specific to any one industry or service Software specific
Outwardly focused from the firm Inwardly focused to the firm
Registration Document No Documentation
Continual Audits No follow up audits

Conclusion

In many ways it is very difficult for the casual reader to get beyond the notion of thinking of CMM or ISO-900 Standards as nothing more than a set of prescription's you endure to get a participate with so to get a rating or certification. However, to accept this would be to completely misunderstand the CMM or ISO 9000 models. In a paper sponsored by the State of Washington regarding the Capability Maturity Model, the state said, "CMM is the foundation for systematically building a set of tools, including a maturity questionnaire, which is useful in software process improvement. The essential point to remember is that the model, not the questionnaire, is the basis for improving the software process." (http://www.wa.gov/) In Mark C. Paulk with SEI's review of the differences and similarities of ISO and CMM, he noted first off, both "have a common concern of quality and process management." So, in essence both models are concerned with that to produce quality in software, whether implicitly stated or not, the underlying organizations processes and systems are what dictates whether an organization can deliver a quality software product. That, is much must be considered and reviewed first before an organization can ever hope to produce repeatable, quality driven results.

Bibliography


[1]

Saiedian, Hossein and Richard Kuzara, "SEI Capability Maturity Model's Impact on Contractors," IEEE Computer, January 1995, p. 16-25.

 

[2]

Alter, Steven. Information Systems: A Management Perspective. 3rd ed. New York: Addison-Wesley, 1999. p. 428.

 

[3]

Carnegie Mellon University-Software Engineering Institute: About the SEI-Welcome: (http://www.sei.cmu.edu/about/about.html)

 

[4]

Carnegie Mellon University-Software Engineering Institute: Capability Maturity Model for Software (SW-CMM): (http://www.sei.cmu.edu/cmm/cmm.sum.html)

 

[5]

Paulk, Mark C., "A Comparison of ISO 9001 and the Capability Maturity Model for Software." Software Engineering Institute, July 1994.

 

[6]

Saiedian, Hossein and Richard Kuzara, "SEI Capability Maturity Model's Impact on Contractors," IEEE Computer, January 1995, p. 16-25.

 

[7]

Goldenson, Dennis R., and James D. Herbsleb., "After the Appraisal: A Systematic Survey of Process Improvement, its Benefits, and Factors that Influence Success." Software Engineering Institute, August 1995.

 

[8]

Herbsleb, James., Anita Carleton, James Rozum, Jane Siegel and David Zubrow., "Benefits of CMM-Based Software Process Improvement: Executive Summary of Initial Results." Software Engineering Institute. September 1994.

 

[9]

Dawood, Mark., "It's Time for ISO 9000." CrossTalk. (http://www.stsc.hill.af.mil/CrossTalk/1994/mar/xt94d03i.asp). October 22, 2001.

 

[10]

Paulk, Mark C., "How ISO 9001 Compares with The CMM." Software Engineering Institute, January 1995.

 

[11]

Brokaw, Leslie,"ISO 9000: Making the Grade," INC, June 1993.

 

[12]

Craft, Dave. "ISO-CMM: Similarites, Differences." Slides 16 and 17. (http://www.umsl.edu/~sauter/analysis/cmm_iso/iso-cmm 11-22/index.htm)