This Capability Area covers the following 7 Roadmaps:
- Development Process,
- Development Techniques,
- Internal Documentation,
- Verification & Validation,
- Configuration Management,
- Re-Use, and
- Reliability Management
It addresses most of the processes, procedures and activities central to
product development.
Implementation activities (e.g., construction, coding) are conducted
according to documented procedures [ISO 9000-3 5.6.3] [Trillium].
Adequate resources and funding are provided for performing the engineering
tasks [SEI SPE Ability 1] [ISO 9001 4.1.2.2] [Trillium].
The project follows a written organizational policy for performing
development activities [SEI SPE Commitment 1] [Bellcore TR-NWT-000179 2.5.1-1
& 3.4.2-5,6,7,8,9,10 & 3.7.2-2].
The organization's standard product development process defines the development methods to be used and the intermediate products generated during each development phase [ISO 9001 4.4.1] [ISO 9000-3 5.4.2.1, 5.4.3, 5.4.5 & 5.6.1] [Bellcore TR-NWT-000179 3.4-1 & 3.4.2-5 & 3.6-1 & 4.1.1-3 & 4.6-2,7] [Trillium] (Ref. IEEE Std. 1074).
The system requirements specification is reviewed and agreed formally [SEI
SPE Activity 2].
The design specification is developed formally [SEI SPE Activity 3] [ISO 9001
4.4.4] [ISO 9000-3 5.6.2] [Bellcore TR-NWT-000179 3.6.1-1 & 3.6.2-1].
Code is developed formally according to a documented procedure to ensure that
the design is met [SEI SPE Activity 4] [ISO 9000-3 5.6.3] [Bellcore
TR-NWT-000179 3.6.1-1,2].
The product development process requires the development of a well-defined
and consistent product architecture
[Roadmap Trillium 6.1.3.7].
The organization defines a group that owns and is accountable for the
architecture of each product or product line throughout its entire life-cycle
[Bellcore TR-NWT-000179 3.4.2-6,7,8,9] [Trillium].
A process model is maintained and used to evaluate the effect of process
changes on the organization
[Roadmap Trillium 6.1.3.2].
A process model is maintained and used to evaluate the effect of process
changes on customer satisfaction
[Roadmap Trillium 6.1.3.2].
State of the practice high-level languages (e.g., PROTEL, C, Pascal, Ada) are
used appropriately for all software products [Bellcore TR-NWT-000179 2.5.1-1]
[Trillium].
Analysis and design (e.g., SA/SD, OOA/OOD) are done according to a documented
procedure [ISO 9001 4.4.1] [Trillium].
The module, process and data dependencies are documented for the project
[Bellcore TR-NWT-000179 3.6.2-1] [Trillium].
Analysis and design (e.g., SA/SD, OOA/OOD,) are done formally according to a
documented procedure [ISO 9000-3 5.6.2] [Bellcore TR-NWT-000179 3.6.2-1 &
3.10.3-4] [Trillium].
Modelling techniques are used to assess product performance during the
product development process [Trillium].
Prototyping techniques are used to verify the correctness of critical
specification items (e.g., user interface specification) with the customer
[Trillium].
Prototyping techniques are used to explore design options and validate design
decisions prior to implementation [Trillium].
Code skeletons are automatically generated from designs by CASE tools
[Roadmap Trillium 6.2.3.1].
Source code metrics (e.g., code complexity) are taken and compared to a
predefined set of permissible values according to a documented procedure
[Trillium].
Formal methods (e.g., VDM, Z, LOTOS) are used for the development of critical
software components
[Roadmap Trillium 6.2.3.1].
Proof of correctness is performed for critical software components
[Roadmap Trillium 6.2.3.1].
CASE tools are used to automatically generate prototypes and design
specifications
[Roadmap Trillium 6.2.3.5].
Design complexity is measured and compared to the organizational baseline
according to a documented procedure [Trillium].
Constructive development methods (i.e., methods in which concrete
intermediate products are constructed from abstract intermediate products)
are used for most development tasks [Trillium].
The majority of source code (>80%) is automatically generated by CASE
tools [Trillium].
Executables are automatically generated by CASE tools [Trillium].
Requirements are documented according to a documented procedure [ISO 9001
4.4.1] [Bellcore TR-NWT-000179 3.4.2-9 & 4.2-1] (Ref. IEEE Std. 1016).
Design is documented according to a documented procedure [Bellcore
TR-NWT-000179 3.4.2-9 & 4.2-1] (Ref. IEEE Std. 1016).
Code is documented according to a documented procedure [Bellcore
TR-NWT-000179 3.4.2-9 & 4.2-1] [Trillium].
Test cases are documented according to a documented procedure [Bellcore
TR-NWT-000179 3.4.2-9 & 3.7.2-5 & 4.2-1] (Ref. IEEE Std. 829).
The documentation that will be used to operate and maintain the
product/service is developed and maintained according to the project's
defined process [SEI SPE Activity 8] [ISO 9001 4.5.1, 4.5.2] [Bellcore
TR-NWT-000179 3.4.2-9 & 3.7.2-2 & 4.2-1] [Trillium].
Requirements are documented formally
[Roadmap Trillium 6.3.2.1].
The design is documented formally
[Roadmap Trillium 6.3.2.2].
The rationale for all aspects of design is formally documented [Trillium].
Code is documented formally
[Roadmap Trillium 6.3.2.3].
Test cases are documented formally according to a documented procedure
[Bellcore TR-NWT-000179 3.7.2-3] [Trillium].
Conformance to requirement, design, coding, and test case standards is
verified by a peer review process [ISO 9001 4.4.7] [Trillium].
The standard for requirements and design documents specifies the use of
structured and formal notations where appropriate [Trillium].
When the product is modified (e.g., to correct a defect or add a feature) the
requirements and design documents for affected units are updated if needed to
reflect the change [Trillium].
The verification and validation activities (e.g., testing, peer reviews) are
formally planned and the plans are documented according to a documented
procedure [SEI PR Activity 1] [ISO 9001 4.4.1, 4.4.8, 4.10.1, 4.11] [ISO
9000-3 4.1.1.2.2, 5.4.6, 5.7.1, 5.7.2] [Bellcore TR-NWT-000179 2.1.2-1 &
3.7-1 & 3.7.1-6 & 3.7.2-6 & 3.8-2] [Trillium] (Ref. IEEE Std.
1012).
Adequate resources and funding are provided for performing peer reviews on
each work product to be reviewed [SEI PR Ability 1] [ISO 9001 4.1.2.2] [ISO
9000-3 4.1.1.2.2, 5.7.2] [Bellcore TR-NWT-000179 3.7.1-6 & 3.8-5].
Peer reviews are conducted on intermediate work products [ISO 9000-3 4.10.2,
5.4.6, 5.6.4] [Bellcore TR-NWT-000179 3.4.2-12 & 3.6.2-1& 3.7.2-6]
[Trillium].
System and acceptance testing are planned and performed formally to
demonstrate that the product satisfies its requirements [SEI SPE Activity 7]
[ISO 9001 4.4.1, 4.10.3, 4.10.4, 4.10.5] [ISO 9000-3 5.7.1, 5.7.3 & 5.8]
[Bellcore TR-NWT-000179 3.7.1-1,8 & 3.7.2-7 & 3.8-1] (Ref. IEEE Std.
1012).
Where testing under field conditions is required, the following are
considered: the features to be tested, responsibilities of both parties, and
post-test restoration of the user environment [ISO 9001 4.10.3, 4.10.4, 4.11]
[ISO 9000-3 5.7.1, 5.7.3, 5.7.4, 5.7.5] [Bellcore TR-NWT-000179 3.7.1-1,8
& 3.8-3] [Trillium].
Product areas that are impacted by any modifications are identified and
retested [ISO 9001 4.10.3, 4.13.2] [ISO 9000-3 5.7.1, 5.7.3] [Bellcore
TR-NWT-000179 3.7.1-1,4,8 & 3.7.2-3] [Trillium].
All testing activities are performed according to the project's defined
process [SEI SPE Activity 5] [ISO 9001 4.9, 4.11, 4.12] [ISO 9000-3
4.1.1.2.2, 5.4.6, 5.7.2] [Bellcore TR-NWT-000179 3.7.1-3,5,8,9 & 3.7.3-1
& 3.7.4.1,2 & 3.7.6-1 & 3.8-2 & 3.10.3-4] [Trillium].
Test documentation, test cases & testing procedures (e.g., unit,
integration, system & regression testing) are developed and documented
according to a documented procedure [Bellcore TR-NWT-000179 3.7.4.1,2] [ISO
9001 4.11] (Ref. IEEE Std. 1012 1.2(4), 3.5(1, 2 & 3)) (Ref. IEEE Std.
829) [Trillium].
The application of test cases, principally for regression testing and where
practical, is automated [ISO 9000-3 4.1.1.2.2] [Bellcore TR-NWT-000179
3.7.1-2,8] [Trillium].
Integration testing activities are planned and performed according to the
project's defined process [SEI SPE Activity 6] [Bellcore TR-NWT-000179
3.7.1-1,7,8 & 3.7.3-1] [Trillium].
Peer reviews are performed on all intermediate products according to a
documented procedure [SEI PR Activity 2] [ISO 9001 4.4.6] [ISO 9000-3 5.4.6]
[Bellcore TR-NWT-000179 3.4.2-12& 3.6.2-1] (Ref. IEEE 1012 figure 1 &
table 1, Std. 1028) [Trillium].
Information on the results of peer reviews is recorded in an organizational
repository [SEI PR Activity 3].
Data on defects identified during peer reviews, integration testing, system
testing, operational testing and defects reported by customers are collected,
documented, analyzed and tracked to closure [SEI SPE Activity 9] [ISO 9001
4.12, 4.13.1] [ISO 9000-3 5.7.3] (Ref. IEEE Std. 1012 3.5(3), 3.6 & 3.7)
[Trillium].
Verification and validation efficiency is measured and tracked according to a
documented procedure [ISO 9001 4.12] [ISO 9000-3 5.7.3] [Bellcore
TR-NWT-000179 3.7.1-9 & 3.7.2-1,4 & 3.7.4-3] [Trillium].
Verification and validation adequacy (e.g., test coverage) is measured and
tracked according to a documented procedure [ISO 9001 4.4.7, 4.4.8, 4.12]
[ISO 9000-3 5.7.3] [Bellcore TR-NWT-000179 3.7.1-9 & 3.7.2-1,4]
[Trillium].
The generation of test cases is automated from requirements and designs
[Trillium].
Verification and validation efficiency goals are defined and monitored
[Trillium].
Verification and validation adequacy (e.g., test coverage,% NCSS inspected)
goals are defined and monitored [Trillium].
Defect density is projected and compared with actuals [Trillium].
Source code is under Configuration Management (CM) control [Bellcore
TR-NWT-000179 4.1.1-1 & 4.1.2-2,3] [Trillium].
All project and product (internal and external) documents are under CM
control [ISO 9001 4.5.1, 4.5.2, 4.5.3] [ISO 9000-3 6.2.1, 6.2.3 & 6.2.4]
[Bellcore TR-NWT-000179 4.1.1-1,3 & 4.1.2-1].
A board having the authority for managing the project's product baselines
(i.e., a product configuration control board - PCCB) exists or is established
[SEI SCM Ability 1] [Trillium].
There is a function responsible for coordinating and implementing CM for the
project [SEI SCM Ability 2] [IEEE Std. 828 2.2] [Trillium].
Adequate resources and funding are provided for performing the CM activities
[SEI SCM Ability 3] [ISO 9001 4.1.2.2] [Bellcore TR-NWT-000179 4.1.3-5]
[Trillium].
A CM plan is prepared for each project according to a documented procedure
[SEI SCM Activity 1] [ISO 9000-3 6.1.1, 6.1.2, 6.1.3.1, 6.1.3.2] [Bellcore
TR-NWT-000179 3.7.3-2 & 3.8-4 & 4.1.3-5 & 4.6-7] [Trillium].
A documented and approved CM plan is used as the basis for performing the CM
activities [SEI SCM Activity 2] [ISO 9001 4.5.3, 4.8] [ISO 9000-3 6.1.1,
6.1.2, 6.1.3.1, 6.1.3.2] [Bellcore TR-NWT-000179 4.1.3-5] [Trillium].
A CM system is established as a repository for product baselines [SEI SCM
Activity 3] [ISO 9000-3 6.1.3.1, 6.1.3.3].
The product repository ensures secure storage of configuration items (e.g.,
code units, design documents) and the secure and controlled retrieval of
current and previous versions of configuration items [SEI SCM Activity 4]
[Bellcore TR-NWT-000179 3.7.3-2 & 4.1.1-1,2 & 4.1.3-7 &
4.1.4-1,3].
The product repository ensures the secure and controlled retrieval of current
and previous baselines [Bellcore TR-NWT-000179 4.1.4-1,2,3] [Trillium].
The status of configuration items/units is recorded according to a documented
procedure [SEI SCM Activity 8] [ISO 9000-3 6.1.3.1, 6.1.3.3] [Bellcore
TR-NWT-000179 4.1.3-2,3].
The product repository maintains records of the status and change history of
all configuration items and baselines [Bellcore TR-NWT-000179 4.1.3-2,3,4]
[Trillium].
There is traceability between design specifications and source code and also
between design specifications and integration test cases [ISO 9001 4.8] [ISO
9000-3 6.1.3.1] [Bellcore TR-NWT-000179 3.7.3-2 & 3.8-4 & 4.1.3-4]
[Trillium].
Change requests and problem reports for all configuration items/units are
initiated, recorded, reviewed, approved and tracked according to a documented
procedure [SEI SCM Activity 5] [ISO 9001 4.5.3] [ISO 9000-3 6.1.1, 6.1.3.3]
[Bellcore TR-NWT-000179 4.1.1-1,3 & 4.1.2-1] [IEEE Std. 828 2.3.2].
Configuration items and baselines are changed formally according to a
documented procedure [SEI SCM Activity 6] [ISO 9001 4.5.3] [ISO 9000-3 6.1]
[Bellcore TR-NWT-000179 3.7.3-3 & 4.1.1-1,2,3 & 4.1.2-1] (Ref. IEEE
Std. 730 3.4.2.6) (Ref. IEEE Std. 828 & 1052).
Baseline(s) are created and released formally [SEI SCM Activity 7] [Bellcore
TR-NWT-000179 4.1.3-1].
Product baseline audits are conducted according to a documented procedure
[SEI SCM Activity 10] [Trillium].
Standard reports documenting the CM activities and the contents of the
product baselines are developed and made available to affected groups and
individuals [SEI SCM Activity 9] [Trillium].
Plans, descriptions, product test procedures, requirements specifications,
design specifications, review results and test cases (e.g., integration,
system, operation) are under CM control [SEI SPE Activity 10] [ISO 9001
4.11] [Bellcore TR-NWT-000179 3.7.3-1] [Trillium].
All development tools are under CM control [Trillium].
There is full forward and backward traceability between all configuration
items (e.g., design specification forward to code units, design specification
backward to requirement specification) [Bellcore TR-NWT-000179 3.7.2-7] (Ref.
IEEE Std. 1012).
The complete development history (e.g., design decisions, design rationale)
is captured and maintained under CM control
[Roadmap Trillium 6.5.3.1].
This Roadmap covers practices that are related to reuse. A few practices have
been taken from the "Reuse Adoption Guidebook", SPC-92051-CMC,
version 02.00.05 November 1993.
Cloning (i.e., copying and modifying) of existing software units in new
designs is tracked [Trillium].
Tools are provided to aid propagation of changes from one software unit to
cloned units or to cloned from units [Trillium].
Third-party components are selected, verified, validated and tracked formally
according to a documented procedure [ISO 9000-3 6.8] [Bellcore TR-NWT-000179
4.6-7 & 4.8-1] [Trillium].
Standard reuse processes are defined and integrated with the organization's
standard development process [SPC-92051-CMC PI-1] [Trillium].
Performance of the standard reuse processes is measured [SPC-92051-CMC CI-1]
[Trillium].
Management and staff are aware of new and evolving technologies and standards
that may affect their products and reusable assets [SPC-92051-CMC TI-1].
Pre-developed and certified source code templates are provided in a template
repository as a basis for cloning [Trillium].
Pre-developed and certified source code units are provided in a software
component repository [Trillium].
A component development function exists to develop, certify and maintain the
items in the template and component repository [Trillium].
Specific steps are included in the organization's standard product
development process for the purpose of maximizing component re-use
[Trillium].
Re-use of pre-developed components is measured, encouraged, and rewarded by
the organization's reward system [Trillium].
Product line reuse strategies are developed to maximize the benefits of reuse
over sets of related products [SPC-92051-CMC PD-2].
Plans are established to systematically address weaknesses identified in the
standard reuse processes [SPC-92051-CMC CI-2].
Pre-developed and certified designs are provided in a software component
repository for the use of developers [Trillium].
CASE tools are provided to support re-use of software components in product
developments [Trillium].
Product costing and funding strategies take into account expected costs and
anticipated benefits of reuse over the product line [SPC-92051-CMC CP-3].
Knowledge based CASE tools are provided to automate re-use of components in new product developments [Trillium].
Product and service reliability and maintainability plans and objectives are
established [Trillium] [IEC 300].
Product and system availability criteria are defined during the preparation
of the requirements specification. They include but are not limited to
failure definition/classification (e.g., critical, major, minor) and
operational modes definition (e.g., nonstop, mission-oriented, batch)
[Trillium] [IEC 300].
Software availability criteria are defined during the preparation of the
requirements specification. They include but are not limited to product or
system availability during enhancements and upgrades, failures, and the
maximum number of failures allowed for the various classes of failure
[Trillium].
The following data is collected and analyzed prior to product release (Ref.
IEEE Std. 982.1 & 982.2):
- Test Log (Test Execution Time),
- Failure Log (including failure classification), and
- Cumulative Execution Time per release.
The following field data is collected and analyzed (Ref. IEEE Std. 982.1
& 982.2):
- Cumulative Execution Time per release.
- Failure-related down time,
- Enhancement-related down time.
The Operational Profile is determined during the preparation of the
requirements specification. The profile includes I/O Classes Definition and
I/O Distribution per Class [Trillium].
Product design guidelines consistent with reliability objectives are
established and enforced [Trillium] [IEC 300].
FMECA and Fault Tree Analysis are performed on all critical modules (H/W,
S/W) as part of the design activities and are part of the design reviews
[Trillium] [IEC 300].
Reliability modelling influences the selection of design alternatives for
non-software sub-systems, and at system level for software [Trillium] [IEC
300].
Reliability growth models are used for resources and/or schedule estimation
using past project parameters (size, initial failure intensity and intensity
decay parameters). These models are continually updated [Trillium].
Field reliability estimation is based on the determination of the testing
acceleration factor and the reliability growth model [Trillium].
Reliability modelling influences software design alternative selection from
both static code properties and dynamic code behavior data (in-line with
re-use philosophy) i.e., Reliability Block Diagram, Markov modelling, Fault
Tree Analysis, Strength-Stress Analysis, Stochastic Petri-Net Modelling
[Trillium] [IEC 300].
![[*]](up.gif)
Back to the Trillium home page.
![[TOC]](toc.gif)
To the detailed Trillium table of contents.
![[TOC]](back.gif)
Back to capability area 5.
![[TOC]](fwd.gif)
Forward to capability area 7.
![[UHCL]](uhcl-stamp.gif)