One or Two page paper
kevin_225
M A N A G I N G THE D E V E L O P M E N T OF LARGE S O F T W A R E SYSTEMS
Dr. Winston W. Rovce
I N T R O D U C T I O N l am going t o describe my pe,-.~onal views about managing large software developments. I have had
various assignments during the past nit,.: years, mostly concerned w i t h the development of software packages
for spacecraft mission planning, commanding and post-flight analysis. In these assignments I have experienced
d i f f e r e n t degrees o f successwith respect to arriving at an operational state, on-time, and w i t h i n costs. I have
become prejudiced by my experiences and I am going to relate some o f these prejudices in this presentation.
COMPUTER PROGRAM D E V E L O P M E N T F U N C T I O N S There are t w o essential steps c o m m o n to all c o m p u t e r program developments, regardless o f size or
c o m p l e x i t y . There is first an analysis step, f o l l o w e d second by a coding step as depicted in Figure 1. This sort
of very simple i m p l e m e n t a t i o n concept is in fact all that is required if the e f f o r t is sufficiently small and if the
final product is to be operated by those w h o b u i l t it - as is t y p i c a l l y done w i t h c o m p u t e r programs for internal
use. It is also the kind of development e f f o r t for w h i c h most customers are happy to pay, since both steps
involve genuinely creative w o r k which d i r e c t l y contributes to the usefulness of the final product. A n
i m p l e m e n t a t i o n plan to manufacture 13rger software systems, and keyed o n l y to these steps, however, is d o o m e d
• t o f a i l u r e . Many additional development steps are required, none c o n t r i b u t e as d i r e c t l y to the final product as
analysis and coding, and all drive up the development costs. Customer personnel t y p i c a l l y w o u l d rather not pay
for them, and development personnel w o u l d rather not implement them. The prime f u n c t i o n of management
is to sell these concepts to b o t h groups and then enforce compliance on the part of development personnel.
A N A L Y S I S
C O D I N G
Figure 1. I m p l e m e n t a t i o n steps to deliver a small c o m p u t e r program f o r internal operations.
A more grandiose approach to software development is illustrated in Figure 2. The analysis and coding
steps are still in the picture, but they are preceded by t w o levels o f requirements analysis, are separated by a
program design step, and f o l l o w e d by a testing step. These additions are treated separately f r o m analysis and
coding because they are distinctly d i f f e r e n t in the way they are executed. They must be planned and staffed
d i f f e r e n t l y for best u t i l i z a t i o n of program resources.
Figure 3 portrays the iterative relationship between successive development phases for this scheme.
The ordering o f steps is based on the f o l l o w i n g concept: that as each step progresses and the design is further
detailed, there is an iteration w i t h the preceding and succeeding steps but rarely w i t h the more remote steps in
the sequence. The virtue of all o f this is that as the design proceeds the change process is scoped d o w n to
manageable limits. A t any p o i n t in the design process after the requirements analysis is completed there exists
a f i r m and c~seup~ moving baseline to whi(:h to ~ t u r n in the event o f unforeseen design difficulties. What we
have is an effective fallback position that tends to maximize the e x t e n t o f early w o r k that is salvageable and
preserved. Reprinted from Proceedings, IEEE WESCON, August 1970, pages 1-9. Co_pyright © 1_9_70 by The Institute of Electrical and Electronics Et)gineers,, .328 Inc. Originally published by TRW.
I SYSTE M
I ANALYSIS PROGRAM
DESIGN
I c o o , . o TESTING
I OPERATIONS
Figure 2. I m p l e m e n t a t i o n steps to develop a large computer program f o r delivery t o a customer.
I believe in this concept, but the i m p l e m e n t a t i o n described above is risky and invites failure. The
problem is illustrated in Figure 4. The testing phase which occurs at the end o f the development cycle is the
first event f o r w h i c h timing, storage, i n p u t / o u t p u t transfers, etc., are experienced as distinguished f r o m
analyzed. These phenomena are not precisely analyzable. They are not the solutions to the standard partial
differential equations of mathematical physics for instance. Yet if these phenomena fail to satisfy the various
external constraints, then invariably a major redesign is required. A simple octal patch or redo o f some isolated
code w i l l not f i x these kinds o f difficulties. The required design changes are likely to be so disruptive that the
software requirements upon which the design is based and w h i c h provides the rationale for everything are
violated. Either the requirements must be m o d i f i e d , or a substantial change in the design is required. In effect
the development process has returned to the origin and one can expect up to a lO0-percent overrun in schedule
and/or costs.
One might note that there has been a skipping-over o f the analysis and code phases. One cannot, of
course, produce software w i t h o u t these steps, but generally these phases are managed w i t h relative ease and
have little impact on requirements, design, and testing. In my experience there are w h o l e departments
consumed w i t h the analysis o f o r b i t mechanics, spacecraft a t t i t u d e determination, mathematical o p t i m i z a t i o n
of payload activity and so f o r t h , but when these departments have completed their d i f f i c u l t and c o m p l e x w o r k ,
the resultant program steps i n v o l v e a few lines of serial a r i t h m e t i c code. If in the e x e c u t i o n of their d i f f i c u l t
and complex w o r k the analysts have made a mistake, the correction is invariably implemented by a m i n o r
change in the code w i t h no disruptive feedback into the other development bases.
However, I believe the illustrated approach to be f u n d a m e n t a l l y sound. The remainder of this
discussion presents five a d d i t i o n a l features that must be added to this basic approach to eliminate most o f the
development risks.
329
I SYSTEM ! REQUIREMENTSIBI~
~ " ' i so,w.,~ I
ANALYSIS
~1111~ I I pRI~OGRAM
~ l l l I CODING Ii
TESTING
OPERATIONS
Figure 3. Hopefully, the ~terat=ve interact=on between the various phases is confined to successive steps.
I SYSTEM "1 .~oo,.~-,..Sl.,~ I so,w..~ !.
I ANALYSIS
PROGRAM DESIGN
I coo,.G I , ~ ! TESTING I
I O .ATO.S ! Figure 4. Unfortunately, for the process illustrated, the design iterations are never confined to the successive steps.
330
STEP 1: P R O G R A M D E S I G N COMES F I R S T
The first step towards a f i x is illustrated in Figure 5. A preliminary program design phase has been
inserted between the software requirements generation phase and the analysis phase. This procedure can be
criticized on the basis that the program designer is forced to design in the relative vacuum of initial software
requirements w i t h o u t any existing analysis..As a result, his preliminary design may be substantially in error as
compared to his design if he were to w a i t until the analysis was complete. This criticism is correct but it misses
the point. By this technique the program designer assures that the software w i l l not fail because of storage,
timing, and data f l u x reasons. As the analysis proceeds in the succeeding phase the program designer must
impose on the analyst the storage, timing, and operational constraints in such a way that he senses the
consequences. When he justifiably requires more of this kind o f resource in order to implement his equations
it must be simultaneously snatched f r o m his analyst compatriots. In this way all the analysts and all the
program designers w i l l c o n t r i b u t e to a meaningful design process w h i c h w i l l culminate in the proper allocation
of execution time and storage resources. If the total resources to be applied are insufficient or if the e m b r y o
operational design is wrong it w i l l be recognized at this earlier stage and the iteration w i t h requhements and
preliminary design can be redone before final design, coding and test commences.
How is this procedure implemented? The f o l l o w i n g steps are required.
1) Begin the design process w i t h program designers, not analysts or programmers.
2) Design, define and allocate the data processing modes even at the risk o f being wrong. Allocate
processing, functions, design the data base, define data base processing, allocate execution time, define
interfaces and processing modes w i t h the operating system, describe input and o u t p u t processing, and define
preliminary operating procedures.
3) Write an overview d o c u m e n t that is understandable, informative and current. Each and every
w o r k e r must have an elemental understanding o f the system. A t least one person must have a deep understand-
ing o f the system which comes partially f r o m having had to w r i t e an overview document.
/ ALLOCATE ~ A DESCRIBE / sO..oOO,,. / c %
I
Figure 5. Step 1 : Insure that a p r e l i m i n a r y program design is complete before analysis begins.
331
S T E P 2 : D O C U M E N T THE DESIGN
At this p o i n t it is appropriate to raise the issue of - " h o w much d o c u m e n t a t i o n ? " My own view is
" q u i t e a l o t ; " certainly more than most programmers, analysts, or program designers are w i l l i n g to do if left to
their own devices. The first rule of managing software development is ruthless enforcement of d o c u m e n t a t i o n
requirements.
Occasionally I am called upon to review the progress of other software design efforts. My first step is
to investigate the state of the d o c u m e n t a t i o n , If the d o c u m e n t a t i o n is in serious default my first
recommendation is simple. Replace project management. Stop all activities not related to d o c u m e n t a t i o n .
Bring the d o c u m e n t a t i o n up to acceptable standards. Management of software is simply impossible w i t h o u t a
very high degree of d o c u m e n t a t i o n . As an example, let me offer the f o l l o w i n g estimates for comparison. In
order to procure a 5 m i l l i o n dollar hardware device, I w o u l d expect that a 30 page specification w o u l d provide
adequate detail to control the procurement. In order to p r o c u r e 5 m i l l i o n dollars of software I w o u l d estimate
~ 1[,00 pa~e specification is about right in order to achieve comparable control,
Why so much d o c u m e n t a t i o n ?
1) Each designer must communicate w i t h interfacing designers, w i t h his management and possibly
w i t h thecustorner. A verbal record is t o o intangible to provide an adequate basis for an interface or manage-
m e n t d e c i s i o n . An acceptable w r i t t e n description forces the designer to take an unequivocal position and
provide tangible evidence o f c o m p l e t i o n . It prevents the designer from hiding behind t h e - " l a m 9 0 - p e r c e n t
f i n i s h e d " - syndrome m o n t h after month.
2) During the early phase of software development the d o c u m e n t a t i o n i . s t h e specification and i._~.s the
design. U n t i l coding begins these three nouns (documentation, specification, design) d e n o t e a s i n g t e t h i n g . If
the d o c u m e n t a t i o n is bad the design is bad. If the d o c u m e n t a t i o n does not yet exist there is as yet no design,
o n l y people t h i n k i n g and talking about the design w h i c h is of some value, but not much.
3) The real monetary value of good d o c u m e n t a t i o n begins downstream in the d e v e l o p m e n t process
during the testing phase and continues through operations and redesign. The value of d o c u m e n t a t i o n can be
described in terms of three concrete, tangible situations that every program manager faces.
a) During the testing phase, w i t h good d o c u m e n t a t i o n the manager can concentrate personnel on the
mistakes in the program. W i t h o u t good d o c u m e n t a t i o n every mistake, large or small, is analyzed by one man
w h o p r o b a b l y made the mistake in the first place because he is the o n l y man w h o understands the program area.
b) During the operational phase, w i t h good d o c u m e n t a t i o n the manager can use o p e r a t i o n - o r i e n t e d
personnel to operate the program and to do a better job, cheaper. W i t h o u t good d o c u m e n t a t i o n the software
must be operated by those w h o b u i l t it. Generally these people are relatively disinterested in operations and do
not do as effective a job as operations-oriented personnel. It should be p o i n t e d o u t in this connection that in
an operational situation, if there is some hangup the software is always blamed first. In order either to absolve
the software or to f i x the blame, the software d o c u m e n t a t i o n must speak clearly.
c) F o l l o w i n g initial operations, when system improvements are in order, good d o c u m e n t a t i o n permits
effective redesign, updating, and r e t r o f i t t i n g in the field. If d o c u m e n t a t i o n does n o t exist, generally the entire
existing f r a m e w o r k o f operating software must be junked, even for relatively modest changes.
Figure 6 shows a d o c u m e n t a t i o n plan w h i c h is keyed to the steps previously shown. Note that six
documents are produced, and at the time of delivery o f the final product, Documents No, 1, No. 3, No. 4,
No. 5, and No. 6 are updated and current.
332
/ ,
I0: w Z /oo i ,~ g ~
Irl . . o i 0 . i IIII ~,- ,,*,1 =
• . ~
illl ~ $ ~ m z ~ _ ~ u,
E
X
E 8
" 0
Ill N ~ , .~- r" .2
/ " z_ ,,,. ~ ~ E ~OLU
a . . ~
N
N I Z ,~,- w i - , < ~
t -
LL
333
STEP 3: DO IT TWICE A f t e r d o c u m e n t a t i o n , the second most i m p o r t a n t criterion for success revolves around whether the
product is t o t a l l y original. If the computer program in question is being developed for the first time, arrange
matters so that the version f i n a l l y delivered to the customer for operational d e p l o y m e n t is actually the second
version insofar as critical design/operations areas are concerned. Figure 7 iltustrates how this might be carried
out by means o f a simulation. Note that it is simply the entire process done in miniature, t o a t i m e scale that
is relatively small w i t h respect to the overall e f f o r t . The nature of this e f f o r t can vary w i d e l y depending
primarily on the overall time scale and the nature o f the critical problem areas to be modeled. If the e f f o r t runs
30 months then this early development o f a p i l o t model might be scheduled for 10 months. For this schedule,
fairly formal controls, d o c u m e n t a t i o n procedures, etc., can be utilized. If, however, the overall e f f o r t were
reduced to 12 months, then the p i l o t e f f o r t could be compressed to three months perhaps, in order to gain
sufficient leverage on the mainline development. In this case a very special kind o f broad competence is
required on the part of the personnel involved. T h e y must have an intuitive feel for analysis, coding, and
program design. T h e y must q u i c k l y sense the t r o u b l e spots in the design, model them, model their alternatives,
forget the straightforward aspects of the design which aren't w o r t h studying at this early point, and f i n a l l y
arrive at an error-free program. In either case the p o i n t of all this, as w i t h a simulation, is that questions of
timing, storage, etc. w h i c h are otherwise matters of judgment, can now be studied w i t h precision. W i t h o u t
this simulation the project manager is at the mercy of human judgment. With the simulation he can at least
perform experimental tests of some key hypotheses and scope d o w n w h a t remains for human judgment, which
in the area o f computer program design (as in the estimation of t a k e o f f gross weight, costs to complete, or the
daily double) is invariably and seriously optimistic.
I I,,, 1 I ANALYSIS I
! PROGRAM I I DES,GN I
- U coo,.o I LI .,s.,.o
USAGE
PRELIMINARY I % PROGRAM DESIGN
ANALYSIS
i PROGRAM DESIGN
TESTING
[ OPERATIONS Figure 7. Step 3: A t t e m p t t o do the job twice - the first result provides an early simulation o f the final product.
334
STEP 4: PLAN, C O N T R O L A N D M O N I T O R T E S T I N G
Without question the biggest user of project resources, whether it be manpower, computer time, or
management judgment, is the test phase. It is the phase of greatest risk in terms o f dollars and schedule. It
occurs at the latest p o i n t in the schedule when backup alternatives are least available, if at all.
The previous three recommendations to design the program before beginning analysis and coding, to
document it completely, and to build a p i l o t model are all aimed at uncovering and solving problems before
entering the test phase. However, even after doing these things there is s t i l l a t e s t phase and there are still
i m p o r t a n t things to be done. Figure 81ists some additional aspects to testing. In planning for testing, I w o u l d
suggest the f o l l o w i n g for consideration.
1) Many parts of the test process are best handled by test specialists w h o did not necessarily
contribute to the original design. If it is argued that o n l y the designer can perform a thorough test because
o n l y he understands the area he built, this is a sure sign o f a failure to document properly. With good
d o c u m e n t a t i o n it is feasible to use specialists in software product assurance w h o w i l l , in my judgment, do a
better job of testing than the designer.
2) Most errors a r e o f an obvious nature that carl be easily spotted by visual inspection. Every bit
of an analysis and every bit of code should be subjected to a simple visual scan by a second party w h o did not
do the original analysis or code but w h o w o u l d spot things like dropped minus signs, missing factors of two,
jumps to w r o n g addresses, etc., which are in the nature o f proofrea0ing the analysis and code. Do not use the
computer to detect this kind o f thing - it is t o o expensive.
3) Test every logic path in the computer program at least once w i t h some kind of numerical check. If
I w e r e a c u s t o m e r , I w o u l d not accept delivery until this procedure was completed and certified. This step w i l l
uncover the m a j o r i t y of coding errors.
While this test procedure sounds simple, for a large, complex computer program it is relatively d i f f i c u l t
to plow through every logic path w i t h controlled values of input. In fact there are those w h o w i l l argue that it
is very nearly impossible. In spite of this I w o u l d persist in my recommendation that every logic path be
subjected to at least one authentic check.
4) A f t e r the simple errors (which are in the majority, and which obscure the big mistakes) are removed,
then it is time to turn over the software to the test area for checkout purposes. A t the proper time during the
course of development and in the hands of the proper person the computer itself is the best device for
checkout. Key management decisions are: when is the time and w h o is the person to do final checkout?
STEP 5: I N V O L V E THE CUSTOMER
For some reason w h a t a software design is going to do is subject to wide interpretation even after
previous agreement. It is i m p o r t a n t to involve the customer i n a formal way so that he has c o m m i t t e d
himself at earlier points before final delivery. To give the contractor free rein between requirement
d e f i n i t i o n and operation is inviting trouble. Figure g indicates three points f o l l o w i n g r e q u i r e m e n t s d e f i n i t i o n
where the insight, judgment, and c o m m i t m e n t of the customer carl bolster the development effort.
S U M M A R Y
Figure 10 summarizes the five steps that I feel necessary to transform a risky development process
into one that w i l l provide the desired product. I w o u l d emphasize that each item costs some a d d i t i o n a l sum
of money. If the relatively simpler process w i t h o u t the five complexities described here w o u l d w o r k
successfully, then o f course the additional money is not well spent. Ii, my experience, however, the simpler
method has never w o r k e d on large software development efforts and the costs to recover far exceeded those
required to finance the five-step process listed.
335
l ~
~ m ~ I T _ ~ L_ ~.L
I w S i g I ~ o _ ~ E ,
_ I " o ~ .~ O . . v a W
r
/ ,
336
r -
E
2
o
' 1
E 8
t - O
E " O E
o E
8 t -
e,.
Q..
, m L L
C
l Q. /
( / )
I-- z ~
i , , . u a ,~L r.n r r n
I n . J
i i 1
,< z <
E
337
Z o_ I.- < r , - i l l Q .
o
/'L__J (.-
r -
0 ( J
f -
t.-
0 . E ~n
E E
>o
+.~
I
E o
E 4..,
o r -
Q .
c ~
i i
I |
I ' I I
: i ] . ~ ' l l e ~$ ~ ~ i
n | ~ ~ u 8 (
I I .. I s " "
O0 0@'
0 O° ~
d
p@@@@@@~S.
I w
R
I.L.
338