"Everything fails," people say. "Hey, especially software."

• People who say that include journalists, of course. They enjoy reporting the adversities people have suffered since the beginning of the 'software age.'
• Then, too, people who have more than a little knowledge of software -- software experts, even -- say that software fails.
• Writers of software engineering textbooks say that software fails.
• Universities employ lecturers in mathematics and logic design, in systems engineering and software engineering all telling their students that software fails.
• Decision makers at every level of management in industry and every department of government say that software fails.{HyperNote 1}

They are not sophisticated.{HyperNote 2}

To be said to work, software must do what it is supposed to do. A specification determines what software is supposed to do -- and by implication what software is not supposed to do.{HyperNote 3}

To be said to fail, software would have to work in the first place then not work in the second place. Software does not do that.{HypeNote 4}

If software ever worked, it will always work. Always.

• Software does not wear out.
• Software does not decay.
• Software is not vulnerable to environmental insults.
• Temperature, hot or cold, has no effect on software.
• Software needs no protection from shock and vibration.
• Neither moth nor dust doth corrupt software.

They are not sophisticated.

To work, software must run, which means commanding hardware to do what hardware is supposed to do. Software, then, depends upon hardware. And hardware...

• does wear out,
• does decay,
• does suffer from temperature extremes,
• does need protection from shock and vibration,
• does get eaten by moths and clogged up by dust.

It is possible for hardware to work and then to stop working.

It is more than possible, it is certain.

Repair Applies to Failures

A hardware failure can be fixed by replacement of the hardware in whole or in part -- replacement by identical hardware, in fact. That's what 'repair' is.

Since software does not fail, 'repair' has no meaning.

Reliability Applies to Failures

Reliability is a term which has meaning only for hardware not software. Reliability is defined as the probability of not failing within a given period of time.

Software does not fail within any period of time. The statistical expression 'mean time between failures' (MTBF) has no meaning for software, inasmuch as the inverse, 'failure rate,' is zero for software.{HyperNote 6}

Concrete Versus Abstract

Hardware is 'hard,' which is to say concrete, not abstract. Hardware can be made, in some sense harder, more reliable. Hardware can be...

• insulated,
• hermetically sealed,
• ruggedized,
• bullet-proofed.

By the way, humans are hardware.

Truth be known: It is because of its softness that software does not fail.

There is nothing softer than software. Other things may be as abstract as software but hardly softer.

A 'procedure,' for example, is abstract and therefore as soft as software.

Software runs on hardware; other procedures run on a special kind of hardware: humans.

They are not sophisticated.

The specification for either hardware or software is abstract and therefore soft. The specification does not do what software does, however. So the specification cannot be said to work or not work.

The specification, which is derived from functional requirements, can be dichotomized many ways:

• right or wrong,
• complete or incomplete,
• appropriate or inappropriate,
• useful or useless,
• safe or dangerous.

One Dichotomy

Only one dichotomy applies to software: either it works or it does not work, which is determined by whether the software complies with the specification. The same can be said for hardware; however, a failure takes hardware from working to not working; conversely, repair takes hardware from not working to working.

• To go from working to not working, software would have to be redesigned. There would be no reason to do that, of course.
• To go from not working to working, software would have to be redesigned. There is a reason to do that.

• wrong,
• incomplete,
• inappropriate,
• useless, or
• dangerous

Objects
 

• Objects can be sets of variables or constants.
• Objects can be inputs from hardware or from software.
• Objects can be outputs to hardware or to software.
• Objects can reside in memory, which is hardware. {HyperNote 9}

Internal States

Software references its 'internal state' and changes its 'internal state.'

• Accessing objects in memory is how software references its internal state.
• Assigning values to objects in memory is how software changes its internal state.

Cases

To be said to work, software must comply with the specification in all 'cases.' Not some, all.

A 'case' is merely a combination of input values.

Sequences

Each internal state is reached as the result of software being acted upon by a sequence of input values -- cases. Software changes its internal state whenever necessary to comply with the specification. Thus, a sequence of cases is necessary to determine what software does.

A sequence of cases will be necessary but not sufficient to determine what software does -- unless the sequence begins with the software in a known state such as 'reset' and the sequence is long enough.{HyperNote 10}

Doing the Right Thing

Software necessarily does the same thing in each sequence of cases -- the right thing. If what the software does in a given sequence of cases complies with the specification, the software is said to work -- for that sequence of cases.

Not doing what it is supposed to do in a given sequence of cases does not mean the software has failed. Instead, it means software did not work in the first place.

Nothing can be left to chance, here.

To do what it does, software must run on hardware, which means commanding the hardware on which it runs.

Commands are codes, and 'code' results from a procedure called design, which is what 'software engineers' do.

A given code can be different from another code that works and still work.

But a specification does not determine the sequence of internal states. If a specification did that, the specification would be more than a specification. It would be code.{HyperNote 12}

Design is a rigorous process guided by knowledge.{HyperNote 11}

he expression 'quality assurance' applies to software. The term also applies to hardware.

Quality assurance differs from 'quality control,' which applies only to hardware -- specifically to hardware manufacturing. Being abstract, software is not, in the conventional sense, manufactured.

Quality assurance means proving that a given code works. Which may not be easy. It is worth the effort, though.  Once proven to work in all sequences of cases, software does not fail.

The previous sentence deserves to be read again.

Quality assurance is not a matter of degree. It is all or none. And 'all' can be plenty.

• Complexity means size, for one thing: number of lines of code.  Each line of code applies to some number of cases and sequences of cases for which the software must be proven to work.
• Complexity increases with the number of input objects and the range of values each variable can take on.
• Complexity increases with the number of possible internal states, too.
• Complexity increases with the number of sequences in which those states can occur.

Testing: Myth and Reality

Testing is the best way to assure quality, some people say. They are not sophisticated.

Whereas proving that software works in a given sequence of cases can indeed be accomplished by testing, testing one sequence may or may not prove that the software works in another sequence.

Quality assurance requires all possible tests. That may not be practical.

Tests generally come in great numbers, but often tests do relate to each other.

A clever selection of tests might make quality assurance by testing practical.

In the previous sentence, 'might' appears in the predicate. That's because 'clever' appears in the nominative.

Even simple software.

Consider the following simple specification:

1. Software is required to assign a value to one output object based on the value of one input object. The output object is a bit, and the input object is a byte.{HyperNote 13}
2. Starting out in a 'reset' state, the output is set to a value of zero.
3. Whenever the binary value of the input is greater than one hundred, the output is set to the value one, which is sustained until...
4. The binary value of the input equals zero, whereupon the output is assigned a value of zero.

• pressure in a vessel or
• traffic on a communications network or
• radiation from a reactor or
• strength of a wireless signal or
• speed of a vehicle.

Software does not get much simpler, and testing that software would not be exceedingly difficult.

There are merely two 'internal states' required to fulfill the specification in the Simple Example: Call them 'on' and 'off,' which, in conjunction with the input object, determine the value of the output object to be one and zero.

• Starting out in the 'off' state following a reset, an input with value zero must produce an output with value zero. If the software outputs a value one, the test will have proven that the software does not work. A zero on the output does not prove that the software works.
• Same for an input value of one, two, three, on up to 100 -- all with an output value of zero.

• At least one more test is needed -- a test having an input value greater than 100.  If the output value is still zero, that test will have proven that the software does not work. Nevertheless, observing a value of one on the output does not prove that the software works (only that the software is not utterly ignoring the input values).

• Once the internal state switches from 'off' to 'on,' the output remains at a value of one for all values of the input except zero, whereupon the software changes its internal state from 'on' to 'off.'

For the Simple Example, tests do indeed relate to each other. There are four sets of cases:

1. those which are specified to leave the internal state 'off,'
2. those which are specified to switch the internal state from 'off' to 'on,'
3. those which are specified to leave the internal state 'on,'
4. those which are specified to switch the internal state from 'on' to 'off'

That will not work. Here's why.

Assume there to be a rational software engineer who, given the specification for the Simple Example, would yawn and then design his or her software to perform a simple arithmetic comparison between the binary value of the input byte and a constant value of 100.

A rational software engineer can make a mistake in the coding, though.

• Thus, for input values from zero through 127, the software would do what it is supposed to do, leaving the internal state 'off' for inputs from zero through 100 then switching the internal state 'on' for inputs above 100.
• Surprise: For all input values from 128 through 255, the software does not do what it is supposed to do: Instead the software leaves the internal state 'off,' and the output stays at zero.

Finding Out the Hard Way

For that particular mistake in code, then, a test having an input value larger than 127 would prove that the software does not work. On the other hand, quality assurance based only on a 'clever' selection of tests near the value 100 would allow software with a design mistake to be said to work.

Then one of the untested input values comes along and poof:

• the vessel explodes or
• the network saturates or
• the reactor melts down or
• the radio signal gets lost or
• the vehicle crashes.{HyperNote 16}

Quality Assurance Did Not Fail

Some people will say it was the quality assurance that failed.

They are not sophisticated.

• that testing with input values less than 128 did not assure quality of the software and
• that a test of merely one input value larger than 127 would have worked

It is not appropriate to say that quality assurance worked for a few tests and then stop working. That would be the evidence of failure.  Instead, quality assurance did not work, period.

Cleverness: The Impossibility

If performing all possible tests is not practical, then quality-assurance-by-testing will work if and only if the selected tests are the ones that do not allow software that does not work to be said to work.

The 'clever' selection of tests to assure quality can be exceedingly difficult.

Quality assurance is not a matter of probability.

The Reasonable Software Engineer

Consider the same Simple Example that was coded by the rational software engineer in the hands of a different person -- a reasonable software engineer. After yawning, he or she designs the software to use a table look-up procedure.

Rather than arithmetically comparing the binary value of each input byte to a constant 100, the software is coded to treat the byte as a 'pointer' which references an object in memory called a 'bit-map.'

• The first 101 entries are zeros, the rest are ones.
• The zeros are used to leave the internal state 'off,' the ones are used to switch the internal state from 'off' to 'on'.

Note that the code is different, but the specification is the same.

• The reasonable software engineer might make the same coding mistake as the rational software engineer.

• The reasonable software engineer might make a different mistake, though, like putting a one in the table where a zero should be or vice versa.

Quality-assurance-by-testing would not work unless the selected tests happen -- by pure chance -- to include the one or more in which the software does not work. After the software is put into use and the untested input value arises, the consequences would be attributed to a software failure by some people -- and to quality assurance failure by other people.

A sophisticated person would say...

• that through a mistake in the design of the code, the software did not work, and
• that through what might be called 'bad luck' in testing, quality assurance did not work.

The Simple Example is not uncommon and the coding errors postulated above are not far-fetched. The 'software age' is with us. What ever shall we do? {HyperNote 19}
 

ome will say that testing does not assure quality unless all possible tests are performed.

Maybe that's what people mean when they say, "You cannot test quality into something."

With enough knowledge of the code, testing is not necessary.{HyperNote 20}

Setting aside quality-assurance-by-testing, a sophisticated review of First Principles would include...

1. Assuring that a given thing will work requires knowledge of the specification for that thing plus...
2. Assuring that a given thing will work requires knowledge of how the thing works.
3. Knowledge of anything requires study.
4. Knowledge of how things work requires a special kind of study, called the 'Design Review.'
5. There needs to be a procedure for conducting the Design Review.
6. The procedure used for conducting the Design Review of software is called the 'Code Walk-Through.'
7. As a procedure, the Code Walk-Through is abstract and therefore soft.
8. Being abstract and soft means that the Code Walk-Through either works or it does not work.
9. To be said to work, the Code Walk-Through must find all mistakes in the code. Not some, all.
10. The Code Walk-Through runs on a special kind of hardware: humans.
11. The Code Walk-Through does not have the power to command the hardware on which it runs.
12. The Code Walk-Through, therefore, is not software; nevertheless,...
13. The Code Walk-Through does not fail. Instead,..
14. The Code Walk-Through which does not find all mistakes did not work.

The human conducting the code walk-through can make an 'undersight,' of course, much as a software engineer designing software can make a mistake. {HyperNote 21}

• An undersight during the Code Walk-Through will result in undiscovered mistakes in the code.
• A different human conducting the same Code Walk-Through may not make an undersight.
• For that matter, the same human repeating the Code Walk-Through may not make an undersight.

• Performing the same Code Walk-Through will not make it work.
• The Code Walk-Through, which is a procedure, needs to be revised to make it work.

• Compilers identify a few classes of mistakes. Other tools are less effective.
• Automatic monitoring for 'code coverage,' for example, is utterly inconclusive.

Soft Things Made Soft

Quality assurance by the Code Walk-Through can be exceedingly difficult. But no more difficult than designing the software itself. And far less difficult than testing all possible sequences of cases.

Consider the Simple Example again. Only a few lines of code were needed to meet the specification.

• The rational software engineer made a mistake in one of those lines: assigning the wrong 'type' to a variable. The Code Walk-Through that checks to make sure all variables have been 'typed' correctly would have found the mistake -- assuming the human performing the code walk-through does not make an undersight.
• The reasonable software engineer made a mistake in coding the bit-map. A code walk-through that includes the inspection of all entries in tables would have found that mistake -- again, assuming the human does not make an undersight.

1. preparing a complete set of values as input objects,
2. causing each to be accessed by the software,
3. independently predicting the values that are to be assigned by the software to the output, and
4. comparing the predicted outputs with those observed in the consequent outputs.

Managing Complexity

Some people will say that complexity makes the Code Walk-Through not practical.

They are not sophisticated.

The sophisticated person knows that complexity must be managed.

One way to do that is to partition the code into 'modules.' A specification must then be prepared for what each module is supposed to do and for what all the modules together are supposed to do.

Unfortunately, quality-assurance-by-testing requires preparation of sequences of cases for each module and some method to intercept the consequences -- 'stubs.' All software modules need to be tested together, which means that testing all possible sequences of cases will be impractical, even with modular software.

The practical becomes more practical.

The sophisticated Code Walk-Through must include procedures...

• that perform detailed analysis of branch topology and case statements,
• that evaluate the effect of processor loading on realtime performance,
• that calculate demands and limitations of data rates and processing delays,
• that validate each flag and pointer in function calls,
• that postulate hardware failures to ascertain remedies and responses,
• that review all interrupt processing for hygienic and timely context switching,
• that analyze asynchronous events for vulnerabilities,
• that examine each loop for accuracy in count and closure,
• that verify the initialization of each iterative sequence,
• that review internal and hoisted processing of loop parameters,
• that check for inclusions and exclusions in code topology,
• that estimate loop timing,
• that validate re-entrant code,
• that review each coded logic decision for correctness,
• that audit the allocation and ranging of heap resources,
• that review the stack management algorithm for proper function,
• that authenticate matrix design for hierarchical consistency,
• that make sure all n-tuples are well-formed and all don't-care cases are properly terminated,
• that check the base address and offset range for pointers,
• that study software partitioning for matched interfaces between modules and coordinate processing,
• that check for violations of enterprise-specific policies,
• that assure compliance with protocols and absolute correspondence between requests and services,
• that check typical and extreme values in recursion formulas,
• that audit for adherence to software design practices of professional societies and standards organizations,
• that confirm all entrances and exits conform to structured programming conventions,
• that inspect table entries for accuracy in coding,
• that inspect all variable attributes for conformity to specification,
• that assure all variables have been typed and scoped correctly, and
• that inspect to make sure that the code has been annotated clearly and completely -- hey, and correctly.

Whatever the ultimate result, though, software does not fail.

{1} References That Fail

See for example Pressman, Roger S., Software Engineering (1987, McGraw-Hill) or von Mayrhauser, Anneliese, Software Engineering(1990, Harcourt Brace Jovanovich). As for hearsay references, suffice it to say that forcible expressions of these sentiments have reached my ears from a hundred sources since the mid-fifties. {Return}
 

My license for bluntness here is appropriated from Ralph Waldo Emerson (1803-1882).

"Speak what you think in words as hard as cannon-balls..." {Return}

Permit me to quote William Makepeace Thackeray (1811-1863):

"For the wicked are wicked indeed and they go astray and they fall and they come to their just desserts, but who can tell the mischief that the very virtuous do!"

Some people will say that not working in the first place is just another expression for 'failure.'

They are not sophisticated.

Terms selected here are somewhat arbitrary but intended to serve clarity by virtue of consistency, thus one might...

'amend' requirements,
'revise' specifications,
'perform' tests,
'repair' hardware,
'redesign' software.

what and who,
how and why

Unhelpful indeed are statements such as are found in Siewiorek, Daniel P. and Swarz, Robert S., The Theory and Practice of Reliable System Design (1982, Digital Press, Bedford, MA):

"A failure causing a [computer] crash may be the result of either hardware or software failure."

Ironically, hardware gets effectively harder when more of its functions are given over to software, as in embedded controllers or, in the extreme, the 'reduced instruction set computer' (RISC).

The term 'firmware' means software that resides in read-only-memory. Nota bene, firmware is as 'soft' as software. {Return}

The distinction being drawn is between 'software' and 'hardware,' the latter can be taken to mean a 'computer,' 'processor,' 'controller,' whatever.

Other words have been eliminated altogether as not relevant to the present scope 'program' and 'subprogram,' 'routine' and 'subroutine.' The word 'system' long ago got stretched beyond its elastic limits and lost its strength.

The sophisticated term 'object,' which is general in the extreme, gives renewed evidence of a linguistic struggle dating back to the dawn of the 'software age' and the realization that calling 'software engineering' 'computer science' is tantamount to calling 'mechanical engineering' 'automobile science.' {Return}

Not all cases can arise in real life, thus we have the term 'don't-care' for those cases.

The sophisticated person cares about those cases: first making sure that they cannot indeed arise and second making sure that if they do arise, the software works anyway. {Return}

In Cultural Literacy (Random House, 1988), E. D. Hirsch frames a compelling picture of the knowledge-bound character of all cognitive skills. {Return}

The term 'code' is taken here to mean 'source code,' which is what software engineers produce.

Development software -- compiler or assembler -- produces 'object code', which is what actually commands hardware. The distinction is not germane to the present subject.

Unless the development software does not work. {Return}

The word 'byte' may be the purest neologism of the 'software age,' giving evidence of the forlorn and tardy impact of 'computer science' on the language we speak. I dare to use the term here without explanation. {Return}

The 256 test cases in the Simple Example are not too many to do, even by hand. A minor tweak of the example, though, to 16 bits on the input will change that.

Automating the testing with a 'driver' means, for the Simple Example, writing more code to do the testing than the code in the first place. Another tweak or two, and the number of test cases -- even for the Simple Example -- will exceed the number of subatomic particles in the Milky Way Galaxy.

The leftmost digit of a binary integer of a given length can be treated either as a binary bit or as the sign of a binary integer one bit shorter in length.

In byte-land, the former interpretation will support a range of values from 0 through +255, the latter, values from -128 through +127. {Return}

A compendium of the most sensational consequences of -- and misguided attributions to -- software failures appears in Leonard Lee's The Day the Phones Stopped (1991 Donald T. Fine, Inc), which inspired the present polemic. The book is listed by the Library of Congress under the category 'Computer software -- Reliability.' {Return}

Most people like to say that if software works in some cases but does not work in other cases, the software has a 'bug' in it. That may be more meaningful than saying that the software failed. But not by a lot. The software did not get a bug in it. The software engineer made a mistake.

A software engineer bemoaning a bug in his or her software is tantamount to a tennis player lamenting flaws in his or her own backhand -- hey, or concentration.

When 'glitch' gets into a sophisticated dictionary, it will be given a synonym, 'intermittent.' {Return}

One handy software engineering tool is the 'debugger,' which is a misnomer tantamount to calling a 'magnifying glass' an 'insecticide.' {Return}

The Simple Example appeared as a portion of a real-life specification, which called for the software to use the binary value of the input byte to select an object from memory for reading.

• The software engineer designed the software to treat the input byte as a 'pointer' to objects in memory.
• Making matters worse, each object was the beginning of a different procedure within the software.
• All possible values of the pointer were properly accounted for in the code.
• The pointer itself, however, was unintentionally coded as a signed integer and then treated as a conventional 'offset' added to a constant to form a memory address.
• The software passed quality assurance, having worked for all test cases and released.
• The software appeared to work at installations all over the world.

However inconclusive, testing of software will continue to be practiced until the sun flickers from the sky.

Software 'bureaucracies' brim with testing. And what is known as SVVP (Software Validation and Verification Plan, abbreviated "V&V"), an orthodoxy wherein both Vs are mandated: The first assuring that the specification is correct and the second testing to make sure the software does what the specification mandates -- in other words, works. {Return}

The word 'oversight' like 'cleave' can cut both ways ("split apart" or "cling together").

As "watchful care," the Code Walk- Through is 'oversight' at its best.
As an "unintended omission," even one 'oversight' in the Code Walk-Through means disaster.