Wednesday, October 6, 2010

What is Software's Jurisdiction?

Computers will seduce you, they will work perfectly every time for ten years. And then one day. They'll kill you.
-Doctor Art Draut,
Test Pilot 
Professor of Aerodynamics and Computer Programming

As a flight instructor, one of the first things I teach my students in the Cessna 172 is how to "lean" the "mixture" correctly. Most aircraft have separate controls for the throttle (black) and the mixture (red). The throttle controls how much fuel and air (unless the mixture is full out, in which case the cylinders get all air and no fuel) get sucked into each cylinder during the intake stroke of the Cessna's four-stroke reciprocating engine. The mixture controls exactly how much fuel is added to the air on its way into the cylinder. Cars originally didn't have mixture controls because they usually stay at around the same altitude, this meant that at higher elevations or on hotter days the cars were burning fuel inefficiently because they were adding the same amount of fuel at 4,000' as they did at sea level; at 4,000' there is less air and so less fuel is required to achieve the optimum fuel/air ratio. Later with the advent of electronic fuel injection automakers quickly realized that they could add a chip to the engine that sensed air density and adjusted the mixture perfectly to ambient conditions for each individual intake stroke of the engine. This significantly increased both engine performance and engine efficiency.

The auto industry in the United States is about 700 times larger than the light aircraft industry (by revenue) and the makers of aircraft engines did not have enough money to redesign their engines with electronic fuel (mixture) control, as a result of this, airplanes are stuck with vintage 1950s engines that still have mixture controls, many old school pilots are very happy about this because they simply do not trust electronics. There is a fundamental trade-off that occurs when you put a computer in charge of something. Performance goes up (hopefully), but complexity goes up as well; when you add electronic fuel control to an engine you are adding one more part that can break and kill your engine. Electronics however, are very reliable, a friend of mine is an engineer employed by Cessna and he reports that mechanical components on Cessna aircraft must have a reliability rating of 10^-5 (1 failure in 100,000 cycles) or better, while electrical components must have a reliability rating of 10^-9 (1 failure in one billion cycles) or better. Electronics are especially reliable when they only have to do one thing and cannot be told to do other things, such as the chips that control the mixture in jet engines (which all jet engines in the sky today have). In the case of the Cessna's engine, I believe handing the job of leaning the mixture to a computer is a no-brainer, I lean the mixture several times a flight but a computer will do it more than a 1,000 times a second. Whenever the engine is at a less than optimal mixture setting it makes the engine more likely to suffer failure in the future so my relatively infrequent leaning is comparatively bad for engine health, thus in this specific case the risk of failure will likely go down.

Moving on from the Cessna to more serious hardware, the problem with electronics is that the more things you want them to do the more likely it is that they will fail. The chips controlling the mixture on a GE-90 have never failed according to the NTSB database but Windows (which is designed to do pretty much everything) fails everyday. Windows not only has the disadvantage of being more complex because it's designed to do everything but it also has the disadvantage of being externally accessible. If your computer is physically connected to the internet then it is possible for your computer to get hacked.

Continuing to even more serious hardware, the flight management system on modern fighter jets is both complex (it must be aware of and correct for temperature, pressure, landing gear position, payload door position, payload weight, fuel weight, gps data, flight regime, other aircraft, pitch, bank, roll, yaw, radar data and much much more) and accessible. Modern fighter jet FMS systems actually run an operating system that is upgradeable and to be upgraded it must be accessible. It is only "internally" accessible however, (to my knowledge anyway, it is really just an educated guess) you must physically connect to the aircraft to modify its software. It seems comical that software, something we usually associate with Microsoft Windows and unicorns named Charlie; could cause something as deadly serious as an air superiority fighter jet to crash, but that is exactly what happened to the world's best fighter jet (a prototype) in 1992 and then to a production aircraft in 2004.

Continuing to even more serious hardware the american electrical grid is planning to "get smart." Last year the Obama administration set aside 3.4 billion dollars (for a total of 11 billion now) to make the electrical grid smarter, a move which the DOE calls a necessity. It is estimated that intelligent grid control could make the system 10% more efficient... and guess what communications network is to be used to control this new dynamic system? That's right, the internet.
Let's pause for a second and think about America's electrical grid... Generating about 4 Trillion Kilowatt hours of electricity a year and powering hundreds of millions of appliances the electrical grid quite literally keeps us alive. Think I'm exaggerating? Let's think about what electricity does for us: it powers our computers, phones and lights, our freezers, refrigerators and home heating/cooling systems, our traffic lights, gas stations and air traffic control system. Maybe you don't need any more convincing but the electrical grid is a serious piece of hardware. If you dabble in the safety sciences you will frequently see a chart like this one:
Software first crept its way from the bottom left of this chart to the bottom right of this chart. Now it is climbing up up from the bottom right toward the top right corner. While very few dispute that software makes cars, Cessna's and jet engines better, it's use in fighter jets might give you pause. There is a concept in the safety sciences called "blood priority," it basically states that even though people know a problem exists, they won't fix it until it kills someone. As people get more comfortable with software, they allow it more leeway, as a (amateur) programmer myself I love software, but it cannot solve everything, and it cannot be held responsible for anything. At some point or another, if we continue to allow software to expand it's jurisdiction into more and more of our lives we will reach a point of dangerous reliance and/or vulnerability. This is exactly what Doc Draut meant when he uttered the quote at the top of this page.
He does not mean that everyone will get killed by a T1000 in ten years, he simply means that we humans need to remain ultimately in control of the software that runs our lives. And nobody is in full control of the internet so I don't believe allowing it to run a life-critical system like the electrical grid is an acceptable risk. Referring to the above chart again I think everyone agrees that the potential severity of a nation-wide electrical failure belongs in the "catastrophic" row. The question is, what column does it belong in? Lockheed Martin seems to think it is not in the far right column.

In the interests of not sounding like a prophet of doom I would like to end this post with this link.

No comments:

Post a Comment