Return to blog

I would like to respond to Bruce Schneier's post "Should the Government Stop Outsourcing Code Development?" I think Bruce is close to the mark in his answer, but misses the point on both defense in depth and critical infrastructure.

Fundamentally, most security flaws are no more than bugs - sometimes in requirements, sometimes in design, and often in implementation, but bugs nonetheless. Of course, most authors do not want their software to be vulnerable. Bugs/defects are defined as any unplanned, undesirable behavior in software/hardware, so by definition, most security vulnerabilities are bugs. That this makes security researchers like me nothing more than a glorified QA department is a topic for another post.

The point I want to make here is twofold:

  • In complex software, there will always be bugs, so there will always be vulnerabilities.
  • Defense in depth recognizes this and is the natural, logical response to the assumption that everything is broken and will remain broken in spite of our best efforts.

Complex software first - I very much believe designs of control systems for critical infrastructure must become much more paranoid and that greater assurance is needed. But, I don't believe we will ever see control system design return to the "old days" of custom compilers on custom operating systems, on custom computing hardware and custom network hardware. Nobody I know wants to return to those days - they were too hard: everything cost more than was reasonable, everything took longer to learn and do, and you got so much less for your effort than stakeholders expected. It certainly wasn't fundamentally any more secure than today, though there was rather greater diversity and much greater obscurity.

The way forward is to simplify - simplify control system designs to use less software, and to make use of hardened, reduced-functionality operating systems. Simplified design means looking at your designs and re-phrasing components that use complex facilities like DCOM that your designers will never fully understand and are almost impossible to secure. Hardening of course, means removing all components that your control system does not use, and modern operating systems have a lot of stuff that should be removed. I like the idea of simplifying even the OS kernel. For example, do control system components need a network stack tolerant of every possible variant of network communication ever observed on the open internet? In my opinion, control system components can afford to throw out most mal-formed packets, but the cost of maintaining parallel kernels will likely throw out that idea.

Simplicity and paranoia will make control systems more secure and will reduce the cost of assurance, and quite a bit of simplification and paranoia is appropriate for SP-99 level 2 and level 1 control system components. But modern control system software is still going to be large and complex enough to have security vulnerabilities / bugs, in spite of much better attention paid to security going forward. Security researchers continue to uncover new kinds of vulnerabilities, kinds of vulnerabilities the designers did not imagine when coding their products in the first place. Even if that were not true, modern control systems are too big to be bug free. Yes, there have been isolated examples of significant amounts of "bug free" code in the history of software development held up as examples of what is possible if we pay enough attention to quality. But even hardened, modern control systems are at least two orders of magnitude bigger and more complex than the most complex of these examples - maybe three orders of magnitude. That's big enough that I just don't buy the "it can be vulnerability-free if you try hard enough" argument.

I also don't buy the argument that given the vulnerabilities, we should pick one mode of defense and call it a day. There are too many attack modes for one defense. Perimeter firewalls will not protect you against insiders. Host firewalls will not protect you against insiders with USB sticks and access to the hardware. Application control and the "no execute" bit will not protect you against scripted attacks. Patching will not protect you against zero-day attacks. And intrusion prevention will not protect you against trojans planted in your trusted computing stack.

And so the only reasonable response to a threat landscape this varied is to become paranoid, and to deploy a reasonable spectrum of preventive, detective, and forensic defenses to protect your most important assets. And yet you need to keep the big picture in mind too - every additional layer of security technology adds costs and reduces the flexibility of your control system. Go overboard and you wind up with an ulta-secure solution which can no longer control your physical process.

In short, there is no silver bullet. I agree with Mr. Schneier that assurance will make things better, but there will never be enough assurance to eliminate all defects. Security vulnerabilities will always be with us. Defense in depth is a reasonable response to the array of threats control system operators must be concerned with and does not deserve to be maligned to make a point about other security approaches or technologies.

Our power grids, our chemical plants, our water plants and our factories producing foodstuffs are too important to entrust to control systems with limited protections. Yes, we need more assurance in the control system space, but we can't rely on any one approach to - or technology for - defense.