Richard Clarke was on a panel at RSA this week. The discussions was centered around improving our nation's cyber defense. He insisted that what we needed was ISP DPI on the major Internet backbones. I don't see how this could work.
The NSA has traditionally had access to a great deal of traffic, foreign and domestic. Clarke said that, given their mission, it was inappropriate to rely on them for the security of the nation's infrastructure. I agree. (I also think that it is time to acknowledge their access at a policy level. The Opposition assumes they have it, and the data can be very useful to their mission, which is an honorable and useful one. Public policy could (and does, to some extent) lay out the limits of use of the results beyond the Fort.
I was unable to corner Richard and find out why he thinks the ISPs are the right place for national cyber defense, and why he thinks it would work. It appears to me that it is unworkable at the technical, economic, and political levels.
What are we to look for in all this data? Deep Packet Inspection gives access to=20 data when it is unencrypted and in a reasonably typical packet format. There is no time for packet reassembly, a fraught endeavor at best (see Vern Paxson's work on packet normalization.) It is easy to hide all but the endpoint addresses with crypto, and as a cyber security policy, we should be tending towards end-to-end encryption.
What would DPI look for? Viruses? "Dirty words" like "plutonium" or "we attack on Friday"?=20 Collect Google queries? Many (most?) Internet sessions have asymmetric routes. Are we to go find the return flows, which might go through a different carrier?
AT&T is the largest IP backbone provider in the US. We install and manage 10Gb lines by the fist-full. Even faster lines are available at higher per-bit rates. Devices to monitor such traffic are expensive. The data volumes are vast: a thousand Costcos couldn't sell us enough disks to keep all the data, or even much data, for very long. Who is going to pay for all the equipment and processing?
Finally, what are the customers going to say about this? Privacy is a touchy subject, and rightfully so. The phone company has always had the ability to eavesdrop on communications, but only in the pursuit of proper network maintenance. This was thoroughly covered when I was trained for central office duty: clearly a central office employee has the tools and access to monitor phone conversations, but we are to check a phone line only to see if it is in use before performing maintenance duties.
I am not sure what Richard has in mind: perhaps he has published more detailed thoughts. I would start national cybersecurity improvements with requirements for strong host security for government machine. If we create a market for more-solid clients, the nation can benefit.