Flickr photo from mlibrarianus

Abstract

Cryptanalysis always gets better. It never gets worse.

Many in the public policy, defense, technology, and security communities have long known, and Edward Snowden gave his freedom to remind us, that the United States government is progressively expanding a comprehensive, intrusive intelligence-gathering system — a Panopticon with capabilities the average citizen can neither comprehend nor resist — a system with unprecedented power for tyranny and repression — and it will be far harder to dismantle than it was to build. There are ways legislators, technologists, and the public can push back, and we must do so as soon as possible to avert a terrible future.

Preface

A few key facts to set the scene:

• 9/11 was publicly labeled a failure of intelligence by a wide cross-section of the American media and political establishment.
• We have a cultural need to place blame on individuals or groups for bad events, even those that result from systemic failure.
• The United States — hereafter “we”, though much of this applies to the “five eyes” (viz. Australia, Canada, New Zealand, the UK, and the US) as well — has made massive investments in “homeland security” signals intelligence for a variety of purposes, including but not limited to counter-terrorism.
• The United States National Security Agency (NSA) probably spends more on cryptography research than everyone else put together. They publish very little research. Thus, the cutting edge of public research is likely an inaccurate predictor of the NSA’s cryptologic capabilities.
• The prospect of collateral damage does not, in practice, deter the American intelligence apparatus from using its capabilities to identify, capture, and kill terrorists.
• Through Edward Snowden and others, we have significant evidence that the NSA has spent significant effort building a comprehensive apparatus for surveilling both Americans and foreigners. Through the PRISM program and other systems, they have acquired unprecedented access to the private information of citizens worldwide. They have made significant investment in systems to automatically analyze this data. The public does not know the full extent of their capability in this regard.
• The head of the NSA, Gen. Keith Alexander, believes that only by collecting all available data can the NSA effectively thwart terrorism.
• The public state of the art in data storage and analysis — storage capacity, processing speed, statistical techniques, available software, automation capability — is several orders of magnitude better than it was 20 years ago. We have no reason to expect this trend will not continue.

Where we could go

Imagine you’re the director of US national intelligence. It’s a few years from now, and you’ve been given an enormous budget and an effectively unlimited supply of smart people to end the terrorism problem. With these resources you build Deep Thought — a machine with access to an unfathomably deep database, capable of answering arbitrarily-formed questions about the population of the US.

Billions of dollars and thousands of man-years later, with the President and the Joint Chiefs watching, you stand in front of this machine and say,

Show me the terrorists.

Obligingly, Deep Thought displays a list of Islamist radicals, anarchist cranks, would-be Unabombers, and so on. The dedicated men and women of our national security agencies round up these dangerous criminals and lock them away. The nation rejoices. The TSA lets you fly with shampoo again. Everyone’s happy.

There is just one catch: Deep Thought is still sitting there, waiting to accept input. The nation has already sunk the cost of building it, and it’s not about to shut it off any more than, say, unilaterally destroy the entire strategic nuclear arsenal. But since we have it:

Show me the murderers.

Show me the child molestors.

Show me the kidnappers.

The New York Times has a field day. Fox News runs a slide show of missing little girls and their rabid killers, finally brought to justice. Government approval soars.

And Deep Thought is still sitting there, waiting for the next question.

Show me the drug dealers.

Hundreds of thousands, ranging from die-hard methheads to small-time marijuana dealers to club kids to ancient hippies passing out LSD. In one swift stroke, the drug problem in the United States is over. Despite the collateral damage (a few politicians’ sons, how embarrassing) support is broad.

Show me anyone with the capability and intent to bring down the United States government.

Show me the leakers.

Show me the hackers.

It is still pretty easy to justify. These are dangerous people — traitors, even. “Crazies.” Nobody really complains.

Who is left? Anyone you want.

Show me the dissidents.

Show me the activists.

Show me…

Frequently exclaimed “but!”s

That system you described is impossible.

I agree — there’s no way we could ever build a system with zero error rate. Either we would miss some terrorists by ensuring P(terrorist)=1 (and if you think we would take this road, I have a deterministic random bit generator to sell you) or we would end up including innocents in the results, like dolphins in a net of tuna. See: this guy.

A 100%-accurate system like this is impossible with the tech we have today. Much more likely is a system — much like ones we have today — that gives a probability of being a terrorist. Unfortunately, decision-makers aren’t often big on fuzzy probabilities:

“I don’t need this,” Harris reports that a senior CIA officer working
on the agency’s drone program once told an NSA analyst who showed up
with a big, nebulous graph. “I just need you to tell me whose ass to
put a Hellfire missile on.”

They might be able to do this in the future, but not now.

Public companies, especially marketers and advertisers, are experts at this sort of behavior. Web advertising companies jumping on the big-data bandwagon have discovered Target et al. already there. What can the NSA accomplish with Google/Facebook/Comcast/Verizon/AT&T’s knowledge of your online habits, plus their own sources? What profile could they build of you? In theory, they can suborn your laptop, your smartphone (a remotely-operable listening device with a location tracker built in, great!), your landline, every Internet connection in your house and business, your physical mail, your email, your OnStar-enabled car, every online profile you have unless you adopt the life of a complete paranoiac — and even then it only takes one misstep for them to get you. Ask John McAfee.

Certainly there are technical capabilities that are out of their reach at this moment; that’s why they’re storing all the data for future analysis. Data that currently seems inconsequential might be far less so later. Currently-unbreakable crypto protocols (and remember, they spend more on crypto research than the entire rest of the world put together) may not be so unbreakable later. Good thing for them it’s all on disk.

This is unconstitutional.

Probably, but that hasn’t stopped them. Their interpretation of the law and the Constitution most likely doesn’t match yours.

The NSA doesn’t consider automated interception and bulk analysis a “collection”, meaning it can build as large of a profile of you as it wants, no approval required, as long as no human intentionally instructs it to do so. Should it ever decide it needs approval, the Foreign Intelligence Surveillance Court’s approval rate is 99.97%.

We would know if this were happening.

The NSA, DEA, and FBI have demonstrated willingness to lie to the courts (through the method of “parallel construction”, also known as intelligence laundering) about the source of their information. They have repeatedly misled Congress about their activities. They have redefined words like “incidental,” “relevant,” and “targeted” in positively Clintonian fashion. They believe that the disclosure of the existence and operation of this technical apparatus is tantamount to treason against the United States.

You are free to believe, as President Obama claims, that we would be having this debate even if Snowden hadn’t done what he did. I do not believe this.

My life is not interesting enough to be surveilled (often rendered as “I’m too boring.”)

The background assumptions there:

• it costs the government some nontrivial amount to watch me, and
• they don’t gain anything of worth by doing so (after all, I am not a terrorist!), so
• it’s not worth it to them to do so.

The first assumption no longer holds.

Before the age of fast automated analysis, surveillance agencies were limited by money and very expensive human time in the scope of their surveillance. If they wanted to watch you, an actual human had to spend time listening to you, following you, reading your physical mail, and so on — with the associated Fourth Amendment implications, to the degree the government followed them.

This is dramatically less true in an era of supercomputers and data centers, where the entire metadata output of, say, Verizon can be recorded by a few racks of servers. In other words, once the infrastructure is in place, the marginal cost to the government to surveil you is effectively zero, and the downside risk for failing to surveil the right people is high. They may have more to lose by not watching you than the other way around.

What harm does it do? If you’re doing nothing wrong, you have nothing to hide.

No. If I have done nothing wrong, you have no reason to watch me.

Just because Google, Facebook, Skype, Verizon and other companies are
routinely monitored by the CIA doesn’t mean that somebody is watching
you every time you order groceries online or voice-chat your sister in
Seoul. It just means that they could if you gave them a reason to do
so. That means you can relax – right up until the time when you want
to go to a protest, or your sister does, or you support the fact that
several thousand complete strangers did.

If you believe you are being watched (or are uncertain but think you may be), you change your behavior. You take fewer risks: the fear of appearing guilty, even with the most innocent of motives, is enough to deter you from taking actions that might appear such to a hostile audience. You watch what you Google, you watch what you say on the phone. In small ways, your life begins to resemble an airport security line (“don’t joke about bombs — they might be listening!”). In order to thrive, our democracy requires that discourse. We need the ability to hold discussions current (or future!) administrations might not like. We need to be able to hold opinions others might detest. We need privacy to flourish, and I do not believe we can be fully human, fully free, without it.

I don’t care. If the government says they need it to catch terrorists, I trust them.

Even with several very generous assumptions about the honesty, integrity, and competence of the current administration (for whom I voted twice, and about which enough has been said) — governments change. We should be deeply cautious of such a powerful tool, potentially the most powerful instrument of tyranny and oppression a government has ever possessed, in the hands of any administration, no matter how benign. All it takes is one election. If the transition from the Bush to Obama administrations has taught us anything, it’s that a change in party in no way necessitates a lessening of executive power.

What we can do

In a very real way, we, the American people, did this to ourselves. We built this machine out of fear and now, at least in the short term, we are stuck with it. By demanding that no terrorist attacks ever happen, and placing the blame on our intelligence apparatus when they do, we have strongly incentivized the creation of a system exactly like this.

In the short term, we can demand radical transparency and oversight for our clandestine security agencies, starting with a top-to-bottom audit from Congress’ loudest complainers (Senators Ron Wyden and Mark Udall come to mind). We can make this a campaign issue: as a society, we must accept this may make them less effective (though who wholeheartedly accepts the doomsaying coming from the NSA in response to the Snowden leaks?) and that we are willing to make that tradeoff. Remember, as easy as it may be for them to forget, our government works for us.

In the medium term, engineers and technologists must refuse to build or support systems that engage in privacy- and liberty-destroying mass surveillance. Transparency reports are a good start, but those with the clout and resources to fight these orders must stand up and do so — large companies, mostly, but individuals can as well. We must establish a strong ethical basis to guide those in my profession as we develop the technology of the future.

In the long term, we must dismantle the current architecture of secret courts, secret orders, and secret law enforcement. As a society we must stand up and say: this is not who we are. Though it may come at a price, we demand freedom of thought and conscience and association. We demand that our messages to our loved ones, our health, our finances, our lives, retain the privacy the Constitution specifically affords them. We demand what Voltaire called “the right most valued by all civilized men”: to be left alone.

I would love to hear your comments; I am @ternus on Twitter.