MUNICH — Tech companies are obsessed with predicting human behavior. Auto-correct knows what word we're about to type and Amazon knows which book we'll buy next. But for a while now, governments have also been using algorithms and predictive analytics to forecast their citizens' behavior.
In many countries, the police already have so-called "predictive policing software," which uses data to estimate the probability of crimes being committed in the future. In the U.S., the city of Chicago has developed an algorithm to predict food hygiene violations in restaurants. In the UK, authorities are using big data analysis to predict cases of child abuse. In Germany, the government plans to use an artificial intelligence system to predict "potential" crises before they arise.
This privatization and mechanization of politics is an extension of libertarian utopian states.
A few months ago, the management consultancy firm Deloitte published a report describing various government trends, most striking of which is "anticipatory government". "The latest developments in neuro-linguistic programming, machine learning and speech and image recognition have made it possible for governments to anticipate and predict problems instead of reacting to them," says the report. They can avoid a whole range of problems, from crime to homelessness to accidents. As the use of this technology grows, governments will turn into a technocratic risk and process management system: input, output, done.
This new approach to politics will not only establish a new basis for exercising power (namely data) but will also lead to new rules being introduced. It won't be decision-makers or politicians deciding what constitutes a crisis — it will be software developers. When a software developed by IBM – or SAP in the case of the German army – identifies any kind of crisis that politicians must react to, it constitutes a kind of lobbying, as developers gain influence over politics through a supposedly evidence-based system. This privatization and mechanization of politics is an extension of libertarian utopian states, a phenomenon that experts predicted decades ago where government structures are influenced by the market.
It is a system is based on a very particular understanding of politics in which political issue is denied legitimacy before it even becomes politicized. The predictive government essentially means preventing social phenomena such as criminality, homelessness or unemployment, as though they are the consequences of delayed or ineffective government policy.
Big Data Is Watching — Photo: Shahadat Rahman
This depoliticizing technology uses a mathematical approach to suck all the air from the political system. From a democratic perspective, political issues are not a "problem", in the sense that they are not bothersome or disruptive. Rather, they are jumping-off points for a political process which, according to the classic model, begins with identifying and defining the problem, then progresses to setting the agenda and formulating and implementing policies. Ending a policy is part of the political process, but it isn't the same as depoliticization. Preventing political problems in advance ultimately means an end to the entire democratic process.
Would citizens have a right to appeal against the software's decision?
We need to let go of the idea that the state is a kind of thermostat you can use to adjust the wellbeing level of society. Democracy is always an incomplete system, of which the core principle is transparency: Issues will be publicly negotiated and resolved. Preemptive government and its fixation on the future actually constitute a conservative obsession with the present: everything must remain as it is. Political problems are treated as disturbances to the normal functioning of society.
In a modern bureaucratic state, there are, of course, legal disturbances that must be resolved (disturbances of the peace, an illegally parked car, etc.). When police are faced with a demonstration, they must consider hypothetical elements and predict whether an escalation will occur. They don't use computers to do this — they use their own judgment and experience. It is a perfect illustration of the difference between a democracy and a preemptive government, which intervenes much earlier and closes off certain avenues before citizens even have a chance to explore them. Such instruments can easily lend themselves to authoritarianism.
A software company develops a computer program that analyses variables such as the number of participants, noise levels and the degree of aggression of geotagged tweets to calculate the probability of a demonstration descending into violence. If the police use this software and deny demonstrators the right to protest, would the computer-generated prediction still be valid? Would protestors have a right to appeal against the software's decision? If the computer produces an unreliable result, would authorities be obliged to act on it? These are only a few of the questions that show how computer code could override the law.
In New Orleans, data company Palantir tested a predictive policing software without the local authorities having any idea it was in place. The model identified connections between people, places, cars, weapons and social media posts, and used these alongside other data to determine who might commit a crime or who might be a victim of crime. Their findings will not be made public, so people will not know who has been identified as a potential criminal. The information is locked away.
The most frustrating thing about this data-driven, deterministic government model is not just that it shuts down all political debate. The big underlying problem is that this system doesn't consider the future a malleable realm of possibilities, but as a latent threat, a risk to be managed. Even utopias are suspicious entities in the computerized regime, as they can't be reduced to a predictable data set.
