Two years ago, I finally sat down to read my grandfather, Eliahu Grebler’s, memoirs. In it he talks about his and my grandmother’s harrowing escape from Poland in the face of the Nazi threat during the Blitzkrieg. They barely escaped the carnage and strafing by the Luftwaffe on columns of fleeing refugees. Being exposed to the stories of the Holocaust from a young age coloured my opinion of freedom and political systems. Freedom can be transient, political systems can rapidly destabilize and rely on a belief that people act in good faith, and bigotry is inversely correlated to economics. Despite realizing at a young age about the atrocities of the past, I remained a hardcore optimist, especially when it came to technology. Maybe you can place me in the camp of believers that “technology solves all” – Ray Kurzweil and Peter Diamandis fall into this camp.
However, with technological advances that we’ve seen up to now about to be completely eclipsed by the advances we’re about to see over the next two decades – especially those around robotics, medicine, and artificial intelligence. The concern is that it will only take a small group of bad actors to control a majority of the population, and that those with access to advanced technology will be at such a huge advantage that those without might as well be considered powerless.
Technology has no morality of its own but is an instrument of the one who wields it. The issue that will arise is that so many of these new instruments will be so powerful that if someone chooses to use them for evil, the impact will be devastating.
Philosopher Nick Bostrom refers to these potentially calamity-inducing technologies as “black balls” in his Vulnerable World Hypothesis. If humanity discovers such a powerful technology, it’s bound to destroy itself with it. He proposes an argument for either curtailing technological advancement or putting a framework around technologies that could potentially lead to a black ball being drawn.
While I’m not convinced that we’re so vulnerable, I do fear what those wishing to do harm, even if for a self-professed higher purpose, could do so easily with what’s available today or what will be available soon.
It’s with this in mind that we should first take a look at what type of relationship we’re having with technology and what information we’re providing about ourselves. We need to understand what could happen if governments or bad actors decided to use this data for nefarious purposes. We need to also be mindful of what might be some new ways surveillance could be adapted to control or influence us even further.
Before we dive in, we need to acknowledge that most governments likely have full access to our private information. Backdoors have been found in everything from phones, routers, laptops, and even built into low level hardware control chips. These hacks not only affect consumer devices but mean that anything that passes through compromised infrastructure is susceptible to espionage. Exchanges, telecommunication infrastructure, and data centers have all been compromised and used for surveillance.
Maybe it’s possible to tolerate government agencies acting outside of the law when the political system overall is stable, there are checks and balances, a healthy and strong judiciary, and the economy is doing well.
But what about when those circumstances change?
The low hanging fruit for bad actors to pick on is anything spoken publicly that’s of negative sentiment against those in power. Any post on Facebook, any tweet, any Medium article. This doesn’t require a large army of people to oversee. Scripts that scrape these posts and new tools around language sentiment analyse can quickly automate this process. Posts can be flagged for follow up by humans to confirm. This effort can scale quickly and be implemented for continuous monitoring. Post something bad about someone in power and they’ll know about it.
This same public information can be used to influence. We’ve already had a small taste of this in the 2016 US Elections. Then, only a small group (several dozens to low hundreds), was able to sow enough dissent at the right time to harden political views and manipulate media and people into spreading misinformation or even running political events. This was just a smart deployment of bad actors – it wasn’t even fueled by AI or automation. What would happen if this were done at a larger scale? What if bots were able to do this?
Beyond what’s disclosed publicly by a population, surveillance can also be scaled through automation to make it very difficult for any dissident to walk in public without detection. Facial recognition is getting more accurate, faster, and requiring less computing. Now, you can scan a crowd and get every face identified within seconds. Unrecognized faces can be registered into the system to be tracked as they move to different camera views. Additional information on the age, gender, and emotional state of the person being tracked can also be assigned to the user. Evading such a surveillance system may require special makeup or disguises.
But you better not speak…
In 1984, microphones were everywhere. The characters were even concerned about microphones that were in the fields eavesdropping on them. Orwell might not have thought of a future where free-living people put far field microphone arrays throughout their homes and offices and carry around with them a 360 degree camera that also has microphones (heck, and wearing headsets and earbuds with mics on them just to make sure that government surveillance doesn’t miss a word).
When the Echo came out (and two years before that, when my fellow co-founders and I launched the Ubi), it’s commitment to privacy came through the fact that the device didn’t actually stream sound outside your home until you said “Alexa”. That meant nothing you said before “Alexa” was sent to the cloud or to Amazon for processing. The microphone would run software locally that would listen for the wake word, wake up the device, and the device would stream audio.
Since that original release, Amazon updated this local wake word to “cloud-based wake word verification”. Amazon can improve the performance of the wake word (fewer false triggers / false rejections) by buffering some of the audio on the device and then shipping this audio, along with the spoken word “Alexa” to Amazon. If it turns out that the device locally detected “Alexa” but the cloud service didn’t verify “Alexa” was heard, it shuts down the audio stream. However, half second of what you spoke before “Alexa” was heard is also shipped and retained by Amazon.
Recently, Amazon also announced “Alexa Guard”, which listens for audio events locally on the device when someone’s away from home and potentially alerts the person if something suspicious is heard (e.g. glass breaking). More capability to hear and understand is being brought to the “edge”, meaning that we can no longer rest assured that even if devices like an Echo aren’t streaming our live audio, that they can’t understand us and ship that context to an outside service.
Today, voice can be used for identification. Soon, it can be used to also determine your emotional state, gender, and in the not too distance, your health.
What’s also something to consider in using these devices is that, yes, Amazon has a clear intent of selling you things to make money, but it wouldn’t take much for a government to seize the operation of these devices, require them in all homes, and then update firmware so that audio is constantly being streamed and analyzed by a service for anti-government activities.
Google Home, Apple HomePod and Earpods, Amazon Echos, Ring and Nest Cams, and all sorts of connected devices provide a very large set of benefits for people. Our phones are magical connectivity devices bringing us instant information. Social media allow us to learn more about our friends and family and share what’s on our mind. All of these benefits come from the basis that we live an open and free society. However, we have to keep our eyes to the horizon and be ready to adjust how we relate to technologies when the ground shifts under our feet. The same benefit giving devices can quickly be turned into tools of imprisonment.
