Would you like Google to know how long you brush your teeth for? Unearthed patent reveals how Google plans to monitor and report household activities
In recent months, there are several reports of Google Home and Amazon Alexa ‘spying’ on users. Google has even filed a patent application that describes using a future release of its smart Home system to monitor and control everything from screen time and hygiene habits, to meal and travel schedules and other activities. According to the unearthed Google patent, the new application gives a detailed account of what can be deduced from household noises, including “the duration of teeth brushing, a speed of teeth brushing, etc.)”
Welcome to the brave new world. We now live in a world where technology has become an integral part of our everyday lives. Just less than two decades ago, there was no iPhone, no Facebook, no YouTube, no smart home appliances like Amazon Alexa and Google Home, and best of it all, most people didn’t even know what a “tweet” was.
Today, there are more smartphones and gadgets on the planet than there are people. What most people may not know is that these smart devices are powered by artificial intelligence (AI). For example, smartphone “personal assistants” like Siri and Google Assistant are powered by AI. These personal assistants can now send your messages, make dinner reservations for you, and even book appointments on your behalf.
In recent years, advances in AI are fundamentally transforming the way we live. Algorithms can already detect Parkinson’s disease and cancer, and control both cars and aircraft. The questions are: How will AI change our society in the future?
The Dark Side of AI
As is the case with the development of any new technology, many experts are worried about the negative aspects of AI, ranging from job loss to privacy concerns, machine ethics, AI, abuse, security issues, putting the power of AI into the wrong hands, general concern about new technology, and negative developments of superintelligence.
In 2019, the German public broadcast Deutsche Welle (DW) did a documentary on the pros and cons of artificial intelligence, machine learning, and algorithms. In one of the segments in the documentary, Physicist and Science Journalist Ranganathan Yogeshwar had an interview with the longest-serving Google employees in Munich Germany, Jens Redmer. As the interview went on, Redmer talked about how AI will transform society and makes lives better in the years to come. He also addressed the issue of AI transparency and why the users must have control of their own data.
That was when Yogeshwar raised the question about Google Home technology and the smart microphone sitting in people’s living rooms. In response, Redmer pointed out that Google Home isn’t eavesdropping. Redmer explained that there’s a small chip in Google Home that listens out for the so-called “hot words” and waiting for the command: “Ok Google or Hey Google” and “only then is the microphone switched on to send a voice command for search requests into the Internet and Google server, it then presents the results.”
Yogeshwar later showed Redmer a Google patent application filed with the United States Patent and Trademark Office (USPTO) on September 8, 2016. Redmer denied having any knowledge of the patent application. He went on to say Google files new patent applications every year. Redmer added that most of them are for imaginary or fictitious services.
Titled “Monitoring and reporting household activities in the smart home according to a household policy,” Google describes a future release of its Smart Home device as follows:
“Embodiments provided herein relate to monitoring and reporting household activities. In one embodiment, a method includes: monitoring, via a smart device, one or more activities associated with: a household; analyzing, via the smart device, a processor, or both, at least one characteristic of the one or more activities to discern information about the household; and reporting, via the device, the discerned information.”
However, as you read further down into the patent, things start to get really creepy. The Google NEST application programming interface (API) will connect to the internet, and also connect to Utilities, Businesses, Government, Academic Institutions, and Charities. Besides, the word “teeth” appears 14 times in the patent. Below are excerpts from the patent about how the new release of the Google Home appliance will detect and collect data on teeth brushing.
“As will be discussed in more detail below, in one embodiment, the pattern detection circuitry 204 may detect that water is typically left on when a member of the household brushes their teeth. This pattern may be provided to the household policy manager 202, which may suggest and/or automatically implement a policy to provide notification to members of the household when they leave the water on when a member of the household brushes their teeth.”
That’s not all. The patent also includes demographic information such as the age and the number of adults in a home, age and the number of children, number of pets, and many more. Then on page 20 of the 54-page document, the patent describes the bathroom and the detection of running water, sink, and shower before the appliance records and reports its findings.
The next page (page 23) of the patent shows how the future Google Home appliance detects teeth brushing (882 below).
Below is how Google explained the above diagram.
“FIG. 33 illustrates an example of process 880 for awareness regarding a bathroom Zone 618 activity. Specifically, the embodiment of FIG.33 begins by detecting that an occupant of the Zone 618 is brushing their teeth (block 882). For example, an audio signature and/or video signature may be associated with the Sounds and/or images of teeth brushing in the Zone 618. Next, additional characteristics may be determined (e.g., the sink 834 being left on, duration of teeth brushing, a speed of teeth brushing, etc.) (block 884). These findings may be reported and/or recorded within the system (e.g., for Subsequent control and/or reporting by the household policy manager 154) (block 886).”
There is more information in the patent than we shared in this article. As Science Journalist Yogeshwar shares in his documentary, the Google patent application also includes information about whether there’s an argument going on or whether a housemate is ill. You can watch the video below starting with Yogeshwar interview at Google Munich Office.
In closing, AI has immense potential to revolutionize every aspect of our daily life from medicine to mobility. However, there are many ethical questions about AI that still remained unanswered.