{"id":340678,"date":"2017-05-11T22:03:52","date_gmt":"2017-05-11T14:03:52","guid":{"rendered":"http:\/\/techcrunch.com\/?p=1488324"},"modified":"2017-05-11T22:03:52","modified_gmt":"2017-05-11T14:03:52","slug":"google-funded-super-sensor-project-brings-iot-powers-to-dumb-appliances","status":"publish","type":"post","link":"https:\/\/people.utm.my\/asmawisham\/google-funded-super-sensor-project-brings-iot-powers-to-dumb-appliances\/","title":{"rendered":"Google-funded \u2018super sensor\u2019 project brings IoT powers to dumb appliances"},"content":{"rendered":"<p><img decoding=\"async\" src=\"https:\/\/tctechcrunch2011.files.wordpress.com\/2017\/05\/screen-shot-2017-05-11-at-1-27-08-pm.png?w=738\" class=\"\"\/><\/p>\n<p id=\"speakable-summary\">The so-called \u2018smart home\u2019 often comes across looking incredibly dumb. Either you have to shell out lots\u00a0of money to replace perfectly functional appliances for their Internet-connected equivalents \u2014 which might then be vulnerable to hacking or whose functionality could\u00a0be bricked at manufacturer whim.<\/p>\n<p>Or you go around manually affixing\u00a0sensors to each\u00a0appliance and moveable\u00a0fixture in your home \u2014 and end up with the equivalent of interior pebble-dashing comprised\u00a0of stick-on\u00a0gadgetry; a\u00a0motion sensor and\/or ugly-looking <a target=\"_blank\" href=\"https:\/\/techcrunch.com\/2016\/11\/22\/amazon-announces-new-dash-powered-devices-that-can-auto-reorder-your-coffee-air-filters-and-more\/\" rel=\"noopener noreferrer\">Dash-style button<\/a>\u00a0on\u00a0everything.<\/p>\n<p>And that\u2019s before you even consider how, in inviting this bevy of connected device makers into your home, you\u2019re typically letting out a flow of\u00a0what can be\u00a0highly\u00a0sensitive personal\u00a0data to be sucked into the cloud for profit-seeking entities to pore over.<\/p>\n<p>Researchers at CMU\u2019s <a target=\"_blank\" href=\"http:\/\/www.figlab.com\" rel=\"noopener noreferrer\">Future Interfaces Group<\/a>\u00a0are taking a different approach to enable the sensing of\u00a0indoor environments, and reckon there\u2019s a quicker, less expensive and less cumbersome way to create what\u2019s at least a <em>smarter<\/em>\u00a0interior. And one that might have some privacy benefits too, depending on the deploying entity.<\/p>\n<p>What they\u2019ve built so far does not\u00a0offer as many\u00a0remote control options as a\u00a0fully fledged, IoT-enabled appliance scenario could. But if it\u2019s mostly signals\u00a0intelligence on what\u2019s going on indoors that you want \u2014 plus the ability to leverage that accrued real-time intel to support\u00a0contextually aware apps for the lived environment \u2014 their\u00a0approach looks very promising.<\/p>\n<p>The team is presenting\u00a0their <a target=\"_blank\" href=\"http:\/\/chrisharrison.net\/projects\/syntheticsensors\/syntheticsensors.pdf\">research<\/a> at the\u00a0ACM CHI Conference in Denver this week. They\u2019ve also produced the below demo video showing their test system in action.<\/p>\n<p><span class=\"embed-youtube\"><iframe loading=\"lazy\" class=\"youtube-player\" type=\"text\/html\" width=\"640\" height=\"390\" src=\"https:\/\/www.youtube.com\/embed\/aqbKrrru2co?version=3&amp;rel=1&amp;fs=1&amp;autohide=2&amp;showsearch=0&amp;showinfo=1&amp;iv_load_policy=1&amp;wmode=transparent\" allowfullscreen=\"true\">[embedded content]<\/iframe><\/span><\/p>\n<p>The system involves using a single custom plug-in sensor board\u00a0that\u2019s packed with multiple individual sensors \u2014 but, crucially from a privacy point of view, no camera. The custom sensor (shown\u00a0in the diagram below) uses\u00a0machine learning algorithms to process the data it\u2019s picking up, so it can be trained to identify various types of domestic activity, such as (non-smart)\u00a0appliances\u00a0being turned on \u2014 like a faucet, cooker or blender. It can\u00a0even identify things like cupboard doors or a microwave door being opened and closed; know which burner on your hob is on; and identify that\u00a0a\u00a0toilet has been flushed.<\/p>\n<p>So it\u2019s effectively a device that enables multiple synthetic sensors that are able to track lots of different\u00a0types of in-room activity \u2014 thereby getting around the tedium and unsightliness of needing to stick sensors on everything, while also eradicating all those potential points of failure (i.e. when physical sensors come unstuck or break or run out of battery power).<\/p>\n<p><a href=\"https:\/\/techcrunch.com\/2017\/05\/11\/google-funded-super-sensor-project-brings-iot-powers-to-dumb-appliances\/screen-shot-2017-05-11-at-7-39-34-am\/\" rel=\"attachment wp-att-1488426\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-1488426\" src=\"https:\/\/tctechcrunch2011.files.wordpress.com\/2017\/05\/screen-shot-2017-05-11-at-7-39-34-am.png?w=665&amp;h=357\" alt=\"\" width=\"665\" height=\"357\" srcset=\"https:\/\/tctechcrunch2011.files.wordpress.com\/2017\/05\/screen-shot-2017-05-11-at-7-39-34-am.png 665w, https:\/\/tctechcrunch2011.files.wordpress.com\/2017\/05\/screen-shot-2017-05-11-at-7-39-34-am.png?w=150&amp;h=81 150w, https:\/\/tctechcrunch2011.files.wordpress.com\/2017\/05\/screen-shot-2017-05-11-at-7-39-34-am.png?w=300&amp;h=161 300w\" sizes=\"(max-width: 665px) 100vw, 665px\"\/><\/a><\/p>\n<p>The idea is a \u201cquick and dirty\u201d smart home system that\u2019s aiming for general-purpose sensing in each\u00a0room it\u2019s located in, says CMU researcher Chris Harrison. And while <a target=\"_blank\" href=\"https:\/\/techcrunch.com\/2015\/02\/13\/scarab\/\" rel=\"noopener noreferrer\">others have also been thinking along similar multi-sensor lines<\/a>\u00a0this project has benefited from uplift by being part of a $500,000+ Google-funded IoT ecosystem research effort\u00a0aimed at encouraging the\u00a0development of an open ecosystem for connected devices.<\/p>\n<p>Google\u2019s\u00a02015\u00a0<a target=\"_blank\" href=\"http:\/\/services.google.com\/fh\/files\/misc\/opencall-openwebofthings.pdf\" rel=\"noopener noreferrer\">research proposal<\/a>\u00a0for that, which the CMU \u2018super sensor\u2019 project forms a part of, describes the main goal and priorities\u00a0as follows:<\/p>\n<blockquote readability=\"10\">\n<p>The mission of this program is to enable effective use and broad adoption of the Internet of Things by making it as easy to discover and interact with connected devices as it is to find and use information on the open web. The resulting open ecosystem should facilitate usability, ensure privacy and security, and above all guarantee interoperability.<\/p>\n<\/blockquote>\n<p>Harrison says he can\u2019t\u00a0discuss any specific plans\u00a0Google might have to commercialize the super sensor research. But there are some\u00a0pretty obvious\u00a0potential avenues for\u00a0the company to\u00a0plug something like this into its own product portfolio \u2014 say by using its <a target=\"_blank\" href=\"https:\/\/techcrunch.com\/2017\/04\/20\/google-home-can-now-recognize-up-to-six-voices-and-give-personalized-responses\/\" rel=\"noopener noreferrer\">Google Home voice-driven AI speaker<\/a>\u00a0as the central in-home interface that\u2019s being fed intelligence by a system of\u00a0super sensors. The\u00a0homeowner would\u00a0then be in a position to be informed of and ask about\u00a0domestic goings on via\u00a0that central IoT device.<\/p>\n<p>When I suggest\u00a0to Harrison a Google Home connected speaker could utilize the system to provide a layer of domestic intelligence for homeowners \u2014 such as by piping up to verbally warn them they\u2019ve left a tap on, or by keeping an internal running tally of the number of cups of coffee they\u2019ve brewed this month in case they want to know\u00a0\u2014 he agrees there\u2019s some clear\u00a0potential here, telling TechCrunch: \u201cYes certainly. Our sensor suite could go right into that product (or a Nest, Chromecast, etc.).\u201d<\/p>\n<p>As well as being able to sense primary indoor events \u2014 e.g. that a tap is running \u2014 the system allows for secondary inferences to be made, such as calculating how much water is being used because it knows how long the tap has been\u00a0running.<\/p>\n<p>Or monitoring a more complex chain of events (e.g.: a microwave door being opened and closed; cooking commencing; the door being opened and closed again; cooking continuing; and\u00a0cooking ending) in order to support the ability to create smart alerts for whether an appliance is available for use, for example. Or whether a dispenser\u00a0item might need restocking\u00a0or another type of appliance be in need of servicing \u2014 based on monitoring cumulative use over time.<\/p>\n<p>The demo video shows various scenarios for utilizing the system that don\u2019t involve a smart home at all \u2014 but rather applications that could\u00a0enable smart facilities management in an office or public bathroom setting, such as counting paper towels dispensed to send an alert to replenish a unit, or estimating when white board pens might run out of ink based on tracking how much ink is being used. Or to monitor activity\u00a0in an industrial workshop environment where the system is\u00a0able to distinguish between different\u00a0tools in use \u2014 with obvious potential safety benefits.<\/p>\n<p>General-purpose tracking in a commercial setting certainly has plenty of possible advantages \u2014 be it alerts to replenish supplies before they run out, or to notify service staff\u00a0when an appliance isn\u2019t\u00a0functioning properly. And generally to keep the environment running smoothly and efficiently.<\/p>\n<p>But inside the home such\u00a0persistent, continuous and potentially powerful activity monitoring can\u00a0start to look a bit, well, creepy.<\/p>\n<p>On the privacy front a feature baked\u00a0into the system means\u00a0that raw sensor data at least never leaves the board \u2014 so there\u2019s no raw audio being sucked into the cloud, for example. \u201cWe featurize everything on the board so that the signal is not recoverable,\u201d says Harrison when I suggest this vision of an all-knowing smart home could be\u00a0a bit dystopic.\u201dThere is no audio or anything transmitted to the cloud.<\/p>\n<p>\u201cIf a developer wants to build an app that does something when e.g., your coffee is ready, they don\u2019t ever get to see raw data. \u00a0Instead, they subscribe to that synthetic sensor feed of \u201ccoffee ready\u201d \u2014 and thats all they get, which helps to protect privacy.\u201d<\/p>\n<p>But he\u00a0also agrees the commercial and industrial use-cases are \u201cparticularly powerful\u201d, with the clear potential\u00a0for safety benefits and cost-savings across an entire workforce, adding:\u00a0\u201cImagine if a restaurant or supermarket knew what was going on automatically with smart sensors \u2014 currently, they know nothing \u2014 \u00a0to \u2018sense\u2019 anything, they have humans walk around.\u201d<\/p>\n<p>The system does have some limitations of course. Not least the lack of ability to remote control appliances given they\u00a0are not themselves wired to the Internet (though that\u2019s arguably\u00a0a potential benefit if you\u2019re worried about hackers breaking into and wreaking havoc via\u00a0your Internet-connected oven.)<\/p>\n<p>Another limitation is domestic chaos. So if lots of appliances and domestic activity is going on at once things could get pretty confusing for the detection system. On this Harrison confirms: \u201cIt can degrade if there are lots of noisy things going on.\u201d Though he also says different appliances can trigger different sensing channels \u2014 so some types of activity would presumably still be able to cut through the noise.<\/p>\n<p>\u201cIf you are running your dishwasher, and coffee grinder and toaster and blender all at the same time, it is likely to only recognize a few of those at the same time (though it\u2019ll recognize the high level state that the kitchen is in use),\u201d he adds.<\/p>\n<p>The CMU team kitted out five different rooms with sensor boards (one per room) for the\u00a0demo system. And each board powered on average eight\u00a0synthetic sensors, according to Harrison, who says\u00a0the average accuracy across all of those deployments \u2014 after about a week of learning signals \u2014 was a pretty impressive 98 per cent.<\/p>\n<p>Of course the system does also need\u00a0to be trained. So that\u2019s another potential limitation \u2014 in that there might\u00a0need to be\u00a0a pretty involved set up process\u00a0during which people have to introduce various appliances and features\u00a0of their home so the algorithms can get to know what they\u2019re sensing. But\u00a0Harrison says a library of known appliances can also be hosted\u00a0in the cloud to take some of the strain.<\/p>\n<p>\u201cOnce the machine learning knows what a blender sounds like, it can rain that classifier down to everyone (so users don\u2019t have to train anything themselves),\u201d he notes.<\/p>\n<p>How easy would it be for something like this synthetic sensor system to be commercialized? Harrison says the team has already built a \u201cpretty tightly integrated\u201d board and a \u201ccomprehensive backend\u201d so while \u201cit\u2019s not commercializable yet\u201d he reckons \u201cwe are well on our way\u201d.<\/p>\n<p>Albeit, he\u2019s not giving any possible timeframes for a marketplace deployment \u2014 perhaps given Google\u2019s involvement.<\/p>\n<p>He says the team is continuing to work on the project, with what sounds like continuing financial backing from Mountain View \u2014 although, again, he says he can\u2019t say too much about \u201cnext steps\u201d. So set your Alphabetic assumptions accordingly.<\/p>\n<p>\u201cWhat we are focusing on now is moving to whole-building deployments, where a sparse sensor network (a la one board per room) and sense everything going on,\u201d he adds. \u201cWe\u2019re also using deep learning to automatically identify appliances\/devices, so users never have to configure anything. \u00a0Truly plug and play.\u201d<\/p>\n","protected":false},"excerpt":{"rendered":"<p><img loading=\"lazy\" decoding=\"async\" width=\"680\" height=\"376\" src=\"https:\/\/tctechcrunch2011.files.wordpress.com\/2017\/05\/screen-shot-2017-05-11-at-1-27-08-pm.png?w=680\" class=\"attachment-large size-large wp-post-image\" alt=\"\" srcset=\"https:\/\/tctechcrunch2011.files.wordpress.com\/2017\/05\/screen-shot-2017-05-11-at-1-27-08-pm.png?w=680 680w, https:\/\/tctechcrunch2011.files.wordpress.com\/2017\/05\/screen-shot-2017-05-11-at-1-27-08-pm.png?w=150 150w, https:\/\/tctechcrunch2011.files.wordpress.com\/2017\/05\/screen-shot-2017-05-11-at-1-27-08-pm.png?w=300 300w, https:\/\/tctechcrunch2011.files.wordpress.com\/2017\/05\/screen-shot-2017-05-11-at-1-27-08-pm.png?w=768 768w, https:\/\/tctechcrunch2011.files.wordpress.com\/2017\/05\/screen-shot-2017-05-11-at-1-27-08-pm.png 840w\" sizes=\"(max-width: 680px) 100vw, 680px\" \/>&nbsp;Researchers at CMU&#8217;s Future Interfaces Group reckon they&#8217;ve come up with a quicker, less expensive and less cumbersome way to create a smart home. And one that might have some privacy benefits too. Just plug in their single &#8216;super sensor&#8217; and play&#8230; <a href=\"https:\/\/techcrunch.com\/2017\/05\/11\/google-funded-super-sensor-project-brings-iot-powers-to-dumb-appliances\/?ncid=rss\">Read More<\/a><\/p>\n<div class=\"feedflare\">\n<a href=\"http:\/\/feeds.feedburner.com\/~ff\/Techcrunch?a=Q8nTRkJ3nY0:Pu8jLFT8llE:2mJPEYqXBVI\"><img decoding=\"async\" src=\"http:\/\/feeds.feedburner.com\/~ff\/Techcrunch?d=2mJPEYqXBVI\" border=\"0\"><\/img><\/a> <a href=\"http:\/\/feeds.feedburner.com\/~ff\/Techcrunch?a=Q8nTRkJ3nY0:Pu8jLFT8llE:7Q72WNTAKBA\"><img decoding=\"async\" src=\"http:\/\/feeds.feedburner.com\/~ff\/Techcrunch?d=7Q72WNTAKBA\" border=\"0\"><\/img><\/a> <a href=\"http:\/\/feeds.feedburner.com\/~ff\/Techcrunch?a=Q8nTRkJ3nY0:Pu8jLFT8llE:yIl2AUoC8zA\"><img decoding=\"async\" src=\"http:\/\/feeds.feedburner.com\/~ff\/Techcrunch?d=yIl2AUoC8zA\" border=\"0\"><\/img><\/a> <a href=\"http:\/\/feeds.feedburner.com\/~ff\/Techcrunch?a=Q8nTRkJ3nY0:Pu8jLFT8llE:-BTjWOF_DHI\"><img decoding=\"async\" src=\"http:\/\/feeds.feedburner.com\/~ff\/Techcrunch?i=Q8nTRkJ3nY0:Pu8jLFT8llE:-BTjWOF_DHI\" border=\"0\"><\/img><\/a> <a href=\"http:\/\/feeds.feedburner.com\/~ff\/Techcrunch?a=Q8nTRkJ3nY0:Pu8jLFT8llE:D7DqB2pKExk\"><img decoding=\"async\" src=\"http:\/\/feeds.feedburner.com\/~ff\/Techcrunch?i=Q8nTRkJ3nY0:Pu8jLFT8llE:D7DqB2pKExk\" border=\"0\"><\/img><\/a> <a href=\"http:\/\/feeds.feedburner.com\/~ff\/Techcrunch?a=Q8nTRkJ3nY0:Pu8jLFT8llE:qj6IDK7rITs\"><img decoding=\"async\" src=\"http:\/\/feeds.feedburner.com\/~ff\/Techcrunch?d=qj6IDK7rITs\" border=\"0\"><\/img><\/a>\n<\/div>\n<p><img loading=\"lazy\" decoding=\"async\" src=\"http:\/\/feeds.feedburner.com\/~r\/Techcrunch\/~4\/Q8nTRkJ3nY0\" height=\"1\" width=\"1\" alt=\"\"\/> <\/p>\n","protected":false},"author":5817,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_et_pb_use_builder":"","_et_pb_old_content":"","_et_gb_content_width":"","footnotes":""},"categories":[25],"tags":[59,66,26],"class_list":["post-340678","post","type-post","status-publish","format-standard","hentry","category-technology","tag-media","tag-techcrunch","tag-technology"],"_links":{"self":[{"href":"https:\/\/people.utm.my\/asmawisham\/wp-json\/wp\/v2\/posts\/340678","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/people.utm.my\/asmawisham\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/people.utm.my\/asmawisham\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/people.utm.my\/asmawisham\/wp-json\/wp\/v2\/users\/5817"}],"replies":[{"embeddable":true,"href":"https:\/\/people.utm.my\/asmawisham\/wp-json\/wp\/v2\/comments?post=340678"}],"version-history":[{"count":0,"href":"https:\/\/people.utm.my\/asmawisham\/wp-json\/wp\/v2\/posts\/340678\/revisions"}],"wp:attachment":[{"href":"https:\/\/people.utm.my\/asmawisham\/wp-json\/wp\/v2\/media?parent=340678"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/people.utm.my\/asmawisham\/wp-json\/wp\/v2\/categories?post=340678"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/people.utm.my\/asmawisham\/wp-json\/wp\/v2\/tags?post=340678"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}