{"id":337851,"date":"2017-04-07T03:45:35","date_gmt":"2017-04-06T19:45:35","guid":{"rendered":"http:\/\/techcrunch.com\/?p=1473350"},"modified":"2017-04-07T03:45:35","modified_gmt":"2017-04-06T19:45:35","slug":"microsoft-maluuba-teaches-management-101-to-machines-in-its-first-paper-since-being-acquired","status":"publish","type":"post","link":"https:\/\/people.utm.my\/asmawisham\/microsoft-maluuba-teaches-management-101-to-machines-in-its-first-paper-since-being-acquired\/","title":{"rendered":"Microsoft Maluuba teaches management 101 to machines in its first paper since being acquired"},"content":{"rendered":"

<\/p>\n

In mid-January, the ongoing race for AI put Montreal-based Maluuba<\/a> on our radar. Microsoft acquired the startup<\/a> and its team of researchers to build better machine intelligence tools for analyzing unstructured text to enable more natural human computer interaction \u2014 think bots that can actually respond with reasonable intelligence to a text you send. The team dropped its first paper<\/a> since being acquired\u00a0and it sheds light on the group\u2019s priorities.<\/p>\n

The paper outlines a method for multi-advisor reinforcement learning that breaks problems down to be\u00a0simpler and more easily computable.\u00a0In oversimplified terms, Maluuba is effectively trying to teach leadership to groups of machines working to solve problems.<\/p>\n

Problem<\/h2>\n

Existing conversational interfaces are rigid\u00a0and easily broken. Siri, Alexa and Cortana\u00a0are miles ahead of old-fashioned dialog trees, but they still are a far cry from generalized intelligence. From a computational standpoint, a complete model of the world would be infeasible to create so instead engineers create specialized machine intelligence tools that can perform well on a smaller number of\u00a0tasks. This is why you can ask Siri to make a phone call but can\u2019t ask it to organize a large dinner event.<\/p>\n

A lot of attention is being given to reinforcement learning, a specialized branch of machine learning. As I have explained previously<\/a>, reinforcement learning steals the idea of utility from economists in an effort to quantify and iteratively evaluate decision making. Instead of explicitly telling an autonomous car every rule of the road, it can be more effective to gamify the problem and assign figurative \u201cpoints\u201d that the intelligent system can optimize. The system could hypothetically\u00a0lose points for driving over a double yellow line and gain them for maintaining the speed limit.<\/p>\n

This allows for a much more adaptable system, but it unfortunately is still a rather complex problem requiring a lot of compute. This is where multi-advisor reinforcement learning comes in.<\/p>\n

\n
    \n
  1. \n
    \"\"<\/div>\n

    IMG_2149<\/h3>\n

    The Maluuba team working at their office.<\/p>\n<\/li>\n

  2. \n
    \"\"<\/div>\n

    IMG_5356<\/h3>\n

    The Maluuba team working at their office.<\/p>\n<\/li>\n

  3. \n
    \"\"<\/div>\n

    IMG_2183<\/h3>\n

    The Maluuba team working at their office.<\/p>\n<\/li>\n<\/ol><\/div>\n

    Solution<\/h2>\n

    The Maluuba team is\u00a0trying to solve these complexity problems that face reinforcement learning. Their approach is to use multiple \u201cadvisors\u201d to break the problem down into smaller, more digestible, chunks. Traditionally, a single virtual agent is\u00a0used for reinforcement learning but in recent years multi-agent approaches have become more common.<\/p>\n

    In a conversation, the group presented the example of an intelligent scheduling assistant. Rather than have a single agent learn to schedule every kind of optimal meeting, it could someday make sense to assign a different agent to different classes of meetings. The challenge is getting all these agents to work together in consonance.<\/p>\n

    Intuitively it\u2019s easy to imagine these agents as humans splitting up a task. Getting people to work together efficiently is no small task even though a divide and conquer strategy can outperform the lone wolf mentality.<\/p>\n

    The solution is to have an aggregator sit on top of all the \u201cadvisors\u201d to make a decision. Each advisor in Maluuba\u2019s paper has a different focus with respect to the grand problem being solved. Each agent gets a different reward for the action it specializes in. If agents take different positions, the aggregator steps in and arbitrates.<\/p>\n

    Maluuba used a simplified version of Ms. Pac-Man, called Pac-Boy to test different methods for it multi-advisor reinforcement aggregating learning framework. The team wants to study\u00a0the process of breaking down problems. Ideally there is\u00a0some universality in how problems can be organized around a number of optimal aggregators. This is another place where it\u2019s interesting to think about how humans decompose problems, often inefficiently \u2014 think leadership 101 for machines.<\/p>\n

    \n

    Why you should care<\/h2>\n

    Multi-advisor reinforcement learning can save CPU and GPU power. Breaking down a problem also makes it more easily\u00a0distributed to different servers for\u00a0paralyzed processing. Reduced complexity\u00a0is universally helpful\u00a0for all\u00a0reinforcement learning problems.<\/p>\n

    The research team explained to me that it\u2019s still early days for\u00a0working alongside\u00a0Microsoft. They\u2019re transitioning to Azure and building out communication channels between exiting machine learning teams. But when that process is complete, it strikes me that Maluuba will play a huge role in analyzing text and the language held within it.<\/p>\n

    While reinforcement learning itself isn\u2019t novel, Maluuba is pouring a lot of resources into it. We have already seen the potential of reinforcement learning in DeepMind\u2019s AlphaGo<\/a>. Future joint research projects could bring more efficient and adaptable reinforcement learning into new\u00a0consumer and enterprise facing dialog products for Microsoft.<\/p>\n<\/div>\n","protected":false},"excerpt":{"rendered":"

    \"\" In mid-January, the ongoing race for AI put Montreal-based Maluuba on our radar. Microsoft acquired the startup and its team of researchers to build better machine intelligence tools for analyzing unstructured text to enable more natural human computer interaction — think bots that can actually respond with reasonable intelligence to a text you send. The team dropped its first paper… Read More<\/a><\/p>\n

    \n<\/img><\/a> <\/img><\/a> <\/img><\/a> <\/img><\/a> <\/img><\/a> <\/img><\/a>\n<\/div>\n

    \"\"\/ <\/p>\n","protected":false},"author":5817,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_et_pb_use_builder":"","_et_pb_old_content":"","_et_gb_content_width":"","footnotes":""},"categories":[25],"tags":[59,66,26],"class_list":["post-337851","post","type-post","status-publish","format-standard","hentry","category-technology","tag-media","tag-techcrunch","tag-technology"],"_links":{"self":[{"href":"https:\/\/people.utm.my\/asmawisham\/wp-json\/wp\/v2\/posts\/337851","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/people.utm.my\/asmawisham\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/people.utm.my\/asmawisham\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/people.utm.my\/asmawisham\/wp-json\/wp\/v2\/users\/5817"}],"replies":[{"embeddable":true,"href":"https:\/\/people.utm.my\/asmawisham\/wp-json\/wp\/v2\/comments?post=337851"}],"version-history":[{"count":0,"href":"https:\/\/people.utm.my\/asmawisham\/wp-json\/wp\/v2\/posts\/337851\/revisions"}],"wp:attachment":[{"href":"https:\/\/people.utm.my\/asmawisham\/wp-json\/wp\/v2\/media?parent=337851"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/people.utm.my\/asmawisham\/wp-json\/wp\/v2\/categories?post=337851"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/people.utm.my\/asmawisham\/wp-json\/wp\/v2\/tags?post=337851"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}