{"id":340469,"date":"2017-05-10T00:35:25","date_gmt":"2017-05-09T16:35:25","guid":{"rendered":"http:\/\/techcrunch.com\/?p=1487153"},"modified":"2017-05-10T00:35:25","modified_gmt":"2017-05-09T16:35:25","slug":"facebook-posts-its-fast-and-accurate-convnet-models-for-machine-translation-on-github","status":"publish","type":"post","link":"https:\/\/people.utm.my\/asmawisham\/facebook-posts-its-fast-and-accurate-convnet-models-for-machine-translation-on-github\/","title":{"rendered":"Facebook posts its fast and accurate ConvNet models for machine translation on GitHub"},"content":{"rendered":"

<\/p>\n

In its latest paper<\/a>, the Facebook AI Research (FAIR)\u00a0team dropped some impressive results<\/a> for its\u00a0implementation\u00a0of a modified convolutional neural network for machine translation. Facebook says it has achieved a small bump in\u00a0accuracy at nine times the speed of\u00a0traditional recurrent network models. And to complement its research, the company is releasing its pre-trained models on GitHub along with all the tools needed to replicate the results on your own.<\/a><\/p>\n

When most of us think of machine translation, we think of Google Translate (sorry Facebook and my 8th grade Spanish teacher). But while that is certainly the most well known\u00a0implementation, Facebook relies on the technology extensively for translating posts on News Feed, among other uses.<\/p>\n

In these use cases accuracy is important to a deliver competitive\u00a0experience, but arguably speed is even more important to Facebook. With nearly two billion users, every incremental improvement in speed is magnified. And Facebook isn\u2019t just promising an incremental improvement, they\u2019re promising an improvement of 900 percent.\u00a0\"\"<\/p>\n

In terms of accuracy, the\u00a0gold standard for evaluating the quality of machine translation is BLEU, bilingual evaluation understudy.\u00a0Facebook benchmarked\u00a0its sequence to sequence convolutional approach on three tasks\u00a0\u2014 \u00a0translating English to Romanian, translating English to German and translating English to French.<\/p>\n

\n
\n
\n

Denis Yarats and Michael Auli, two of the\u00a0authors on the paper, explained to me that these tasks weren\u2019t selected because they are\u00a0the hardest languages to translate, but because they are the most competitive and others have already achieved impressive BLEU scores using alternative methods.<\/p>\n<\/div>\n<\/div>\n<\/div>\n

Based on these numbers, applying ConvNets to machine translation is a worthy pursuit, but it\u2019s worth stepping back a bit to explore why recurrent nets are typically used over ConvNets for translation in the first place. Recurrent networks take into account time series information which makes them ideal for handling sequential tasks \u2014 reading left to right is a great example of this.<\/p>\n

ConvNets on the other hand have risen in prominence in recent years because of how useful they are for analyzing visual information. They process information simultaneously rather than sequentially which presents barriers if you want to use them for machine translation. To make it work, Facebook implemented what it calls \u201cmulti-hop attention.\u201d<\/p>\n

\"\"<\/p>\n

Machine translation is a two step process. As humans we take for granted the process of understanding a\u00a0sentence in our own language, but machines have to first put resources into this before they can output to another language.<\/p>\n

Another thing we don\u2019t realize is that everything we do is governed by probabilities.\u00a0\u201cBait\u201d for example can be both a noun and a verb and when we evaluate a sentence we are subconsciously assigning\u00a0likelihoods that help us interpret meaning. This requires us to reference other parts of a sentence\u00a0at different times to develop understanding.<\/p>\n

In something of a twist on this process, Facebook\u2019s multi-hop takes advantage of the simultaneous nature of\u00a0CovNets\u00a0to allow machines to reference different parts of text to build understanding during encoding. Once this is done and a vector representation is created, a translation can be outputted one word at a time until its complete.<\/p>\n

Yarats and Auli believe that their models can be engineered to do more than simply machine translation. Their ConvNet could be used in any scenario where a computer needs to understand text and express something, so this could include summarizing text or even interpreting a reading and then asking questions.<\/p>\n

Both reinforcement learning and adversarial networks have the potential to improve upon the results achieved by Facebook \u2014 each of these could become a standalone paper. Additionally, the team hopes to further experiment with the applications of\u00a0multi-hop attention.<\/p>\n

Featured Image: mariusz_prusaczyk\/Getty Images<\/small><\/p>\n","protected":false},"excerpt":{"rendered":"

\"\" In its latest paper, the Facebook AI Research (FAIR)\u00a0team dropped some impressive results for its\u00a0implementation\u00a0of a modified convolutional neural network for machine translation. Facebook says it has achieved a small bump in\u00a0accuracy at nine times the speed of\u00a0traditional recurrent network models. And to complement its research, the company is releasing its pre-trained models on… Read More<\/a><\/p>\n

\n<\/img><\/a> <\/img><\/a> <\/img><\/a> <\/img><\/a> <\/img><\/a> <\/img><\/a>\n<\/div>\n

\"\"\/ <\/p>\n","protected":false},"author":5817,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_et_pb_use_builder":"","_et_pb_old_content":"","_et_gb_content_width":"","footnotes":""},"categories":[25],"tags":[59,66,26],"class_list":["post-340469","post","type-post","status-publish","format-standard","hentry","category-technology","tag-media","tag-techcrunch","tag-technology"],"_links":{"self":[{"href":"https:\/\/people.utm.my\/asmawisham\/wp-json\/wp\/v2\/posts\/340469","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/people.utm.my\/asmawisham\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/people.utm.my\/asmawisham\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/people.utm.my\/asmawisham\/wp-json\/wp\/v2\/users\/5817"}],"replies":[{"embeddable":true,"href":"https:\/\/people.utm.my\/asmawisham\/wp-json\/wp\/v2\/comments?post=340469"}],"version-history":[{"count":0,"href":"https:\/\/people.utm.my\/asmawisham\/wp-json\/wp\/v2\/posts\/340469\/revisions"}],"wp:attachment":[{"href":"https:\/\/people.utm.my\/asmawisham\/wp-json\/wp\/v2\/media?parent=340469"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/people.utm.my\/asmawisham\/wp-json\/wp\/v2\/categories?post=340469"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/people.utm.my\/asmawisham\/wp-json\/wp\/v2\/tags?post=340469"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}