{"id":337697,"date":"2017-04-06T01:48:22","date_gmt":"2017-04-05T17:48:22","guid":{"rendered":"http:\/\/techcrunch.com\/?p=1473211"},"modified":"2017-04-06T01:48:22","modified_gmt":"2017-04-05T17:48:22","slug":"google-says-its-custom-machine-learning-chips-are-often-15-30x-faster-than-gpus-and-cpus","status":"publish","type":"post","link":"https:\/\/people.utm.my\/asmawisham\/google-says-its-custom-machine-learning-chips-are-often-15-30x-faster-than-gpus-and-cpus\/","title":{"rendered":"Google says its custom machine learning chips are often 15-30x faster than GPUs and CPUs"},"content":{"rendered":"

<\/p>\n

It\u2019s no secret<\/a> that Google has developed its own custom chips to accelerate its machine learning algorithms. The company first revealed those chips, called Tensor Processing Units (TPUs), at its I\/O developer conference back in May 2016, but it never went into all that many details about them, except for saying that they were optimized around the company\u2019s own TensorFlow<\/a> machine-learning framework. Today, for the first time, it\u2019s sharing more details<\/a> and benchmarks<\/a> about the project.<\/p>\n

If you\u2019re a chip designer, you can find all the gory<\/del> glorious details of how the TPU works in Google\u2019s paper<\/a>. The numbers that matter most here, though, are that based on Google\u2019s own benchmarks (and it\u2019s worth keeping in mind that this is Google evaluating its own chip), the TPUs are on average 15x to 30x faster in executing Google\u2019s regular machine learning workloads than a standard GPU\/CPU combination (in this case, Intel Haswell processors and Nvidia K80 GPUs). And because power consumption counts in a data center, the TPUs also offer 30x to 80x higher TeraOps\/Watt (and\u00a0with using faster memory in the future, those numbers will probably increase).<\/p>\n

\"\"<\/a><\/p>\n

It\u2019s worth noting that these numbers are about using machine learning models in production, by the way \u2014 not about creating the model in the first place.<\/p>\n

Google also notes that while most architects optimize their chips for convolutional neural networks<\/a> (a specific type of neural network that\u2019s works well for image recognition, for example). Google, however, says, those networks only account for about 5 percent of its own data center workload while the majority of its applications use multi-layer perceptrons<\/a>.<\/p>\n

Google says it started looking into how it could use GPUs, FPGAs and custom ASICS (which is\u00a0essentially what the TPUs are) in its data centers back in 2006. At the time, though, there weren\u2019t all that many applications that could really benefit from this special hardware because most of the heavy workloads they required could just\u00a0make use of the excess hardware that was already available in the data center anyway. \u201cThe conversation changed in 2013 when we projected that DNNs could become so popular that they might double computation demands on our data centers, which would be very expensive to satisfy with conventional CPUs,\u201d the authors of Google\u2019s paper write. \u201cThus, we started a high-priority project to quickly produce a custom ASIC for inference (and bought off-the-shelf GPUs for training).\u201d The goal here, Google\u2019s researchers say, \u201cwas to improve cost-performance by 10x over GPUs.\u201d<\/p>\n

Google isn\u2019t likely to\u00a0make the TPUs available outside of its own cloud, but the company notes that it expects that others will take what it has learned and \u201cbuild successors that will raise the bar even higher.\u201d<\/p>\n

\"\"<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

\"\" It’s no secret that Google has developed its own custom chips to accelerate its machine learning algorithms. The company first revealed those chips, called Tensor Processing Units (TPUs), at its I\/O developer conference back in May 2016, but it never went into all that many details about them, except for saying that they were optimized around the company’s own TensorFlow… Read More<\/a><\/p>\n

\n<\/img><\/a> <\/img><\/a> <\/img><\/a> <\/img><\/a> <\/img><\/a> <\/img><\/a>\n<\/div>\n

\"\"\/ <\/p>\n","protected":false},"author":5817,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_et_pb_use_builder":"","_et_pb_old_content":"","_et_gb_content_width":"","footnotes":""},"categories":[25],"tags":[59,66,26],"class_list":["post-337697","post","type-post","status-publish","format-standard","hentry","category-technology","tag-media","tag-techcrunch","tag-technology"],"_links":{"self":[{"href":"https:\/\/people.utm.my\/asmawisham\/wp-json\/wp\/v2\/posts\/337697","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/people.utm.my\/asmawisham\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/people.utm.my\/asmawisham\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/people.utm.my\/asmawisham\/wp-json\/wp\/v2\/users\/5817"}],"replies":[{"embeddable":true,"href":"https:\/\/people.utm.my\/asmawisham\/wp-json\/wp\/v2\/comments?post=337697"}],"version-history":[{"count":0,"href":"https:\/\/people.utm.my\/asmawisham\/wp-json\/wp\/v2\/posts\/337697\/revisions"}],"wp:attachment":[{"href":"https:\/\/people.utm.my\/asmawisham\/wp-json\/wp\/v2\/media?parent=337697"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/people.utm.my\/asmawisham\/wp-json\/wp\/v2\/categories?post=337697"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/people.utm.my\/asmawisham\/wp-json\/wp\/v2\/tags?post=337697"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}