When someone talks about AI, or machine learning, or deep convolutional networks, what they\u2019re really talking about is \u2014 as is the case for so many computing concepts \u2014 a lot of carefully manicured math. At the heart of these versatile and powerful networks is a volume of calculation only achievable by the equivalent of supercomputers. More than anything else, this computational cost is what is holding back applying AI in devices of comparatively little brain: phones, embedded sensors, cameras.<\/p>\n
If that cost could be cut by a couple orders of magnitude, AI would be unfettered from its banks of parallel processors and free to inhabit practically any device \u2014 which is exactly what a breakthrough at the Allen Institute for AI makes possible.<\/p>\n
XNOR.ai is, essentially, a bit of clever computer-native math that enables AI-like models for vision, speech recognition, to run practically anywhere. It has the potential to be transformative for the industry.<\/p>\n
\u201cThere\u2019s a disconnect between state of the art AI and common computing,\u201d explained Ali Farhadi, senior research manager on the project at AI2, as they call the place. Nestled on the waterfront nearby Gasworks Park in Seattle, AI2 is focused on \u201cAI for the common good\u201d; it\u2019s small, yet even so is the largest not-for-profit AI research institution in the country.<\/p>\n
Machine learning, Farhadi continued, tends to rely on convolutional neural networks; these involve repeatedly performing simple but extremely numerous operations on good-sized matrices of numbers. But because of the nature of the operations, many have to be performed serially rather than in parallel. (Whether machine learning models truly constitute AI<\/a> is another, so far unanswered question<\/a>, but for now we\u2019ll use AI in its broader sense.)<\/p>\n For example: it\u2019s simple to multiply each in a set of a thousand numbers by two, since all those operations are independent from one another and can be performed at the same time by a thousand processors or threads working in parallel.<\/p>\n But imagine each operation depends on the result of the previous one \u2014 you need to add the last digit of the product of the previous operation to the next one, say. That means you have to go through one by one, which means only one processor can work on it \u2014 which means it could take a thousand times as long as the other one, even though really, it\u2019s still simple arithmetic. (I\u2019m oversimplifying, but this is the essential nature of the problem.)<\/p>\n It\u2019s the unfortunate reality of both training and running the machine learning systems performing all these interesting feats of AI that they phenomenally computationally expensive.<\/p>\n \u201cIt\u2019s hard to scale when you need that much processing power,\u201d Farhadi said. Even if you could fit the \u201cbeefy\u201d \u2014 his preferred epithet for the GPU-packed servers and workstations to which machine learning models are restricted \u2014 specs into a phone, it would suck the battery dry in a minute. Meanwhile, the accepted workaround is almost comically clumsy when you think about it: you take a load of data you want to analyze, send it over the internet to a datacenter where the AI actually lives, and computers perhaps a thousand miles away work at top speed to calculate the result, hopefully getting back to you within a second or two.<\/p>\n It\u2019s not such a problem if you don\u2019t need that result right away, but imagine if you had to do that in order to play a game on the highest graphical settings; you want to get those video frames up ASAP, and it\u2019s impractical (not to mention inelegant) to send them off to be resolved remotely. But improvements to both software and hardware have made it unnecessary, and our ray-traced shadows and normal maps are applied without resorting to distant datacenters.<\/p>\n Farhadi and his team wanted to make this possible for more sophisticated AI models. But how could they cut the time required to do billions of serial operations?<\/p>\nWhere\u2019s the beef?<\/h2>\n