Intel Neural Compute Stick 2 Launched for AI, Deep Learning Without the Cloud

Intel has launched the next version of its Neural Compute Stick, a simple USB device that the company says can greatly accelerate AI processing and deep learning inference on consumer PCs. It is aimed at use cases where neural networks need to be deployed without a connection to cloud-based computing resources. The new Neural Compute Stick 2 uses the Intel Movidius Myriad X VPU (visual processing unit). Intel acquired Movidius, a computer vision processing startup, in 2016. The device is priced at $99 (approximately Rs. 7,137).

The Neural Compute Stick 2 is slightly larger than a thumb drive and plugs into a standard USB 3.0 port. The device is fanless and does not require additional power. It can be used with pre-trained neural networks for smart cameras, IoT devices, robotics, drones, and VR hardware. It can also be used to test AI models “at the edge”, which means on a local machine rather than in the cloud, for quick turnaround or when sensitive data cannot be sent to a remote server. Intel has previously showed demos of the first-gen Neural Compute Stick recognising and classifying people and objects in a video stream in real-time, which could be used in security applications or for traffic management when low latency is important.

Intel claims up to 8X better performance in certain situations with the Neural Compute Stick 2 compared to its predecessor. The Myriad X VPU has a dedicated neural compute engine, 16 programmable compute cores, and logic for processing dual 720p video streams. The device currently works with Linux on standard PCs or Raspberry Pi computers, but Intel has said that support for Windows ML is coming soon. The Tensorflow and Caffe frameworks are supported, and Intel distributes its own version of the OpenVino toolkit for accelerating computer vision performance.

According to Intel, 43 percent of AI tasks will need to be processed at the edge rather than in the cloud by 2023, and there will be a 15X growth in the number of edge devices that support AI processing. The company expects AI capabilities to become an important part of client PC workloads in the future. Nearly 70 percent of Indian companies might deploy AI by 2020, according to the company's research.

In addition to its acquisition of Movidius, Intel also bought AI startup Nervana Systems and deep learning company Vertex.AI over the past few years. Intel also recently said it had sold over $1 billion worth of AI-capable processors in 2017. The company recently held an AI DevCon conference in Bengaluru where it announced that it has trained over 99,000 Indian students, teachers and developers in AI and is working on improving the accessibility of AI development tools in India.