In the fields of mobile phones, security monitoring, automobiles, and augmented reality/virtual reality (AR/VR), all types of embedded applications that are being developed or planned involve neural networks, and neural network applications are exploding. The field of neural network innovation is extraordinary, its own architecture is constantly updated, and new networks, new applications and markets are also emerging in endlessly. With the deepening and complexity of neural network applications, the requirements for computing performance are increasing day by day. In less than 4 years, the MAC/frame computing requirements have increased approximately 16 times (see Figure 1).
Figure 1 The growth of MAC/frame computing requirements
With the development of neural networks, the demand for embedding processors (rather than using CPUs and GPUs) in devices continues to increase. However, the processing power and operating speed of the network have not kept up with the development requirements of neural network applications. This conflict is particularly obvious in the field of vision applications. Up to now, meeting the needs of neural network applications can only rely on the resources of traditional data centers. However, as security and latency become important considerations, it is becoming more common to implement neural networks through embedded systems for real-time data processing. Although most neural network training can be done offline, applications that use neural networks must embed it in the system.
In all embedded applications, AR/VR or mixed reality faces unique challenges. Most of the devices in the above fields are wearable devices such as smart helmets, earphones, or smart glasses. They rely on battery power and are one of the most important considerations when choosing a neural network solution for energy consumption. Another important requirement for AR/VR applications is to reduce latency, so neural networks must implement device embedding. All these devices require some kind of image recognition, gesture recognition, stereo camera segmentation, 3D sensing, head tracking, eye detection, and eye tracking capabilities. There are many different imaging technologies, but over time, some of these functions, such as semantic environment understanding, gesture recognition, or image recognition, will all be realized through neural networks. In addition to imaging/visual neural networks, these devices also place requirements on sound/audio neural networks to receive voice commands.
In today's rapidly changing technological environment, AR/VR equipment manufacturers need to immediately select platforms for products to be marketed in 2019, 2020, and even later. After the introduction of the new neural network, due to the continuous changes in its architecture, we cannot ensure the effectiveness of the current effective work platform in the future system. In addition, these applications also require low latency and low power consumption, which is also particularly important; but in view of the continuous growth of neural network requirements and the continuous progress of this trend, we still need to ensure a certain degree of flexibility and forward-looking.
Currently, there are two main options for implementing neural networks: CPU/GPU or using hardware accelerators and matching imaging DSP. These two options can each solve some of the challenges faced by designers; but both have some unsatisfactory compromises in terms of ease of development, power efficiency, latency, future upgrade space or performance. Hardware accelerator and matching imaging DSP is one of the choices of embedded devices, but this combination is inefficient and will generate unnecessary power consumption. In addition to development difficulties, the software must also be partitioned between the DSP and the accelerator. Only unloading the convolutional layer will significantly increase the burden of data transmission and affect efficiency. In addition, the hardware is fixed at the time of tapeout, so these accelerators will not have room for future upgrades.
Neural network DSP solutions that meet the needs of embedded applications must meet the following requirements: easy to develop, able to handle huge amounts of data, have room for future upgrades, use energy efficiently and minimize delay.
Cadence solution: Tensilica Vision C5 digital signal processor (DSP)
As an optimized solution for vision and fusion sensor applications, Cadence Tensilica Vision C5 DSP is the industry's first DSP dedicated to neural network processing and suitable for multi-processor architecture. This solution achieves unprecedented speed and low power consumption, and meets all the requirements of high-end neural network technology.
The solution is based on nearly 20 years of Xtensa multi-processor experience, with features such as memory structure sharing, interrupts allowed, synchronized queues, and synchronized multi-processor debugging. Vision C5 DSP can realize the calculation acceleration of all neural network layers (convolutional layer, fully connected layer, pooling layer and normalization layer), not just the convolutional layer function. Therefore, the ability of the main vision processing DSP is released to run image enhancement applications independently; while the Vision C5 DSP runs inference tasks. By removing the redundant data transmission of the hardware accelerator, the power consumption of the Vision C5 DSP is much lower than that of the existing neural network accelerator.
Vision C5 DSP has a computing power of 1TMAC/sec, which can meet the ever-increasing computing requirements of neural networks; it also has accurate calculations, has a multi-core design architecture, and supports multi-TMAC embedded solutions. Vision C5 DSP is aimed at applications that often run multiple neural networks. Due to its programmable characteristics, the solution has room for future upgrades and can support new layers as the design changes.
The vision processing system must be designed comprehensively, applicable to all platforms, and simultaneously develop hardware and software. In order to develop this technology, designers must use tools and IP that support efficient algorithms, and the hardware platform used must also meet the target cost and power consumption requirements of each application. From a system perspective, Cadence can support designers of embedded vision devices to develop transformative products as quickly and efficiently as possible.
Our other product:
