We hosted a very informative and engaging session on ‘Semiconductor Sustainability’ with Sean Varley, Chief Evangelist, Ampere Computing, a designer of energy-efficient, cloud native processors. Here is a summary of key points made.
End of the road for conventional Processors?
The vast majority of compute in data centers today is based on old technology developed when power consumption was not a factor and code was mostly monolithic. Today with the move to cloud, applications are built on microservices architecture; a typical web service may touch thousands of servers and may encounter millions of requests per second. Further, power consumption of processors has been rising steeply since at least the year 2020, and has crossed 500W for CPUs and over 1200W for GPUs. This calls for a new approach to computing that is both more energy efficient, and allows scaling-out.
Enter processors with cloud-native architecture
Processors based on cloud native architecture, such as those designed by Ampere, offer up to double the performance at half the energy consumption, potentially delivering 4X shift in efficiency. Thus for web workloads with service level constraints, only a third as many racks deploying cloud-native processors may be needed compared with the legacy X86 servers. This reduces both the operational carbon (from lower energy consumption to process the workload) and embodied carbon emissions (by requiring fewer racks) by nearly 60% each.
Why is cloud-native architecture more efficient?
The cloud-native architecture is single-threaded, providing the entire compute infrastructure – cache, ISA pipeline and Vector Unit – in each core – unlike the dual-threaded architecture of legacy x86, where two threads share the same infrastructure. These smaller cores consume less energy and allow microservice to run to completion, allowing faster processing time and lower energy consumption. Further, each core has two vector units (primary computational unit), allowing further optimization and therefore energy efficiency.
Why is this relevant for sustainable computing?
Data centers have been getting larger and larger, creating an ever greater strain on the grids. This is likely to force data centers to start shrinking again. However, typically smaller data centers are also less energy efficient (higher PUE). Since cloud-native processors are more energy efficient, they can more than compensate for this loss of energy efficiency. Further, as AI workloads shift from training to inferencing (reaching 20-80 split), compute will need to move closer to the users, leading to a proliferation of smaller data centers. Cloud-native processors are ideally suited for inference type of workloads and can help in bending the energy curve of AI.
Where will further gain in performance and efficiency come from?
Future gains in performance and efficiency will come from using a combination of general purpose processor and domain-specific processor, which are optimized for specific workloads, such as graphics, tensor processing, video, data / network and AI graph. Such a pairing can have a big impact, if the total workload has more than 30% workload a particular type, say graphics.