PCIe is a standard interface for connecting PC components, such as graphics cards or storage devices, and is used in a wide range of devices, from embedded systems, handheld devices, industrial applications, and servers to supercomputers, etc. For consumers, upgrading the advantage of the later PCIe is that it can support faster memory.
What is PCle?
PCIe, also known as PCI Express® or Peripheral Component Interconnect Express, is a high-speed serial bus used in computers. This is a physical connection that transfers information and data from one device within a computer to another, or between a computer and a peripheral. The PCIe interface specification is developed and maintained by the PCI Special Interest Group, an alliance of more than 900 companies working together to achieve a common standard.
The application scenarios of PCIe are becoming more and more abundant, whether it is chip-to-chip or network card-to-network connection. Therefore, PCIe 6.0-related products have wide market demand, whether it is data centers, IoT, automobiles, or some government-related applications. As an important backbone for fast data transfer between various computing nodes such as CPUs, GPUs, FPGAs, and workload-specific accelerators, the PCI Express (PCIe) interface has become an important interface between chips in data centers and computing applications since its official creation in the early 21st century. The industry standard for data transfer. With the rapid development of the PCIe standard, in January 2022, the relevant specifications of PCIe 6.0 were officially released, marking the entry into the era of PCIe 6.0.
PCle performance:
One of the most important features of this standard is the multi-lane nature of the PCle connection. A channel is a single serial data connection, similar to a SATA connection. PCIe uses a four-lane connection on storage devices, allowing data transfer up to four times faster than a SATA connection.
All enhancements come with other benefits and reduce power consumption. The combination of NVME and PCle, coupled with the fact that SSDs have no moving parts, makes storage disks drain power slowly, increasing battery life in laptops and tablets. Using the PCle connector with the NVMe protocol creates faster SSDs.
Why is PCIe Needed? Which Drives Market Demand?
From existing applications to emerging applications represented by artificial intelligence and machine learning, more data is being generated continuously. According to IDC data, in the first half of 2021, the overall market size of China's big data platforms reached RMB 5.42 billion, a year-on-year increase of 43.5%.
To support the transmission and storage of such a huge amount of data, the data center must not only have a complete infrastructure structure but also need to adopt new computing models, such as disaggregated computing or composable computing. It also puts forward higher requirements for the improvement of the data transmission rate and the security of data transmission.
Specifically, computing models can become very specific to specific workloads, such as the difference between AI training and inference workloads, and the computing resources required in each case are specific. It can be seen that computing resources and computing models are becoming more homogeneous or decentralized. In this case, standard CPUs and AI accelerators are used in combination, and data is required to be transmitted between them at high speed and safety. This is A core requirement for the PCIe standard to move towards higher data transfer rates. Therefore, as specific workloads or computing resources are distributed or partitioned, the industry must move data over greater distances as quickly and securely as possible, such as the distance between a CPU and an AI accelerator, but requires a fast secure exchange of information.
PCIe has grown tremendously over time. From the launch of PCIe 1.0 to the current PCIe 6.0 era, its data transfer rate has been rising. Especially from PCIe 4.0, 5.0 to 6.0, the increase in data transfer rate is very significant. Analyzing the reasons behind it are mainly due to the explosive growth of data, and this growth is derived from a series of applications driven by artificial intelligence and machine learning.
Update Cycle of PCIe Specification:
- Release of new specifications: The starting point for the application of each generation of PCIe specifications is the completion and release of the new generation of PCIe specifications by the PCI-SIG organization. Before and after the release of the new specification, related component manufacturers will also launch control component products that match the new PCIe specification.
- Promotion period: About 12 to 18 months after the completion of the new PCIe specification, manufacturers will start to launch peripheral device products that support the new PCIe specification, such as SSD, high-speed network card, or GPU accelerator card, to promote and drive the new PCIe specification. Application of the first generation PCIe specification.
- Entering mainstream applications: 12 to 18 months after peripheral devices began to support the new PCIe specification, the processors and chipsets of computers and servers also began to support the new PCIe specification, allowing the new PCIe specification to enter computers and servers, thereby Started to provide applications for mainstream core platforms.
- Maturity period: 12 to 18 months after the server starts to support the new PCIe specification, external storage devices such as storage arrays also start to introduce the new PCIe specification. The update of the core architecture of storage devices is a generation later than that of servers. Therefore, when storage devices also start to adopt the new PCIe specification, it also means that this generation of PCIe specifications has entered a mature stage and has begun to be popularized.
Nowadays: The PCIe 6.0 Specification
The PCI-SIG organization released the official specification of PCIe 6.0, which can reach a transmission capacity of 64 GT/s, twice that of the PCIe 5.0 specification, and is compatible with the PCIe specification from the first generation to the fifth generation. PCIe 6.0 can provide high bandwidth, low power consumption, and low latency interconnection, so it is an ideal solution for data centers and many other data applications.
To further meet the requirements of PCIe 6.0 to achieve a data transfer rate of 64GT/s, its architecture must be re-optimized and adjusted. At the same time, there are huge differences in the ratio of the entire circuit and the transmission mechanism of the circuit. One of the biggest changes is the transition from NRZ to PAM4. NRZ and PAM4 have obvious differences. Under the NRZ system, the dual-level mode is adopted, but in PAM4, it is upgraded to the four-level mode, and its transmission rate reaches two-bit clock cycles.
New Features of PCIe 6.0:
- The data rate doubled from 32GT/s to 64GT/s.
- Switching from NRZ encoding to PAM-4 encoding, and the resulting error correction impact.
- From variable-size packets for transmission (TLP) to fixed-size packets (FLIT).
In the field of PCIe, GT/s is used as its unit of measurement. The theoretical data transfer rate of PCIe 1.0 and 2.0 era is about 2.5 and 5GT/s, but considering the specific encoding technology and signal transmission mode, the actual data transfer rate is only 2 and 4GT/s. In this way, PCIe 6.0 with a rate of up to 64GT/s has realized the vision and goal of doubling the rate of each generation standard.
To better achieve a data transmission rate of 64GT/s under the PCIe technical specification and overcome the limitation of the entire channel transmission length and distance, PCIe 6.0 uses a new PAM4 modulation signal, which is very critical to the realization of PCIe 6.0. Before PCIe 6.0, all generations of PCIe used NRZ modulation signals, that is, non-return-to-zero coding. It uses two voltage levels of 0 or 1, and each clock cycle can only transmit 1-bit signals. It only uses two signal levels, high and low. Therefore, compared to the four levels used by PAM4, NRZ is also called PAM2.
To achieve a data transfer rate of up to 64GT/s, PCIe 6.0 uses PAM4 modulation signals. Through PAM4, data transmission per clock cycle can reach 2 bits, not just single-bit data transmission. PAM4 uses four different level levels, which can express 2 numbers per clock cycle, that is, from 00, 01, 10 to 11. This means that within the same voltage fluctuation range and the same clock cycle, because the voltage level of PAM4 is two higher than that of PAM2, it brings a lower voltage margin and higher bit error rate, so signal integrity is guaranteed in the device.
Considering that PAM4 is the key to ensuring that PCIe 6.0 reaches a data transmission rate of 64GT/s, PCIe 6.0 uses forward error correction technology. FEC is essentially an algorithmic technology that can ensure the integrity of all signals in a data transmission link. At the same time, the adoption of FEC technology has also changed the situation of the data flow control unit, requiring developers to make adjustments and changes to the size of the data package itself.
Several generations of specifications before PCIe 6.0 used variable-size data packets, but due to the adoption of FEC technology, PCIe 6.0 must use fixed-size data packets (FLIT) to better ensure the implementation and operation of FEC technology. In addition, considering the increase in the data transmission rate and the PAM4 technology itself will increase the power consumption of the device, the energy consumption per unit of data caused by each bit of data transmission will also increase. To reduce the energy consumption of the overall system, PCIe 6.0 adopts the subversive L0p mode, which essentially allows each channel to be closed or opened through dynamic channel allocation to achieve systemic energy saving.
The Future: PCIe 7.0 Specification
The PCIe 7.0 specification will be designed with low latency and high reliability as the design goals, while reducing power consumption, and will also ensure compatibility with previous PCIe design specification connection modes. PCI-SIG initially established several PCIe 7.0-related specifications, one of which is to have a speed of 128GT/s and a bidirectional transmission performance of 512GB/s with an x16 configuration. While using PAM4 signals, and continue to provide low latency and high-reliability goals, as well as improved energy efficiency, in addition to requirements for backward compatibility with previous generations. PCI-SIG will provide reliable, high-speed, low-latency I/O interconnection function as the goal of PCIe 7.0, and the application market includes 800 Gig Ethernet, AI, machine learning, HPC, quantum computing, and hyper-scale data centers with cloud applications.
Key features of the PCIe 7.0 specification:
- Offers 128 GT/s in x16 configuration, and 512 GB/s in both directions.
- Uses PAM4 Pulse Amplitude Modulation signals.
- Focus on channel parameters and ranges.
- Reduce latency and improve reliability.
- Improve energy efficiency.
- Compatible with all previous PCIe versions.
The PCIe 7.0 specification will focus on 800G Ethernet connection transmission, and include artificial intelligence, machine learning, cloud computing, and quantum computing, and accelerate the promotion of data computing-oriented large-scale data centers, supercomputing systems, or military and space Exploration and other computing application requirements.
If you look at the needs of the consumer market, in terms of user behavior and hardware performance, even if the possibility of using PCIe 7.0 performance by 2025 is not high. The current transmission performance of PCIe 4.0 can already meet the bandwidth of top graphics cards and high-speed SSDs demand, even though Microsoft will officially release the Direct Storage technology that enables the GPU to directly access SSD game texture data recently, the consumer market should still stay at PCIe 5.0 for a long time.
However, the PCIe 7.0 specification, which is a unified standard in the industry, has an urgent need for professional fields, especially large-scale data centers, HPC, heterogeneous acceleration, etc. Although such as AMD, Intel and NVIDIA may develop higher-speed transmission for their chips However, PCIe is still an important and mainstream standard technology in the market across different brand product portfolios. At present, the demand for high-performance computing, large-scale data centers, and even quantum computing is increasing, so PCI-SIG has to accelerate the formulation of standards.
