GDDR6：Infinite Possibilities from Graphics Memory to Edge AI Inference.
As the demand for data continues to grow, even in areas where market growth has slowed down, artificial intelligence applications such as ChatGPT have proven to be surprising in their speed of development and the amount of data they require. For example, ChatGPT-3 requires 175 billion parameters during training. These rapid advancements in AI applications have raised higher requirements and standards for processors and bandwidth.
However, there has long been a misconception in the industry: Despite the significant growth in computing power, bandwidth advancements have not improved. This means that many GPU resources are not being fully utilized on top of existing high computing power, leading to current challenges. Frank Ferro, Senior Director of Product Marketing for Rambus IP Cores, points out that "we need to reexamine this issue and find a solution." He believes that by improving bandwidth and optimizing processor architectures, we can better utilize GPU resources and promote the development of AI applications.
Artificial intelligence applications consist of training and inference components, each with different requirements. Training requires extensive data analysis and is a scenario with high computing power and high time consumption. Inference layers, on the other hand, have lower computing power requirements but are more sensitive to cost and power consumption. As AI inference moves towards edge devices, the advantages of reduced data transmission volume, improved edge device performance, and reduced latency will become apparent. In this process, GDDR's high bandwidth and low latency features make it stand out on the edge side.
GDDR was originally developed for high-performance graphics calculations, but it is now appearing in data centers and network applications as well. Although GDDR is mainly used in the graphics field, its excellent data transfer rate is suitable for many edge-side AI inference scenarios and network applications. In contrast, DDR devices face challenges in terms of quantity, cost, and power consumption. Take DDR4 as an example; its operating speed is 3.2 Gb/s, while GDDR has already achieved 16 Gb/s bandwidth, which is 3-4 times faster than DDR4. Additionally, both have significant differences in terms of capacity density and power consumption. Therefore, GDDR is the best choice when considering the main standards and most important measures; however, DDR is a better option when considering storage density and cost sensitivity.
When discussing the application differences between HBM and GDDR6, Frank Ferro pointed out that although HBM can provide a bandwidth of 800Gb, it exceeds the required 400-500Gb bandwidth for AI inference and increases costs by 3-4 times. In contrast, for AI inference scenarios that require larger capacity and higher bandwidth, GDDR6 is a more suitable choice.
Furthermore, Rambus denied any plans to develop GDDR6X technology or products. Although both GDDR6 and GDDR6X are excellent technologies with very high performance, the former is a JEDEC standard, while the latter is protected by patents. Therefore, Rambus prefers to develop specific technologies based on customer needs.
Recently, Rambus launched a 24Gb/s GDDR6 PHY product that provides a bandwidth of 96 GB/s per GDDR6 memory device. This product is designed to provide high cost-effectiveness and high bandwidth memory performance for artificial intelligence, graphics, and network applications.
The main features of this PHY IP include:
● Providing up to 24 Gb/s data transmission rate with a maximum bandwidth of 96 GB/s;
● Can be combined with Rambus GDDR6 digital controller IP to form a complete memory subsystem solution, allowing users to customize the system directly;
● Provides LabStation™ development environment for quick system establishment, characteristic analysis, and debugging;
● Based on Rambus' 30 years of leadership in high-speed signal integrity and power integrity (SI/PI) expertise, it can provide system-level signal integrity;
● Provides reference designs and support for packaging and PCB;
In practical applications, the PHY physical layer connects directly to DRAM through two 16-bit slots, while the other side connects to the memory controller via DFI interface to ensure that the controller accesses the logic control of the entire system directly. Frank Ferro explained that the trend of using 16-bit dual read-write channels has become mainstream, and the significant increase in data width of 32 bits can significantly increase data transmission speed and efficiency. Meanwhile, GDDR6 memory also has eight such dual read-write channels, bringing together a total of 256 bits of data transmission width, which can bring about very large data transmission rates and system efficiency improvements, as well as further optimization in power management.
For AI inference applications, each GDDR6 device can achieve a bandwidth of 96 Gb/s, so combining four to five GDDR6 devices together can easily meet bandwidth requirements of 500 Gb/s or below.
lu/Tg2D68eZGv1a7uJ/EyhfXsQfmqTrWaX0YR7QOBsGsKJ8z+qQwPwxcpWLR9NBvLPwzsfUF9JXOtboN/XrBh/6nzDxMTExMzkeXDEKK/ZVRm4bPEFx9fPHX6wIbV08e9+4pRKxqloLDrido+R47pjxOLvH/2yMap37T1lXiEnEDamSa1qn0MhSzoRRKN3POnluPQLQmLX8nGP4ZBwduqU6b1T4fuWPNl5JzViGgTqmNh2t1t0cGSJjSzibNfGMr8J7Hf1fdk1+CKVpTfgGLmNibpzw/zVwQxa34dCWPLMIHlDAo/V5nKg7alOVHEjGAVQkTZfv6zQC0IyVQ5RHQ8nvGanlX9yxrh+0eeIHlNSPoJgI7NoYxq7EPSO6+xWcpAnqFTE2OvPTYUNCNInRAxe80rGhCsAYjyjcG+mmYYnYWCFvQmiUgTAJ2xLZiUW+0kskWz+VWmwxN+KhI0JRmFVTGUL7aV3ALE3PkhGqaLSkNR7f1Iqh8TAfFqF4nO8yc7kn/H5otcBhlR6tEkRkZXAPFSuJuA8s462tXxFgrrnCQRwNA8IfDBh0Ya/RfpSLnSl0lbKwM8WEMtWp6U0UXAPRXdBHSuLq9VTc6jwNsMJBX2iIEpQ3wp+iQi7blgJhVvs3CuDVaH8HMotMo5fje4CVj4s1GbKm5HkW9UJoF2N8XAjD8CiFrfQOrMpkw8/mGBjl2dTCpQYx+6EJjYyV3A5AhNKjPPIVTau0RSt4tiYN7iqgT1jyC9cyAT6QsnC5Sta96vXzWwnKeZVCea31yHS4Gr/ATI/cIlwt2VNcg4LhvFnmUiAQg7LQbKMY2kouruRpZzDCygwWMmiOh4fGL39rmzSSeOHDly5IiB773Xp33LehUkbtKATHQtUroIUPBneEuuzWa7CLYBGtTtCQp+qzGZ1HCHLAQ64iI8AcDYdJfM5KSFid9qVszzMzOfP7p35cT2OV+3reApcah5FbnKLFOrc5JpWcjFMsN5Jn5ySvw9rvGpPIJS5WJZyOQYQ5aOMnsOuLWM5oQcQ9HlCQYigOB1BUIgWpdHduw9JcGJTJ80ZAL9s8QilPOe31rat6aelXGqk1XG/XMHYlZMZfj3SB8uOZumTCWfsiiVKmne1CJnbjpw8amTyaNa/BRI4jPq76lFL8+kSl3511TSKb2Javw8lf2szTG7j5+7/czBIrOF1ui/tQuH8aEUEDA/XwxEp82OrDP7sil/QCFF2uMmN9WzaX4XWcrJu3+MbBViAQE3UsUHAq3pNNVaCYo11+o6fF0yA1t/VSP1PUW1zgxK1ZcPea3byD35dI7vJY0JvY7iywu8KcB/wnNBuP7OBiKzFIRojx9TkYXhJ2SZML1VeSMIOlWmuW2h0p+kmk8AAJJf+5gCKpyhET4HqRaAsqUKI1Oo8JCHtujGygrAtI90FOAxJFFxO/zZeK9QFGLhnvYSXchVBgW72+hB3F+cCgOAKpvodvlqxCGVATANzqC621Bb/M+gIm+2owFT1D2l3WjCBhqcVRbinb4Gqqh8OsfqIBBZDaBVEtX5ui4a+G6myuirLW2tysArbSQK0HW8KCsr9y1G0OuBwjDpIz2FfgXSb6oEWuOxgepGE1cNPsijwe+15Zs8heDFrjoKgEa7HIrCIaykqEcKwye9KCpcobsbDmq3hE43Vqa51cxlq/GEaq5RU/5ExT4YZKaBoOW5ivrRwAgMfRMUhifrkXV5RuWcpFe9fToqGOSkedbFZQs4TLXJX1OWKAdTfyxLA2UnPlfSJD0r0EeccihLnudNNKyA6m44qN5RBm9SpfZw2SwLqY5Uc5Ewd0kVGjB/mqhGAHWnpCgKU7sRzUDqrWYtauWg6u6ywSiqUyGaMsGhIHRsbyxRgL7bSadiBgNPz/ZrnzgVhP9YCPw2URUMg5cfBrNX+cCKNWozDZ1Oda+hpnySrSTEM2/QAIRuLFRI3ltcADxbTtz7VFaMtSNB0FGq9Eaa1DzHZdLX6jpw2I+zF+/cvf/yndssHxdSPWysKe2tysI7HxhpoPISuzIu1ecEoKvU6sNf18feslqtqRm02Xl5dpkL/kkQeonqejlNqv/INdIHRK26lJSPYj/Qlsr3FIZPx/rSQOBipyKWenB70ejlFxAYWL5O+3bkHT747NPxU5bvOXU9ndnRasW1SKT6x0eTQuOFc1zbGcN1d4Iq+fY7nYPia4znEaVhzl8VaSD4iBLS3wTl6ys3fPuX2zKbJ+2La/Uv1VSTNt0TLu9L4DxZjYJnZ6ISNcbwl6w0zNsaSgNdkxSwxVsFXjSFLrQxwUHFdbdTjdW5BGk9FTdGhWpscqDrA5EOVgWPLifYxUA81c5A4TlbvKSuoJo+P+Qw+dNUTG+kHuMaOPpp1Cojh8qbZXSFwjLY2M981ap+i0H788WQb/b3IIO2iaLlTTarB5Rbz2Sjr4uDn2hUTAA7j78c6BLVuMAk44/qEgBIFUYlCoGY9kMAmWmJYPLKcqCm4U9ZnK/AbrRbt5NDZAq6Rn5rWNjGWKBoXZ/LshCYu7Q2EfQuEOt4bVBVv3UsnlUvpksu1STDS4nAXegiwQ8MCv70guL1zQ7IQqDjYD2ikFtC3ekIKvulzCCzdjGtk6gWe76UeD/HZYp8TuVcGQjEVVblCoG4syqJz3KR0gfo1aablUFOl2Ka3ac6UuZlhNdedJka36Y6XBsoA/+wiVHwtUQAYwSy/2AGtQ2/wyKimDpxVI+qvYxok+Q6WQ7SXGwG1N6Dk4XA44Ek0fnCFC4qA6r72i0e5WOobD1fRkxwMHA8vXmeZWzsM62D2RTx3YChKTpBiOwWJBHPhNkZAurb5gEPw3wqecZLCP9tSC1fm/h6VZORocE4TvMG5BKlDdSzAH23KyIUfEbS/K4oF5uCCkflsOhaDIy00+D5mu6T5zqqGFZ1z9Eda6EH1tGa1ySdxDbKBIxbnXbys48gee2mIAkRoMLmmcgwq3FxHZOo8odJbpPHHKrjwYzaP6W6Hw7sta/cBQL7TD9g3nCPYLfESB+kU6PGD1kkVSvO/wIVXqjrCmUNUKc7zRj1LqD62+TCmOcVJ6+tBBzr7nXyyoskiXgmRO5kT1BhvyUyi7jA4vQzZCrnbIsLZO2uCoYfqaydGL2H1G+BCwOfFHeoJnB9ZY/MyfoqSVSeCIXzfEGFy/yRiywXWYqDdllUmDXcy/V51lUVpCFUBe+L0t2l6Zxe1JUWwDnsAae1PiTfoYi7qoP6GustsSHTIUAYuJ8OU4db3CPoQ4XLvQQZKrkytU4X8aCrxEs/UebyLAIIy2wS4VR9UFfJUrVBlyl3ZWT6oBUJfJpLh5nT60huUTsbVdb7eiaRdCcquzKmFS+kfiwB95ZpPPIne5I0eCTAjY4goiVqoKiDvvtt6aEEO7Le4U9U7QgDdJ7+opo7VP8mFT7sZ2YRkUlln1HRhYExhYi2b83Av0ECB3lVABBKYxz8nvaVhKicmJMtaI6MPPO+AfKBNgaI2XEz3wmtFBgYGOjnTW+RtM/aQx0q7qbD5GU9QyoElvX29vb20hcTFk+FuUe/blK5nI83Q9PHLkDPFLRP8wUB22axcx4IAdJaV5B75tcGELJKGqri1eoU3huYIDpzM+Pj4uLitq9aTT03UPtyP1cHw2wGKOckX4zbv2b16tWrBxZT9TAdYoHt/vENqxmuPOkCVLssb64AAuonyuwuhQGpz7RCbvbfLKAhzh90FNDuPhue2W20L+9LdYCv8hkQ7y9Gt4CFuFqkf+vLYBCx4X1kfreDRGIYmoG8HYvKgZbEBgOtYahNsJw33Iam98SA/lkuFYAEIpZbJTN7HK0HQl3kM+S+LQi0JL0v0JdZKIuV9arbYN4gSIUzLpaQ5h9zkHXWMCOQtruJ3C80Bi1xzPFhALUOyUJlBLkN0CtbDGlIobth/CQdWef/4gmkjc4g99udQFP2VAOmTY7K7lmZTWJAtWPuRuRTZC0vKwukVbch99RInaacaAaMmx53z6DLIzGg97/uRZe7yHxnMJAGLCjk9nyoATREjm0KzBvusGtYoyTtMg6ziWEakelOhJ1H5mcbAqnHBBvyzp1kAQ2xLQsCjlX/StWuVoXaBT5TbEKAecJz9yEoBpkndAVS6aM05O1cGgDa4bz/TQBwtbx30q5ZDtUZJxD4Dk8UAnxGWd2F8isczDK+MBB1voe85d3VQTPst2eE6oCzrsrws9nugccskcDUYe1TWQDw6BGb6xZ4/FqArJ3TLEDaNA65xzUGjXA+3jG2gQQiVhu04p5DsF9VoUU+r/bCec4TCsDSdeYtOz+A6oP3ZsnCPXQ5TEMykfmB6kBaZTtyj+8Eole8nWxlnfiQdfyxvTELxr/TqLwORPVp0OfXbTcfPbXyvFaFaPi/VvKUg2a6mDQreep6ia7O6WQrcXLi2zRNz6RYyVNuduUEg5OtlMlLDFwAPEJ7T9504WGSlThlERVIldoNXnUqIfGZVdyUk3UpqmxNtVKO0WuM1CcRmd/vDKQBCx3cnkZJwpmahDGv7sfa18tiMelAcIOnb7mG4WEc6xiIyjQNowyWqKR64WGUIUCvrxkeRhweaqbR1wyjDH/VwMuvcRhlMz/gbzD7+NUNDyNtVokOAHQe3r4VG4eJG17bQCFVC6P1B41tfR2Z5483kJjG5yDv7LEeUNK29iFkv70sEOr7pSLvwrneUNK2wjpkf6ExkL6egLwLlpSDkraWvwvYxXcH0oankXfh4gpQ0tZjpA2Z3+qhI6myG3nnLCwPJW11UUnIunB/Wx0QBswq5JX4bVkocdvwLjK2x0+oDKSek7KRq/x8TwczlLx991lhxrPirTlF2W7uGB+qB1Lpk2zkmX9jVR9/KIlbtnOfiI6EvaIGT5gwbkCPpmUlIDesRpQL83JsGRnpt+PIz2xcNPHdRhYoGSzp9TpgKPXeuXfu+P4RLUOCgioFUvpaDFDiWdJJUOr/Uv+X+r/U/6X+L/V/qf9L/V/q/1L/u9kAVlA4IBoHAABQWACdASq8AvcAPm0ylkkkIqGhIbFqSIANiWdu4XEej/4B+AHw4wAoUIgH4AfoB/ADezfwH5O69/2f8fPzA6FHgTwF+R/CA0v52/iP5h/kvuH9/HqH8wP9Lf9R1D/MB+y/7ge9D/oP1m92X+A9QX+d/5H//9gf+2/sAfuF6YP7EfBP+3P7bfAl+xH/s6wDqv+gH8A/AD9mf4/+/vf4NHd3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3dzFgPvzDdBgzBatVnMOh5wj13d3d25f7WzLXjzFk+UxXkXFVRgdsHdeam1lKAJnndldfwPPHgXoksk0sZmZmY7Dd+AjxCwZlFRMkKHnzq3fxWJXA/Z5VLpvgfCIkJOozDvTpPf1+tIuDMkQymmIe/uq67qk2s0sYnwlBgF/xTX2LcLZjnpvlOumLWGahu7cmW9SbN7YtoV0dG2MSOrumUZbXq8ej1uHWlG9urpVE+sH4V867gBSzIv+Tvt+ak0T5VKGJVkSQfQ8ftqmRkq+kzMw+DrxrTBrGYhbVPRJynSYW0Up6dW9k6TThvSoJ86YXj63hwDOxnFS7FgAPjbZmZZF03ZQCmCIV4l0kb19QohQ3PvogF7W6iI50PcnH6HXy/NQarfQ4E98j4tfvUv+4bWCvn1I64qJpJCjNR9r5DyvSEVa2W5wr/y+uKiaQ1iffUEXoSShd4A2XlRKPDAeEf3t08GlhXFRNRT4zKiUeGA8I/vbp4NLCuKiainSp3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3dzAAP6vdAAAAAAAAAAAABKlpqnWvGKt1VAHOGvMbhagJn5Ejhn72+LBi8VajN0vpFWkSXTP30z5LSJDoQO29V+S666uHjOVqz3sDZ81gL+lsQqOmLHltxSZmx2ZOoIf8n3/iKNQLcM23oFyQqf86B5q/nQPM79DNZ4GlxKpgNEqQbvmb5ywCnr9M/oCYcMgeaDM8EjyDagWf8adWF/RpXceSPAzlCu9Uu5Ks0AYcPdZ/w3dDnfRBOWL4vkmt23uNCOEb63xEhiV3WqX/gtUok7QG+xj6hqPMs30uPMtpOugdiVtvGAf9YeHtHSCLOOns17iNjy6ppUwtQODzswPgHkBFUgMODUzSedzfb6iudxY7W0+5mh//1Zf/+q4v//VOQ78gR5fX2ArPw5Y6L/MuuzlFmUNwmQn//td7xPUzTgF/Bm1oI3zcLebKT2ggw6zfjkfUzZaD1sv8sDaTkrbDVAIzPNRHZM8CO+yrS3SrmM+obk3PiwidycCt1cTLMGaCZMszo5cpzJ1aYzhQrYR5Ej24ipprtGTASDmGXAlYK0tv6vZWhMHVpRtCQLAaX9ad3FYr1U4CO3UNBsqh6bxu4ivE/W9KCLvxQ9gNMn50H5iFHP//nAQCOy5hnI4AC4p5vAUR7sg74dhreP5wYdcRYkXpVccOckQx7Ayd5Y2s9EZhkdVvDtVe7VVQdHdnOev5kDLSXvEo98rvElgqfGzqk5ab36eEY7mM61SB/FveIH+KKu9vVtlofAqu/5TJkvzo5nYXf52a/nVZfnZ3+d7g0QR0fE4bCT6tAGqg0MaYl6lccNLv97l7Xad5j64VzDtYpa+9fx7lArvOfhbkw3uiWzfXU9I5JoUKiLsRnqFpjzMFC+V3CqluBZiY3cRxLXoYOAbDwvapg3AccYgtg6IZQ9q0TKdWYHC8WUu4kSj/zgL9MD+GSF8fK6sBDACYYuEtyb9A9Giycm7f7uy1K5aWLPo1MUbjLWlhaS6DcJwSIKW39/P/D1P//OAjQeHHuQFx9kIFMRfD1cLoNXak9aHvgrpgG0NLkXigun0jFi6gTNAQGR0Cz0vk3mQSgDt1ftg1RbuwCN7WCpmoQPcfJhuEa+w5A4s/mwMn//gqO/ID5LCk+j5uFOGn6hJWs8BiVi3aOy38/RXS11YOrLp/UhjxJ+kWLk1vTqjV7hxhckuIzLzAE9tFrDaaaDO7yW8NPUP508E6LxGtdg2jMdgwpQ1X0jXMc0tvX+l82z6HxEgXcnP/Iu/5pPTZJoU8VUH+9Hm3PvNa663qkfLglQ+AaidU22I/4XUyjHB+fGXCSe2kPaN+wID1etR7NF5DQyM5ADd5r5DHO4kmutsaByqfp2qnNPU0Abb3N8KcxXVYrwl6mC54SxX5OorGH6vDfuy0/sjOw9tInsYMq7sYtMHp61LqEW5ejIE5wi4hXfncBjAaf51BARb5QAAAAAAAAAAAAAAAAA=" src-original="//31753717.s21i.faiusr.com/4/ABUIABAEGAAgu-3ZqQYopKWSqgEwywc41gI.png" natural-width="971" natural-height="342" data-loaded="true">