Servers Designed for Single-Phase Immersion Cooling
Unlike air cooling, servers used in single-phase immersion cooling have some special considerations:
Server installation mechanism
Chemical phase changes
Steady flow of coolant in the system
By partnering with the major single-phase immersion cooling suppliers in the market, GIGABYTE's single-phase immersion cooling servers can be installed directly in their off-the-shelf tanks from these partners, eliminating the need for product compatibility testing and customization. These systematic modifications greatly reduce the time for verification and adoption of single-phase immersion cooling technology.
Structural Integrity and Simplified Management
Servers must be installed vertically in the tank, and this lowering or raising of servers is due to an optimized tank design for single-phase immersion cooling. Therefore, the server installed upright needs to have rigid and firm structural integrity to ensure the chassis will not deform. Also, due to the vertical design, the cable routing and maintenance are also different than traditional data centers. Therefore, we re-examine the design and strengthen the server chassis for immersion cooling and reroute the cables. For instance, I/O ports are all arranged on the rear side facing up, and the network cables need to be rerouted to the sides so that maintenance can be quick and easy. Also, to make it more convenient for IT to install or remove a server, a bracket was added to allow hooks to move the server.
Design to Ensure Steady Fluid Flow in the Tank
When a server is submerged in a coolant, an important condition is that the fluid is circulated well, and the warm fluid is pumped out as cooler fluid is pumped in. Only by maintaining a stable coolant flow can the heat absorbed by the fluid be managed, and the system can automatically adjust the coolant flow rate according to the operation needs. To ensure efficient heat removal, the server design for single-phase immersion cooling not only must remove any components that negatively affect the fluid flow in the system, but they also must reduce the dead space in the chassis to avoid heat accumulation. Based on the viscosity of the coolant and the specific heat capacity of the coolant, the heat dissipation rate of the server must be constantly monitored and adjusted to avoid issues . Servers developed for single-phase immersion cooling can significantly reduce unnecessary barriers that users may encounter when adopting new technologies, allowing them to focus on productivity.
Made with Materials Suitable for Various Coolants
Single-phase immersion cooling submerges the server in a cooling fluid that removes heat by directly coming into contact with the heating elements. GIGABYTE had to evaluate how the chemical composition of the coolant may affect the components as well as the effect of the temperature and process on the coolant. Through theory and experimental experience, we perfected the design and will continue to explore materials that are suitable for immersion cooling.
G292-Z43 Product Overview
G292-Z43 Block Diagram
Architecture for Modern Data Centers
AMD EPYC™ 7003 Series Processors
GIGABYTE servers and AMD EPYC 7003 Series Processors deliver the highest possible I/O throughput to pave the way for fast communication between CPU and accelerator in HPC and data analytics.
Up to 64 Cores
8 Channels of DDR4-3200
Up to 4TB Memory Capacity
128 lanes PCIe 4.0
2-way SMT & Turbo Boost
4,6,8-channel Memory Interleave
Synchronized Fabric and Memory Clock Speeds
Secure Memory Encryption
Secure Encrypted Virtualization
Hardware Security
Optional TPM 2.0 Module
For hardware-based authentication, the passwords, encryption keys, and digital certificates are stored in a TPM module to prevent unwanted users from gaining access to your data. GIGABYTE TPM modules come in either a Serial Peripheral Interface or Low Pin Count bus.
High Availability
Smart Crises Management and Protection (SCMP)
SCMP is a GIGABYTE patented feature which is deployed in servers with non-fully redundant PSU design. With SCMP, in the event of faulty PSU or overheated system, the system will force the CPU into an ultra-low power mode that reduces the power load, which prevents the system from unexpected shutdown and avoids component damage or data loss.
Smart Ride Through (SmaRT)
To prevent server downtime and data loss as a result of loss of AC power, GIGABYTE implements SmaRT in all our server platforms. When such an event occurs, the system will throttle while maintaining availability and reducing power load. Capacitors within the power supply can supply power for 10-20ms, which is enough time to transition to a backup power source for continued operation.
User Friendly
Certified Ready with Software Partners
Being a member of key software alliance partner programs enables GIGABYTE to rapidly develop and validate joint solutions, enabling our customers to modernize their data centers and implement IT infrastructure and application services with speed, agility, and cost optimization.
Tool-less Drive Bays Design
Clipping mechanism secures the drive in place. Install or replace a new drive in seconds.
Value-added Management
GIGABYTE offers free-of-charge management applications via a specialized small processor built on the server.
GIGABYTE Management Console
For management and maintenance of a server or a small cluster, users can use the GIGABYTE Management Console, which is pre-installed on each server. Once the servers are running, IT staff can perform real-time health monitoring and management on each server through the browser-based graphical user interface. In addition, the GIGABYTE Management Console also provides:
Support for standard IPMI specifications that allows users to integrate services into a single platform through an open interface
Automatic event recording, which can record system behavior 30 seconds before an event occurs, making it easier to determine subsequent actions
Integrate SAS/SATA/NVMe devices and RAID controller firmware into GIGABYTE Management Console to monitor and control Broadcom® MegaRAID adapters.
GIGABYTE Server Management (GSM)
GSM is a software suite that can manage clusters of servers simultaneously over the internet. GSM can be run on all GIGABYTE servers and has support for Windows and Linux. GSM can be downloaded from GIGABYTE website and complies with IPMI and Redfish standards. GSM includes a complete range of system management functions that includes the following utilities:
GSM Server: A software program that provides real-time, remote control using a graphical user interface through an administrator’s computer or through a server in the cluster. The software allows ease of maintenance for large clusters of servers.
GSM CLI: A command-line interface for monitoring and managing remotely.
GSM Agent: A software program installed on each GIGABYTE server node that retrieves information from each system and devices through the OS, and this software integrates with GSM Server or GSM CLI.
GSM Mobile: A mobile app for both Android and iOS that provides admins with real-time system information.
GSM Plugin: An application program interface that allows users to use VMware vCenter for real-time monitoring and management of server clusters.
AMD EPYC™ 7003 Series Processors
AMD EPYC™ 7002 Series Processors
Dual processor, 7nm technology
Up to 64 cores, 128 threads per processor
cTDP up to 240W with 35°C oil inlet
Note: If only 1 CPU is installed, some PCIe or memory functions might be unavailable.
Socket
2 x LGA 4094
Socket SP3
Chipset
System on Chip
Memory
16 x DIMM slots
DDR4 memory supported only
8-Channel memory architecture
RDIMM up to 128GB supported
LRDIMM up to 128GB supported
3DS RDIMM/LRDIMM up to 256GB supported
Memory speed: Up to 3200 MT/s
LAN
Top side:
2 x 10Gb/s LAN ports (1 x Intel® X550-AT2)
Support NCSI function
1 x 10/100/1000 Mbps Management LAN
Video
Integrated in Aspeed® AST2500
2D Video Graphic Adapter with PCIe bus interface
1920x1200@60Hz 32bpp
Storage
Bottom side:
4 x 2.5" Gen4 NVMe/SATA/SAS devices
4 x 2.5" SATA/SAS devices
- (2 x NVMe from CPU_0, 2 x NVMe from CPU_1)
- (SATA from CPU_0)
SAS card is required for SAS devices support
Recommended 12Gb/s SAS cards:
CRA4448
CRA4548
SAS
Depends on SAS add-in cards
RAID
Depends on SAS add-in cards
Expansion Slots
16 x PCIe x16 (Gen4 x16) FHFL slots, from PEX88096, for single slot GPUs (Broadcom solution)
1 x PCIe x16 (Gen4 x16) low-profile slot, from CPU_0
1 x PCIe x16 (Gen4 x16) low-profile slot, from CPU_1
- Maximum limitation of GPU cards: 285mm (L) x 111.5mm (W) x 39.5mm (H).
- System is validated for population with a uniform GPU model.
- Support is not provided for mixed GPU populations.
- For the latest GPU cards QVL, please contact your GIGABYTE representative.
Internal I/O
1 x TPM header
Top I/O
2 x USB 3.2 Gen1
1 x VGA
2 x RJ45
1 x MLAN
1 x Power button with LED
1 x ID button with LED
1 x NMI button
1 x Reset button
1 x System status LED
Backplane Board
Speed and bandwidth:
PCIe Gen4 x4 or SATA 6Gb/s or SAS 12Gb/s
TPM
1 x TPM header with SPI interface
- Optional TPM2.0 kit: CTM010
Power Supply
Dual 2200W 80 PLUS Platinum power supply
AC Input:
- 100-127V~/ 14A, 47-63Hz
- 200-240V~/ 12.6A, 47-63Hz
DC Input: (Only for China)
- 240Vdc/ 12.6A
DC Output:
- Max 1200W/ 100-127V~
+12.12V/ 95.6A
+12Vsb/ 3.5A
- Max 2200W/ 200-240V~ or 240Vdc Input
+12.12V/ 178.1A
+12Vsb/ 3.5A
Note: The system power supply requires C19 power cord.