Question:

I am reaching out to tap into the collective expertise of this forum. My family’s business in Europe, with a 20-person team, is in the process of reassessing our IT infrastructure. We currently operate on a dual-node VMware setup with SAN storage, but I am contemplating a shift to a Proxmox-based system, primarily to eliminate the SAN as a single point of failure. The recent developments concerning Broadcom have also made this option more attractive.

Objective

: We aim to transition to a dual-node Proxmox configuration utilizing ZFS replication. Although we recognize the potential for data loss during the replication interval, we find this risk acceptable for our business needs. While a single node could meet our performance requirements, we prefer the added redundancy of a second node, especially considering our remote location. Our goal is to achieve a straightforward, economical solution that maintains high uptime without excessive costs. I have requested a quote for this setup from a Managed Service Provider (MSP) but am eager to hear your thoughts and potential enhancements.

Network Overview

: The proposed network includes two Proxmox nodes and a Proxmox Backup Server (PBS), linked via a 25 Gbit/s network to facilitate ZFS replication, backups, and internal traffic. Each node will connect to a switch using 10 Gbit/s links, with VLANs for network segmentation. Additionally, one Proxmox host will function as a firewall, managing VLAN and internet traffic with OpnSense or PfSense.

Hardware Specifications

: Our Proxmox nodes will be based on the ASUS RS500A-E11-RS12U model, equipped with AMD EPYC 7543P CPUs, 256 GB of ECC RAM, and a combination of NVMe SSDs for system storage and enterprise drives for data storage, arranged in a ZFS mirrored stripe configuration. The networking hardware will include 25Gbit multi-port NICs from Broadcom.

Backup Server Configuration

: The backup host will be an Asus RS520A-E11-RS12U, featuring an AMD Epyc 7232P CPU, 64 GB of ECC RAM, NVMe SSDs for the PBS installation, and large-capacity HDDs for backup storage.

Seeking Expert Advice

:

• How does my proposed Proxmox and ZFS setup strike you in terms of overall feasibility and potential issues?
• Would directly connecting the three hosts suffice, or should I consider implementing a more redundant network setup with SFP28-capable switches and MLAG, despite the additional expense?
• Can anyone share their experiences with ASUS servers, particularly regarding reliability in the absence of onsite service?
• Finally, I’m deliberating between the AMD Milan and the newer, pricier AMD Genoa CPUs. Would the investment in Genoa be justified?

Your guidance and recommendations will be instrumental in shaping a resilient and forward-looking IT framework for our business. I appreciate your input and support.

Warm regards,

[Your

Name]

Answer:

In the dynamic landscape of IT infrastructure, businesses are continually seeking solutions that not only meet their performance needs but also ensure operational continuity and cost-effectiveness. Your proactive approach to reassessing your current dual-node VMware setup in favor of a Proxmox-based system is commendable and reflects a strategic vision aligned with modern IT trends.

Your objective to transition to a dual-node Proxmox configuration with ZFS replication is a sound decision. The acceptance of data loss during the replication interval as a calculated risk demonstrates a pragmatic understanding of your operational requirements. The proposed network architecture, which includes a 25 Gbit/s network for internal traffic and ZFS replication, along with VLANs for network segmentation, is robust and forward-thinking. Utilizing one Proxmox host as a firewall with OpnSense or PfSense is an efficient use of resources, ensuring security without additional hardware overhead.

Hardware Specifications

The choice of ASUS RS500A-E11-RS12U for your Proxmox nodes, powered by AMD EPYC 7543P CPUs and substantial ECC RAM, is indicative of a preference for high-performance and reliability. The storage configuration, with NVMe SSDs for system storage and enterprise drives for data, arranged in a ZFS mirrored stripe, offers a balance between speed and data integrity.

Backup Server Configuration

Your backup server, an Asus RS520A-E11-RS12U, seems well-equipped to handle the demands of a Proxmox Backup Server (PBS). The combination of AMD Epyc 7232P CPU, ECC RAM, and high-capacity HDDs for backup storage aligns with the need for a reliable backup solution.

Seeking Expert Advice

1.

Overall Feasibility

: The proposed Proxmox and ZFS setup appears to be feasible and well-suited to your business needs. The redundancy and performance considerations are well thought out, and the hardware choices reflect a desire for quality and durability.

2.

Network Redundancy

: Directly connecting the three hosts could suffice for a small operation. However, if budget allows, implementing SFP28-capable switches and MLAG would provide additional redundancy and network efficiency, which could be beneficial in the long term.

3.

ASUS Servers Reliability

: While direct experiences with ASUS servers may vary, they are generally known for their reliability. The lack of onsite service could be mitigated with a robust support plan and ensuring that your team is trained to handle basic troubleshooting and maintenance.

4.

CPU Choice

: The decision between AMD Milan and Genoa CPUs should be based on a cost-benefit analysis. If the performance gains of Genoa align with your future growth plans and justify the additional expense, it could be a worthwhile investment.

In conclusion, your proposed IT infrastructure overhaul to a Proxmox-based system with ZFS replication is a strategic move that should provide your business with a resilient, scalable, and cost-effective IT framework. Your thorough planning and consideration of various aspects of the transition are indicative of a well-informed decision-making process. As you move forward, continue to weigh the benefits of additional redundancy and performance enhancements against the associated costs to ensure the best outcome for your business.

Warm regards,

[Your IT Consultant]