Question:

How to migrate from VMware to Proxmox with ZFS replication for a small business?

Hello experts,

I am a part-time administrator for a small family business in Europe with 20 employees. We are currently using a dual-node VMware setup with a SAN for VM storage, but we want to switch to a Proxmox-based solution with ZFS replication for better reliability and cost-efficiency. We have a remote location and need high uptime, so we prefer a dual-node configuration with a backup server as a third quorum device. We have obtained a quote from an MSP for this project, but we would like to hear your opinions and suggestions before we proceed.

Here are the details of our proposed solution:

• Network Diagram: We plan to connect two Proxmox nodes (PVE 1, PVE 2) and a PBS server with 25 Gbit/s links for ZFS replication, backups, and internal traffic. Each node will also have a 10 Gbit/s connection to a switch, with VLANs for network segmentation. One Proxmox node will act as a firewall with OpnSense (or PfSense), managing VLAN and internet traffic. You can see our network proposal here: https://imgur.com/a/1fVJsSy
• Proxmox Node Specifications:
• Model: ASUS RS500A-E11-RS12U (Product Page)
• CPU: AMD EPYC 7543P – 32 Cores, 64 Threads, 2.8GHz Base, 3.7GHz Max Turbo
• RAM: 256 GB DDR4 3200 ECC Registered, 32GB Sticks | Brand: Micron
• System Storage: Two 512GB NVMe SSDs for Proxmox installation
• Data Storage: Four 6.4TB NVMe Enterprise Drives, 2.5″ | Kioxia CD8-V SIE | 3 DWPD, configured in ZFS as a stripe of mirrors (equivalent to RAID 10), providing a total of 12.8 TB usable space
• Networking:
• 25Gbit Quad-Port | Broadcom NetXtreme N425G | OCP 3.0
• 25Gbit Dual-Port | Broadcom NetXtreme P225p
• Service: Next Business Day parts service (excluding onsite service)
• A Proxmox subscription with support will be bought.
• Backup Server Specifications:
• Model: Asus RS520A-E11-RS12U (Product Page)
• CPU: AMD Epyc 7232P – 8 Cores, 16 Threads, 3.1 GHz Base, 3.2GHz Max Turbo
• RAM: 64 GB DDR4 3200 ECC Registered, 16GB Sticks | Brand: Micron
• System Storage: Two 512GB NVMe SSDs for PBS installation
• Data Storage: Four 12TB HDDs 3.5″ SATA-3 | Toshiba MG07ACA12TE | 7.2K RPM for backup storage
• Networking:

25Gbit Quad-Port NIC

We have the following questions for you:

• Do you see any major flaws or risks in our proposed solution?
• Is it a good idea to directly connect the three hosts, or should we use two switches with SFP28 and MLAG for better redundancy and performance, even if it costs more?
• How reliable are ASUS servers, especially without onsite service?
• Is it worth upgrading to AMD Genoa CPUs, or are AMD Milan CPUs sufficient for our needs?

We appreciate your feedback and advice on how to achieve a robust and future-proof setup. Thank you for your time and expertise!

Best regards,

phiob

Answer:

Thank you for your question and for sharing the details of your proposed solution. I will try to answer your questions based on the information you provided and some online sources.

• Do you see any major flaws or risks in our proposed solution?

Your solution seems well-designed and suitable for your small business needs. You have chosen a powerful and efficient CPU, a large and fast RAM, a reliable and scalable storage, and a flexible and secure network. Proxmox VE is a great choice for virtualization, as it offers many features and benefits, such as easy management, high availability, live migration, backup and restore, and more. ZFS replication is also a good option for data protection and disaster recovery, as it allows you to synchronize your VMs or directories between two servers with minimal network traffic and storage overhead.

However, there are some potential risks and challenges that you should be aware of and prepare for. For example:

• You may encounter compatibility issues with some hardware or software components, especially if they are not certified or supported by Proxmox VE or ZFS. You should check the compatibility lists and forums before purchasing or installing any component .
• You may face performance degradation or instability if you do not configure or tune your system properly, especially for ZFS, which requires adequate resources and settings to perform optimally. You should follow the best practices and recommendations for ZFS configuration and optimization .
• You may experience data loss or corruption if you do not backup your data regularly and properly, especially for ZFS, which does not have a built-in checksumming or scrubbing mechanism to detect and repair errors. You should use Proxmox Backup Server (PBS) or another backup solution to backup your data to a separate location, and perform periodic ZFS scrubs to verify the integrity of your data .
• Is it a good idea to directly connect the three hosts, or should we use two switches with SFP28 and MLAG for better redundancy and performance, even if it costs more?

Directly connecting the three hosts is a simple and cost-effective method, but it has some limitations and drawbacks. For example:

• It does not provide any redundancy or failover in case of a link failure or a host failure. If one of the links or hosts goes down, the replication and backup will be interrupted, and you may lose data or access to your VMs.
• It does not allow you to scale up or add more hosts in the future, as you will run out of ports or bandwidth. If you want to expand your infrastructure or increase your capacity, you will need to replace your NICs or add more NICs to your hosts.
• It does not support any advanced network features or protocols, such as VLANs, LACP, or CXL. If you want to use these features or protocols, you will need to use switches that support them.

Using two switches with SFP28 and MLAG is a more complex and expensive method, but it has some advantages and benefits. For example:

• It provides redundancy and failover in case of a link failure or a switch failure. If one of the links or switches goes down, the replication and backup will continue, and you will not lose data or access to your VMs.
• It allows you to scale up or add more hosts in the future, as you will have more ports and bandwidth. If you want to expand your infrastructure or increase your capacity, you can simply connect more hosts to the switches.
• It supports advanced network features and protocols, such as VLANs, LACP, or CXL. If you want to use these features or protocols, you can configure them on the switches.

Therefore, the choice between directly connecting the three hosts or using two switches depends on your budget, your requirements, and your preferences. If you want a simple, cheap, and easy solution, you can go with direct connection. If you want a robust, fast, and flexible solution, you can go with switches.

• How reliable are ASUS servers, especially without onsite service?

ASUS servers are known for their high quality, performance, and reliability. They have passed stringent quality tests and have achieved many world records and awards. They also offer a range of service and support options, such as next business day parts service, warranty extension, and technical support.

However, without onsite service, you may face some challenges or delays in case of a hardware failure or a technical issue. You may have to wait for the replacement parts to arrive, or troubleshoot the problem yourself, or contact the technical support remotely. Therefore, you should consider the availability and cost of onsite service in your region, and weigh the benefits and risks of having or not having it.

• Is it worth upgrading to AMD Genoa CPUs, or are AMD Milan CPUs sufficient for our needs?

AMD Genoa CPUs are the next generation of AMD EPYC CPUs, based on the Zen 4 architecture and the 5nm process node. They are expected to offer significant improvements over AMD Milan CPUs, such as more cores, higher frequencies, larger caches, faster memory, and more features. They are also compatible with the latest technologies, such as DDR5, PCIe 5.0, and CXL.

However, AMD Genoa CPUs are not yet available in the market, and they may not be compatible with your current hardware or software components. They may also be more expensive and more power-hungry than AMD Milan CPUs. Therefore, you should consider the availability and cost of AMD Genoa CPUs, and compare their performance and features with AMD Milan CPUs, before deciding whether to upgrade or not.

In conclusion, your proposed solution of migrating from VMware to Proxmox with ZFS replication for a small business is a viable and feasible solution, but it also involves some risks and challenges that you should be prepared for. You should also evaluate the pros and cons of different options for your network, your service, and your CPU, and choose the ones that best suit your needs and budget. I hope this article has answered your questions and provided you with some useful information and advice. Thank you for your time and expertise.

Best regards,

Copilot