Retired Techie

Introduction

Over the past six months, I’ve been building, and destroying, a large number of Containers and Virtual Machines (VM). I’ve now arrived at several I actually want to keep.

The problem is that when I was setting them up, I completely mishandled storage planning, at least in the long run.

All the VMs and Containers I was building and deleting on my Proxmox VE server were basically tests. Each one was set up with its own storage assumptions, without an overall plan.

In the end, several of these were worth keeping, but my storage planning (or lack there of) wasn’t. I allocated a lot of storage unevenly; Plex got 1 TB, Kiwix got 1 TB, File Browser got 512 GB, and Calibre server got 256 GB.

None of those services have ever got close to their allocated space. On average they were running with less than 50% space utilization. So I effectively limited myself from using approximately 1.5 TB of storage tied up in my various Servers.

The fix is to consolidate storage by moving most data to a single file server, and have the other VMs access it for their content.

My Initial Server Setup

Without my server setup, I can’t give an accurate picture of what I’m doing. Your environment will be differ in both hardware and software. So everything I describe must be interpreted in the context of my system.

I do want to point out for Mr. Average user out there, that I don’t have a dedicated server rack, and I don’t even have a proper NAS (Network Attached Storage). Everything here is running on a mini PC. Having said that, the mini PC I am using is not a bargain basement one. It does have some chops, probably a lot more than it needs for this job. The point I want to make is that you don’t need fancy hardware or complex setups to run this stuff. It can be done simply and does not have to cost an arm and a leg. As an example, my other Proxmox mini PC is about six years old and I picked it up in a thrift shop for $20 US (did upgrade memory and ssd for another $100 US).

This mini PC is a BeeLink SER6 MAX mini PC (about 4 years old, so have not seen many in the used market). It has a Ryzen 9 6900HX (8 cores/16 threads), 32 GB RAM, and 3 TB of total NVMe storage. My two NVMe drives are combined into a single LVM volume group (VG), and each VM/Container uses its own logical volume (LV).

All guest VMs, with one exception, are on a base Debian 13 (Trixie) installation. The exception is my Mine Test server running Arch. This was mainly to meet a couple of application requirements about a year ago. Will need to check if Debian Back-ports can fill them now.

This setup has several inherent flaws. Some are related to storage (like not using thin provisioning) others to networking (like changing IP addresses of containers). These issues initially seemed minor, but as I planned for a long-term server farm, they became more significant.

Something else I need to clarify, none of my PC’s are server grade, they are all consumer models. There are a lot of discussions (arguments) about server vs consumer PC. But for mos thome users it shouldneve rbe an issue as long as they maintain good backups.

In this presentation, I’ll cover these issues and my mitigation, along with overall strategies.

LVM Provisioning

A common suggestion is to use thin provisioning. To understand why, it helps to first compare the two provisioning modes in LVM; thick and thin.

Thick (could almost call this default) provisioning is straightforward. For example, when you create a 512 GB logical volume, it reserves and uses 512 GB of real storage space from the underlying volume group immediately. The logical volume’s size is effectively the physical space you’ve committed.

Thin provisioning works differently. Instead of consuming physical space up front, a thin logical volume is created with a virtual size limit (512 GB in this example). Physical storage is consumed only as data is actually written. This allows you to overcommit capacity for workloads that rarely use their full allocated size.

Thin logical volumes also depend on additional infrastructure, as they are managed inside a thin pool. The thin pool tracks and allocates real storage as writes occur and also requires dedicated space for metadata, which is essential for operation.

Thin provisioning can be a great fit for test and development environments because it improves storage utilization. In production, though, the main risk is over-commitment. If the thin pool runs out of space in either the data area or its metadata, the system may be unable to write new data, and applications can experience I/O errors. For that reason, I’m generally hesitant to use thin provisioning in production unless the thin pool is correctly sized, actively monitored, and has a clear expansion plan.

Containers and IP Addresses

This one caught me by surprise. In Proxmox, a container’s network configuration, including the IP address, is applied during container creation as part of the container networking setup. Which is different from a full VM, where the installed Operating System (OS) is used to set the IP address.

For example, with Debian VMs I typically change the IP by editing /etc/network/interfaces. In a Debian container, though, that usual in-guest method doesn’t behave the same way because the container’s networking is being managed differently (and may be overwritten/controlled by how Proxmox provisions container networking).

It’s not a big problem, just something to be aware of. You can still change the IP in Proxmox by updating the container’s network settings, but it surprised me because my normal workflow is to log in and change networking inside the guest.

Pre-Planning the Build

I got into my current situation by using the mini PC for testing, without thinking much about future needs. This time, since this is going to be production, I’m approaching things from the “lets make a plan” perspective.

My plan is to build five main services:

1. A file server that provides both HTML and SMB access
2. A media server that stores its media on the file server
3. A wiki server, also storing its data on the file server
4. A E-book server, also using storage on the file server
5. A stand alone Gaming Server

I’m still deciding whether to run these services in containers or VMs, but before that I want to do some capacity planning based on what I measured during previous testing.

For a Debian 13 basic install, I’ve found that it uses about 1.2 GB. With the extra packages I expect to install (for example unzip, openssh-server, sudo, curl, etc.), I’ll budget closer to 1.4 GB.

The applications I’ll be installing generally fall in the 256 MB to 1.5 GB range. Conservatively, I’m estimating about 3 GB per instance for the core system and application footprint. Then, accounting for logs, databases, configuration files, and possible add-ons, I’m budgeting 5–6 GB, and then rounding up. I’ll plan for 10 GB for the media, wiki, and E-book servers.

One exception is the file server. Based on what I collected during testing, I’m projecting initial usage of 0.8 TB to 1 TB. But I also expect it to expand as I create/acquire additional data. Also since this is production, I don’t expect to be adding anymore test servers. Considering future expansion, I will allocating 2.5 TB to this server.

The Gaming server is a bit trickier to estimate. I used Arch instead of Debian because of library requirements not available in Debian at the time. A basic Arch install has a smaller footprint than Debian. However, if Debian Back Ports now has the correct Library versions available, I will strongly consider going that direction. But this is secondary to user data, which is variable depending on where each player is, what they are doing, and how much of the world has been explored. My existing server hosts four players and is currently using 24 GB of space. But this has been expanding over the last year. Since I don’t expect a sudden influx of players, I think 50 GB should be good.

I also want to note that there will be measurable storage-size differences between VMs and containers. Since I’m intentionally overestimating, this shouldn’t impact my overall planning.

The Build Plan

The next phase is to develop a build plan. This needs to include capturing any data that will need to be restored. A good example of this is the Kiwix Full Wiki file, which weighs in at 115 GB. While at some point I will need to download a later version, it should not be a priority for the rebuild.

So my plan is as follows;

1. Make sure I have good backups copies of all my data files (they are backed up to the cloud and my Proxmox backup server, but a backup is not a backup until it is tested).
2. Take down my Proxmox VE, and do some long overdue hardware maintenance on the mini PC while I have the chance.
3. Delete all existing servers and their corresponding files and data. I am considering reinstalling Proxmox, which would mean creating a new api key for the backup server.
4. Build the file server first, and restore other server data to server specific directories. This process also includes configuring SAMBA to grant server access but restrict servers to their own data files. While not necessary up front, html access can be setup later if needed.
5. Build the media server next as it is the one that gets used the most.
6. The wiki and E-Book servers can be built later, as I am the primary user (about 90%), though my niece has been digging into my SiFi and fantasy Libraries lately.
7. Finish up with the game server.

Conclusion

That is about the extent of my plan. I hope to carry it out over the next couple of weeks. Expect future articles and updates on my progress.

I have tried to cover all the technical stuff one should think about up front. I skipped over things that are more administrative, like good passwords, User names, server names, access methods, etc. Those are probably better handled in the individual setups.

As with other articles, where appropriate, videos will follow. So watch for them.

I do want to reiterate that this type of setup/home lab does not have to be expensive, even with the spiraling hardware prices. One can do amazing things with low end mini PCs these days. Pretty much any affordable NAS unit out there is running on a low end PC. If you intend to buy used hardware, then for a good experience, let me make a few suggestions. I can’t comment on storage space as only you know how much you need. Don’t worry about graphics, integrated is fine for most work loads. However for CPUs you will probably want a minimum of four core threaded (this yields 8 processors). Obviously more is better. For memory you can get buy with 16 GB. 32 GB is better, but it will depend on how many VMs you want to run.