Retired Techie
Getting older, not necessarily wiser!
Tagaytay Network Part 4: Backups and Archives
Published on October 30, 2023 at 2:28 pm by LEWIntroduction
Backups is another subject I want to take a few minutes to explore. While not specifically related to the Local Area networks (LAN) I will be working on, it is something that most of us do not give much thought to until it is to late. While I do backups on my computer, I probably don’t do them as often as the experts would recommend. Nor do I exactly follow any of the methodologies that I see being proposed. It is not that I necessarily agree or disagree with the methodologies, but rather that I have some understanding of the law of diminishing returns.
Additionally I would like to discuss what I backup and why. This necessitates that I define how I differentiate between a backup and an archives. This is probably not an important concept to the average user, but it does, to some extent, affect how and what I backup on my own systems.
Before moving forward, I want to state that there are some things that I do not, as a general rule, backup. At least not on a regular basis. For example Operating Systems (OS) and Application. But I will get into that more below.
Backups vs Archive
One thing many people do not consider when thinking about backups is archival data. I tend to break my personal data into two categories, backup and archive.
Backups are copies of items I am currently working on and /or need to access on a regular basis. I generally need to access them relatively quickly, so they must be ready available.
Archives are items that I am no longer working on and that I access only rarely, but want to keep for a variety of reasons. It is not critical to have immediate access. So they are stored with conservation of space, rather than speed of access, being the primary concern.
For example, a legal document, like a three year old tax return, might be something you want to hold onto, but are unlikely to access without specific reason. To save some space I would store this in an archival format like zip.
To access the document would be a two step process rather than one step. Pull it from your backup media and unzip it. While the space savings of individual documents may not be great, it adds up with volume over time.
Backup Methods
Now I want to discuss are the various methods of backing up data. Over the past several days I have reviewed a lot of information on backups that is out there on the internet, and there is a lot of options on backup methods. And in fact I see a lot of negative commentary on perfectly legitimate backup methods.
In the end there are two basic methods used. The recommended method is copy your data to some other media. Another type of method is to use multiple disks and something like striping and partiality to allow data to be recalculated when a specific single component fails. This is the one that is usually not recommended for a couple of specific reasons.
While parity and striping is a valid method, it has a down side. To say it is not a legitimate method is incorrect as long as one understands the issues. The major issue being rebuild time after a failure. For a modest system of 10 terabytes you are probably looking at around five to six days.
My historical backup method has been two external drives. I alternate drives when backing up all my personal data once a week. One drive (the week old copy stays in my car. The other drive (the current one) stays in my house.
Backup Strategy
A backup strategy is separate from method, which is how you do a backup. A backup strategy involves three parts; what, how, and when.
The first decision is what you will back up. The two things I normally backup are my personal files and various configuration files. This involves selecting directories and files, and using programs like rsync (Linux) or robocopy (Windows).
Some people prefer to backup their whole system. In these cases you can do things like snapshots, restore points or drive clones.
After deciding what to backup you need to decide how to do it. Part of how will be the number of backups you want to have.
There are multiple methods to chose from such as full clone, full backup , incremental backup, and differential backup. There are also many variations on these methods.
A Full Clone is making an image of a partition with a program like Clonezilla. A good use of the clone strategy is on a clean install of an Operating System (OS) after all the applications have been installed, and before use. Then when a drive breaks, you can restore the clone to get a functional OS before restoring data. Another use of a clone is to propagate multiple identical systems fairy quickly.
A Full backup is just like it sounds, backing up all the selected files. While similar to a clone, this can span multiple partitions and drives as sources.
An Incremental backup backs up only files that have changed since the last backup. This is usually done by looking at the date stamps of the backup files and overwriting backup files that have been updated.
A Differential backup looks at the last full backup and saves any changed files since then to a new backup. Restoring from a differential backup means restoring from the full backup and all the differential files in order.
Another part of how is how many copies. This is where the law of diminishing returns also comes into play. I know people who keep four or five backups. Now while storage drives do fail, the odds of multiple storage drives failing is very small. However this does not preclude physical damage if you keep your backup next to your computer. House burns down, they are both gone. And keeping drives in remote locations can significantly increase recovery times.
My recommendation is to keep one backup in house and one backup remote. Some would also suggest keeping backups in the cloud. That will depend on either your technical ability, or how much you trust tech companies with your data.
Future Plans
This time around I want to create a backup server on the LAN. It will basically be a file server that I can copy various files to from different devices on the network. It will basically need lots of storage and various means of access.
For multi device access I am thinking of exploring Network File System (NFS) as opposed to Server message Block (SMB), the default on Windows Clients. While SMB is a at best an average method of connecting shares, NFS is more definitely more robust. The ability to enable NFS has become available in recent versions of Windows, and I want to give it a try.
While I can enable NFS on my Windows clients, that may not be something that guests want to do or are able to do. So I might possibly still be stuck with SMB.
Also, I need to give thought to the directory structure on the backup File Server. Since I plan on scripting various backup routines, while the directory structure needs to accommodate different devices, it should also be as flat as possible (I am thinking no more than thee layers at the moment).
Once the various scripts are written and tested, the final step will be automation. Basically the scripts need to run automatically at specific times.
Conclusion
Still in the rambling phase, just tossing a few ideas around. I still have a few more areas to explore before getting down to drafting a project plan
Add New Comment