How I manage my backups
Go through the intro before you think you are not concerned #TodayWisdom
The need of backup
A few reasons we need backups, from the most known to the least expected
- 1. Disaster recovery. Your hard disk broke, your computer was flooded with water or stolen. That’s the most obvious use case. It won’t apply to notes in online services. It’s very rare for hard drives too. Restoration requires a complete backup.
- 2. Bad manoeuver. Like all software designers know the weak part of the system is the interface between the screen and the keyboard AKA the user. A wrong succession of gestures can lead to the disappearance of files. If it gets unnoticed long enough (like a few days) most undo systems or trash basket will lose track of lost documents. The more so if it was an edit. This will happen with services as well.
- 3. Intrusion and vandalism. Someone gained access to your system and by a joke or bad intent caused the destruction of some documents. Although it’s not so common it’s by far more likely than number 1. It could be a cat taking a nap on your keyboard (it’s warm, happened to me). It could be a nasty kid of a visitor (not ours of course) or an intruder (happened to me too). That’s a good reason to keep awake times short on devices.
- 4. General intrusion on the service. This could happen only in hosted services. It happens on regular basis. Security is a vast field and pirates are ready to waste years of their lives to find a breach. Like for nuclear power plants, it’s only a question of time before your data become accessible to them. If they steal it, good enough. If they remove it or encrypt it, you are in real trouble. It can take months before we know our data has been stolen. This happened again this morning with Uber (70 B$ valuation) who took 11 months to disclose it. Evernote is far from this size.
- 5. Changes in ToS, human errors on the service side. If your data is on a service you never know if they decide to take down a portion of your data because it breaches new ToS for reasons you can’t control and can be totally absurd (for pictures of “origin of the world” https://news.artnet.com/art-world/french-art-lovers-sue-facebook-426308 ). Another problem is that when you cross borders you may be asked to release access to your data for a long period of time without knowing what will be done with it. There will be time AI Bots will be routinely used to parse user data in search of some disturbing coincidence.
Bottom line
We need
- Complete backups of ours notes (the computer/OS backup is another story)
- Continuous backup, up to last hours
- Version based backups
- On site and offsites backups, off service if we use a service
On Site and versions
For this I have a simple portable 4 To HD. A small script copy various branches. Old files or vanished ones are placed in a versioned branch named with the date. This didn’t solve the sorting case, files being moved instead of removed. For this purpose, large files are checked for unicity before being kept in the archive. This is especially handy for video files or scanned papers. I prefer to upgrade my HDs to larger size instead of adding a new one to keep it minimal when I travel. I archive very old branches (1 year) in the last larger drive. Since the portable USB HD is not always connected and should stay away from my desk as much as possible it can’t be a solution for continuous backup.
Off site and continuous backup
For this I use a leased virtual server with a lot of disk in the cloud. Various suppliers offer this service. It’s easier to use than service likes Amazon Glacier because it includes a CPU and direct access storage (aka a disk). I rent 500 GB for $5 a month. It’s running Centos 7 which is a security-oriented version of Linux. A script using rsync is running every 2 hours on my machine. It’s a super optimized data transfer tool. It is sending differences only by constant size slices and zipped. This means that only portions of very large files are transferred if the file is partially changed. The initial backup is slow but incremental backups are fast and don’t use massive bandwidth. I don’t serve or share files from this computer to keep it totally stealth. There is no name or web server on it. I have a shell access and I can download files from there. It happens occasionally when I travel. If someone tells me to download quickly a large file my connection wouldn’t allow me to download, I first download it on this server and extract what interests me. It allows me also to prepare files to upload, like videos.
Alternatives to this tactic exist. You could use Google Drive and let it sync automatically or iCloud drive. Both are limited in size or not cheap when you exceed the free allowance (15 GB on Google Drive). I need 100 GB minimum, it’s $2 on Google Drive, $10 for 1To. The benefit is to have tools to access, modify and view some documents using G-suite. Note that gmail is also taking up space there. Worth noting that above risks related to services apply as well to such services. However, used as backups it’s only a fraction of the risk
Sabrina did a nice periscope on this topic https://www.pscp.tv/SabrinaCadini/1yNGamYQjNXGj. She listed Carbonite ($6/mo https://www.carbonite.com/backup-software/buy-carbonite-safe/) and SugarSync ($10/mo) https://www.sugarsync.com/pricing. Check her explanation on respective value of services.
https://twitter.com/brunowinck/status/932518988038524928