Backing Up Data - The Redrobes way.

[Redrobes]

A surprising number of people post on this forum about a total hard drive wipe out or loss of work through some mishap. Nobody is immune to the effects of data loss right up to the infamous case of the near total loss of the film Toy Story 2. I have also experienced the total loss of recent work when a PSU on my machine blew out (smoke and all !) taking out the whole machine and all peripherals and hard drives within it. I made the mistake of using a second hard drive within the machine to keep it backed up. So henceforth, I changed my strategy and took it all a little more seriously.

This tutorial is for Windows only just one way of doing things which has prevented me from losing anything important since that date. I have lost a hard drive since my PC blew up and I did lose a few days worth of camera photos but that was the limit to the damage. If your on linux then generally you know what your doing and if on apple then you can use the cloud if you trust it.

I would also like to add that there are a multitude of ways you can backup and anybody who wishes to propose a different or better way of doing things is welcome but this is one way to show how you can do this where you should be able to hedge against most scenarios with about 10 mins of instruction and no cost. To prevent more traumatic data loss scenarios you probably need to spend a little and get some off computer storage.

The first thing I would like to say about back ups is that it had better be easy to do once set up. Even better would be if it were to be automated but I found that setting that up to be more difficult and that if not done with some manual trigger you don't have confidence that it is truly backing up the data.

There is just two halves to this tut. One is saving files whilst you work so that you don't lose files from a bad save. The other is saving all files on your PC once finished in case a drive fails or it blows up.

The first thing I would like to say is that data storage is cheap. Well, in my view its a lot cheaper than my time to recreate something I have lost. So before you start a new map or piece of art just ensure you have ample space on your hard drive to store any copies of the WIP. If your down to your last gig then its time to back up and find some off hard drive storage to keep things on or buy a bigger drive.

[Redrobes]

So for the first half this is what I would recommend when making a new piece of work.

When you start your map you can name it and then add a two digit number at the end starting 01. So MyNewMap_01.png is a good option. Then every change you make save it with increasing number. Now some maps with lots of layers are big files. Well then you need more space for them. Its just no good to keep saving over old copies. If its really that bad then when you get to 10 then delete 01 and keep a backlog of the last 10. But don't save over the last copy over and over again. When your ready to show the world your map then you can take the latest MyNewMap_15.png, merge all of the layers and then rescale it down to a sensible size and then save out another copy as say Guild_MyNewMap_01.png. I have heard of people merging layers to reduce the file size for publishing and then want to mod the map but cant without the information in layers again.

My advice is that whenever you give out a copy of a map or some file, like when you want to post a WIP on the guild, then you make a copy and label it so that you can refer to that version of it.

[Redrobes]

The second half is about backing up all your files.

My recommendation here is to have two similar sized hard drives in your machine. Then every now and again you run a little script which will copy everything in the drive to the other one by comparing all the files and updating anything that has changed. You can do that in one line which I will show in a mo. With the script that I run, if you have new files then it will copy them over. If you have an existing file which has changed then it will copy over the old one with the new one. If you have old files which have not changes then it will skip them.

Now obviously if you have an existing file and modify it so that it then becomes incorrect then when you back up it will overwrite the old correct one with the new incorrect one. That is a risk but if you modify your work and save out the new file with the increasing file post fix number then that is not an issue any more.

So this strategy will prevent you losing data from a hard drive failure. If your backup hard drive fails then you need to buy a new one and run the back up as soon as possible and get a new copy set. If your main working drive fails then you need to buy another, you lose files from the period between the last backup. Hence backup often and whenever you feel like you wouldn't want to redo whatever your doing.

The other downside to this is that the backup drive also has copies of old files which you might have deleted or moved in directory around. You can manually go in and delete stuff later on the backup drive which you don't want to keep any more. My experience tho is that this hardly ever happens and I am happy to accumulate the extra cruft and not worry about it being there.

If your PC does blow up spectacularly like mine did then it can take out both hard drives at the same time. So I think its a good idea to have an external USB hard drive. You can plug this in now and again and then run a similar one line script to copy the working hard drive to this one. With USB2 or USB3 now the data rate to the hard drive is faster that the write speeds of almost every drive so its not an issue. It used to be with USB1 or tape drives tho in days gone by. If you have a NAS then this is a similar option and your sorted but this tutorial is for those without special backup systems in place.

Now, if your PC blows up your drive is on a shelf and that's your monthly back up. I also have a couple of drives which I fill once in a while and take out of the building which are my last resorts. If a meteor hits the place then I lose a lot but not my life's work. Instead of using a dedicated USB drive, I have a USB drive caddy and I plug in raw drives into that and use them like USB drives - its the same thing but the caddy is shared across all the the raw drives so is cheaper. I know you can do cloud storage but I don't trust anything I don't own.

[Redrobes]

So that's both halves talked about so finally I am going to show you how to create a desktop icon to kick off a backup which you can hit once you have done enough work that you are nervous about losing it.

When you install the operating system it is best to assign it to one partition on the drive and have a second (or third etc) partitions for all the work and stuff you want on it. Generally tho, most people have a PC pre installed with Windows on it and you have the remaining space on the drive all as one giant partition. Usually thats the C drive. My advice tho is even with this you should create a new top level directory called "work" or similar and put everything you do, everything you install and everything thats not windows under that.

But assuming you haven't done that and you have a PC with a C drive and a DVD drive on D and that you have fitted a new hard drive and it comes up as E. Then you need to copy everything on C drive to E. If you do not have a second hard drive, cant fit one then you should plug in an external USB drive and see what letter that comes up as. But assuming that one comes up as E instead then its the same thing. You would just have to make sure you plug it in before back up and take it out afterwards.

You need to create two new text files and save them at the top of your main working drive. So create a new text file called BackupMainDrive_01.txt and another called ExcludeDirs_01.txt

If your not sure how to do that then use the right mouse button whilst the cursor is over the start button and use Explore All Users or the File Explorer. Then in the folder view click on Local Disk (C:). On the right hand panel are some file and all the top level directories. In a blank space under them you use the right mouse button and with a click it brings up a floating menu with one of the items being New and then it opens up the secondary menu with file types, one of which is Text Document. When that is selected then it puts up a "New Text Document.txt" and highlights it so that you can type in the file name. So create two of them and name them whatever you like but best not to keep any spaces in the filename.

So along with the other OS files you have:

BackupMainDrive_01.txt
ExcludeDirs_01.txt

You can now just double left click on them to open them with some kind of text editor - usually Notepad.

Now you should be able to cut-n-paste the text below straight in otherwise you have to type it.

Into BackupMainDrive_01.txt you put in

xcopy C:\*.* E: /K /O /X /E /Y /D /C /R /F /H /EXCLUDE:ExcludeDirs_01.txt

then save it and quit.

Into ExcludeDirs_01.txt file you put in all the directories you DONT want to back up one per line. For example you might your own temporary working dir. And there is the normal Windows temp dir which is unlikely to have anything useful in it.

C:\Temp
C:\WINDOWS\Temp

Save and quit that too.

Just to be clear if your main working drive is not C then change the xcopy line, if your USB drive comes up as something other than E then change that too. I.e. if you have two main hard drives E and F and want to back them both up to one large USB drive U then put in two lines like this.

xcopy E:\*.* U:\PC_E_Drive /K /O /X /E /Y /D /C /R /F /H /EXCLUDE:ExcludeDirs_01.txt
xcopy F:\*.* U:\PC_F_Drive /K /O /X /E /Y /D /C /R /F /H /EXCLUDE:ExcludeDirs_01.txt

In the exclude list you list all the excluded dirs from both drives in the one file.

Then on the USB drive create two dirs:
PC_E_Drive
PC_F_Drive

Now before I continue this bit is important. Just double check that the xcopy line you have in your script points to the correct drive letters and dirs. The C:\*.* is the from and the E: is the to. You dont want to copy the whole hard drive to itself so its real important that your from and to drives are correct.

Once done, click on the text file to highlight it so it turns blue, right click and use the rename menu option whereby it will put a box around it and give you a cursor to edit the name. Take off the final .txt and replace it with .bat
It may say that you have changed the file extension and may be unusable are you sure. You just hit yes.
The icon for the file will change from a little text pad to a little window thing because its now treated as a batch file.

Right click on it again and use the menu option of Create Shortcut. A new file will appear with a similar name but called "Shortcut to BackupMainDrive.bat"
Drag the shortcut out of the explorer and onto your desktop. You can rename it to something else if you wish like say "RunBackupMainDriveTo_USB_E" and you can assign it a nicer icon if you wanted too. It makes little difference.

So to back up your whole machine plug in your USB drive and then double click that shortcut on the desktop and a black command window will appear with a title of the shortcut and start to back up the whole machine. When its finished it will just go away. So there is nothing to do for it.

Now presumably people will post lots of messages saying how it doesn't work for them so ill try to instruct what to do. If this seems very alien then maybe wait till a few people have tried it and see how they get on and what their opinion is.

Finally - this is intended for information only and you do it at your own risk which you can balance against the risk of hardware failure.

[Hilded]

Very good advice. Thanks for the tutorial. I've learned from experience that back-ups of folders and drives are very important in preserving what we create in digital form.