[Starlab] PANIC: deleted files from ext filesystem by mistake

(This post belongs to the series ‘from Starlab.net’, cross copies from my non-public blog at Starlab)

[Listening: keyboard typing]

This is the story of a dissaster, which could be half recovered:

On the premises of Stargym project, some tests are being run. Those tests are specified in a plain ascii config file for each test, and take several hours to finish each. After each test is finished, you obtain several files of different type: some PNG graphic files, some .out ascii files with results, some .sav ascii files with stage snap-shots… The png files are quite small, but the .out and .sav have a size of a fer hundreds of MB.

Therefore, in order to distribute tests’ results, you need to distribute hundreds of MBs. And here is where the real disaster story begins…

After weeks of tests preparation and runs, the time of collecting the results had come. In order to provide the files, an SSH account was set up on my machine, so it was possible to download the files through the scp command. But, I haven’t been able to figure out why, on one of those copy runs, the whole results directory disappeared. All the results of weeks of executions lost.

It’s been well known that recovering deleted files from ext filesystems has been almost impossible since ever. Therefore, my feelings were not coming really calm once i realized that the directory was actually lost (and not moved).

Luckly, I’m using the new ext4 filesystem on my machine. From the implementation of journalling (since v3 of ext filesystem), it is a-bit-less-than-impossible to recover deleted files. Looking for information about it, i found a piece of software claiming to be able to recover files from ext4 making use of new features on ext4’s journalling schema:

http://extundelete.sourceforge.net/

So I gave it a chance: I remounted the whole drive where the data used to be (a good point of having different disks for system/data) on read-only mode, and compiled the soft. And then, crossed the fingers…

And it worked! It told me that everything was recovered!!! Too much for a simple piece of new code… The ascii config files were correctly recovered, with all the contents intact. However, large .out files were not recovered right. They were dumped from the hard drive as binary data, rather than ascii data. Therefore, the data was useless for us. (yes, i could try further recovering options, or coding a binary-to-ascii converter, but time was really a factor and didn’t want to lose any more hours on a try I wasn’t sure it was going to work).

So, at the end, the results of the executions were lost, but not the configurations for the executions. I lost the machine-consuming work done, but i could recover the human-consuming work. Not too bad, at the end. It was only a matter of running again the tests and waiting the results (starblast is, as i write this, a huge help for this…).

First mini-point for ext recovery software.

[Listening: footsteps, door slams]

Anuncis

Deixa un comentari

Fill in your details below or click an icon to log in:

WordPress.com Logo

Esteu comentant fent servir el compte WordPress.com. Log Out / Canvia )

Twitter picture

Esteu comentant fent servir el compte Twitter. Log Out / Canvia )

Facebook photo

Esteu comentant fent servir el compte Facebook. Log Out / Canvia )

Google+ photo

Esteu comentant fent servir el compte Google+. Log Out / Canvia )

Connecting to %s

%d bloggers like this: