I figured I would chronicle my latest journey here, because I know some people will get a kick out it it. The theme: Retro computing with a side of data recovery.. or is that data recovery with a side of retro computing..... first in a series 🧵
The beginning: Recently I have gotten back to some old projects, and I started to re-hydrate some OLD data files. One of these was curiously corrupt. After tracking down how (looks like it got truncated in one of the copies), I remember I still had a SUPER old copy on a backup from WAY back when (think 10 + years)
This old copy was on USB external disks, striped (2 of them) geli encrypted and written to on SPARC64/FreeBSD.. I think now you know the trajectory of the story (Maybe I should have opened with that)
So, first I decide to just bring them over to x86, I know UFS will be a problem with the endianness, but I can just get a good poke at the data, make sure it is even there. Get the disks attach, geli attach... accepts the passwords, but the stripe doesn't show up?!
I look at the raw data from geli.. look at drive 1.. totally random.. I GUESS it could be in the middle of a data block.. look at drive 2.. a clear UFS header... OK.. so this clearly works. Look at a few more blocks... randomness. I SHOULD be seeing inode blocks, I know what they look like.. what gives?.. and which is the GEOM_STRIPE footer missing?
Could it be the IV for geli is endian specific.... YES IT IS. Patch kernel to force endianness on it.. BOOM it works.. I have access.
my sparc64 hardware is OLD. *OOOOOLD*. ok, qemu has sparc64 support... lets do that, that will be easier than dealing with hardware...
Step one... compile... get the following error:
warning: POSIX Yacc does not support %define [
the build doesn't even exit cleanly.
wtf... search... search... search, find:
https://bugs.freebsd.org/bugzilla//show_bug.cgi?id=256536
Ok, implement the workaround (set environment variable.. .BUILD. Success. run...
IT BOOTS. yay..
wait.
interrupt storm detected on "vec2016:"; throttling interrupt source
after kernel takes over. search, find:
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=239068
Try recompiling as suggested, try turning off DMA on FreeBSD... try booting virtio.. sparc64 doesn't support virtio. try booting scsi, qemu doesn't support SCSI on sparc64 (non virtio)
Relent. give up. Hardware time.
Go downstairs, find most convenient box pick it up, clean it off, bring it up, plug in, turn on.
WTF.. It boots. Its is sitting at a login prompt?! EVEN THE NVRAM IS VALID?!?!?!
ah, yes.. where did we leave off... oh yes.. it boots, it works.. wow even attempting the emulation was a waste of time. running FreeBSD 11.. so even pretty recent (for SPARC)... check it out.. 1G RAM.. but no swap. and there's some data on the drive I should preserve before I nuke it. Start the tar.. go to bed.
Morning. power outage.. crap.. go to check, the tar finished, great.. system reset, sitting at fsck.. unexpected fileystem error.. crap. run fsck ... EVERYTHING is corrupt.. double crap.. vague memory that I knew this.. that for some reason fsck on 11 on sparc would flag everything as bad even though filesystem was consistent, but fsck on 10 would show it all as good... crap...
No point trying to recover, and it is old.. there's no way this data is valuable, I look at it again.. yeah.. toast it. download fresh 12.2-RELEASE-SPARC.disc1.iso, check chesksums and signatures (ALWAYS DO THIS).. now to find CDR media... ZOMG found some... ZOMG, cdrecord works.. boots.. errors on sector way at the end. whatever burn again.. slower speed. new disk fails the EXACT SAME sector. whatever.. just boot and go
zero out drive for good measure. reboot.... garbage on the serial console.... power cycle. garbage. power cycle.. garbage... vague memories that this is why I pulled one of these machines out and gave up on the series for anything real...
Now I begin my trek through my spares....
Look at the data.. yup, its there, lets see what I can do.. this will be maddening to walk by hand.. screw it. sparc64 we go....