37

I written an ISO image to my hard disk using sudo dd if=image.iso of=/dev/sda. I meant /dev/sdc, and now running sudo fdisk -l returns only one partition on /dev/sda while there is supposed to be 6. Is it possible to recover my data?

I didn't shut down (reboot) my computer yet.

Baaing Cow
  • 489
  • 25
    dd has shown its ugly 'Data Destroyer' face again :-( It has overwritten the head end of the drive (corresponding to the size of the iso file). The data that were there are lost. Have you got a fairly recent backup of the files that you cannot afford to lose? Otherwise you must prepare for hard work to repair the partition table and file systems, best on a cloned copy using Testdisk, and if that fails, even harder work to recover file data to be found 'behind' the part of the drive that is overwritten. You can use PhotoRec for that purpose. See this link, cgsecurity.org – sudodus Dec 02 '17 at 20:44
  • 1
    Please tell us if you have a backup, or if you have to try hard to recover the data, that might still be possible to recover. – sudodus Dec 02 '17 at 21:05
  • Are there more descriptive commands about how exactly to recover a partititon table? – Baaing Cow Dec 02 '17 at 21:07
  • It's possible you can recover using testdisk I wrote an answer to a similar problem here that you might find useful. – Elder Geek Dec 02 '17 at 21:10
  • 6
    Is drive gpt partitioned, it has the advantage of a backup partition table at end of drive. Only the beginning of drive to size of ISO was totally overwritten and is lost. Depending on sizes of partitions then you may be able to recover some or most of the other partitions. – oldfred Dec 02 '17 at 21:51
  • 10
    @sudodus "dd has shown its ugly 'Data Destroyer' face again" - Except it's nothing to do with dd, it's to do with directly accessing a disk device as root. Any other tool could've had the same effect. cat image.iso > /dev/sda would have had the exact same effect. Or pv .... Or, to some extent, even vi /dev/sda. In fact, this reminds me of the story where Linus Torvalds tried to dial in to a network using his hard drive device rather than the modem device, overwriting the start of his drive with the dial-in handshake. Nothing to do with dd. – marcelm Dec 02 '17 at 23:51
  • 4
    How large exactly is your .iso file? Did you interrupt dd, or did it run to completion? – marcelm Dec 02 '17 at 23:52
  • 4
    @marcelm, You are right. But in real life, people are using dd like this (for this purpose), so I think the 'Data Destroyer' warning is relevant, although the completely correct warning should be longer: "Do not use a command line like sudo dd if=image.iso of=/dev/sdx to clone to a mass storage device. Use a tool that helps you identify and select the correct target device and that has a final checkpoint, so that you can double-check that you will clone from the iso file to the correct target device, e.g. StartupDiskCreator, Disks, mkusb, Win32DiskImager." – sudodus Dec 03 '17 at 03:08
  • @sudodus "But in real life, people are using dd like this ..." - But making people afraid of the wrong thing (dd - by calling it "data destroyer") will achieve nothing. Then they'll see a command like gunzip -c foo.gz > /dev/sda and think it's fine because it doesn't have the "data destroyer" in it, and they'll hose their drive again. Better to make them afraid of the right thing, writing directly to device files as root :) – marcelm Dec 04 '17 at 11:05
  • 2
    Please clarify the file system used by those six partitions, in what order they are and what their approximate size is. – Andrea Lazzarotto Dec 04 '17 at 11:24
  • 1
    @marcelm, If we can make this happen, to make people recognize writing directly to device files as root, and be very careful when doing it, fine :-) – sudodus Dec 04 '17 at 11:54
  • Baaing Cow, please give us some feedback :-) Tell us details about your partition table and file systems (that you want to recover)! It will help us help you. Otherwise we can only guess. And what progress can you report? – sudodus Dec 04 '17 at 12:42
  • Did you ctrl-c the dd command soon after issuing it or did you let it copy the complete ISO? – Peter Green Dec 05 '17 at 15:09

6 Answers6

65

Since your computer is still on, you have a good chance of saving things.

First, sysfs exposes the kernel's in memory view of partition tables. Right now, your kernel has not reread the disk's table and it remembers your partitioning. Inside /sys/block/sda/, there should be one directory for each partition. Each such subdirectory contains a file start and a file size, which represent the partition's location and size, in sectors. Run fdisk and then recreate each partition (in order!) with the information in its sysfs directory. Note that you will have to create a new partition table, too. Make sure your new partition table is the same as the one before. You can check this in /sys/block/sda. If you see sda1 up to sda7 while you only have 6 partitions, you likely had MBR, and otherwise it was probably GPT. You also need to set flags and types for these partitions. Look at /etc/fstab and try to glean information about the types of partitions you have, then check Wikipedia to convert those to codes for your partitioning scheme.

Since you appear to be able to use your filesystem, it's likely that your dd did not get to overwrite any of it, since most of the important bits of a filesystem are at the front. Check /etc/fstab to see which partitions correspond to what part of your system, and then use the amount of data dd copied to get an estimate of how much you nuked. If you overwrote only part of a filesystem a good fsck might be able to fix it a bit (but you've lost a good chunk of data).

Most likely, you overwrote /boot. You'll have to reinstall your kernel (with aptitude) and reinstall GRUB2 (with grub-install) (which got both its first and second stage destroyed). GRUB2 is configured with files in /etc, so you won't have to manually recreate anything.

HTNW
  • 646
  • Output that is easier to decipher, type these at the shell prompt with a RETURN after each ...............: sudo parted, unit s, print all, quit – Hannu Dec 03 '17 at 19:16
  • 3
    Copy any valuable info to another drive (e.g. a USB stick) as you discover it. In case any of the partition-recreation steps don't go perfectly, you want to have a record of what was in memory of the still-running computer. Also, copy your most valuable and/or small files (including stuff like .bashrc and other dot files from your home directory, and all of /etc, besides your real data) to another disk while your running computer is still working. tar.gz is a good format for copying whole directories (because it preserves permissions and stuff). – Peter Cordes Dec 04 '17 at 02:58
  • He will also likely need to reformat what was /boot, which is kind of dangerous when there is uncertainty what the kernel thinks of your partition layout... – rackandboneman Dec 05 '17 at 10:01
  • 1
    @rackandboneman The kernel refuses to reread partition tables on a disk with mounted partitions. Reformatting /boot like this is perfectly "safe" (modulo literally everything else about this situation) because the kernel isn't uncertain about anything. – HTNW Dec 05 '17 at 13:20
24

Sorry to read about your tough luck

I guess that you have no current backup, and I am sorry to read about your tough luck using

sudo dd if=image.iso of=/dev/sdx

where you used the drive letter x = a so that you were overwriting the beginning of your hard disk drive where you stored a lot of valuable data.

You are not the first person and I am afraid not the last person affected by this problem. dd is recommended at many web sites to be used like this. It is a powerful but dangerous method, because it does what you tell it to do without questions. For this reason it is often nicknamed 'Data Destroyer'.

@marcelm comments, that this nickname is unfair. He is right that writing directly to a device file as root is causing the risk. So please be aware that you can easily overwrite a drive and destroy valuable data by writing to the corresponding device file /dev/sdx. (Safer tools are listed at the end of this answer.)

Recover the partition table, file systems and/or file content

  • Do as little as possible with the drive, and above all, do not write anything to it, because it can make the situation worse by overwriting data, that are still possible to recover.

  • If your data are valuable, you should work on a cloned copy, not the original (partly overwritten) drive. See this link, scroll down to 'Advanced repair of a partition table, file system and/or recovery of files'

    Repair the partition table and file system of a pendrive

  • If you had a GUID partition table, GPT, there is a backup of the partition table at the end of the drive /dev/sda, and you may be able to restore the main partition table from this backup table using the tool gdisk. See the manual man gdisk for details.

  • You may manage to restore the partition table and file systems using Testdisk from

    http://cgsecurity.org

    Read the instructions at the CGSecurity web site and also the link provided by @ElderGeek, Partitions disappeared after power loss while installing

  • You may also manage to restore the partition table and file systems using gpart/gparted according to the link provided by @CSM,

    HOWTO: Restore Lost Partitions to a Deleted or Corrupt Partition Table

  • PhotoRec: If you fail to recover the partition table and file systems, you can still recover some file data. The files at the head end, maybe 1-1.5 GB, are overwritten and lost, but files behind that part of the drive are still there. PhotoRec from http://cgsecurity.org is a tool, that can recover data 'from the drive surface' without a file system. Typical data in the beginning of common file types are used for identification.

    The following link provides details about PhotoRec,

    https://www.cgsecurity.org/wiki/PhotoRec

    You should save the files to a partition in a separate drive.

    It is not possible to recover the directory structure and it is usually not possible to recover the file names (and not the permissions and ownership), and there are problems, when files are fragmented, but many files can be recovered by PhotoRec (not only photos, also many other common file types). But, and this is a big but, it is a lot of hard work to wade through the huge amount of unsorted files to identify what are really important files, that you should rename to meaningful file names.

Safer tools to create USB boot drives with Ubuntu

In the future, please use a tool that helps you identify and select the correct target device and that has a final checkpoint, so that you can double-check that you will clone from the iso file to the correct target device, for example

  • In Ubuntu: the Ubuntu Startup Disk Creator (in Ubuntu 16.04 LTS and newer versions),

  • In Linux:

    • Disks alias gnome-disks,
    • mkusb,

  • In Windows: Win32DiskImager.

You can also use extracting tools, that provide a final checkpoint, for example

sudodus
  • 46,324
  • 5
  • 88
  • 152
  • 1
    i can vouch for cgsecurity's Testdisk, works much better than other alternatives i've tried in the past, where Testdisk was the only one who managed to salvage anything useful. but be careful, the little fucker defaults to try to recover your partition on the drive itself!! x.x – hanshenrik Dec 04 '17 at 14:37
  • 1
    +1. I can also vouch for testdisk. Also a wrapper script to prevent dd from writing to your /sda /sdb /whatever is a good idea too. – WinEunuuchs2Unix Jan 17 '18 at 02:27
  • so what is the proper way to do this? I purposefully backed up /dev/sda and want to restore /dev/sda but when I try to restore I just end up nuking the system because it overwrites everything including the disk head – CQM Jan 04 '21 at 21:51
  • @CQM, Your case is probably different from that of the question in this thread. Please create an own question and describe with as many details as possible your case. That way many people will see it, and you have better chances to get useful advice :-) – sudodus Jan 05 '21 at 10:06
18

By your dd command, you overwrote the partition table of /dev/sda and all data as far as the size of image.iso goes.

So the best you can achieve is to restore your partition table (with exact sizes) to recover at least the partitions towards the end of /dev/sda.

muclux
  • 5,154
16

Short of restoring from a backup, there is no way to fully recover. dd has overwritten the partition table and some of the data of the device.

BeastOfCaerbannog
  • 14,585
stumblebee
  • 3,547
  • 2
    This isn't even an answer, there can't be no way to restore the data. – Baaing Cow Dec 02 '17 at 21:02
  • 19
    I guess I should throw out all my disaster recovery backups then. – stumblebee Dec 02 '17 at 21:10
  • 23
    @Baaing Cow, of course it is an answer and of course there can be no way to recover data. That's what it means when you overwrite data: it's gone. – psusi Dec 02 '17 at 21:32
  • 2
    @psusi Overwritten data can still sometimes be recovered by data recovery specialists. – Chai T. Rex Dec 02 '17 at 22:04
  • 22
    @ChaiT.Rex That seems to be a myth, at least on any equipment you're likely to find in a PC made within the past decade. – David Schwartz Dec 02 '17 at 22:56
  • 19
    @ChaiT.Rex That only applies on not-fully-overwritten disks where only part of the disk is erased. If the entire disk has been erased or overwritten, however, there's no data recovery. Specialists or not. – Thomas Ward Dec 02 '17 at 22:59
  • 3
    @DavidSchwartz and Thomas Ward I've looked into it, and it looks like you're correct. – Chai T. Rex Dec 02 '17 at 23:13
  • @stumblebee - we keep backups because restoring data is hard, not always impossible. So hard that it's not worth the effort. If using a hard disk for example, there can be residual magnetic field imbalances corresponding to the original data, even if the data has been overwritten. See this video. There's a reason people recommend writing and rewriting random data over and over again for maximum security when clearing a drive. – Myridium Dec 03 '17 at 05:51
  • 8
    @BaaingCow As you know that there must be a way to restore the data, please enlighten us. Many people will be willing to pay money for your technique! – Tim Dec 03 '17 at 07:42
  • @Myridium as said above, that (4 year old video) is talking about older hardware. Modern HDDs are not nearly as vulnerable I believe – Tim Dec 03 '17 at 11:51
  • Quite possible for someone to have a 4 year old HDD, isn't it? My main one is 4 years old. – Myridium Dec 03 '17 at 11:55
  • 2
    @Myridium: If I recall the papers examining residual magnetic fields of hard disk drives correctly, the reliability is extremely low for anything beyond a handful of bits even for old (before 2000) drives. This is relevant for cryptographic keys but not general data recovery. – David Foerster Dec 04 '17 at 01:01
  • @David Foerster Ooh do you have a link to the paper(s)? – Myridium Dec 04 '17 at 01:03
  • 2
    @Myridium: I can't find it but I found similar ones, Overwriting Hard Drive Data: The Great Wiping Controversy and Can Intelligence Agencies Read Overwritten Data? A response to Gutmann.. – David Foerster Dec 04 '17 at 01:31
  • 1
    This answer on its own is slightly dangerous, even if it's technically correct in narrow sense, because it could make a reader assume all hope is lost. If the partition table type is GPT (which is suggested by the 6 partitions mentioned by OP) there's a secondary copy of the partition table at the end of the disk which is useful for recovery tools. If the ISO file written is smaller than the EFI system partition, the system partition was not damaged at all, and no file system level recovery is needed for that, and essentially nothing of value was lost. – nitro2k01 Dec 04 '17 at 08:54
  • 1
    @Myridium please stop spreading that false myth... just... please. – Andrea Lazzarotto Dec 04 '17 at 11:24
9

If you drive is partitioned into separate / and /home partitions, and the /home partition was after the / partition, then you can recover your data. The MBR which lists where most partitions are has been overwritten. However, gparted can recover your /home partition.

This post on Ubuntu Forums gives some ideas on how to do it; I found that by searching for "gparted recover partition".

Once you've found your lost partition (in might be inside an extended partition), you should add it as the only partition in the partition table.

Once you've done that, you'll have to reinstall Linux, telling to keep the recovered partition as /home.

Zanna
  • 70,465
CSM
  • 201
2

Testdisk to the rescue

I can personally vouch for Testdisk which saved my data after I was repeatedly copying and pasting instructions for /dev/sdb and changing it to /dev/sdd which was my USB. Only one time I forgot to change it and there went my /dev/sdb into the dustbin.

Setup your system to never write to /dev/sda

I strongly recommend creating a wrapper script: Prevent `dd` from destroying SSD or HDD. This ensures dd never writes to /dev/sda or possibly /dev/sdb or any other of your mass storage devices.

Community
  • 1
WinEunuuchs2Unix
  • 102,282