Recovering a half-encrypted disk

Question

Recently I filed a bug against what, from my perspective, was a majorly annoying bug with the Ubuntu installer.

It would appear, however, that my troubles installing Ubuntu are not related to Ubuntu but rather that when trying out an installation of Manjaro Linux a few days ago, I accidentally selected "Encrypt disk" and "Use LVM", and I assumed that a simple fresh partition table would make the encryptedness go away.

Now that I've tried to install Ubuntu 14.04 with Ubiquity crashing in the same place, I see that there's obviously an error with the disk in some way.

It's not physically damaged, as far as I can tell from the SMART status (included below), and gparted doesn't complain about IO errors formatting or writing the disk.

I've also verified the hashes and integrity of all the many LiveCDs I used.

~~Assuming that the encryption is the issue, what can I do to recover my disk so that Ubuntu will install? Do I write every sector full of zeroes? (Wouldn't that take days?)~~

If encryption isn't the issue, what is?

Assuming encryption isn't the issue, what is?

here's smartctl -a /dev/sda, yielding, as expected, no errors:

smartctl 6.2 2013-07-26 r3841 [x86_64-linux-3.13.0-24-generic] (local build)
Copyright (C) 2002-13, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Device Model:     SanDisk SDSSDA240G
Serial Number:    153655402044
LU WWN Device Id: 5 001b44 ed2285a3c
Firmware Version: U21010RL
User Capacity:    240,057,409,536 bytes [240 GB]
Sector Size:      512 bytes logical/physical
Rotation Rate:    Solid State Device
Device is:        Not in smartctl database [for details use: -P showall]
ATA Version is:   ACS-2 T13/2015-D revision 3
SATA Version is:  SATA >3.1, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Sat Dec 26 21:37:31 2015 EST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled
AAM feature is:   Unavailable
APM level is:     254 (maximum performance)
Rd look-ahead is: Disabled
Write cache is:   Enabled
ATA Security is:  ENABLED, PW level MAX, not locked, frozen [SEC6]
Wt Cache Reorder: Unavailable

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x00) Offline data collection activity
                    was never started.
                    Auto Offline Data Collection: Disabled.
Self-test execution status:      (   0) The previous self-test routine completed
                    without error or no self-test has ever 
                    been run.
Total time to complete Offline 
data collection:        (    0) seconds.
Offline data collection
capabilities:            (0x71) SMART execute Offline immediate.
                    No Auto Offline data collection support.
                    Suspend Offline collection upon new
                    command.
                    No Offline surface scan supported.
                    Self-test supported.
                    Conveyance Self-test supported.
                    Selective Self-test supported.
SMART capabilities:            (0x0002) Does not save SMART data before
                    entering power-saving mode.
                    Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                    General Purpose Logging supported.
Short self-test routine 
recommended polling time:    (   2) minutes.
Extended self-test routine
recommended polling time:    (  10) minutes.
Conveyance self-test routine
recommended polling time:    (   2) minutes.

SMART Attributes Data Structure revision number: 1
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAGS    VALUE WORST THRESH FAIL RAW_VALUE
  5 Reallocated_Sector_Ct   -O--CK   100   100   000    -    0
  9 Power_On_Hours          -O--CK   143   100   000    -    143
 12 Power_Cycle_Count       -O--CK   100   100   000    -    833
166 Unknown_Attribute       -O--CK   100   100   000    -    1
167 Unknown_Attribute       -O--CK   100   100   000    -    0
168 Unknown_Attribute       -O--CK   100   100   000    -    12
169 Unknown_Attribute       -O--CK   100   100   000    -    46
170 Unknown_Attribute       -O--CK   100   100   000    -    0
171 Unknown_Attribute       -O--CK   100   100   000    -    0
172 Unknown_Attribute       -O--CK   100   100   000    -    0
173 Unknown_Attribute       -O--CK   100   100   ---    -    7
174 Unknown_Attribute       -O--CK   100   100   000    -    57
187 Reported_Uncorrect      -O--CK   100   100   000    -    0
194 Temperature_Celsius     -O---K   061   100   000    -    39 (0 52 0 0 0)
199 UDMA_CRC_Error_Count    -O--CK   100   100   000    -    0
230 Unknown_SSD_Attribute   -O--CK   100   100   000    -    0
232 Available_Reservd_Space PO--CK   100   100   004    -    100
233 Media_Wearout_Indicator -O--CK   100   100   000    -    1872
241 Total_LBAs_Written      ----CK   253   253   000    -    1687
242 Total_LBAs_Read         ----CK   253   253   000    -    891
                            ||||||_ K auto-keep
                            |||||__ C event count
                            ||||___ R error rate
                            |||____ S speed/performance
                            ||_____ O updated online
                            |______ P prefailure warning

General Purpose Log Directory Version 1
SMART           Log Directory Version 1 [multi-sector log support]
Address    Access  R/W   Size  Description
0x00       GPL,SL  R/O      1  Log Directory
0x01       GPL,SL  R/O      1  Summary SMART error log
0x02       GPL,SL  R/O      1  Comprehensive SMART error log
0x03       GPL,SL  R/O      1  Ext. Comprehensive SMART error log
0x04       GPL,SL  R/O      8  Device Statistics log
0x06       GPL,SL  R/O      1  SMART self-test log
0x07       GPL,SL  R/O      1  Extended self-test log
0x09       GPL,SL  R/W      1  Selective self-test log
0x10       GPL,SL  R/O      1  NCQ Command Error log
0x11       GPL,SL  R/O      1  SATA Phy Event Counters
0x30       GPL,SL  R/O      9  IDENTIFY DEVICE data log
0x80-0x9f  GPL,SL  R/W     16  Host vendor specific log
0xe0       GPL,SL  R/W      1  SCT Command/Status
0xe1       GPL,SL  R/W      1  SCT Data Transfer

SMART Extended Comprehensive Error Log Version: 1 (1 sectors)
No Errors Logged

SMART Error Log Version: 1
No Errors Logged

SMART Extended Self-test Log Version: 1 (1 sectors)
No self-tests have been logged.  [To run self-tests, use: smartctl -t]

SMART Self-test log structure revision number 1
No self-tests have been logged.  [To run self-tests, use: smartctl -t]


SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

SCT Commands not supported

Device Statistics (GP Log 0x04)
Page Offset Size         Value  Description
  1  =====  =                =  == General Statistics (rev 2) ==
  1  0x008  4              833  Lifetime Power-On Resets
  1  0x010  4              143  Power-on Hours
  1  0x018  6             1687  Logical Sectors Written
  1  0x020  6         67940491  Number of Write Commands
  1  0x028  6              891  Logical Sectors Read
  1  0x030  6         42663463  Number of Read Commands
  2  =====  =                =  == Free-Fall Statistics (empty) ==
  3  =====  =                =  == Rotating Media Statistics (empty) ==
  4  =====  =                =  == General Errors Statistics (rev 1) ==
  4  0x008  4                0  Number of Reported Uncorrectable Errors
  4  0x010  4               57  Resets Between Cmd Acceptance and Completion
  5  =====  =                =  == Temperature Statistics (rev 1) ==
  5  0x008  1               39  Current Temperature
  5  0x010  1             -110  Average Short Term Temperature
  5  0x018  1             -124  Average Long Term Temperature
  5  0x020  1               51  Highest Temperature
  5  0x028  1               10  Lowest Temperature
  5  0x030  1             -110  Highest Average Short Term Temperature
  5  0x038  1                1  Lowest Average Short Term Temperature
  5  0x040  1               -1  Highest Average Long Term Temperature
  5  0x048  1                0  Lowest Average Long Term Temperature
  5  0x050  4                0  Time in Over-Temperature
  5  0x058  1              100  Specified Maximum Operating Temperature
  5  0x060  4                0  Time in Under-Temperature
  5  0x068  1                0  Specified Minimum Operating Temperature
  6  =====  =                =  == Transport Statistics (rev 1) ==
  6  0x008  4             4442  Number of Hardware Resets
  6  0x018  4                0  Number of Interface CRC Errors
  7  =====  =                =  == Solid State Device Statistics (rev 1) ==
  7  0x008  1                0  Percentage Used Endurance Indicator

SATA Phy Event Counters (GP Log 0x11)
ID      Size     Value  Description
0x0001  2            0  Command failed due to ICRC error
0x0002  2            0  R_ERR response for data FIS
0x0003  2            0  R_ERR response for device-to-host data FIS
0x0004  2            0  R_ERR response for host-to-device data FIS
0x0005  2            0  R_ERR response for non-data FIS
0x0006  2            0  R_ERR response for device-to-host non-data FIS
0x0007  2            0  R_ERR response for host-to-device non-data FIS
0x0008  2            0  Device-to-host non-data FIS retries
0x0009  2            0  Transition from drive PhyRdy to drive PhyNRdy
0x000a  2            4  Device-to-host register FISes sent due to a COMRESET
0x000b  2            0  CRC errors within host-to-device FIS
0x000d  2            0  Non-CRC errors within host-to-device FIS
0x000f  2            0  R_ERR response for host-to-device data FIS, CRC
0x0010  2            0  R_ERR response for host-to-device data FIS, non-CRC
0x0012  2            0  R_ERR response for host-to-device non-data FIS, CRC
0x0013  2            0  R_ERR response for host-to-device non-data FIS, non-CRC

You're correct. Writing a new partition table to the drive should get rid of any interference with previous partitions or data. I don't believe that the (half-)encrypted partition is related to your installation issue. Did you verify the integrity of the disk image(s) used as source to create the installation media? Did you verify the integrity of the installation medium itself with the appropriate boot menu option? — David Foerster, Dec 29 '15 at 13:33
@DavidFoerster I did indeed both verify the md5 and check the integrity at boot of each live cd of the pile that i've used. — cat, Dec 29 '15 at 14:54
Thanks. It would be best to clarify that in your question for future readers, since it's the first go-to for installation issue. It's best to have everything relevant in one place. :-) Also, comments may be deleted for various reasons. Unfortunately I don't have any other suggestions on the issue to offer. — David Foerster, Dec 29 '15 at 15:00

score 1 · Accepted Answer · edited Apr 13 '17 at 12:24

At this point what I would recommend is a full wipe of the drive. As a disclaimer this really does a full wipe making anything there impossible to recover! For HDD's you'd use dd if=/dev/zero of=/dev/sdX but appears you've got an SSD so it's a tad more involved.

Start by checking that security is not enabled (FYI: this all must be done as root/sudo and I'm using /dev/sda as that's what your output above shows. You can replace that with whatever the drive in question is.):

# hdparm -I /dev/sda
[...truncated...]
Security: 
    Master password revision code = 65534
            supported
    not     enabled
    not     locked
            frozen
    not     expired: security count
            supported: enhanced erase
    2min for SECURITY ERASE UNIT. 2min for ENHANCED SECURITY ERASE UNIT.

That not enabled line means we can proceed with ease, we now enable security:

# hdparm --user-master u --security-set-pass tempPassword /dev/sda

The password doesn't matter it'll be cleared too. Let's do a quick sanity check to make sure that worked.

#hdparm -I /dev/sda
[...]
Security: 
    Master password revision code = 65534
            supported
            enabled
[...]

It now says enabled, so we can now perform the actual wipe:

# hdparm --user-master u --security-erase tempPassword /dev/sda

One last sanity check with # hdparm -I /dev/sda should reveal a not enabled like in the first check.

Reference: What is the recommended way to empty a SSD?

I actually solved the problem on my own; it was because the SSD was password-protected! However, you are sorely in need of rep and I greatly appreciate your effort, so congrats! :D — cat, Jan 01 '16 at 05:29
It should be noted that the command to set the password for the SSD will fail with an Input/Output Error if there's a hardware password on the disk. — cat, Jan 01 '16 at 05:31
First, thanks. Second, now I'm curious you're saying the above command would fail if there is a password set... where? bios? — snorp, Jan 05 '16 at 01:24
The command failed for me on my Toshiba, with all the OEM BIOS (from 2012) passwords enabled. Works On My Machine™, but maybe not so much for other machines. — cat, Jan 05 '16 at 01:56
Awesome thanks, that is helpful. I imagine that you're not the only one where a BIOS password on a disk could cause these issues. Although usually by the time you're making changes to a disk it is unlocked so this is definitely interesting. — snorp, Jan 05 '16 at 15:57

Recovering a half-encrypted disk

1 Answers1

Linked