5

I have two SSDs in my laptop:

  • Crucial MX300 725GB --> /dev/sda
  • SanDisk SSD Plus 240GB --> /dev/sdb

Their performance reads on Linux and Windows like this:

Crucial MX300 --> same on both OSs

sudo hdparm -tT /dev/sda # Crucial
Timing cached reads:   13700 MB in  2.00 seconds = 6854.30 MB/sec
Timing buffered disk reads: 1440 MB in  3.00 seconds = 479.58 MB/sec

Crucial MX300 725GB

SanDisk Plus --> way faster on Windows!

sudo hdparm -tT /dev/sdb # SanDisk
Timing cached reads:   7668 MB in  2.00 seconds = 3834.92 MB/sec
Timing buffered disk reads: 798 MB in  3.00 seconds = 265.78 MB/sec # TOO LOW !!

SanDisk

The sequential read performance of the SanDisk on Linux is about half of its performance on Windows!

My Question is of course: Why and can that be fixed? Is this due to the SanDisk SSD Plus being handled as a SCSI drive?

From syslog:

~$ grep SDSSD /var/log/syslog
systemd[1]: Found device SanDisk_SDSSDA240G
kernel: [    2.152138] ata2.00: ATA-9: SanDisk SDSSDA240G, Z32070RL, max UDMA/133
kernel: [    2.174689] scsi 1:0:0:0: Direct-Access     ATA      SanDisk SDSSDA24 70RL PQ: 0 ANSI: 5
smartd[1035]: Device: /dev/sdb [SAT], SanDisk SDSSDA240G, S/N:162783441004, WWN:5-001b44-4a404e4f0, FW:Z32070RL, 240 GB
smartd[1035]: Device: /dev/sdb [SAT], state read from /var/lib/smartmontools/smartd.SanDisk_SDSSDA240G-162783441004.ata.state
smartd[1035]: Device: /dev/sdb [SAT], state written to /var/lib/smartmontools/smartd.SanDisk_SDSSDA240G-162783441004.ata.state

Compared to the Crucial MX300 which has on linux almost the same performance as on Windows:

~$ grep MX300 /var/log/syslog
systemd[1]: Found device Crucial_CT750MX300SSD1
kernel: [    1.775520] ata1.00: ATA-10: Crucial_CT750MX300SSD1,  M0CR050, max UDMA/133
smartd[1035]: Device: /dev/sda [SAT], Crucial_CT750MX300SSD1, S/N:16251486AC40, WWN:5-00a075-11486ac40, FW:M0CR050, 750 GB
smartd[1035]: Device: /dev/sda [SAT], state read from /var/lib/smartmontools/smartd.Crucial_CT750MX300SSD1-16251486AC40.ata.state
smartd[1035]: Device: /dev/sda [SAT], state written to /var/lib/smartmontools/smartd.Crucial_CT750MX300SSD1-16251486AC40.ata.state


Any help is very welcome!

Edit:

The difference that hdparm is showing on Linux is very real. I created two identical directories, one in each of the two drives, each directory containing about 25Gb of files (36395 files), and ran the exact same hashdeep checksum creation script on both dirs (the script just creates a md5-checksum for every file in the test dirs and stores all the checksums in one single file). These are the results:

test-sandisk# time create-file-integrity-md5sums.sh .
real    1m49.000s
user    1m24.868s
sys 0m15.808s

test-mx300# time create-file-integrity-md5sums.sh .
real    0m54.180s
user    1m4.628s
sys 0m11.640s

Same test with a single 7Gb file:

test-sandisk# time create-file-integrity-md5sums.sh .
real    0m26.986s
user    0m19.168s
sys 0m3.232s


test-mx300# time create-file-integrity-md5sums.sh .
real    0m17.285s
user    0m16.248s
sys 0m1.368s

Edit 2:

The partitions are "optimal" aligned and the only difference in /sys/block/$disk/queue is discard_zeroes_data (1 on Crucial, 0 on SanDisk). File system and mount options used: type ext4 (rw,nosuid,nodev,relatime,data=ordered,uhelper=udisks2)

dmesg | grep -i sata | grep 'link up'
[    1.936764] ata1: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
[    2.304548] ata2: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
JLTD
  • 629
  • 1
    I think you run in to a benchmark issue here. I would never compare two different benchmarks with each other. Moreover, the values are artificial. You should run a benchmark for the workload you use instead of artificial benchmarks. – user61664 May 23 '17 at 11:25
  • Thank you for your thoughts. Do you know of any benchmark software that runs on Windows and on Linux? I can't compare the workload I use in Windows because the software I use only runs on Linux. – JLTD May 23 '17 at 21:50
  • Why would the sequential read speed numbers match for the CX300 on Windows and Linux and not for the SanDisk Plus? I think that sequential read benchmarks should deliver comparable results, don't you think? – JLTD May 23 '17 at 21:52
  • What file-system are you using, and what mount options are being used?

    Do, $ sudo parted /dev/$disk_partition align-check minimal 1' andsudo parted /dev/$disk_partition align-check optimal 1 return 1 aligned?

    There are a lot of variables here, but you may be running into differences on alignment or enabled features. You can check cache types, and other statistics by looking in '/sys/block/$device_name/queue/'

    Unfortunately I am using NVMe and cannot provide examples but you can use cat on most of the files to see how the disk is configured.

    – gdahlm May 24 '17 at 01:31
  • @gdahlm, thank you very much. The partitions are "optimal" aligned and the only difference in /sys/block/$disk/queue is discard_zeroes_data (1 on Crucial, 0 on SanDisk). File system and mount options used: type ext4 (rw,nosuid,nodev,relatime,data=ordered,uhelper=udisks2) – JLTD May 24 '17 at 08:22
  • @user61664 forgot to tag you. – JLTD May 24 '17 at 08:25
  • 1
    Can you reproduce the issue with an up-to-date mainline kernel – possibly with a different Linux distribution? This is probably a kernel or firmware bug. – David Foerster May 25 '17 at 17:28
  • Does dmesg | grep -i sata | grep 'link up' reveal both are operating on Sata III channel (6 Gbps)? – WinEunuuchs2Unix May 25 '17 at 23:52
  • Would be possible, for any reason, the APM level too low for this device? References: Ubuntu SSD - Was fast, is now extremely slow and Default on battery APM level causes SSD runing very slow. – pa4080 May 27 '17 at 09:01
  • @DavidFoerster I am using 4.8.0-53-generic without any mod on Ubuntu 16.04. I could reproduce the hdparm measurements on a Ubuntu live cd, bot have not tried with another distribution. Thank you for the idea! – JLTD May 27 '17 at 14:04
  • @WinEunuuchs2Unix Yes, both report SATA III, thank you. – JLTD May 27 '17 at 14:05
  • @SpasSpasov Good point but I'm using the laptop plugged to AC all the time. – JLTD May 27 '17 at 14:05
  • The Ubuntu kernel is not the mainline kernel and it's not up-to-date either. The current upstream kernel version is 4.11. See https://wiki.ubuntu.com/Kernel/MainlineBuilds for instructions on how to build mainline kernels in Ubuntu. – David Foerster May 27 '17 at 22:26
  • Could it be that the USB is formated with FAT or some windows File System? I think that might effect the speed. – jack0da May 30 '17 at 04:42
  • @JLTD - fio runs on both windows and linux, and is pretty well-regarded linux disk benchmark software: https://github.com/axboe/fio – aggieNick02 Dec 30 '21 at 04:06

3 Answers3

1

This may not be a good enough answer, but I am new and can't "comment", and wanted to share with you in case it helps:

I used to work with eMMC flash memories, similar hardware underpinnings as SSD. discard_zeroes_data sounds like it could matter. There were very slow features called Erase and especially Trim (like Erase, but on a 4kB read-block basis instead of the much larger Erase Group Block needed for performance. Later a "Discard" feature was introduced which didn't erase the data to all zeros (really all-ones under the hood, but inverted for your convenience), but just changed the state of the data to "don't care".

So if you did a discard on 4kB of data, it was faster because didn't do an erase, but let the firmware know that you didn't need the data anymore, and it kept track of that physical area in a table. This allowed the garbage collection type of mechanism to reclaim that data location (likely re-map it to a different address) later when enough of its neighbours could be erased, copying any needed data to another area as needed and then doing the expensive "Erase" operation in the background.

TBH we ended up disabling discard at first, because it didn't work so well in and caused us performance issues when too much data was left in discard state, and the physical erases never happened.

So I can't tell what "discard_zeros_data" exactly means, but if I were you I'd definitely try changing it to the opposite state and see what happens. If it reads like "subject verb object", then it could be a security feature where it takes the time to forcibly erase even small bits of data (expensive) so that there's no chance that someone can reclaim your old files after you think you've deleted them. I would think that setting this to "Zero" would actually increase performance, if that's how you read it, but you're seeing the opposite.

Also try doing the biggest forcible "erase" that you are able to, whether using special SSD tools, or a full format, or something, and see if the performance is better right after this erase. That will at least give you some information about the problem.

The filesystem type should definitely matter also, but I think this is a constant in your two experiments.

Starman
  • 298
1

Comparison to USB pendrives

Simple USB pendrives get slower gradually after being used for a while. Such drives have no discard feature (as far as I know; I am not talking about advanced and very expensive pendrives). Wiping the whole drives, overwriting with zeros (or ones internally) will restore the speed again. From my own experience I know that this applies to Sandisk Extreme pendrives, which are fast when new (compared to other simple USB pendrives).

So I can expect, that the discard method (or lack of it) can be responsible for the performance drop also in your Sandisk SSD. I think that @Starman's answer adds valuable information to solve your problem.

You can

  • try to leave the system running idle overnight and check if it has used the idle time to catch up discarding what should be discarded. If no luck, you can

  • wipe the free space of the drive and check if that improves the performance,

  • try to find some mount option in linux, or some setting of the SSD, that will improve the performance.

Tools

  • zerofree is an efficient tool for ext file systems. See this link,

    manpages.ubuntu.com/manpages/zesty/man8/zerofree.8.html

  • Otherwise (if you check and doublecheck, that everything is correct) you can use dd to create a huge file blank containing zeros and wipe it afterwards (slow but works in all file systems).

    Boot from another drive, for example a live USB drive

    cd <mountpoint-of-the-target-partition>  # for example: cd /mnt
    sudo dd if=/dev/zero of=blank bs=4096
    

    Let it run until it stops because the drive is full and after that remove the file blank. This assumes that there is no useful file with the name blank already.

    sudo rm blank
    

    Warning: dd is a powerful but dangerous tool, so check and double-check that you will not destroy valuable data. To play safe in this case is to backup everything valuable on the connected drives to another location (for example to an external drive that is disconnected afterwards).

Wipe the whole device

My method for USB pendrives is to wipe the whole device (not only the free space between files in a partly filled partition). I think this is the most efficient method for pendrives, but I think there should be some discard setting, that works efficiently in an SSD without wiping the whole device. Anyway, if you want to test, you can try it, and after that create a new partition table and an ext4 partition. See this link

help.ubuntu.com/community/mkusb/wipe#Wipe_the_whole_device

sudodus
  • 46,324
  • 5
  • 88
  • 152
1

I wanted to add one comment in response to the excellent post by @sudodus, but I'm still currently 4 rep points shy of being allowed to comment, so sticking it in here:

As for leaving things idle - with the eMMC technology I worked on (again, should be similar but not identical to yours), this would not have worked, as the device had to assume that the system designers might cut the core power and only leave the IO power alive.

There is actually a sleep mode to coordinate power savings, but I do remember the eMMC vendors saying they didn't want to just start doing background operations in the middle of no-time, they only run these cleanup type operations in between accepting a command you send it, and before returning the status (result).

It's possible that SSD's are different, having a controller that specifically, during idle time, sends commands to the NAND flash back end that leads to cleanup and re-org of the data blocks inside it. But I wouldn't necessarily expect that to happen, based on my experience.

Starman
  • 298
  • I see. So we can expect that these cleanup type operations will add time to the commands that you send to the memory unit. – sudodus Jun 02 '17 at 19:15
  • Yes, that was one of the reasons we had to do so much work on it. As the complexity of the background operations developed, you'd find that occasionally, some small data-access command you sent would take an order of magnitude longer to complete than you expected. We had to push back on the vendor a bit, but also push software to be able to accomodate occasional instances of this, within limits.
    (P.S. Yay! I can comment now. Thx)
    – Starman Jun 02 '17 at 20:24
  • You are welcome, and thank you for sharing valuable knowledge :-) – sudodus Jun 02 '17 at 20:35