2

For scientific computing, we need fast read/write speeds. So I upgraded one of our machines with 2 SSDs. I used the command

hdparm -Tt /dev/sda

to test the read speed. As reference, our standard HDDs have a read speed of 250MB/s. Each SSD separately has a tested read speed of 2000MB/s (roughly expected).

However, when I connect the two SSDs to one logical volume (following the top-voted answer here: How to set up multiple hard drives as one volume?), then the read speed of the logical volume is only 200MB/s – much slower than the individual SSDs and even slower than the HDDs…

Is this normal? My understanding of LVM is very basic. I thought files are written to both SSDs simultaneously and therefore the read speed of the combined logical volume might even be higher than that of one single SSD. I also expect some overhead of the LVM, so I do not expect the read speed to be twice that of a single SSD. But only 10% the speed of a single SSD?

I guess my question is: is this normal or did I do something wrong? What is the expected read speed for the logical volume and how can I achieve it? As solution, we could just use the two SSDs as individual volumes, but it would be slightly less convenient. And for the expected I/O speed, I do not expect an exact numerical value, but rather if it is smaller, similar or greater than that of a single SSD.

The two SSDs are

Samsung 980 1TB PCIe Gen 3.0 x 4 NVMe M.2

and they are connected via

Ableconn PEXM2-130 Dual PCIe NVMe M.2 SSDs Carrier Adapter.

EDIT/SOLVED

I am still not sure what the original problem was, but when I followed this instruction, everything works well: https://www.theurbanpenguin.com/striped-lvm-volumes/ Now, my logical striped volume of two SSDs has a read speed that is slightly above the read speed of a single SSD (~2500MB/s compared to 2000MB/s single). I hope my experience helps other people as well.

Tilman Hartwig
  • 21
  • Can you test the two SSDs *at the same time*, instead of each one separately? Can it be that the controller is the bottleneck? – FedKad Mar 01 '22 at 08:22
  • Thank you very much for this suggestion, @FedKad. I have tried it, but the controller does not seem to be the bottleneck: When I test the read speed for both SSDs simultaneously, they both can read with about 2000MB/s. – Tilman Hartwig Mar 01 '22 at 08:59
  • Also, can you test with a newer version of Ubuntu like 20.04 or 21.10 (live session)? Please give details on how you use LVM (mirroring, striping, etc.). – FedKad Mar 01 '22 at 09:07
  • I am afraid it is difficult to test these SSDs in a live session: the machine is a working node of a cluster and can not be rebooted atm. I can test it during the next cluster maintenance. Mirroring/striping: I am new to LVM and simply followed the guide linked above. So I guess by default I did not set up mirroring/striping. However, I will look into "striping" which might be useful for our purpose. Thanks! Based on your questions, I guess you also find the small read speed of the SSDs suspicious, right? – Tilman Hartwig Mar 01 '22 at 10:47

0 Answers0