1

I'm moving a single large file from /mnt/z to /mnt/y on a bare metal Ubuntu 20.04.3 install.

Both disks are capable of sustaining 2 GB/s in writes, and do in other operations, but mv caps out at ~200 MiB/s.

The disks are PCIe 4.0 Highpoint RAID cards with either U2 or M2 drives in RAID 0. Both with NTFS partitions.

What could I be missing?

BeastOfCaerbannog
  • 14,585
Chris Han
  • 11
  • 1
    Are these USB 2 or 3.1 or 3.2 drives or external SATA drives? If these are SATA drives in USB cages, is UASP working? Please edit your question and add the new information. – user68186 Sep 14 '21 at 21:34
  • 1
    Neither. They're in the machine, directly attached via PCIe lanes. – Chris Han Sep 14 '21 at 23:43
  • 2
    Perhaps because the file system format is ntfs? Would the performance not be better when using a native linux file system like ext4, or perhaps with vfat? Is it also so slow when copying many smaller files? – vanadium Sep 15 '21 at 09:32
  • 1
    +1 to @vanadium's comment. My experience is the NTFS is much slower than ext4 when accessed from Linux. – sudodus Sep 15 '21 at 10:10
  • @vanadium 99% sure that is the issue -- Move them using windows otherwise NTFS is not the best filesystem for this; You can use ext3 for 100% OS support. ext4 might need a 3rd party driver on some other OSes – Rinzwind Sep 15 '21 at 10:52
  • Search this site for "slow copy". e.g. see https://askubuntu.com/questions/1113235/very-slow-copying-files-from-exfat-to-ntfs In a nutshell, the copy process hands off the output to the system (buffers), system buffers fill up, things drag to a crawl. Happens any time the output is slower than the input, and the copy is big enough to fill the systembuffers (memory dependent). – ubfan1 Sep 15 '21 at 15:17
  • It's interesting that writing to either NTFS disk from memory goes at full speed. I can also copy files to different disks simultaneously and the total read from the NTFS disk rises, but each individual mv operation still caps out around 200MiB/s. – Chris Han Sep 15 '21 at 15:19
  • @ubfan1, would this be relevant with a 100GB file and 500GB RAM? Both the read and write are happening at 1/10th the bandwidth. – Chris Han Sep 15 '21 at 15:21
  • Please add additional info to your posting instead of in comments. No experience with 500GB memory, but I see slowdowns when my copy size exceeds my memory. Bad news is the SMR hardware that is just slow -- honest vendors state "not suitable for backups". – ubfan1 Sep 15 '21 at 15:28

0 Answers0