14

Feel free to edit the title to explain better what I am going to write here.

When I copy big files to a pen drive for example, the progress window shows an estimate that most of the time does not fail to show the real time and percent to finish, but there are cases where it says everything is finished and the progress window closes. I go to extract the pen drive and it says it is still in use. After checking the pen drive I see it is still copying the files but there is no progress window showing this.

It does not only happen with big files, it also happens with many small files. If I copy them, the progress bar might say 15 seconds for example and finish in that time, but the real time might be 1 minute and for the next 45 seconds I need to actually look at the light in the pen drive to see if there is real activity on it.

I do not want to know how to fix it since I have read how deep a fix for this could go. What I want to know is why does then the progress window show an estimate that does not correspond with the process of copying.

Is it dependent of the Cache in the external unit?

Is the file size and amount of file influence on the correct estimation. For example 1 file of 4 GB or 1000 files of 4 MB.

Is there configuration options that can change the behavior.

There are other questions similar to this like copying files to usb stick never finished but am more focus on the mechanics into why it would behave like this.

Community
  • 1
Luis Alvarado
  • 211,503

2 Answers2

6

I would suppose you are using Nautilus as your file manager and if so there are long-standing bugs about this. Too numinous to mention effecting Mint, Fedora, Red Hat and all the like. Ubuntu is not without this same issue.

Some suggest turning off thumbnail view helps. Others put their hopes in the "newest kernel" but this still exists.

The problem = Starts off fast, then goes slower This is because when mounted with async it will write to the cache, when the cache is full you see the "Real" write speed.

The work around seems to be sudo cp /filetobecopied /dev/nameofdevice

another posted here says that "copying in chunks" works. Unconfirmed on my part.

Ringtail
  • 16,127
  • 3
    To clarify, the kernel caches the data in RAM instead of writing it directly (for performance reasons). When you click Eject, a sync command is performed in the background, which flushes the cache. For large amounts of data, this can take a while. – thirtythreeforty Jan 07 '13 at 03:46
  • 1
    TIP: sudo cp /filetobecopied /dev/nameofdevice will replace the entire filesystem with your file, usually not what you want.... – Mr. Developerdude Oct 02 '16 at 18:55
2

This is also a nice answer with a solution: https://unix.stackexchange.com/a/181236 It says:

The reason it happens that way is that the program says "write this data" and the linux kernel copies it into a memory buffer that is queued to go to disk, and then says "ok, done". So the program thinks it has copied everything. Then the program closes the file, but suddenly the kernel makes it wait while that buffer is pushed out to disk.

So, unfortunately the program can't tell you how long it will take to flush the buffer because it doesn't know.

If you want to try some power-user tricks, you can reduce the size of the buffer that Linux uses by setting /proc/sys/vm/dirty_bytes to something like 15728640 (15 MB). This means the application can't get more than 15MB ahead of its actual progress.

A side effect is that your computer might have lower data-writing throughput with this setting, but on the whole, I find it helpful to see that a program is running a long time while it writes lots of data vs. the confusion of having a program appear to be done with its job but the system lagging badly as the kernel does the actual work. Setting dirty_bytes to a reasonably small value can also help prevent your system from becoming unresponsive when you're low on free memory and run a program that suddenly writes lots of data.

But, don't set it too small! I use 15MB as a rough estimate that the kernel can flush the buffer to a normal hard drive in 1/4 of a second or less. It keeps my system from feeling "laggy".

Community
  • 1
Cirelli94
  • 616