Does Ubuntu damage USB drives?

Question

Whenever I unmount a USB drive with Nautilus and plug it to a Windows machine, I receive a warning that the drive needs to be repaired.

(the same message like in this screenshot)

For 10+ years working with windows I never had a corrupted or damaged USB drive but during the last two years three of my USB drives became inoperative, so I cannot prove it, but it's obvious that this is related to Ubuntu's (un)mount behavior.

A friend told me I can prevent such damage using udisks and sync, but I hope this is not the way to do it, mounting drives with shell commands in 2016.

I would try two tests to figure out the root-cause: 1) Can you prevent the error from appearing if you call sync in a shell before unmounting in Ubuntu? (unlikely) 2) Try writing only one file at a time, unmount, then check if this file can be read on windows and its contents are up-to-date, despite seeing this warning. 2) means the issue is most likely with Windows being fussy saying "we found errors" about some non real issue. — arielf, Apr 01 '16 at 20:29
Don't believe people when they tell you Linux won't corrupt your drives. It can. On Ubuntu 15, mere file copy followed by reboot consistently gave me errors in chkdsk like this: Stage 2: Examining file name linkage ... Found corrupt basic file structure for "<0x32,0x1e63>" ... queued for offline repair. Found an unneeded link ($FILE_NAME: ???) in index "$I30" of directory "\ <0x5,0x5>" ... queued for offline repair. Found missing Index entry for file "<0x32,0x1e63>" from index "\??\D:\found.000" of directory "$I30" ... queued for offline repair. — user541686, Apr 02 '16 at 06:49
^...not to mention, this wasn't even on a flash drive, it was on my main SSD. A brand-new SSD. And it happened every single time that I didn't add a ton of extra I/O after the file copy. In other words, Linux's implementation of NTFS is broken, as much as its fans will want to deny it and avoid believing. — user541686, Apr 02 '16 at 06:52
Is this a NTFS filesystem? FAT32? Can you reproduce it reliably? — Braiam, Apr 03 '16 at 14:23
What's so bad with mounting drives with shell commands ? ¯＼(o_o)/¯ — ulidtko, Apr 04 '16 at 12:25
I have the same behavior the other way - I must 'repair' USBs under Linux, because Windows gets them broken. — MatthewRock, Apr 04 '16 at 13:24
Are you unmounting the drive, or just pulling it out? If you're properly unmounting it, then all pending writes should be performed and the disk buffers flushed. If you're just pulling the drive out, then I'm not surprised you've got file system corruption. — TMN, Apr 05 '16 at 12:58
I've noticed that if you wait a few (say, 15) seconds after you eject/unmount the USB & then remove it, you don't have this trouble. It seems that even though it shows immediately after unmount that it's "ejected", it actually is still busy? — Tracy LF, Apr 04 '16 at 06:37
No no no ! this is a windows scam project started with windows 7 servicepack 2 put in production with windows 8: It works like this.. Anything accessing a forign ( not exclusively Windows Owned drive - may it be a USB or iscsi Drive (HD, stick memory card) that have been accessed by something not OWNED BY Microsoft Will trigger all KINDS of SCARY WARNINGS and ultimately tell you to format the drive ( before reading / Copying of course)!! Next time try it on a Unix, Machintosh or other Linux distro ( same result - No Complaints What so ever!!) Beware of Microsuft Agressive Marketing. !!! :-((( — Ken Mollerup, Apr 06 '16 at 12:03
@Mehrdad, linux NTFS implementations have very long been experimental when it was about writing. The main trouble with NTFS is its closed spec aspect, meaning Microsoft does not give much information on how to implement NTFS. This is not Linux fault but Microsoft choice. About current NTFS implementations in Linux, there is many different ones. One or many of them are maybe broken about writing, or just obsolete in your current system. Look here for a list under Debian. I guess there is equivalent choices elsewhere for Ubuntu. — Frédéric, Apr 06 '16 at 12:23
@KenMollerup: Tell us why anyone should believe what you're saying when you have provided zero links or technical details. — user541686, Apr 06 '16 at 18:01
@Frédéric: It's Linux fault and Microsoft choice. You can't corrupt user data and then blame it on someone else making their software closed-source. Furthermore, I believe Microsoft does share details of NTFS with its "partners" (those who make partition managers etc.), they just don't do it for free, but for a commercial company like Canonical, it doesn't seem out of the question to try that route. Finally, how do you even know the incompleteness of the current reverse-engineered spec of NTFS is the source of the problem? To me it seems like a completely unrelated bug. — user541686, Apr 06 '16 at 18:06
@Mehrdad I Have 2 Usb sticks that after Windows tried to use them for backup, while I was trying to put my Mail back together, show 0 bytes awailable out of 0! Nothing left out of 16GB Kingston DataTraveler, and a NoBrand 8GB. Nothing can make them come back to life! — Ken Mollerup, Apr 07 '16 at 06:14
related: https://askubuntu.com/questions/955565/does-persistence-kills-usb-sticks — JinSnow, Aug 27 '20 at 15:40

Takkat · Answer 1 · 2016-04-04T13:26:32.653

105

No worries Ubuntu did not damage your USB drive. But we do not use poorly documented bit flags of a FAT32, FAT16, or NTFS filesystem. On Windows these flags indicate a possibly corrupted filesystem when we had not properly unmounted the drive or an I/O error had occured.

Those bits are located in a reserved entry of a FAT partition table. According to an internal Microsoft paper of 2004 the purpose of these bits are:

ClnShutBitMask:
If bit is 1, the volume is “clean”. The volume can be mounted for access. If bit is 0, the volume is “dirty” indicating that a FAT file system driver was unable to dismount the volume properly (during a prior mount operation). The volume contents should be scanned for any damage to file system metadata.

HrdErrBitMask:
If this bit is 1, no disk read/write errors were encountered. If this bit is 0, the file system driver implementation encountered a disk I/O error on the volume the last time it was mounted, which is an indicator that some sectors may have gone bad. The volume contents should be scanned with a disk repair utility that does surface analysis on it looking for new bad sectors.

There was some discussion with the kernel file system developers some years ago on how to overcome this but I was unable to follow up the results. Apparently it did not make it into recent kernels.

As only a bit flag is involved but generally our data should be in perfect shape we can ignore the warning Windows issues on those drives we had previously used in Ubuntu.

edited Apr 04 '16 at 13:26

answered Apr 01 '16 at 14:38

Takkat

142,284

3

I've never had these errors in these drives when I actually safely unmount the drives - are you sure support for these bits is missing? – Thomas Ward Apr 01 '16 at 14:47
5

@ThomasW. These bits are quite obscure but I occasionally have these errors with my Ubuntu-formatted USB drives on my workplace Windows 7. So far I haven't seen any helpful pattern. Only thing I can say is that I always properly unmount/eject the drives. Testing is not easy because at home no Windows, at work no Ubuntu. – Takkat Apr 01 '16 at 15:49
The headline is false. – MatthewRock Apr 04 '16 at 13:25
@MatthewRock: better? – Takkat Apr 04 '16 at 13:27
Yes, it is better. – MatthewRock Apr 04 '16 at 13:49
This seems to be the more helpful answer. – xji Apr 05 '16 at 08:10
Whenever I had problems I just check and fix the drive with GParted - often it says 'Removing dirty bit' which I suppose refers to the above. These problems I usually had with silly windows machines refusing to unmount, or newer USB drives where you have to use 'Safely remove drive' instead. – Wilf Apr 07 '16 at 02:52

score 73 · Answer 2 · edited Apr 07 '16 at 07:07

73

This is mostly an issue with Windows. It thinks it is the only OS in the world and acts out if it detects something it does not understand.

Just because Windows says you must repair the drive does not make it true.

Any of my drives used with Ubuntu get this message from Windows, I just say no and they work fine with Windows.

In short there is nothing wrong with the drive, it is just there is something on there that Windows does not understand and its response is destroy it.

Don't repair the drive, it will format it without telling you and you will lose all the data on the drive.

edited Apr 07 '16 at 07:07

guntbert

13,134

answered Apr 01 '16 at 09:59

Mark Kirby

18,529
19
78
114

41
Repair does not format, repair saves allocated data to a folder. For me it seems as Ubuntu does not finish write process properly.

jan6352781

Apr 01 '16 at 10:19

10

Most USB drives are FAT32 formatted, which was developed by Microsoft, so why should Windows be the only system to "not understand" somethin??

– jan6352781 Apr 01 '16 at 10:22

score 19 · Answer 3 · edited Apr 13 '17 at 12:23

As you suggested in a comment, this is possibly related to Ubuntu not completing the write process before you remove the flash drive. Ubuntu writes the files to RAM during the copying process, and writes these files from the buffer to the USB drive in the background after the copying dialog is closed. There's another answer that suggests this is especially a problem for machines with large amounts of memory, but it's likely that you're removing the drive before the background write is complete. When you eject a drive, this forces the buffer to be written to disk, but it is difficult to tell when the write is actually completed. There's a pop-up in newer versions (15.10 for certain, possibly 15.04) intended to remedy this -- a purple alert appears after some (generally short) time that says something along the lines of "[Drive] can now be removed" once the write is finished.

This is probably the reason your friend suggested sync. Running sync in terminal forces all buffered data to be written to disk, and should return only once the writes are completed (See this answer). See this answer for a utility that will help you determine if this is actually the problem.

I get the popup in 14.04 (Linux Mint KDE edition) whenever I eject a USB drive, and also my 15.10 Kubuntu machines. Not sure about 15.04, but I would assume so. — Jon Bentley, Apr 01 '16 at 13:25
The GUI really doesn't make it obvious? I've been using eject in the terminal for a long time, and there's a noticeable delay before it returns if I've just copied a large file. — Izkata, Apr 01 '16 at 15:10
@Izkata On vanilla Ubuntu 14.04, when you click the eject symbol in Nautilus it disappears immediately. Unlike internal drives, though, USB drives disappear completely from the sidebar when completely unmounted (i.e. write is finished), so there's some amount of time during which the USB drive is being written but appears to be ejected. — alex_d, Apr 01 '16 at 15:22

score 7 · Answer 4 · edited Apr 13 '17 at 12:37

I've seen lots of usb drives, especially flash (SD, etc) cards in USB adapters, with FAT filesystems (virtually every USB drive comes pre-formatted with FAT32) getting corrupted while using Ubuntu & Ubuntu-derived distros. It was almost a regular occurrence every few months of regular daily use.

If it were a hardware problem, then formatting the partitions with ext3 (or 4) shouldn't help, but ext3/4 makes the filesystems virtually bulletproof. (ext2 also was corrupted over the few months I tried it; was supposed to have less writes so a longer life for flash drives, mainly from no journal).

So, Ubuntu should not physically damage USB drives themselves, but I still don't trust it 100% with FAT filesystems.

I think filesystem corruption can be best avoided by:

Never just unplug a USB drive until after it's been unmounted/umount/ejected. Even if the drive's not currently writing anything (if it has a light it's not flashing) and even if you've done a sync the filesystem may still
Wait at least a few seconds after umount/eject before unplugging it. It seems that even after umount the drive's activity light may sometimes keep flashing for a little while. Other users like this guy say it could go on for up to a minute.
Don't only rely on only sync either, like this guy who got file corruption.

Related (generally) links:

score 3 · Answer 5 · answered Apr 04 '16 at 14:07

This has little to do with any operating system. This is a file system error, which in most cases indicates that the drive was not unmounted properly (user error). It is not an absolute indication of damage, it merely indicates a possibility of a damage. If present, the damage is usually limited to the last written file(s). Windows "Repair" command in this case scans the drive for sectors marked as "occupied", but not assigned to any file and then assigns this scraps and pieces to new files.

This error can be also created on Windows, if you just yank out the drive during write without unmounting it first (in Windows "Eject" performs the unmount).

Seeing this message means that the drive was not unmounted properly.

You can verify on your own if the file system was really damaged or is it just false positive: if after running "Repair" a new folder named like "found000" appears with some files in it, it means there really was an unfinished write that got interrupted. Some of your data were lost, and that is a concern.

score 2 · Answer 6 · edited Apr 06 '16 at 08:55

2

I use several years ubuntu and windows with dual-boot, and when i unmount the drive the correct way, in windows always show up the error message, but never was error on my usb drive. In W10 you can turn off this pop-up error message.

edited Apr 06 '16 at 08:55

HadiRj

103
4

answered Apr 05 '16 at 09:53

tviragh

21

score 1 · Answer 7 · edited Apr 02 '16 at 19:59

1

There is almost no way for any OS could damage USB drive, after a normal format (not the quick), there should be no trace left that drive was ever used with Ubuntu.

Check the drives with "H2testw" or "USB Flash Drive Tester" - bad sectors might be source of many weird errors.

edited Apr 02 '16 at 19:59

antivirtel

3,645

answered Apr 01 '16 at 13:55

user158037

75
6

4

The OP is worded badly, but it means "file system", not "flash ROM". – wizzwizz4 Apr 01 '16 at 15:48
It appears that the OP may mean actual USB drive damage, they say " I never had a corrupted or damaged USB drive but during the last two years three of my USB drives became inoperative" If they meant just corrupted they probably would have said corrupted, instead of "inoperative" @wizzwizz4 – Xen2050 Apr 02 '16 at 11:30
3

@Xen2050 Yet all of the other answers, and comments threads including jan6352781 imply, or state, the file system is the issue, and the "broken device" is an extrapolation of them not working properly. – wizzwizz4 Apr 02 '16 at 11:39
@wizzwizz4 Well, in Mark Kirby's answer, the OP comments "3. The last years I skipped the repair almost every day ending up with damaged* files and USB drives. "* I'm guessing it's just from old drives that would have failed, windows or no, but they OP still said it – Xen2050 Apr 02 '16 at 12:02
@Xen2050 Or maybe the OP was referring to the old message "The device is damaged. Do you want Windows to fix it?" (paraphrased) which sometimes comes up when a different dll handles the issue. (I've had it appear myself, although I'm not sure the last sentence is right.) – wizzwizz4 Apr 02 '16 at 12:16
I'm not saying this answer is wrong, and I'm not downvoting it (I've upvoted actually), but I was just providing insight into the interpretation of the question that everybody else, including the question poster, used. – wizzwizz4 Apr 02 '16 at 12:17
@wizzwizz4 That may be true. Unfortunately the question (like most questions ;-) isn't crystal clear. (I upvoted this one too, thought we were the only ones, but it's +4 and -2 right now) – Xen2050 Apr 02 '16 at 12:21
Oh Yes Microsoft Windows have quite a few: F.eks Wait til you start copying a large file to a (Quite Big Enough) Usb stick - then autostart the Backup Anything on the Whole HD! It will shurely destroy any Usb- device with less space than required by this aforementioned operation!!! – Ken Mollerup Apr 06 '16 at 12:13

score 1 · Answer 8 · answered Apr 06 '16 at 09:17

I can't say if the drive was "damaged", maybe it was and maybe it wasn't. But as someone who can say the same: "For 10+ years working with windows...", I can tell you that if you're running Windows 10 that could be the source of your new problems. I ran into a new problem my first day on 10: in 10 a database is created for external drives (it might be the indexing database, I don't remember). If that database doesn't match the drive it will tell you that your drive is damaged, sometimes you can ignore this warning and sometimes you can't (see anecdote). Running "repair" will fix the database.

Anecdote:

I can't remember where I found this information but I found out about it when I moved my Music folder. It complained that the drive was broken instead of just complaining about the folder it was looking for not existing anymore. Before I ran repair the new location appeared on Ubuntu and the old location on Windows 10...but all the files couldn't be opened on 10. After I ran repair the new location of the Music folder appeared in Windows, the old folder disappeared, and it worked on Windows again.

Running sync is also not a bad idea.

Does Ubuntu damage USB drives?

8 Answers8