7

I've been using the characters ":" and "|" as part of file names without any problems in Ubuntu 18.04 to create files in a NTFS external drive. Last week I installed Ubuntu 22.04 and can't write files whose names contain those characters to my NTFS drive. The returned error is "Invalid argument". I've done some research but couldn't find anything relevant for this apparent huge change. So my questions are: does anyone know about this? Is a permanent change? Is just a bug? Should I give up using NTFS? Thanks for any advice.

Raffa
  • 32,237
user263657
  • 327
  • 1
  • 3
  • 10
  • SEE https://askubuntu.com/questions/1150258/can-you-copy-files-with-invalid-filenames-to-ntfs-and-change-the-name-automatica and https://docs.microsoft.com/en-us/windows/win32/fileio/naming-a-file – Rinzwind Jun 10 '22 at 12:43
  • NTFS is used for interoperability with other systems like Windows or Android computers. If this is not case for you, you should consider using ext4. – FedKad Jun 10 '22 at 13:13
  • 1
    "Should I give up using NTFS?" It's your choice, but, personally, I'd rather give up on using : and | in file names. Interoperability is challenging enough already, no need to artificially make it harder. – Heinzi Jun 10 '22 at 21:08
  • Note that a Unicode/full-width colon, like the kind you get when using a Japanese IME, will work even in Windows. Same with question marks; ? is forbidden, but the full-width version is not. – JAB Jun 10 '22 at 22:47
  • @FedKad I believe Android does not detect NTFS, at all. None of my smartphones with OTG till date supported, while ExFAT & FAT32 are readily plug-&-play on both flash/SSDs & HDDs. I am unaware about computers running Android / Chrome OS that might be supporting it though. – Shubham Deshmukh Jul 13 '22 at 09:59
  • @ShubhamDeshmukh I don't know about OTG on smart phones and tablets, but all Android TV boxes and smart TVs can easily read NTFS external (USB) disks. – FedKad Jul 13 '22 at 12:18

2 Answers2

8

You are right, there is a change ... as of kernel 5.15 a new NTFS driver(module) is integrated into the kernel ... It's called NTFS3 which replaces(obsoletes) the old NTFS kernel driver(module) itself and the NTFS-3G(NTFS driver for FUSE).

The old drivers were lenient and allowed filenames that were not allowed by Windows, either because they contain some not allowed character (which are the nine characters " * / : < > ? \ | and those whose code is less than 0x20) or because the last character is a space or a dot ... while the new driver doesn't.

Original Kernel level support for NTFS was contributed into a Linux Kernel back in 2001 and is very limited in functionality, especially write-support. Till now, NTFS-3G was the de facto way to enable NTFS in Linux. However, NTFS-3G is a filesystem in userspace (FUSE). The main drawback of this implementation is performance. On the contrary, NTFS3 is a kernel NTFS implementation, which offers much faster performance than FUSE based implementations.

Raffa
  • 32,237
  • 1
    Thanks. This should have come up in the news. Jesus. Now I've deleted the whole folder from 5.15 and many files remained in the directory with ?????? in the stats, I'm now unable to delete them from 5.15 neither from 5.4. Amazing. Will see how to solve it. – user263657 Jun 10 '22 at 14:10
  • 1
    @user263657 : You can create a virtual machine with Ubuntu 18.04 and mount your drive there. Then, while in Ubuntu 18.04, you can write scripts to rename all your files and directories to names not using these special characters. Or, as another alternative, you can create a new ext4 file system and move your files there. AFAIK ext4 file systems will not allow *only* the character / in names. – FedKad Jun 10 '22 at 14:25
  • @FedKad, yes, I tried that. First I deleted the folder in the external drive with rm -r from Ubuntu 22.04. No errors (I assumed it was OK). Then I renamed all the files in the internal drive changing colon by '-'. Then I copied the renamed files to the external drive. When I checked again, many old files have remained. Tried again but deletion returned error. Tried from Ubuntu 18.04, kernel 5.4, same error (input/output error). So this NTFS3 upgrade allowed me to do something to mess things up (namely, delete files with ":" in their name which weren't going to be properly deleted by the upgr). – user263657 Jun 10 '22 at 14:41
  • I don't care about the files themselves as I have a copy, but now have to see how to fix those inconsistencies in the drive created by the deletion. – user263657 Jun 10 '22 at 14:46
  • Or, perhaps, those files (unabled to be deleted) were really damaged and I haven't noticed until now. So NTFS3 wouldn't be to blame. Because only about 1% of the files weren't deleted (1200 out of 108000). So that's probably the reason. – user263657 Jun 10 '22 at 14:56
  • 2
    @user263657 I haven’t tested it, but unloading the NTFS3 module like so sudo modprobe -r ntfs3 then installing NTFS-3G like so sudo apt install ntfs-3g should bring back the old driver and the old times leniency under 22.04 :) – Raffa Jun 10 '22 at 19:08
0

The characters ':' and '|' are officially forbidden by Microsoft in NTFS filenames. See for example https://docs.microsoft.com/en-us/windows/win32/fileio/naming-a-file#naming-conventions . Linux was able to bypass that restriction and create filenames containing those characters, but this could rightfully be considered a bug and may cause serious problems for Windows systems accessing that filesystem. This bug has been fixed and the Linux NTFS driver now respects the NTFS restrictions.

Tilman
  • 3,599
  • 21
  • 27
  • There are no NTFS filename restrictions, only Windows filename restriction. Linux, or more exactly ntfs-3g, used NTFS in POSIX namespace. So not a bug as it is according to NTFS specs. The new driver only chose to use Win32 namespace. – Futal Jan 26 '24 at 11:20
  • IMHO the distinction between NTFS restrictions and Windows restrictions is invalid. NTFS is a filesystem designed by Microsoft for Windows. Using it in a way that does not conform to Windows rules is outside its specification and will have undefined results. – Tilman Jan 26 '24 at 11:45