126

I don't want any answer saying that there's no need, please!

Fabby
  • 34,259
  • 6
    Yes, finally, a Q&A that actually targets "HOW" and not "why not". Lot of people have to operate on disks near to max capacity without an ability to upgrade the capacity. – jave.web Feb 09 '21 at 11:05
  • Indeed. I feel like I've died and gone to heaven not reading a bunch of responses by people who think they know your system better than you do. – Sridhar Sarnobat Dec 25 '23 at 00:01

6 Answers6

132

Use e4defrag to defrag your files

If your ext4 file system is created with the extent option (it's default in recent distros), you can use the e4defrag utility to check and defragment it online i.e. without umounting.

Just check fragmentation level with something like this (you need to be root to see details):

sudo e4defrag -c /path/to/myfiles

Here's an example of the output you can get:

$ sudo e4defrag -c iso/
<Fragmented files>                             now/best       size/ext
1. /home/gerlos/iso/debian-live-7.5.0-i386-rescue.iso
                                                 7/1         111177 KB
2. /home/gerlos/iso/systemrescuecd-x86-4.4.1.iso
                                                 4/1         100897 KB
3. /home/gerlos/iso/debian-live-7.5.0-amd64-rescue.iso
                                                 6/1         116053 KB
4. /home/gerlos/iso/ubuntu-14.04.2-server-amd64.iso
                                                 8/1          76160 KB
5. /home/gerlos/iso/ubuntu-14.10-desktop-amd64.iso
                                                15/1          75712 KB

 Total/best extents             40/5
 Average size per extent            90577 KB
 Fragmentation score                0
 [0-30 no problem: 31-55 a little bit fragmented: 56- needs defrag]
 This directory (iso/) does not need defragmentation.
 Done.

As in this example, most of the time it will tell you that no defragmentation is needed, but if you want to do it anyway you can use (you don't need to use sudo to defrag your own files):

e4defrag /path/to/myfiles

Your users can even run it on their own files, there's no need to be root unless you want to work on other user's or system files.

e4defrag is in the e2fsprogs package, and I guess it's already installed on your Ubuntu system.

gerlos
  • 2,774
  • 2
    e4defrag -c returns nothing but done. Does that just mean there is no fragmentation? – hrzhu May 11 '15 at 08:44
  • 8
    Sorry, I forgot to mention that you need to run e4defrag -c with sudo or as root to get detailed information on your files fragmentation. But don't need to be root to defrag them. – gerlos May 11 '15 at 16:00
  • @gerlos What if I try to defrag the root file system /? – Pavel Sayekat Nov 13 '16 at 09:34
  • 3
    @PavelSayekat you just need to be root (i.e. use sudo) to run e4defrag /, and don't need to do anything special. e4defrag will check every file there, so be ready to wait for your drive to read everything. Remember that it can be a big waste of time. Most of the times, if you have enough free space in file system, you will have only a few fragmented files, and often they will be in your home dir, related to your web browser activity. For example, on my desktop PC that I use every day e4defrag reports that there are only 5 fragmented files, all in Chromium config and cache dir. – gerlos Nov 15 '16 at 10:56
  • 5
    This defrags only the extents insides the files themselves, but not the files among each other. That is, if I have a directory containing three files at wildly different physical locations on the disk, it will still be slow to use these files simultaneously after running this tool. Is there any way to rearrange the files on the disk, or "defragment the free space" as it is sometimes called? – fuzzyTew Dec 18 '16 at 18:47
  • @fuzzyTew It's true, but it's not a problem in most cases (for sure not with SSDs). We already use preload to speed up os boot, so if we have enough free space (say 20%), disk fragmentation won't affect boot time. Anyways, if we really want a set of files to be written close to each others, we can just move them elsewhere and copy them back. They will be written close to each others. But, anyways, how can the system know what data files we want to open at a time? Being in the same dir isn't enough, cause we can use hard links to put the same file in 2 or 3 different dirs. – gerlos Dec 20 '16 at 15:12
  • @gerlos If the defrag tool supported defragging directories of files together, I could tell it what files I will be accessing simultaneously by passing them to the tool. I like the idea of moving them off the drive and copying them back on, but I don't think ext* tends to write files near each other by default; are you sure of this? I thought it spread them to allow for growth without fragmentation. Similarly if there are no large contiguous blocks of free space on the drive I would have to move all files off and back on which would take an eon. – fuzzyTew Dec 20 '16 at 16:58
  • @fuzzyTew You're right about ext* and the fact that files are usually spread around so they can grow. Actually, moving file off and back on is the best approximation possible at the moment of "put them near each other". Anyways, in my experience, as long you leave enough free space, you can happily ignore fragmentation problems: in a virtualization server I run for a dozen of VMs with dynamically allocated storage, e4drfrag tells me that there's no need to defrag anything, even after 4 years of use. So, from my point of view, it's fine to just don't care about fragmentation. – gerlos Dec 07 '17 at 10:24
  • I notice that it only shows 5 files quickly, but when you run the regular command it chugs over all the files in the system. Looks like it inspected a lot more than 5. – Gringo Suave May 09 '19 at 19:18
  • I'm running a RAID on my Synology with ext4. Any reason this wouldn't work that my type of set up? Synology will only defrag Btrfs so I'm not sure if these commands are safe... – Razor Apr 04 '20 at 23:44
  • 1
    To clarify the obvious, file layout does matter a lot when you are running processes bound by seek time, which is why people reach this article. For example when interleaving data between files by e.g. muxing audio and video streams together, on rotating media the speed will be affected by how distant the files are on disk. Defragging is for rotating media, which is still in common use. – fuzzyTew Dec 11 '20 at 13:35
  • Why does the tool show 1 as best fragmentation scenario for iso files? A maximum extent size is 128 MiB. There's no way for such big files to use only one extent. – pmartycz Dec 20 '20 at 18:06
  • @fuzzyTew, look at my answer below https://askubuntu.com/a/1171593/14413 as it describes e2fsck -D – rjt Apr 17 '22 at 19:24
  • i wish there was some kind of verbose progress indicator rather than what looks like it's just hanging. – Sridhar Sarnobat Dec 24 '23 at 23:59
33

Take a look on e2fsprogs. This is also available in the Ubuntu packages. It provides the program e4defrag.

TIIUNDER
  • 1,203
  • 5
    This is really only useful on ext4 file systems that were created with -O extent, which is not the default. – dobey Nov 23 '12 at 20:43
  • The guy answes about defrag, it could be posible with e4defrag, the cuestion not specity an external journaling or not, he don't deserve a -1. – Felipe Alcacibar Jan 08 '13 at 23:25
  • 8
    @dobey Ubuntu 12.04(.3) LTS contains extent in the ext4 stanza of the [fs_types] section in /etc/mke2fs.conf - thus, it is a "default". Though, regardless of the create-time configuration of the machine to which the filesystem is connected, you're best off to inspect the filesystem itself: tune2fs -l </dev/with/ext4> | grep extent (or look at the "Filesystem features" line in the complete tune2fs output). Mind you, I just ran e4defrag -c ... on a 100% full 1TB filesystem with extent, and there are only 5 fragmented files, and "fragmentation score" 0. YMMV. – Richard Michael Dec 28 '13 at 00:30
  • 1
    @RichardMichael FYI: e2defrag -c /path only prints the first 5 fragmented files found (not even the most fragmented ones). Run e2defrag -v /path to see how it reduces extent counts. – Tino Jul 10 '16 at 22:44
21

Use Gparted to defrag your file system

If you have enough space on your hard drive, you can use Gparted to defrag your file system (ext2, ext 4, nfts, etc.). You have to boot from a CD/DVD/USB boot disc because the drive you're working on has to be unmounted. You also have to have more unused space available than used space for this to work and it may take a while.

  • Boot from a boot disc.
  • Run gparted and shrink the partition that contains the data you want to defrag to just over the amount of your data.
  • Make sure the partition you want to defrag is the last partition on the drive by moving it to the end (you may have to create another blank partition in front of it if there's only one partition on the drive).

With the partition you want to defrag as the last partition on the drive:

  • grow the partition to the left of it to maximum size. This will move your partitions data to the end of the drive.
  • Once it's done, shrink that partition back to it's previous size.
  • If you created a blank partition to have more than one on the disc, you can now delete it.
  • Move your partitions back into the original order and regrow the partition you wanted defragged back to it's full size.

It's now defragged.

And I know you're not interested in why/why not to defrag ubuntu, but I'll post the link to why-is-defragmentation-unnecessary anyway.

Community
  • 1
James
  • 17,106
  • 5
  • 24
  • 38
  • 1
    After shrinking and moving right, why do you grow left then shrink again instead of just moving left? – Jason C Oct 22 '13 at 17:47
  • Also I noticed that after the grow-left step, the data itself was actually moved left again (evidenced by gparted moving data, verified by a direct examination of the block device afterwards), so unless that is version/filesystem-dependent behavior the final shrink + regrow can be skipped. It appears to be sufficient to only do: 1) shrink and move right, 2) apply, 3) grow left. – Jason C Oct 22 '13 at 18:59
  • Simple moving the partition you want to defrag to last does not necessarily move it to the actual end of the HD. Growing the partition to the left to maximum size makes sure that the end partition is actually "moved" to the very end of the hard drive. – James Oct 22 '13 at 22:04
  • Moving it to the right does move it to the end of the hard drive. The partition bar at the top of the gparted GUI is actually an accurate representation of the data layout (well, at least as visible to the OS, flash devices for example map logical to physical locations on the other side of the controller, but for a typical HDD, it is accurate). This can be verified by observing the new start location in the partition table. :) – Jason C Oct 22 '13 at 23:03
  • Ah, we are miscommunicating. You meant reorder the partition so it is last. I meant move the partition to the end of the drive, and that's what I thought you meant as well (you can move partitions around on the drive with gparted independently of reordering them). :) – Jason C Oct 22 '13 at 23:05
  • There is a great reasons to defrag your files regularly: it is much easier to recover contiguous, non-fragmented files in case of a file system failure. – Ivan May 18 '15 at 23:21
  • Any tips on how this interacts with logical partitions? For example, Ubuntu creates a logical partition by default with actual swap and ext4 partitions inside it. What should I do if the actual partition I want to defrag is inside a logical partition? – Shnatsel Oct 08 '15 at 00:43
  • Why do you need to the very end of HDD rather than just enough it does not overlap with its previous location? – abukaj Nov 16 '22 at 21:09
  • Followed your procedure. According to e2defrag output, the number of extents actually increased. Seems that it does not defrag files. It defrags free space. – abukaj Nov 16 '22 at 22:17
  • Moreover, its efficiency depends on the filesystem settings. I have filesystem created with mke2fs -O extent,flex_bg -G 4096 which is 49% full, but gparted can shrink its size to 99% only - thus fragmentation of free space remains almost the same. – abukaj Nov 18 '22 at 09:27
  • I think you should do shrink + grow to defrag the free space and then do e2defrag for the whole filesystem to defrag the files as much as possible. – Mikko Rantalainen Feb 06 '24 at 15:56
8

e2fsck -D /dev/sda1
e2fsck -C 0 -c -c -D /dev/sda1

-D optimizes Directories by sorting and reindexing them which usually speeds things up. Might not be considered a full filesystem defrag, but not all systems have e4defrag.

Sorting directories so that directories and files are in alphabetical order speeds reading up because files are read together physically as well as logically.

I include the following options when running e2fsck. -C 0 outputs to stdout -c -c non-destructive rw test using badblocks

rjt
  • 426
7

Defragmenting ext4 file system in linux I am adding this answer to also show how we can find about various built-in Linux tools to do some commonly done operations. you

Also, note that you need to do as super-user only if you want to defragment without unmounting.

Using apropos command

e4defrag -vc /path #verbose output with count
Sudip Bhandari
  • 271
3

Ubuntu 14.04 LTS does place (at least the primary drive) into EXT4 partition when installed on a clean install (not residing on top of Win-Junk).

In this case, if you have just set up a new system, or formatted fresh on an old system and then did a clean install, then by default with 14.04 LTS, you have an EXT4 Partition at /dev/sda1.

Now, just open a Terminal (CTRL-ALT-T) and type:

sudo e4defrag -c /dev/sda1

and be patient until the 'check' is completed and you get the fragmentation report back.

If you aren't confident about having an EXT4 Partition on your OS drive, simply go HOME and type df -T. Look to see what your sda* partitions are set to (try this in 14.04LTS, and you'll see EXT4 unless you chose another partition type upon harddrive setup).

lance
  • 39