224

The title might not be as descriptive as I would like it to be but couldn't come up with a better one.

My server's file system went into Read-only. And I don't understand why it does so and how to solve it.

I can SSH into the server and when trying to start apache2 for example I get the following :

username@srv1:~$ sudo service apache2 start
[sudo] password for username:
sudo: unable to open /var/lib/sudo/username/1: Read-only file system
 * Starting web server apache2                                                                                                                                                                                                               (30)Read-only file system: apache2: could not open error log file /var/log/apache2/error.log.
Unable to open logs
Action 'start' failed.
The Apache error log may have more information.

When I try restarting the server I get :

username@srv1:~$ sudo shutdown -r now
[sudo] password for username:
sudo: unable to open /var/lib/sudo/username/1: Read-only file system

Once I restart it manually it just start up without any warning or message saying something is wrong.

I hope somebody could point me into the right direction to resolve this issue.

Pro Backup
  • 3,210
  • 3
  • 25
  • 33
John
  • 2,413
  • 2
  • 16
  • 11
  • 1
    I recommend to @John to change the answer to the last posted answer by Bibhas as it actually works where the other answers are not helpful at all actually. – Joshua Robison Nov 09 '14 at 06:30
  • For MicroSD: http://askubuntu.com/questions/213889/microsd-card-is-set-to-read-only-state-how-can-i-write-data-on-it – Ciro Santilli OurBigBook.com Aug 20 '16 at 14:33
  • 1
    Everyone, this question is for a server, not a PC. If you have this issue on your dual boot PC, Please check The **quick start** option can be found in **power options** in the control panel of Windows. I'm having the same issue with an Ubuntu on AWS – vanduc1102 Jul 14 '18 at 08:35
  • I have this problem too, and non of solutions work fo me i use ubuntu 18.04 , and i don't know what makes this happened , I should restart my system and then it show me page that contain (initramfs) there when I run fsck /dev/sda1 -y and reboot os work and again after 30 minutes to 1 hour problem happens. – Parisa.H.R Aug 07 '21 at 03:10
  • if disk is NTFS and used by Windows - run from it shutdown /f /r /t 0, it helped for me – CodeBy Jul 08 '22 at 14:07
  • In my case, the problem was coming from Windows either it was Windows Updates or improper Shutdown. I just went back to Windows and let the Windows install its Updates and Restart. That's it. Not sure, but may be Windows kinda lock Partitions. – Abdul Jabbar Oct 31 '22 at 04:02

20 Answers20

125

The filesystem will usually go into read-only while the system is running if there is a filesystem consistency issue. This is specified in fstab as errors=remount-ro and will occur when a FS access fails or an emergency read-only remount is requested via Alt+SysRq+U. You can run:

sudo fsck -Af -M

to force a check of all filesystems. As one of the other answers states, looking at dmesg is also very helpful.

Edit: Don't forget the -M on the command-line.

NOTE: As mentioned by Bibhas in his answer: If fsck gets stuck after its version banner:

$ sudo fsck -Af -M
fsck from util-linux 2.20.1

you may want to try using the EXT4-specific fsck

$ sudo fsck.ext4 -f /dev/sda1

Provided the partition in question /dev/sda1 was an ext4 filesystem.

nanofarad
  • 20,717
  • 5
    I think you should not force a filesystem check on other r/w mounted filesystems. That will potentially corrupt your data. Add the -M option to skip mounted filesystems. (-M Do not check mounted filesystems and return an exit code of 0 for mounted filesystems. from FSCK(8)) – gertvdijk Jun 24 '13 at 18:35
  • 41
    This doesn't help, I just get the same error when trying to run that command. sudo: unable to open /var/lib/sudo/kuplack/1: Read-only file system fsck from util-linux 2.20.1 – Mark Kramer Dec 08 '13 at 21:47
  • 1
    But... -M will skip mounted file systems, including /dev/sda1 (or what ever mounted drive includes /var/lib/sudo/... ). @MarkKramer: I highly recommend always setting the root password when installing linux, with sudo su; passwd. Then you can use su -c fsck -Af -M. – naught101 Mar 01 '15 at 00:33
  • 3
    Yea for me I needed to remove -M since /dev/sda1 was mounted, and to make your life easier, add -Afy (The y means answer yes to all prompts)... I play fast and loose with VM's so I'm usually ok with this type of solution, but if this is unbacked up hardware, might take a different approach and read dmesg. – FreeSoftwareServers Jul 19 '16 at 21:57
  • running fsck on a mounted filesystem is not a good idea – Darshan Chaudhary Mar 01 '17 at 06:18
  • 3
    @DarshanChaudhary The -M flag causes fsck to skip mounted filesystems. See the fsck (8) manpage. – nanofarad Mar 01 '17 at 19:07
  • command succeed after reboot the system. – Sinan Gül Mar 07 '17 at 12:33
  • UP! For -> "The filesystem will usually go into read-only while the system is running if there is a filesystem consistency issue." – Eduardo Lucio Aug 25 '17 at 01:21
  • This answer does not work for me. After executing the suggested command I receive fsck from util-linux 2.29.2 – birdman May 24 '19 at 07:02
  • 5
    This actually did end up corrupting my filesystem :( – Blairg23 Jul 04 '19 at 01:44
  • @hexafraction many humans don't have "SysRq" keyboard buttom in their laptop keyboard, I am one of them. How do I pull this off? – Pranav Jul 19 '19 at 15:11
  • @PrabeshBhattarai SysRq is the same (physical) key as PrintScreen, even if your laptop manufacturer didn't physically print "SysRq" on it; alternatively you may need Fn+[something]. That said, you don't need to press that key combination; I mention that keyboard shortcut as an example of something that can cause a filesystem to fail read-only, not as a step that you need to take. – nanofarad Jul 19 '19 at 19:30
  • sudo fsck -Af -M gives me: fsck: illegal option -- A fsck: ? option? – ScottyBlades Sep 29 '19 at 00:34
  • fsck from util-linux 2.20.1 gives me: fsck usage: fsck [-fdnypqL] [-l number] – ScottyBlades Sep 29 '19 at 00:35
  • sudo fsck.ext4 -f /dev/sda1 gives me: sudo: fsck.ext4: command not found – ScottyBlades Sep 29 '19 at 00:36
  • @ScottyBlades This answer is seven years old, and the flags have likely changed--I don;t have a copy of fsck from util-linux 2.20.1, and my copy handles -A just fine, so I don't have any great advice, aside from manually invoking on each volume that needs to be checked. – nanofarad Sep 29 '19 at 00:43
  • @hexafraction, it's cool. I'm new to web dev. So I think I'm making more obvious/basic mistakes that would go under most seasoned developer's radars. MY SOLUTION: When I clicked "save as" I was saving to the root directory, not the directory I actually wanted to save in. Making sure I saved to the correct location solved the problem. – ScottyBlades Sep 29 '19 at 00:47
  • 1
    @ScottyBlades Understood. Looks like your situation was mostly unrelated to actual file system corruption then, meaning that there was no need to get fsck involved. – nanofarad Sep 29 '19 at 01:14
  • @hexafraction, good to know – ScottyBlades Sep 29 '19 at 08:12
  • TY, I had a problem with snappy daemon and tryied a lots of things. Your sudo fsck.ext4 -f /dev/sda1 saved me (in my case on sda2). – Òscar Raya Nov 26 '20 at 10:39
67

The answer by hexafraction didn't work for me. Every time I tried executing sudo fsck -Af -M it just showed

$ sudo fsck -Af -M
fsck from util-linux 2.20.1

and nothing else. No error or anything. For me, booting into a live disc and executing this worked -

sudo fsck.ext4 -f /dev/sda1

Provided the partition in question /dev/sda1 was an ext4 filesystem.

notpeter
  • 1,569
Bibhas
  • 1,016
59

Here is the command that solved my problem :

mount -o remount /

better than a reboot or sudo fsck -Af

Sulliwane
  • 2,085
36

If you want to force your root filesystem to remount as rw, you can do the following.

mount -o remount,rw /
17

Try running dmesg | grep "EXT4-fs error" to see if you have any issues related to the filesystem / journaling system itself. I would recommend you to restart your system, then. Also, sudo fsck -Af answer by ObsessiveSSOℲ won't hurt.

thiagowfx
  • 845
  • 6
  • 9
12

Note that sometimes this can be caused by the computer forgetting the system time - disk check fails because the dates in the journal are in THE FUTURE!

Setting the BIOS time (and checking the BIOS battery) fixed this problem for me, without having to do any disk recovery.

Tony Cook
  • 121
  • Welcome to Ask Ubuntu! I recommend [edit]ing this answer to expand it with specific details about how to do this. (See also How do I write a good answer? for general advice about what sorts of answers are considered most valuable on Ask Ubuntu.) – David Foerster Feb 23 '16 at 08:50
  • I am fairly certain this is happening to me right now, given that my computer told me it forgot its time this morning. – nomen Jan 22 '18 at 04:14
8

If you're dual booting your machine with Ubuntu and Windows together and this issue occurs,it's because Windows changes the filesystem,in that case this might do the trick. Try disabling fast startup

Control Panel > Hardware and Sounds > Power Options > (in the left) Choose what closing the lid does > Change settings that are currently unavailable > Untick 'Turn on fast startup'

Now booting into Ubuntu will solve the issue. Hope this helps!

https://youtu.be/KJ-0KPZhAFo

8

If you have the graphical user interface go to the disk application, select the drive with the issue, click on the gears icon and choose the option Repair Filesystem. In less than a second the problem is fixed.

enter image description here

diegosasw
  • 211
  • 3
  • 6
7

(Deleted previous answer)

Edit: The main problem was on the windows side. After updating my Windows 10, the 'quick start' option automatically got enabled. On disabling that option again, and then again re-starting the machine, the problem went away. Windows 10 gave me heavy headache for days :(

The 'quick start' option can be found in 'power options' in the control panel. Disable that!!! :)

4

If you dual boot ubuntu alongside windows 10 it's probably windows 10's fast start-up that's holding onto your filesystem , it doesnt unmount your hard disks properly. to fix this you need to boot into windows 10

  1. Start > Power Settings
  2. click on Additional power setting on the right
  3. click on choose what thepower buttons do on the left
  4. clock on change settings that are currently unavailable
  5. unmark turn on fast-startup
  6. save changes and then reboot into ubuntu everything will work fine!
4

For me,Rebooting a system solving this issue

sudo reboot

as he mentioned about it.

Remember

as System Administrator rebooting should be the latest Solution

Zaman Oof
  • 229
  • 1
  • 7
  • 2
    ¡CAUTION! I was fixing a remote server and it didn't turn on after reboot which made things way harder, I recommend testing other solutions before trying to rebbot specially if you're not physically with the computer. For the first time in the engineering world, turning it off and on again was a big problem instead of a solution :( – Mark E Jul 19 '21 at 17:28
  • @MarkE Rebooting Against Availability so should be the latest solution – Zaman Oof Sep 30 '21 at 10:41
3

Usually linux puts your filesystems in read only when errors occur, especially errors with the disk or the filesystem itself, errors like a wrong journal entry for example.

You better check your dmesg for disk related errors.

Google is full of discussion about this and you can pick the ones that is closer to your configuration, but a look at dmesg is usually enough.

axis
  • 219
2

If you were in situations that can not use live disc, e.g. you are remotely ssh into your system, you can still using the command that @Bibhas had answered:

sudo fsck.ext4 -f /current/filesystem/mount/point

It will prompt for fixing your filesystem error. You also need to reboot your system remotely.

1

It looks like some mounted files have got corrupted, and as a result, the kernel has set the file system to RO to prevent further damage. To find which file system is corrupted, we could run:

cat /proc/mounts | grep -i ro

The output would be similar to the below:

proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0
/dev/sda1 / ext4 ro,relatime,data=ordered 0 0
tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0

One of the solutions for this issue could be to remount the corrupted file system.

1

I have had this problem on my computer for over 1 year and tried everything to solve the problem. Suddenly Linux goes into read-only mode. If you are editing something you are unable to save and have to execute fsck command and reset the computer. The computer is also very slow and freezing all the time. I removed the dual boot and left only Ubuntu, upgraded Ubuntu from version 18.04 LTS to version 20.04 LTS, and it didn't work. What was crucial to solving the problem is the use of the dmesg command. The experience didn't work out for me, just this command. The function of this command is to monitor the computer.

In my case, the problem was related to the SSD incompatibility with Ubuntu. I used HDD and after I switched to SSD the problem came up. The problem was solved by updating the SSD firmware, which was only possible by partitioned Windowns, because Kingston does not have the program to update firmware through Linux. I also installed the dual boot Windowns and Linux, first installing Windows over the entire SSD, then deallocating space through Windowns and installing Ubuntu, but it is very unlikely that this was the solution to the problem.

1

In my case it was down to RAID 1 stabilizing after the initial installation. I have /boot and / on s/w RAID1. Having left the system overnight and rebooted, everything is working fine. Richard

0

I am using the 18.04 LTS release and what I did was, first I unmounted my damaged drives then I went into “Disks” app by seaching in the dash. From there I selected the drive which was damaged. Make sure it is unmounted else it won’t work. In the options menu (the little gear shape), and clicked on “repair file system”. Boom! Worked like a charm.

0

This happens to my laptop quite often as it hangs after sometime and normal shutdown is not possible and I have to hard power off causing damage to my disk

Steps

  1. Use sudo dmesg --level=crit,err to find out which disk say (/dev/sda3) has the problem.

Example output

[ 8696.776775] EXT4-fs error (device sda3): ext4_journal_check_start:83: comm ThreadPoolForeg: Detected aborted journal
[ 8696.986946] EXT4-fs (sda3): Remounting filesystem read-only
  1. Use sudo blkid /dev/sda3 to get the UUID of the disk
sudo blkid /dev/sda3
/dev/sda3: LABEL="home" UUID="7a9689e4-90d9-496f-b889-c4f0ee389798" BLOCK_SIZE="4096" TYPE="ext4" PARTLABEL="home" PARTUUID="5472ae83-2084-4fa1-b1dc-0593d9519d17"

  1. Edit sudo vi /etc/fstab and comment out that UUID so that it does not mount on bootup.
#UUID=7a9689e4-90d9-496f-b889-c4f0ee389798  /home  ext4  noatime,errors=remount-ro  0  0

This is needed because fsck does not work if the disk is mounted. Fortunately, in my case, the disk is mounted to /home. I am not sure if the disk is mounted to the root. If so this may not work and you may have to use a boot CD or boot USB to do the next commands

  1. sudo reboot and when it comes up run sudo fsck.ext4 -y -f /dev/sda3
  2. Uncomment the commented-out drive from fstab (undo step 3 )

reboot again

Alex Punnen
  • 262
  • 3
  • 7
0

Check if you have any faulty hardware.I got this error due to a hardisk loosening. Ran mount -o remount,rw / and it worked fine.

-1

It's misconfigured boot settings (try mounting / as ext2) not drive or partition physical error.

dmesg | grep "error" gave me:

ext3-fs (sda2): error: couldn't mount because of unsupported optional features (240) 
ext2-fs (sda1): error: couldn't mount because of unsupported optional features (240) 
ext4-fs (sda2): mounted file system with ordered data mode: opts: (null)

It mounted / as read only due to the fstab error=mount-ro directive.