Are you trying to to recover from File System Corruption Using FreeBSD’s Single-User Mode?

This guide is for you.

Despite your best efforts, there are certain instances where your server might suffer file corruption and need recovery. Sometimes this happens when your VPS is powered off abruptly, or when a piece of software or hardware malfunctions suddenly.

Here at Ibmi Media, as part of our Server Management Services, we regularly help our Customers to perform FreeBSD related queries.

In this context, we shall look into the steps to boot the FreeBSD droplet into single-user mode and how to use the tools to recover a damaged filesystem.

Important Considerations and Risks

In any circumstance, the best way to prevent data loss is through good backups. So implementing an offsite backup solution and thoroughly testing it on a regular basis is the best way to guarantee the integrity of the important data.

The recovery options like fsck are often useful. But there is no guarantee that it will work correctly every time. The fsck operation can occasionally cause data corruption on active disks. Problems can still occur in cases of severe damage though, so we consider these last-resort methods for data recovery.

Booting to Single User Mode

We run the below command to boot the droplet into single-user mode:

sudo nextboot -o "-s" -k kernel

This will instruct the droplet to enter single-user mode on the next reboot rather than attempting to load the full system. Once we are ready we use the reboot command to reboot the droplet.

sudo reboot

After rebooting the droplet, it will no longer be accessible over the network via ssh.

So, we would need to access the droplet via the console in the control panel. Once the reboot completes, we see the below line:

Enter full pathname of shell or RETURN for /bin/sh:

Then we press Enter to start a shell session.

Alternate Method for File System Corruption

If the droplet is not accessible via ssh we can still boot the droplet to single-user mode using this alternate method.

i. First, in the control panel, we click on the power cycle button to reboot the droplet.

ii. Immediately after this, we open the droplet’s console. After a few seconds we see a screen as shown below:

iii. At this menu, we select item 2 and press enter to continue.

How to Perform a File System check ?

Now we have our droplet in single-user mode. First, we must know the device that we are checking. We run the below command to display the filesystems currently configured.

cat /etc/fstab

Here, the first item in the list shows a ufs filesystem. We were looking for this itself. Now, we run the below command to perform a filesystem check on this disk.

fsck -yf /dev/gpt/rootfs

Checking the Results

Once the filesystem check is complete, we reboot the droplet to leave single-user mode and restart the dropet in normal (Multi-User) mode. For that, we run the below command.

reboot

We connect to the droplet using an SSH client. If the droplet responds to the ssh connection and it was not before then this is a good sign.

If you run into any specific problems that alerted you to the possibility of corruption, you can apply the tips below to resolve.

Check the /lost+found directory. 

This is where fsck puts partially recovered files.

Sometimes, fsck recovers file data, but it cannot find a reference to the file on the filesystem. Basically, it is a file without a name. If fsck faces this situation, then it places these files in the /lost+found directory so that you can manually try to figure out what the file is.

After running the fsck, check if anything has been placed in that directory. Also, the lost+found directory is only available to the root user. So, first change the root account with a sudo su command:
sudo su
cd /lost+found
ls

If there are files in this directory, you can see and identify them. Often, these are files that you had deleted anyways, but were still being used when the system crashed. It is worth checking them though to be sure.

[Need urgent assistance with FreeBSD queries? – We are here to help you.]