30

September
2012

Problems with NTFS filesystem for Slax changes

I'm currently fighting strange problem. When Slax is started from USB device with FAT filesystem, everything works just fine. But if the usb drive uses NTFS, strange things happen.

In Slax, the ntfs drive is mounted using ntfs-3g, and then it is overmounted with posixovl to provide posix compatibility. Then, all system modifications (eg. firefox startup files, cache, etc) are stored in /slax/changes/ directory on the drive. However, it's very slow on ntfs. If you run firefox for example, and 'iostat -m 1' on xterm console, you'll see that the processor is 99% of time iowait-ing.

I have currently no solution for this problem. It is probably a problem of the ntfs-3g driver, since it doesn't happen with FAT (where only posixovl is used). Any suggestion will be really appreciated.

User comments
WolfAlpha 2012-09-30 03:02

btw not using NTFS, I am try to save changes to ext4 filesystem, but failed the changes not saved anything, in boot error message say cannot mounted.

Manfred 2012-09-30 06:55

Hi Tomas,

Some remarks to NTFS on removable media:

I'm never using NTFS on removable media because interoperabilty
with mediaplayers cameras and so on gets lost this way.

Another reason not to use NTFS is that on some systems
the possibility to remove the device without shutting down/rebooting gets lost, too.

Imho we should use a loop mounted xfs file on NTFS systems.
This is working fine for years now...

Sooner or later most removable devices (at least for Vista Sp1 and Windows7) will be using exFat for interoperability.

Don't get me wrong but concentrating on saving changes on NTFS directly imho is wasting time...

Manfred

Tomas M 2012-09-30 08:16

I tend to agree with you regarding ntfs. The problem with xfs filesystem (loopback file on ntfs, or loopback file on any fs in general) is that it's not dynamic size. XFS can be enlarged on the fly, but the user has to perform the operation manually. And creating 1GB big file (or similar size) for changes during the first Slax start is no solution as well.

I am in touch with Junjiro Okajima, the author of AUFS, who could possibly help here, if there is a solution - if aufs detects it's unable to write to the filesystem due to insufficient disk space, it could start userspace program, which would enlarge the xfs filesystem on the fly. But that may not be possible, I don't know, Junjiro suggested some other solutions first before he considers this in aufs. If there was any other way how to make a loopback-mounted filesystem of dynamic size, I would prefer it, since fixed size is not ideal.

aitap 2012-09-30 08:32

A "cheat" available for some file systems (including NTFS @ntfs-3g, but not FAT): sparse files.
$ dd if=/dev/zero of=sparse_file bs=1G count=0 seek=100
$ ls -lh sparse_file
-rwxrwxrwx 1 root root 999G Sep 30 23:28 sparse_file
$ df -h .
Filesystem Size Used Avail Use% Mounted on
/dev/loop0p1 99M 96M 3.4M 97% /mnt

The file will be created even if the filesystem size is less than the file size and will be available for writing as long as there is free space on device. Unfortunately, there is no easy way of reducing such files.

If only there was a way to cache the changes before saving...
What will you say about an option of saving changes (from tmpfs/ramfs to posixovl @ writable filesystem) at the shutdown stage? It can be switched via kernel command line and activated by default on NTFS.

Tomas M 2012-09-30 09:03

I didn't know that sparse files are available on ntfs, that could be used. I'm also considering something more hardcore, like cowloop or lvm2-snapshot of in-memory sparse file, not sure if that could work, needs investigation :)

axion 2012-10-02 23:00

Thomas, try using the kernel based ntfs-driver it is lightning fast as compared to ntfs-3g.

Tomas M 2012-10-03 00:50

As far as I know, kernel based ntfs driver is not usable in any way, it has only support for writing to existing files. The problem is that if the device with ntfs filesystem is mounted in initramfs, it can't be later mounted differently, so if the user wants full access to his data then we have to mount it the best we can from the beginning (in initramfs). And here ntfs-3g is maybe slower but is necessary due to its features.

hellosmile6 2022-04-30 00:13

Version 9.11.x seems to work well with USB boot (NTFS). I use NTFS because I need to use my USB drive with Windows occasionally, and it works perfectly from USB.