Bug 354113

Summary: [SOLVED] NTFS partitions not mountable after update from factory
Product: [openSUSE] openSUSE 11.0 Reporter: Casual J. Programmer <casualprogrammer>
Component: BasesystemAssignee: Bernhard Kaindl <bk>
Status: RESOLVED FIXED QA Contact: E-mail List <qa-bugs>
Severity: Minor    
Priority: P3 - Medium CC: elchevive68, holler, jmolles, Joachim.Reichelt, matz, pearson45j, rastislav.krupansky
Version: Alpha 2   
Target Milestone: ---   
Hardware: x86   
OS: openSUSE 11.0   
Whiteboard:
Found By: Beta-Customer Services Priority:
Business Priority: Blocker: ---
Marketing QA Status: --- IT Deployment: ---
Attachments: Screenshot of the situation
preprocessed source of libntfs-3g/attrib.c (as asked for)
preprocessed source of libntfs-3g/runlist.c (as asked for in #19)
attrib.i, generated with -m32

Description Casual J. Programmer 2008-01-16 09:59:08 UTC
When booting to openSuSe normally the Windows NTFS partitions are mounted automatically in the process. After updating this morning this is not the case anymore.

/etc/fstab looks normal

cat /etc/fstab
/dev/disk/by-id/scsi-SATA_FUJITSU_MHV2080_NW78T662BLKF-part4 /                    ext3       acl,user_xattr        1 1
/dev/disk/by-id/scsi-SATA_FUJITSU_MHV2080_NW78T662BLKF-part3 swap                 swap       defaults              0 0
proc                 /proc                proc       defaults              0 0
sysfs                /sys                 sysfs      noauto                0 0
debugfs              /sys/kernel/debug    debugfs    noauto                0 0
usbfs                /proc/bus/usb        usbfs      noauto                0 0
/dev/disk/by-id/scsi-SATA_FUJITSU_MHV2080_NW78T662BLKF-part1 /windows/C           ntfs-3g    users,gid=users,fmask=133,dmask=022, 0 0
/dev/disk/by-id/scsi-SATA_FUJITSU_MHV2080_NW78T662BLKF-part2 /windows/D           ntfs-3g    users,gid=users,fmask=133,dmask=022, 0 0


when I issue mount -a as root I get:

mount -a
Failed to read $MFTMirr: Input/output error
Failed to mount '/dev/sda1': Input/output error
NTFS is either inconsistent, or you have hardware faults, or you have a
SoftRAID/FakeRAID hardware. In the first case run chkdsk /f on Windows
then reboot into Windows TWICE. The usage of the /f parameter is very
important! If you have SoftRAID/FakeRAID then first you must activate
it and mount a different device under the /dev/mapper/ directory, (e.g.
/dev/mapper/nvidia_eahaabcc1). Please see the 'dmraid' documentation
for the details.
Failed to read $MFTMirr: Input/output error
Failed to mount '/dev/sda2': Input/output error
NTFS is either inconsistent, or you have hardware faults, or you have a
SoftRAID/FakeRAID hardware. In the first case run chkdsk /f on Windows
then reboot into Windows TWICE. The usage of the /f parameter is very
important! If you have SoftRAID/FakeRAID then first you must activate
it and mount a different device under the /dev/mapper/ directory, (e.g.
/dev/mapper/nvidia_eahaabcc1). Please see the 'dmraid' documentation
for the details.

Although pretty sure that Windows is OK, I did a chkdsk -f on both partitions, and as extra measure reset the BIOS to its factory settings. chkdsk did not find anything amiss and both Windows partitions boot without problem.
Comment 1 Casual J. Programmer 2008-01-16 10:06:59 UTC
Notebook: Fujitsu Siemens Amilo Si 1520
Graphics: Fujitsu Siemens Mobile 945GM/GMS/GME, 943/940GML Express
Monitor:  QUANTADISPLAY LCD Monitor 1280x800@60Hz
Wireless: Intel PRO/Wireless 3945ABG Network Connection
Sound:    82801G (ICH7 Family) High Definition Audio Controller
Desktop:  gnome2-SuSE-10.3-138
YaST GUI: yast2-qt-2.16.14-2
OS:       openSUSE 11.0 (i586) Alpha0 VERSION = 11.0
Kernel:   2.6.24-rc7-git3-2-default

rpm -qa | grep ntfs | sort

ntfs-3g-1.913-17
ntfsprogs-2.0.0-2
ntfsprogs-fuse-2.0.0-2
Comment 2 Casual J. Programmer 2008-01-16 10:38:51 UTC
Same happens if I hotplug an external USB disk with NTFS filesystem.

Message box appears, stating:

Cannot mount volume.

Rest of text see screen shot attached.
Comment 3 Casual J. Programmer 2008-01-16 10:39:30 UTC
Created attachment 190690 [details]
Screenshot of the situation
Comment 4 Casual J. Programmer 2008-01-16 11:46:10 UTC
cat /var/log/messages | grep NTFS
Jan  9 10:57:32 workstation6l ntfs-3g[5289]: Mounted /dev/sdb1 (Read-Only, label "WS2 Extern 1", NTFS 3.1)
Jan 15 10:32:43 workstation6l ntfs-3g[12692]: Mounted /dev/sda1 (Read-Only, label "WS6 Intern1", NTFS 3.1)
Jan 15 10:32:46 workstation6l ntfs-3g[12697]: Mounted /dev/sda2 (Read-Write, label "WS6 Intern2", NTFS 3.1)
Jan 16 09:47:26 workstation6l klogd: NTFS driver 2.1.29 [Flags: R/W MODULE].
Jan 16 09:47:26 workstation6l klogd: NTFS-fs error (device sda1): ntfs_read_block(): Failed to read from inode 0x1, attribute type 0x80, vcn 0x0, offset 0x0 because its location on disk could not be determined even after retrying (error code -5).
Jan 16 09:47:26 workstation6l klogd: NTFS-fs error (device sda1): ntfs_read_block(): Failed to read from inode 0x1, attribute type 0x80, vcn 0x0, offset 0x200 because its location on disk could not be determined even after retrying (error code -5).
Jan 16 09:47:26 workstation6l klogd: NTFS-fs error (device sda1): ntfs_read_block(): Failed to read from inode 0x1, attribute type 0x80, vcn 0x0, offset 0x400 because its location on disk could not be determined even after retrying (error code -5).
Jan 16 09:47:26 workstation6l klogd: NTFS-fs error (device sda1): ntfs_read_block(): Failed to read from inode 0x1, attribute type 0x80, vcn 0x0, offset 0x600 because its location on disk could not be determined even after retrying (error code -5).
Jan 16 09:47:26 workstation6l klogd: NTFS-fs error (device sda1): ntfs_read_block(): Failed to read from inode 0x1, attribute type 0x80, vcn 0x0, offset 0x800 because its location on disk could not be determined even after retrying (error code -5).
Jan 16 09:47:26 workstation6l klogd: NTFS-fs error (device sda1): ntfs_read_block(): Failed to read from inode 0x1, attribute type 0x80, vcn 0x0, offset 0xa00 because its location on disk could not be determined even after retrying (error code -5).
Jan 16 09:47:26 workstation6l klogd: NTFS-fs error (device sda1): ntfs_read_block(): Failed to read from inode 0x1, attribute type 0x80, vcn 0x0, offset 0xc00 because its location on disk could not be determined even after retrying (error code -5).
Jan 16 09:47:26 workstation6l klogd: NTFS-fs error (device sda1): ntfs_read_block(): Failed to read from inode 0x1, attribute type 0x80, vcn 0x0, offset 0xe00 because its location on disk could not be determined even after retrying (error code -5).
Jan 16 09:47:26 workstation6l klogd: NTFS-fs error (device sda1): check_mft_mirror(): Failed to read $MFTMirr.
Jan 16 09:47:26 workstation6l klogd: NTFS-fs error (device sda1): load_system_files(): $MFTMirr does not match $MFT.  Mounting read-only.  Run ntfsfix and/or chkdsk.
Comment 5 Hans-Peter Holler 2008-01-17 20:51:16 UTC
As a workaround you can downgrade to ntfs-3g-1.913-4 from 10.3.
Install the rpm with rpm -Uhv --force
ntfs-3g is a user space application.
Changing Component to Basesystem, Assignee to coolo (don't know who's responsible for this package, he should know) and Severity to Blocker

Maybe stupid guess: glibc 2.7 ?
Comment 6 Casual J. Programmer 2008-02-05 13:43:14 UTC
Still not working with ntfs-3g-1.913-24

cat /var/log/messages | grep ntfs provides endless list of:

Feb  5 12:05:52 workstation6l ntfs-3g[1575]: Skipping unrepresentable filename (inode 80367): Invalid or incomplete multibyte or wide character
Feb  5 12:05:52 workstation6l ntfs-3g[1575]: Skipping unrepresentable filename (inode 79860): Invalid or incomplete multibyte or wide character
Feb  5 12:05:52 workstation6l ntfs-3g[1575]: Skipping unrepresentable filename (inode 79861): Invalid or incomplete multibyte or wide character
.
.
.

Suggestion from #5 still OK.
Comment 7 Bernhard Kaindl 2008-02-05 17:47:11 UTC
Hi Casual,

"Skipping unrepresentable filename (inode 80367): Invalid or incomplete multibyte or wide character"

This tells that there was no locale set when the NTFS-partition was mounted. To ensure this, set the locale, e.g. using by setting the environment variable LC_CTYPE to the locale which you use before mounting or if you mount thru /etc/fstab, you can append ",locale=en_US.UTF-8" to the last column of the respective line after "defaults".

About the I/O errors, which are (from an NTFS perspective) an entirely separate matter from setting the locale for the mount, please have a look at this FAQ entry:

http://ntfs-3g.org/support.html#ioerror

I/O errors are errors of the underlying driver or hardware layers and not something which NTFS-3g can be responsible for.

Your kernel messages from Jan 16 in comment #4 are not even from NTFS-3g, but from the in-kernel ntfs filesystem driver, so basically it looks like as if your NTFS partitions are having corrupt filesystem structures or are not readable correctly from Linux.

Is Windows booting fine and not reporting any errors if you let it check your NTFS partitions?
Comment 8 Hans-Peter Holler 2008-02-06 11:44:32 UTC
OK, let's restart:

cat /etc/SuSE-release
openSUSE 11.0 (i586) Alpha2
VERSION = 11.0

rpm -q ntfs-3g
ntfs-3g-1.913-24

ntfs-3g /dev/sda1 /windows/C
Failed to read $MFTMirr: Input/output error
Failed to mount '/dev/sda1': Input/output error
NTFS is either inconsistent, or you have hardware faults, or you have a
SoftRAID/FakeRAID hardware. In the first case run chkdsk /f on Windows
then reboot into Windows TWICE. The usage of the /f parameter is very
important! If you have SoftRAID/FakeRAID then first you must activate
it and mount a different device under the /dev/mapper/ directory, (e.g.
/dev/mapper/nvidia_eahaabcc1). Please see the 'dmraid' documentation
for the details.

rpm -Uhv ntfs-3g-1.913-4.i586.rpm --force
Preparing...                ########################################### [100%]
   1:ntfs-3g                ########################################### [100%]

ntfs-3g /dev/sda1 /windows/C

mount
/dev/sda6 on / type ext3 (rw,acl,user_xattr)
/proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
debugfs on /sys/kernel/debug type debugfs (rw)
udev on /dev type tmpfs (rw)
devpts on /dev/pts type devpts (rw,mode=0620,gid=5)
/dev/sdb1 on /data type ext3 (rw)
/dev/sda7 on /home type ext3 (rw)
usbfs on /proc/bus/usb type usbfs (rw)
fusectl on /sys/fs/fuse/connections type fusectl (rw)
none on /proc/fs/vmblock/mountPoint type vmblock (rw)
/dev/sda1 on /windows/C type fuseblk (rw,nosuid,nodev,noatime,allow_other,blksize=4096)

As you can see: the only difference is ntfs-3g-1.913-24 / ntfs-3g-1.913-4
Comment 9 Casual J. Programmer 2008-02-06 11:55:41 UTC
The interesting fact is: It works perfectly with the package from #5 ( ntfs-3g-1.913-4.i586.rpm from the 10.3 repository ) also without setting a locale in /etc/fstab.

Windows XP is working fine with both partitions and neither CHKDSK /f C: nor CHKDSK /f D: is reporting any errors

After setting the locale and trying to mount with ntfs-3g-1.913-24 installed I get the same results as HPH in comment #8
Comment 10 Hans-Peter Holler 2008-02-06 12:17:58 UTC
*** Bug 355454 has been marked as a duplicate of this bug. ***
Comment 11 Hans-Peter Holler 2008-02-08 09:39:00 UTC
It's gcc 4.3.

- Use Factory/Alpha2 with development environment (gcc 4.3)
- download ntfs-3g-1.913-24.src.rpm and install it with rpm -ihv
- cd /usr/src/packages/SPECS
- rpmbuild -bb ntfs-3g.spec
- cd /usr/src/packages/RPMS/i586
- rpm -Uhv ntfs-3g-1.913-24.i586.rpm --force
- try to mount a ntfs-partition ==> ERROR
- download gcc42-4.2.1_20070724-17.i586.rpm and cpp42-4.2.1_20070724-17.i586.rpm
  from 10.3 and install with rpm -ihv
- cd /usr/bin
- rm cc gcc cpp
- ln -s gcc-4.2 cc
- ln -s gcc-4.2 gcc
- ln -s cpp-4.2 cpp
- cd /usr/src/packages/SPECS
- rpmbuild -bb ntfs-3g.spec
- cd /usr/src/packages/RPMS/i586
- rpm -Uhv ntfs-3g-1.913-24.i586.rpm --force
- try to mount a ntfs-partition ==> SUCCESS
Comment 12 Casual J. Programmer 2008-02-09 16:17:42 UTC
It might be added here, that the suggestion from comment #7, adding a locale to /etc/fstab ntfs partitions does actually provoke errors like:

Feb  9 17:04:00 workstation6l ntfs-3g[1580]: Skipping unrepresentable filename (inode 71332): Invalid or incomplete multibyte or wide character
Feb  9 17:04:00 workstation6l ntfs-3g[1580]: Skipping unrepresentable filename (inode 41028): Invalid or incomplete multibyte or wide character
Feb  9 17:04:00 workstation6l ntfs-3g[1580]: Skipping unrepresentable filename (inode 40726): Invalid or incomplete multibyte or wide character

Removing the locale again returns normal behaviour ( using ntfs-3g-1.913-4.i586.rpm ).

Comment 13 Bernhard Kaindl 2008-02-12 11:21:17 UTC
Hello Richard,

I am sorry that I am throwing this blocker bug at you for now, but I think it should show up in the GCC arena now:

Hans-Peter and I have verfied that the current gcc-4.3 rpm in openSUSE Factory results in a broken ntfs-3g binary on the i586 architecture (not on x86_64)

Here is my verfication:

works: http://download.opensuse.org/repositories/home:/benkai:/ntfs/openSUSE_10.3/i586/ntfs-3g-1.2129-7.1.i586.rpm
does not work: http://download.opensuse.org/repositories/home:/benkai:/ntfs/openSUSE_Factory/i586/ntfs-3g-1.2129-7.1.i586.rpm

Thanks very much for all the data everyone, especially Hans-Peter for his immensely helpful last posting

gcc-4.3 does not produce any compiler warning with -Wall (except for an unused variable) so it did not discover anything (like an uninitialized variable) which might hint to something to improve in the ntfs-3g code itself.

I'll try to narrow it down more to provide some more data.

ntfs-3g is heavvily tested on many archictectures and the bug spans at least from version 1.913 to 1.2129 which covers about 6 months of development during which such (if it were a bug in NTFS-3g) should not have gone unnoticed to be only uncovered by gcc-4.3, but we'll find out where the bug really is.
Comment 14 Richard Biener 2008-02-12 13:20:03 UTC
Try adding -fno-strict-aliasing and/or -fwrapv to the CFLAGS used for building.
Does building with -O0 result in a working ntfs-3g?  If so please reduce the
failure to a single file by individually building selected files with -O2/-O0.
Comment 15 Bernhard Kaindl 2008-02-12 13:46:26 UTC
I now tested with -O0 and -O1, but could not isolate it to a individual optimisation (including -fno-strict-aliasing and -fwrapv) yet, despite
trying even with:

CFLAGS="-O1 -fno-ipa-reference -fno-ipa-pure-const -fno-if-conversion -fno-if-conversion2 -fno-merge-constants \
-fno-wrapv \
-fno-cprop-registers -fno-defer-pop \
-fno-split-wide-types \
-fno-gcse  \
-fmessage-length=0 -Wall \
-fno-defer-pop -fno-guess-branch-probability -fno-unit-at-a-time \
-fno-strict-aliasing -fno-thread-jumps -fno-tree-pre -fno-regmove -fno-expensive-optimizations -fno-peephole2 -fno-optimize-sibling-calls -fno-optimize-register-move -fno-delete-null-pointer-checks -fno-caller-saves -fno-forward-propagate -fno-schedule-insns2 -fno-tree-pre -fno-tree-store-ccp -fno-tree-vrp -fno-align-jumps -fno-align-labels -fno-align-loops -fno-crossjumping  -fno-cse-follow-jumps -fno-inline-small-functions \
-fno-tree-ccp -fno-tree-ch              -fno-tree-copy-prop  -fno-tree-copyrename \
-fno-tree-dce -fno-tree-dse  -fno-tree-fre \
-fno-tree-pre -fno-tree-store-ccp -fno-tree-vrp \
 -fno-forward-propagate -fno-crossjumping -fno-cse-follow-jumps \
-fno-tree-ccp -fno-tree-ch           -fno-tree-copy-prop  -fno-tree-copyrename \
-fno-tree-dce -fno-tree-dominator-opts  -fno-tree-dse  -fno-tree-fre \
-fno-tree-salias \
-fno-tree-sink  -fno-tree-sra  -fno-tree-ter "

and 

CFLAGS="-O0 -fipa-reference -fipa-pure-const -fif-conversion -fif-conversion2 -fmerge-constants \
-fwrapv \
-fcprop-registers -fdefer-pop \
-fsplit-wide-types \
-fgcse  \
-fmessage-length=0 -Wall \
-fdefer-pop -fguess-branch-probability -funit-at-a-time \
-fstrict-aliasing -fthread-jumps -ftree-pre -fregmove -fexpensive-optimizations -fpeephole2 -foptimize-sibling-calls -foptimize-register-move -fdelete-null-pointer-checks -fcaller-saves -fforward-propagate -fschedule-insns2 -ftree-pre -ftree-store-ccp -ftree-vrp -falign-jumps -falign-labels -falign-loops -fcrossjumping  -fcse-follow-jumps -finline-small-functions \
-ftree-ccp -ftree-ch              -ftree-copy-prop  -ftree-copyrename \
-ftree-dce -fno-tree-dse  -fno-tree-fre \
-ftree-pre -ftree-store-ccp -ftree-vrp \
 -fforward-propagate -fcrossjumping -fcse-follow-jumps \
-ftree-ccp -ftree-ch           -ftree-copy-prop  -ftree-copyrename \
-ftree-dce -ftree-dominator-opts  -ftree-dse  -ftree-fre \
-ftree-salias \
-ftree-sink  -ftree-sra  -ftree-ter "

The result was always identical to just -O0 (working) and -O1 (not working):

Here is the result of my debug session, with unpatched ntfs-3g-1.2129 compiled with --enable-debug to get extra debug logging enabled in the code.

I diffed the debug log and single-stepped where the 1st difference occured:

In ntfs_attr_lookup(), libntfs-3g/attrib.c
(gdb) l 493
488     {
489             LCN lcn;
490             ntfs_attr_search_ctx *ctx;
491
492             ntfs_log_trace("Entering for inode 0x%llx, attr 0x%x, vcn
0x%llx.\n",
493                     (unsigned long long)na->ni->mft_no, na->type, (long
long)vcn);
494
495             lcn = ntfs_rl_vcn_to_lcn(na->rl, vcn);
496             if (lcn >= 0 || lcn == LCN_HOLE || lcn == LCN_ENOENT)
497                     return 0;

Stepping into:

ntfs_rl_vcn_to_lcn (rl=0x0, vcn=0) at runlist.c:978
978             if (vcn < (VCN)0)
(gdb) p vcn
$8 = 0
(gdb) s
985             if (!rl)
(gdb)
986                     return (LCN)LCN_RL_NOT_MAPPED;
(gdb) s
ntfs_attr_map_runlist (na=0x8e63630, vcn=0) at attrib.c:496
496             if (lcn >= 0 || lcn == LCN_HOLE || lcn == LCN_ENOENT)
(gdb) p lcn
$9 = -2
(gdb) s
497                     return 0;

LCN_HOLE is -1 and LCN_ENOENT is -3 (both are defined in an enum)

=> AFAICS, "return 0" may not happen in with this value of lcn.

Code compiled with -O0 does not return and continues successfully to mount.

info registers
eax            0x0      0
ecx            0x1      1
edx            0x0      0
ebx            0xf7fb8ff4       -134508556
esp            0xff9cdfc0       0xff9cdfc0
ebp            0xff9ce018       0xff9ce018
esi            0x3      3
edi            0xffffffff       -1
eip            0xf7f75c97       0xf7f75c97 <ntfs_attr_map_runlist+196>
eflags         0x202    [ IF ]
cs             0x23     35
ss             0x2b     43
ds             0x2b     43
es             0x2b     43
fs             0x0      0
gs             0x63     99

Disassembly of the code after the call:
0xf7f75c6b <ntfs_attr_map_runlist+152>: call   0xf7f7464c
<ntfs_rl_vcn_to_lcn@plt>
0xf7f75c70 <ntfs_attr_map_runlist+157>: mov    %eax,%esi
0xf7f75c72 <ntfs_attr_map_runlist+159>: mov    %edx,%edi
0xf7f75c74 <ntfs_attr_map_runlist+161>: mov    $0x1,%ecx
0xf7f75c79 <ntfs_attr_map_runlist+166>: cmp    $0xffffffff,%edi
0xf7f75c7c <ntfs_attr_map_runlist+169>: jge    0xf7f75c83
<ntfs_attr_map_runlist+176>
0xf7f75c7e <ntfs_attr_map_runlist+171>: mov    $0x0,%ecx
0xf7f75c83 <ntfs_attr_map_runlist+176>: mov    %esi,%eax
0xf7f75c85 <ntfs_attr_map_runlist+178>: xor    $0xfffffffd,%eax
0xf7f75c88 <ntfs_attr_map_runlist+181>: mov    %edi,%edx
0xf7f75c8a <ntfs_attr_map_runlist+183>: not    %edx
0xf7f75c8c <ntfs_attr_map_runlist+185>: mov    %eax,%esi
0xf7f75c8e <ntfs_attr_map_runlist+187>: or     %edx,%esi
0xf7f75c90 <ntfs_attr_map_runlist+189>: sete   %al
0xf7f75c93 <ntfs_attr_map_runlist+192>: or     %al,%cl
0xf7f75c95 <ntfs_attr_map_runlist+194>: je     0xf7f75ca1
<ntfs_attr_map_runlist+206>
0xf7f75c97 <ntfs_attr_map_runlist+196>: mov    $0x0,%eax
0xf7f75c9c <ntfs_attr_map_runlist+201>: jmp    0xf7f75d65
<ntfs_attr_map_runlist+402>

The same code, compiled with -O0 (plus the other flags above) looks like this:

0xf7ede08e <ntfs_attr_map_runlist+155>: call   0xf7edc730
<ntfs_rl_vcn_to_lcn@plt>
0xf7ede093 <ntfs_attr_map_runlist+160>: mov    %eax,-0x20(%ebp)
0xf7ede096 <ntfs_attr_map_runlist+163>: mov    %edx,-0x1c(%ebp)
0xf7ede099 <ntfs_attr_map_runlist+166>: cmpl   $0x0,-0x1c(%ebp)
0xf7ede09d <ntfs_attr_map_runlist+170>: jns    0xf7ede0c7
<ntfs_attr_map_runlist+212>
0xf7ede09f <ntfs_attr_map_runlist+172>: mov    -0x1c(%ebp),%eax
0xf7ede0a2 <ntfs_attr_map_runlist+175>: mov    %eax,%edx
0xf7ede0a4 <ntfs_attr_map_runlist+177>: xor    $0xffffffff,%edx
0xf7ede0a7 <ntfs_attr_map_runlist+180>: mov    -0x20(%ebp),%eax
0xf7ede0aa <ntfs_attr_map_runlist+183>: xor    $0xffffffff,%eax
0xf7ede0ad <ntfs_attr_map_runlist+186>: or     %edx,%eax
0xf7ede0af <ntfs_attr_map_runlist+188>: test   %eax,%eax
0xf7ede0b1 <ntfs_attr_map_runlist+190>: je     0xf7ede0c7
<ntfs_attr_map_runlist+212>
0xf7ede0b3 <ntfs_attr_map_runlist+192>: mov    -0x1c(%ebp),%eax
0xf7ede0b6 <ntfs_attr_map_runlist+195>: mov    %eax,%edx
0xf7ede0b8 <ntfs_attr_map_runlist+197>: xor    $0xffffffff,%edx
0xf7ede0bb <ntfs_attr_map_runlist+200>: mov    -0x20(%ebp),%eax
0xf7ede0be <ntfs_attr_map_runlist+203>: xor    $0xfffffffd,%eax
0xf7ede0c1 <ntfs_attr_map_runlist+206>: or     %edx,%eax
0xf7ede0c3 <ntfs_attr_map_runlist+208>: test   %eax,%eax
0xf7ede0c5 <ntfs_attr_map_runlist+210>: jne    0xf7ede0d3
<ntfs_attr_map_runlist+224>
0xf7ede0c7 <ntfs_attr_map_runlist+212>: movl   $0x0,-0x2c(%ebp)
0xf7ede0ce <ntfs_attr_map_runlist+219>: jmp    0xf7ede1ac
<ntfs_attr_map_runlist+441>

The testing above was with -O0 and -O1 with several optimisation options
enabled in -O0 and several disabled with -O1, but the behaviour of ntfs-3g did
not change because of using individial optimisation flags behind -O0 and -O1,
only -O0 and -O1 made a difference in the test wether ntfs3-g mounted the
filesystem or not.

To reproduce:

mkfs.ntfs -F test 6000

cat >.gdbinit <<END
set args test /mnt
file ntfs-3g
b main
run
b ntfs_attr_map_runlist
b attrib.c:492
c
END

gdb
Comment 16 Richard Biener 2008-02-12 14:13:49 UTC
Can you attach preprocessed source of libntfs-3g/attrib.c?
Comment 17 Casual J. Programmer 2008-02-12 16:17:05 UTC
Changing the "Request info from" to Bernhard Kaindl, as following the discussion, he would be the one to be able to answer the request. This is way beyond my "casualness" ;-)
Comment 18 Bernhard Kaindl 2008-02-12 19:30:56 UTC
Created attachment 194532 [details]
preprocessed source of libntfs-3g/attrib.c (as asked for)

Created with gcc -DHAVE_CONFIG_H -I.. -I../include/ntfs-3g/ -E libntfs-3g/attrib.c
Comment 19 Richard Biener 2008-02-13 12:20:33 UTC
Err, actually I want preprocessed source of runlist.c ;)
Comment 20 Bernhard Kaindl 2008-02-13 18:48:12 UTC
Created attachment 194750 [details]
preprocessed source of libntfs-3g/runlist.c (as asked for in #19)

cd ntfs-3g-1.2129/libntfs-3g 
gcc -DHAVE_CONFIG_H -I.. -I../include/ntfs-3g/ -E runlist.c >runlist.i
Comment 21 Richard Biener 2008-02-17 10:47:16 UTC
Also die beiden funktionen sehen wunderbar aus - da kann der fehler nicht
liegen.  Auf i386 sehe ich aber

attrib.c: In function ‘ntfs_external_attr_find’:
attrib.c:2072: warning: right shift count >= width of type
attrib.c: In function ‘ntfs_attr_record_move_to’:
attrib.c:3447: warning: left shift count >= width of type

 ctx->al_entry->mft_reference =
  (((MFT_REF)(((MFT_REF)((u16)((u16)(ni->mrec->sequence_number))) << 48) | ((MFT_REF)(ni->mft_no) & 0x0000ffffffffffffULL))));

und MFT_REF ist 

typedef unsigned long int uint64_t;
typedef uint64_t u64;
typedef u64 MFT_REF;

oder ist dein preprocesster source von x86_64?

Can you verify your analysis by just building attrib.c and runlist.c with
-O0 and the rest with regular optimization?
Comment 22 Bernhard Kaindl 2008-02-18 19:43:22 UTC
Created attachment 195524 [details]
attrib.i, generated with -m32

Indeed, I was on 64-bit and forgot to add -m32, and the shift warnings do
not occur on a normal i586 compile.

Here is the new attrib.i, generated with -m32:

gcc -m32 -DHAVE_CONFIG_H -I.. -I../include/ntfs-3g/ -E attrib.c >attrib.i

Compiling only attrib.c with -O0 is sufficient to work around the problem:

%ifarch %{ix86}
touch libntfs-3g/attrib.c
make CFLAGS="$CFLAGS -O0"
%endif

I added -W -Werror to the CFLAGS which I use, and fixed the resulting errors.
The only warning related to attrib.c was:

attrib.c:144: error: passing argument 1 of 'ntfs_log_redirect' discards qualifiers from pointer target type
(at dozens of places)

That was caused by:
#define ntfs_log_critical(FORMAT, ARGS...) ntfs_log_redirect(__FUNCTION__,__FILE__,__LINE__,NTFS_LOG_LEVEL_CRITICAL,NULL,FORMAT,##ARGS)

The passing argument 1 of 'ntfs_log_redirect' is defined as "const char *'.
I changed arg1 of the macro to: "(const char *)__FUNCTION__" and that fixed the warning but attrib.c still seems broken or miscompiled.
Comment 23 Bernhard Kaindl 2008-02-19 23:17:05 UTC
I did a quick check in the hope that there would be an on/off optimizer option
which could turn the optimizer which causes the problem off:

touch libntfs-3g/attrib.c;make CFLAGS="-O1 `gcc --help=optimizers,^joined| grep '  -f'|cut -d' ' -f 3|sed s/f/fno-/|grep -v -e fno-no-threadsafe-statics |tr '\n' ' '`" && ./src/ntfs-3g.probe --readonly test ; echo $?

This compiles attrib.c with -O1 but with all optimizers which do not take a number argument and which apply to C code turned off (-fthreadsafe-statics is C++ only), but the bug is still triggered.

The other way around, at least some data can be gathered:

-frtl-abstract-sequences causes this ICE:

../include/ntfs-3g/attrlist.h:49: error: unrecognizable insn:
(insn 50 0 0 (set (reg:SI 0 ax)
        (symbol_ref:SI ("*.L5") [flags 0x2])) -1 (nil))
../include/ntfs-3g/attrlist.h:49: internal compiler error: in insn_default_length, at insn-attrtab.c:1339

touch libntfs-3g/attrib.c;make CFLAGS='-O1 -fipa-pta' causes this ICE:

attrib.c: In function 'ntfs_attr_lookup':
attrib.c:2285: internal compiler error: in initialize_flags_in_bb, at tree-into-ssa.c:437

Finally, -O0 with nearly all options on except for

> -fipa-pta
> -fpack-struct  (only a problem if only attrib.c is compiled different)
> -frtl-abstract-sequences
> -fwhole-program

which would cause errors just works:

mv opts opts.old;touch libntfs-3g/attrib.c;make CFLAGS="-O0 `gcc --help=optimizers,^joined| grep '  -f'|cut -d' ' -f 3|grep -v -e whole -e rtl -e pack -e fipa-pt|tee opts |tr '\n' ' '`" && ./src/ntfs-3g.probe --readonly test ; echo $?;diff opts.old opts

I submitted a package with the -O0 workaround to stable, but it would be better to have a gcc fix, because ntfsresize and ntfsclone are also affected,

see bug 355151 :

Program received signal SIGSEGV, Segmentation fault.
0xb7f25f58 in ntfs_volume_startup (dev=0x93f10e0, flags=0) at volume.c:403
403             if (vol->mftmirr_na->rl[0].lcn != vol->mftmirr_lcn ||
(gdb) p vol->mftmirr_lcn
$1 = 16
(gdb) p vol->mftmirr_na->rl[0].lcn
Cannot access memory at address 0x8

--------> That is the same data variable as in this bug: lcn

Even the mount issues from kernel NTFS might be releated, just pasting from bug 355151 :

NTFS-fs error (device sdb1): ntfs_read_block(): Failed to read from inode 0x1,
attribute type 0x80, vcn 0x0, offset 0xe00 because its location on disk could
not be determined even after retrying (error code -5).
NTFS-fs error (device sdb1): check_mft_mirror(): Failed to read $MFTMirr.
NTFS-fs error (device sdb1): load_system_files(): $MFTMirr does not match $MFT.

and from this bug, comment 4 :

Jan 16 09:47:26 workstation6l klogd: NTFS-fs error (device sda1):
ntfs_read_block(): Failed to read from inode 0x1, attribute type 0x80, vcn 0x0,
offset 0xe00 because its location on disk could not be determined even after
retrying (error code -5).
Jan 16 09:47:26 workstation6l klogd: NTFS-fs error (device sda1):
check_mft_mirror(): Failed to read $MFTMirr.
Jan 16 09:47:26 workstation6l klogd: NTFS-fs error (device sda1):
load_system_files(): $MFTMirr does not match $MFT.  Mounting read-only.  Run
ntfsfix and/or chkdsk.

AFAIK, for the kernel, we cannot compile single files with -O0 because it needs optimisations from O2 to produce working code (for inline, I think), so we'd probably have to compile the kernel gcc41 if this bug is not fixed before Alpha3.

Other users also report this issue: bug 360566 bug 362568
bug 359936 looks strange bug has 3 duplicates closed on it which clearly fit
to this bug: bug 356195 bug 359667 and bug 360440

I am going to close those NTFS-mount bugs as duplicates of this bug now and rise priority to high.
Comment 24 Michael Matz 2008-02-20 16:36:11 UTC
This is http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35264 
and will be fixed with the next GCC update.
Comment 25 Bernhard Kaindl 2008-02-21 22:41:46 UTC
Packages with the i586-only -O0 workaround for attrib.c are now at:

http://download.opensuse.org/repositories/home:/benkai:/ntfs/openSUSE_Factory/repodata/

The same openSUSE buildservice project also provides packages of the ntfs-3g-2216 for other distributions like 10.3:

http://download.opensuse.org/repositories/home:/benkai:/ntfs/openSUSE_10.3/repodata/

I'd feel confident with closing this bug, but we can also keep it open until we tested with the next GCC update.

I'm lowering Severity and Priority already packages with a workaround are publically available for further testing now and the workaround is now also added in openSUSE factory until the GCC update is in effect.
Comment 26 Chema Ollés 2008-02-25 17:27:17 UTC
Hi.
Since my last system's upgrade yesterday from factory,I can mount my ntfs 
HD partition.
Is it solved?
Regards
Comment 27 Michael Matz 2008-02-25 17:42:27 UTC
You have a fixed gcc43 version if the last changelog entry of it is:
* Mon Feb 18 2008 rguenther@suse.de
- Update to gcc-4_3-branch head (r132521).  [bnc#355496, bnc#354113]

In that case also ntfs-3g was compiled with that version, and it indeed is
fixed.  I'm closing this proactively as fixed, as even if current factory
should not have that gcc43 it will be there sooner or later.
Comment 28 Richard Biener 2008-02-25 17:47:12 UTC
I believe the current ntfs-3g package contains a workaround.  Bernhard, can
you remove that again?
Comment 29 Bernhard Kaindl 2008-02-26 11:08:28 UTC
Of course I will.

Will reassign to me for removal of workaround when fixed gcc is in the Factory.
Comment 30 Bernhard Kaindl 2008-02-26 11:12:18 UTC
Severity = Minor as workaround is in Factory now

Will close when fixed gcc is in Factory and workaround is removed.
Comment 31 Casual J. Programmer 2008-03-04 18:59:15 UTC
Not sure what the state of affairs is here. Currently I don't seem to be able to mount NTFS partitions even with # 5 in place.

NTFS-fs error (device sda1): ntfs_read_block(): Failed to read from inode 0x1, attribute type 0x80, vcn 0x0, offset 0xe00 because its location on disk could not be determined even after retrying (error code -5).
NTFS-fs error (device sda1): check_mft_mirror(): Failed to read $MFTMirr.
NTFS-fs error (device sda1): load_system_files(): $MFTMirr does not match $MFT.  Mounting read-only.  Run ntfsfix and/or chkdsk.
printk: 19 messages suppressed.
NTFS-fs error (device sda1): ntfs_read_block(): Failed to read from inode 0x1, attribute type 0x80, vcn 0x0, offset 0x0 because its location on disk could not be determined even after retrying (error code -5).
NTFS-fs error (device sda1): ntfs_read_block(): Failed to read from inode 0x1, attribute type 0x80, vcn 0x0, offset 0x200 because its location on disk could not be determined even after retrying (error code -5).
NTFS-fs error (device sda1): ntfs_read_block(): Failed to read from inode 0x1, attribute type 0x80, vcn 0x0, offset 0x400 because its location on disk could not be determined even after retrying (error code -5).

Notebook: Fujitsu Siemens Amilo Si 1520
Graphics: Fujitsu Siemens Mobile 945GM/GMS/GME, 943/940GML Express
Monitor:  QUANTADISPLAY LCD Monitor 1280x800@60Hz
Wireless: Intel PRO/Wireless 3945ABG Network Connection
Sound:    82801G (ICH7 Family) High Definition Audio Controller
Desktop:  gnome2-SuSE-10.3-163
YaST GUI: yast2-qt-2.16.31-5
OS:       openSUSE 11.0 (i586) Alpha2 VERSION = 11.0
Kernel:   2.6.24-6-default

rpm -qa | grep ntfs | sort

ntfs-3g-1.913-4
ntfsprogs-2.0.0-14

Comment 32 Hans-Peter Holler 2008-03-04 19:47:54 UTC
Casual,

1st:
please install ntfs-3g-1.2216-5 from Factory.
The changelog of this version shows:
* Do Feb 28 2008 bk@suse.de
- Workaround for build issue with gcc-4.3 (bnc#354113) is obsolete
- Improve treatment of warnings and integrate -Wformat-security

* Mo Feb 18 2008 bk@suse.de
- Update to 1.2216:
  - adds ntfs3-g.probe to check if filesystem is in mountable state
  - fixes "Operation not supported" when deleting huge directories
  - uses specialized integrated libfuse with fixes for suid usage
- Add a temporary workaround to prevent miscompilation with gcc-4.3
- Add a simple functionality check using ntfs-3g.probe to %check
- Added -Wextra -Werror and fixed the resulting compiler errors

A "mount /dev/sda1 <mountpoint> -t ntfs-3g" will return without error.

2nd:
Don't know if "Kernel:   2.6.24-6-default" is a typo. If you're up to date with Factory a "rpm -q kernel-default" should give "kernel-default-2.6.24.1-6"

With this a "mount /dev/sda1 <mountpoint> -t ntfs" (note the type ntfs instead of ntfs-3g) will return without error, too.

Please check.
Hans-Peter
Comment 33 Casual J. Programmer 2008-03-04 22:17:30 UTC
2.6.24-6-default is not a typo, I currently stick with this kernel, as it seems to be the last one to support ipw3945 in parallel with iwlwifi.

I had the most recent ntfs3-g installed, but reading your advice I assume the kernel might be the problem here. So you make me think I am squeezed between "a rock and a hard place" i.e. either I loose wlan or ntfs :-(
Comment 34 Hans-Peter Holler 2008-03-05 19:34:46 UTC
Closing this as resolved/fixed for ntfs-3g.
Kernel-based ntfs is fixed at least with 2.6.24.1-6.
Please open a new bug with for the wlan item.
Comment 35 Jeff Mahoney 2008-03-13 16:46:22 UTC
*** Bug 355151 has been marked as a duplicate of this bug. ***
Comment 36 Jeff Mahoney 2008-03-13 16:46:43 UTC
*** Bug 360566 has been marked as a duplicate of this bug. ***
Comment 37 Jeff Mahoney 2008-04-22 14:14:31 UTC
*** Bug 354903 has been marked as a duplicate of this bug. ***