Bugzilla – Bug 858233
Patch openSUSE-2013-1034 breaks xfs_growfs
Last modified: 2015-02-19 05:34:58 UTC
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1650.63 Safari/537.36 Recently I migrated all my filesystems to XFS. This works just fine, but I found I could not grow a filesystem while using kernel 3.7.10-1.24.1-desktop. After downgrading to 3.7.10-1.16-desktop xfs_growfs works. Symtoms while using kernel 3.7.10-1.24.1-desktop: xfs_growfs error message: xfs_growfs: XFS_IOC_FSGROWFSDATA xfsctl failed: Cannot allocate memory Message in system log: Jan 07 12:55:58 linux-zvf1.ka.haagnet.net kernel: XFS (dm-5): _xfs_buf_find: Block out of range: block 0x4000001, EOFS 0x4000000 Reproducible: Always Steps to Reproduce: 1.create a logical volume 2.create an xfs filesystem on the volume 3.enlarge the logical volume 4.run xfs_growfs on the filesystem Actual Results: xfs_growfs: XFS_IOC_FSGROWFSDATA xfsctl failed: Cannot allocate memory error in syslog: Jan 07 12:55:58 linux-zvf1.ka.haagnet.net kernel: XFS (dm-5): _xfs_buf_find: Block out of range: block 0x4000001, EOFS 0x4000000 Expected Results: Enlarged filesystem No errors Works with kernel 3.7.10-1.16-desktop
Same problem here, same kernel. corresponding entries in syslog: 2014-01-24T15:37:28.888724+01:00 alpha kernel: [1121931.525327] XFS (dm-7): _xfs_buf_find: Block out of range: block 0xc580001, EOFS 0x8200000 2014-01-24T15:37:28.888758+01:00 alpha kernel: [1121931.525336] XFS (dm-7): _xfs_buf_find: Block out of range: block 0xc580001, EOFS 0x8200000
I had to reboot into the old 3.7.10-1.16-default kernel to grow the xfs filesystem. A fix for this problem is highly appreciated!
Unfortunately also the kernel 3.7.10-1.28.1 compiled on Tue Feb 04, still contains this bug. Still stuck with 3.7.10-1.16.
Does openSUSE 13.1 work for you instead?
(In reply to comment #4) > Does openSUSE 13.1 work for you instead? Sorry, I don't have 13.1 yet. If you can't reproduce the error using the steps described in the bug report it seems like it's fixed. The XFS version on 12.3 is: xfs-1.1.2-5.1.1.x86_64 (in rpm -qi).
openSUSE 13.1 is at the moment not an option due to other issues, but the xfs_growfs works on 13.1. This xfs-"Bug" was also seen in SLES11-SP2 and SLES11-SP3 kernels but got fixed. SR# 10858518821
The package mentioned by wouter koersen "xfs-1.1.2-5.1.1" is not related to the xfs-filesystem. The bug is also not affected by the xfsprogs package, it is just kernel-version related.
OK, since it's not seen in the recent systems, let's close the bug. Please reopen if you still encounter the same problem again. Thanks.
The problem does still exist in openSUSE 12.3. It was never fixed. To summarize again: - kernel 3.7.10-1.16 works - kernel 3.7.10-1.28 does not work Since it seems to be fixed in SLES11-SP2 and SLES11-SP3 kernels, I would be very interested to see what was the patch there? I have reopened the bug again.
The current situation is not a solution. To prevent the problem only the workaround to downgrade the kernel to version 3.7.10-1.16 gives a system which is "maintainable". Every time installing updates the kernel and the corresponding packages need to be downgraded manually.
OK, thanks for updating. Looking at the log, this seems like the result of the security fix commit eb178619f930fa2ba2348de332a1ff1c66a31424 xfs: fix _xfs_buf_find oops on blocks beyond the filesystem end The upstream 3.7.x reverted this patch. commit c3793e0d94af2071c6f3842dcdb5ea08bd011354 Author: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Date: Thu Feb 14 11:22:53 2013 -0800 Revert: xfs: fix _xfs_buf_find oops on blocks beyond the filesystem end This reverts commit a56040731e5b00081c6d6c26b99e6e257a5d63d7 which was commit eb178619f930fa2ba2348de332a1ff1c66a31424 upstream. It has been reported to cause problems: http://bugzilla.redhat.com/show_bug.cgi?id=909602 So, we should do it, too. I reverted the affecting patch now in kernel git openSUSE-12.3 branch. Please try OBS Kernel:openSUSE-12.3 repo later. If the problem still persists with the newer kernel with the commit 60c70ed216, please reopen. Thanks.
original security bug 858233
I would not remove the patch! Revert: xfs: fix _xfs_buf_find oops on blocks beyond the filesystem end it opens you up to NULL pointer dereferences on huge block numbers. xfs_growfs() cannot call xfs_buf_find(). In earlier kernels it did via xfs_buf_get(). The commit, fd23683c, in v3.7-rc1-42-gfd23683 changed those to direct reads.
(In reply to comment #13) > I would not remove the patch! > > Revert: xfs: fix _xfs_buf_find oops on blocks beyond the filesystem end > > it opens you up to NULL pointer dereferences on huge block numbers. > > xfs_growfs() cannot call xfs_buf_find(). In earlier kernels it did via > xfs_buf_get(). The commit, fd23683c, in v3.7-rc1-42-gfd23683 changed those to > direct reads. You mean 3.8-rc1, right? If it were 3.7-rc1, it should have been already fixed :) I backported that commit and the previous one to apply cleanly, and revert the revert. The test kernel packages are being built in OBS home:tiwai:bnc858233 repo. Please test whether this fixes the issue. Once when confirmed, I'll push this to git tree.
Yes, my mistake, I was using "git describe" on the patch from a OSS kernel and it does not always match up with the real releases. I did look on the linux-3.7-stable version on kernel.org and the commit "fd23683c" had not yet made it into Linux-3.7-stable, so yes this patch would be first seen in Linux-3.8. You can check to see which patches were used is SLES11SP3, but these look appropriate: author Dave Chinner <dchinner@redhat.com> 2012-04-23 05:58:55 (GMT) committer Ben Myers <bpm@sgi.com> 2012-05-14 21:20:51 (GMT) commit 7ca790a507a9288ebedab90a8e40b9afa8e4e949 (patch) tree 70ad7655655e3f5e021b3c80f5c102f4399bbd68 /fs/xfs/xfs_fsops.c parent a8acad70731e7d0585f25f33f8a009176f001f70 (diff) xfs: kill xfs_read_buf() xfs_read_buf() is effectively the same as xfs_trans_read_buf() when called outside a transaction context. The error handling is slightly different in that xfs_read_buf stales the errored buffer it gets back, but there is probably good reason for xfs_trans_read_buf() for doing this. Hence update xfs_trans_read_buf() to the same error handling as xfs_read_buf(), and convert all the callers of xfs_read_buf() to use the former function. We can then remove xfs_read_buf(). Signed-off-by: Dave Chinner <dchinner@redhat.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Mark Tinguely <tinguely@sgi.com> Signed-off-by: Ben Myers <bpm@sgi.com> author Dave Chinner <dchinner@redhat.com> 2012-11-12 11:53:59 (GMT) committer Ben Myers <bpm@sgi.com> 2012-11-13 22:40:43 (GMT) commit fd23683c3b1ab905cba61ea2981c156f4bf52845 (patch) tree 1ea38537f14d0ff537ff15a97cedd244a18890e9 /fs/xfs/xfs_fsops.c parent b64f3a390d3477517cbff7d613e551705540769b (diff) xfs: growfs: use uncached buffers for new headers When writing the new AG headers to disk, we can't attach write verifiers because they have a dependency on the struct xfs-perag being attached to the buffer to be fully initialised and growfs can't fully initialise them until later in the process. The simplest way to avoid this problem is to use uncached buffers for writing the new headers. These buffers don't have the xfs-perag attached to them, so it's simple to detect in the write verifier and be able to skip the checks that need the xfs-perag. This enables us to attach the appropriate buffer ops to the buffer and hence calculate CRCs on the way to disk. IT also means that the buffer is torn down immediately, and so the first access to the AG headers will re-read the header from disk and perform full verification of the buffer. This way we also can catch corruptions due to problems that went undetected in growfs. Signed-off-by: Dave Chinner <dchinner@redhat.com> Reviewed-by Rich Johnston <rjohnston@sgi.com> Signed-off-by: Ben Myers <bpm@sgi.com>
Thanks. As mentioned in comment 18, I backported the commit fd23683c3b. But this patch requires another clean up patch b64f3a390d3 (xfs: use btree block initialisation functions in growfs), so I took it, too, instead of taking SP3 one. Unmodified patches are much easier to maintain in the case like this (between 3.7 and 3.8). Let's see whether the test package works.
I'm running a kernel from the http://download.opensuse.org/repositories/home:/tiwai:/bnc858233/standard/ repo and xfs_growfs works for me. Linux linux-zvf1.ka.haagnet.net 3.7.10-1-desktop #1 SMP PREEMPT Thu Mar 20 07:12:37 UTC 2014 (95b5819) x86_64 x86_64 x86_64 GNU/Linux.
Thanks for checking. Now the commits are merged to git tree openSUSE-12.3 branch. You can use the fixed kernel from OBS Kernel:openSUSE-12.3 repo later.
Thanks for fixing this bug. I'm impressed by the bug fixing process.
Just to confirm. The problem is fixed in kernel 3.7.10-95.1.g60c70ed from Kernel:openSUSE-12.3: * Wed Mar 19 2014 tiwai@suse.de - Delete patches.fixes/xfs-fix-xfs_buf_find-oops-on-blocks-beyond-the-filesystem-end. It turned out that this patch causes regressions (bnc#858233) The upstream 3.7.x also reverted it in the end (commit c3793e0d94af2). - commit 60c70ed I'm looking forward to see it in update/12.3. Thanks!
Note that there are other commits again making this patch back but with more proper fixes. It was committed today, so the package should be available in tomorrow or later.
Bug still fixed with kernel 3.7.10-97.gd1d38c4 from Kernel:openSUSE-12.3: * Thu Mar 20 2014 tiwai@suse.de - xfs: growfs: use uncached buffers for new headers (bnc#858233). - xfs: use btree block initialisation functions in growfs (bnc#858233). - commit 4ade8a9 Thanks!
Thanks for confirmation!
openSUSE-SU-2014:0677-1: An update that solves 16 vulnerabilities and has 10 fixes is now available. Category: security (important) Bug References: 733022,811746,833968,837111,851426,852652,852967,858233,858638,858869,858870,858872,860835,862145,863335,864025,866102,868653,869414,869898,871148,871252,871325,873717,875690,875798 CVE References: CVE-2013-4254,CVE-2013-4579,CVE-2013-6885,CVE-2014-0101,CVE-2014-0196,CVE-2014-0691,CVE-2014-1438,CVE-2014-1444,CVE-2014-1445,CVE-2014-1446,CVE-2014-1690,CVE-2014-1737,CVE-2014-1738,CVE-2014-1874,CVE-2014-2523,CVE-2014-2672 Sources used: openSUSE 12.3 (src): kernel-docs-3.7.10-1.32.2, kernel-source-3.7.10-1.32.1, kernel-syms-3.7.10-1.32.1