Bug 1213749

Summary:	fstests btrfs/276 failed
Product:	[SUSE ALP - SUSE Adaptable Linux Platform] Granite	Reporter:	Long An <lan>
Component:	Kernel	Assignee:	Wenruo Qu <wqu>
Status:	IN_PROGRESS ---	QA Contact:
Severity:	Normal
Priority:	P2 - High	CC:	jalausuch, jcheung, rgoldwyn, rtsvetkov, sweiberg, yosun
Version:	unspecified
Target Milestone:	---
Hardware:	Other
OS:	Other
Whiteboard:
Found By:	---	Services Priority:
Business Priority:		Blocker:	---
Marketing QA Status:	---	IT Deployment:	---

Description Long An 2023-07-28 07:46:15 UTC

[description]
Reproducible frame: all
Product version: alp-Dolomite-Build2.4
Regression issue or not: not
http://10.67.133.133/tests/288#step/run/1210

[version]
coreutils-9.3-3.40.x86_64
xfsprogs-6.2.0-4.1.x86_64
e2fsprogs-1.46.5-2.3.x86_64
kernel-default-6.4.1-1.2.x86_64
btrfsprogs-6.1.3-5.1.x86_64
xfstests-20230618.87f90a2d-14.1.x86_64
6.4.1-1-default
Name        : kernel-default
Version     : 6.4.1
Release     : 1.2
Architecture: x86_64
Install Date: Mon Jul 24 22:36:14 2023
Group       : System/Kernel
Size        : 246582709
License     : GPL-2.0-only
Signature   : RSA/SHA256, Mon Jul 24 22:34:20 2023, Key ID fec28eaf09d9ea69
Source RPM  : kernel-default-6.4.1-1.2.nosrc.rpm
Build Date  : Mon Jul 24 22:29:48 2023
Build Host  : goat10
Packager    : http://bugs.opensuse.org
Vendor      : SUSE LINUX Products GmbH, Nuernberg, Germany
URL         : https://www.kernel.org/
Summary     : The Standard Kernel
Description :
The standard kernel for both uniprocessor and multiprocessor systems.


Source Timestamp: 2023-07-07 13:41:37 +0000
GIT Revision: 9531dc6a25eace8fd42efce8d0d18e641da8a7f6
GIT Branch: ALP-current
Distribution: SUSE:ALP:Source:Standard:Core:1.0:Build

[environment]
1 config file for xfstests
export TEST_DEV=/dev/loop0
export TEST_DIR=/mnt/test
export SCRATCH_MNT=/mnt/scratch
export SCRATCH_DEV_POOL="/dev/loop1 /dev/loop2 /dev/loop3 /dev/loop4 /dev/loop5"
export KEEP_DMESG=yes

2. device size and kernel PAGE_SIZE
/dev/loop0    4454M
/dev/loop1    4454M
/dev/loop2    4454M
/dev/loop3    4454M
/dev/loop4    4454M
/dev/loop5    4454M
PAGE_SIZE    4096

[steps to reproduce]
Install xfstests
Prepare 1 TEST_DEV and SCRATCH_DEV_POOL with btrfs
Trigger btrfs/276 and check output

[output]
FSTYP         -- btrfs
PLATFORM      -- Linux/x86_64 localhost 6.4.1-1-default #1 SMP PREEMPT_DYNAMIC Fri Jul  7 13:41:37 UTC 2023 (9531dc6)
MKFS_OPTIONS  -- /dev/loop1
MOUNT_OPTIONS -- -o context=system_u:object_r:root_t:s0 /dev/loop1 /opt/scratch

btrfs/276       QA output created by 276
wrote 17179869184/17179869184 bytes at offset 0
XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
Number of non-shared extents in the whole file: 131082
Create a snapshot of 'SCRATCH_MNT' in 'SCRATCH_MNT/snap'
Number of shared extents in the whole file: 131082
wrote 1048576/1048576 bytes at offset 8388608
XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
wrote 1048576/1048576 bytes at offset 12884901888
XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
Number of non-shared extents in the whole file: 15
Number of shared extents in the whole file: 131067
Number of non-shared extents in range [8M, 9M): 8
Number of non-shared extents in range [12G, 12G + 1M): 7
Delete subvolume (commit): 'SCRATCH_MNT/snap'
Number of non-shared extents in the whole file: 131082
- output mismatch (see /opt/xfstests/results//btrfs/276.out.bad)
    --- tests/btrfs/276.out	2023-07-19 07:24:07.000000000 +0000
    +++ /opt/xfstests/results//btrfs/276.out.bad	2023-07-28 04:15:06.223985372 +0000
    @@ -1,16 +1,16 @@
     QA output created by 276
     wrote 17179869184/17179869184 bytes at offset 0
     XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
    -Number of non-shared extents in the whole file: 131072
    +Number of non-shared extents in the whole file: 131082
     Create a snapshot of 'SCRATCH_MNT' in 'SCRATCH_MNT/snap'
    -Number of shared extents in the whole file: 131072
    ...
    (Run 'diff -u /opt/xfstests/tests/btrfs/276.out /opt/xfstests/results//btrfs/276.out.bad'  to see the entire diff)
Ran: btrfs/276
Failures: btrfs/276
Failed 1 of 1 tests


[what it test]
# Verify that fiemap correctly reports the sharedness of extents for a file with
# a very large number of extents, spanning many b+tree leaves in the fs tree,
# and when the file's subvolume was snapshoted.

Comment 1 Wenruo Qu 2023-08-01 05:46:01 UTC

This seems to be a false alert that the test case doesn't take into consideration.

The test case itself requires 128K compressed extents, but this is done by writing a 16G file with 8M block size.
There is no guarantee that our writeback is going to always create the maximum size of file extents.

If there is any memory pressure (considering the file size is pretty large, it's pretty possible) we can writeback smaller extents and lead to more than expected number of extents.

This seems to be problem in the test case itself, but I failed to reproduce locally with 4G VM RAM.

I can update the test case to handle it more correctly soon.

Comment 2 Wenruo Qu 2023-08-01 05:53:37 UTC

BTW, where can I find the RAM size of the VM?

Comment 3 Long An 2023-08-01 05:58:51 UTC

(In reply to Wenruo Qu from comment #2)
> BTW, where can I find the RAM size of the VM?

http://10.67.133.133/tests/288#settings
In test settings, QEMURAM=4096
And I'll also update test code to print this

Comment 4 Wenruo Qu 2023-08-01 06:12:32 UTC

OK, I also got one hit of the false alert, with the same increased number of extents.

Would be much easier to verify the fix now.

Comment 5 Wenruo Qu 2023-09-26 03:42:29 UTC

Filipe has updated the test case to address the problem, his fix is already merged now: 24cfe625794e ("btrfs/276: make test accurate regarding number of expected extents")

Mind to check if the newer fstests solved the false alerts?