Bug 720150 - rpm: free space calculation is botched
rpm: free space calculation is botched
Status: RESOLVED UPSTREAM
: 777390 1011022 (view as bug list)
Classification: openSUSE
Product: openSUSE Tumbleweed
Classification: openSUSE
Component: Basesystem
Current
x86-64 Linux
: P5 - None : Minor (vote)
: ---
Assigned To: Michael Schröder
E-mail List
:
Depends on:
Blocks: 1188684
  Show dependency treegraph
 
Reported: 2011-09-23 19:33 UTC by Jan Engelhardt
Modified: 2021-07-24 20:21 UTC (History)
6 users (show)

See Also:
Found By: Beta-Customer
Services Priority:
Business Priority:
Blocker: ---
Marketing QA Status: ---
IT Deployment: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Jan Engelhardt 2011-09-23 19:33:52 UTC
Installing glibc-locale in my factory VM just failed, as rpm-4.9.1-29.1 thinks it has no space left:

linux:/var/cache/zypp/packages/openSUSE-12.1-12.1-0/suse/x86_64 # rpm -ihv glibc-locale-2.14-12.3.x86_64.rpm 
Preparing...                ########################################### [100%]
        installing package glibc-locale-2.14-12.3.x86_64 needs 71MB on the / filesystem
linux:/var/cache/zypp/packages/openSUSE-12.1-12.1-0/suse/x86_64 # df /
Filesystem     1K-blocks   Used Available Use% Mounted on
/dev/sda2         948408 464256    435976  52% /

The funny thing: the disk space rpm wants increases linearly with the amount of used bytes on the disk:

linux:/var/cache/zypp/packages/openSUSE-12.1-12.1-0/suse/x86_64 # dd if=/dev/zero of=test bs=1M count=20
20+0 records in
20+0 records out
20971520 bytes (21 MB) copied, 0.0156471 s, 1.3 GB/s
linux:/var/cache/zypp/packages/openSUSE-12.1-12.1-0/suse/x86_64 # rpm -ihv glibc-locale-2.14-12.3.x86_64.rpm 
Preparing...                ########################################### [100%]
        installing package glibc-locale-2.14-12.3.x86_64 needs 91MB on the / filesystem

linux:/var/cache/zypp/packages/openSUSE-12.1-12.1-0/suse/x86_64 # dd if=/dev/zero of=test bs=1M count=40
40+0 records in
40+0 records out
41943040 bytes (42 MB) copied, 0.0359535 s, 1.2 GB/s
linux:/var/cache/zypp/packages/openSUSE-12.1-12.1-0/suse/x86_64 # rpm -ihv glibc-locale-2.14-12.3.x86_64.rpm 
Preparing...                ########################################### [100%]
        installing package glibc-locale-2.14-12.3.x86_64 needs 111MB on the / filesystem

linux:/var/cache/zypp/packages/openSUSE-12.1-12.1-0/suse/x86_64 # dd if=/dev/zero of=test bs=1M count=80
80+0 records in
80+0 records out
83886080 bytes (84 MB) copied, 0.0763985 s, 1.1 GB/s
linux:/var/cache/zypp/packages/openSUSE-12.1-12.1-0/suse/x86_64 # rpm -ihv glibc-locale-2.14-12.3.x86_64.rpm 
Preparing...                ########################################### [100%]
        installing package glibc-locale-2.14-12.3.x86_64 needs 151MB on the / filesystem
linux:/var/cache/zypp/packages/openSUSE-12.1-12.1-0/suse/x86_64 # dd if=/dev/zero of=test bs=1M count=800
dd: writing `test': No space left on device
473+0 records in
472+0 records out
495747072 bytes (496 MB) copied, 0.813001 s, 610 MB/s
linux:/var/cache/zypp/packages/openSUSE-12.1-12.1-0/suse/x86_64 # rpm -ihv glibc-locale-2.14-12.3.x86_64.rpm 
Preparing...                ########################################### [100%]
        installing package glibc-locale-2.14-12.3.x86_64 needs 496MB on the / filesystem
Comment 1 Jan Engelhardt 2012-05-10 04:15:03 UTC
Please consider fixing this.



The following packages are going to be upgraded:
  xen        4.1.2_05-1.1.1 -> 4.1.2_16-1.7.1  x86_64  update  openSUSE
  xen-libs   4.1.2_05-1.1.1 -> 4.1.2_16-1.7.1  x86_64  update  openSUSE
  xen-tools  4.1.2_05-1.1.1 -> 4.1.2_16-1.7.1  x86_64  update  openSUSE


3 packages to upgrade.
Overall download size: 13.1 MiB. After the operation, additional 695.0 KiB will 
be used.
Continue? [y/n/?] (y): 
Retrieving package xen-4.1.2_16-1.7.1.x86_64 (1/3), 7.5 MiB (27.5 MiB unpacked)
Retrieving: xen-4.1.2_16-1.7.1.x86_64.rpm [done (6.2 MiB/s)]
Retrieving package xen-libs-4.1.2_16-1.7.1.x86_64 (2/3), 376.0 KiB (1000.0 KiB unpacked)
Retrieving: xen-libs-4.1.2_16-1.7.1.x86_64.rpm [done (0 B/s)]
Retrieving package xen-tools-4.1.2_16-1.7.1.x86_64 (3/3), 5.2 MiB (12.2 MiB unpacked)
Retrieving: xen-tools-4.1.2_16-1.7.1.x86_64.rpm [done]
Installing: xen-4.1.2_16-1.7.1 [error]
Installation of xen-4.1.2_16-1.7.1 failed:
(with --nodeps --force) Error: Subprocess failed. Error: RPM failed:    installing package xen-4.1.2_16-1.7.1.x86_64 needs 4MB on the /boot filesystem


Abort, retry, ignore? [a/r/i] (a): ^Z
[1]+  Stopped                 zypper up xen'*'
06:10 xen35:~ # df
Filesystem     1K-blocks     Used Available Use% Mounted on
rootfs         139737100 72592820  60051880  55% /
devtmpfs         5879976       76   5879900   1% /dev
tmpfs            6097252     6588   6090664   1% /dev/shm
tmpfs            6097252     1456   6095796   1% /run
/dev/sda3      139737100 72592820  60051880  55% /
tmpfs            6097252        0   6097252   0% /sys/fs/cgroup
tmpfs            6097252     1456   6095796   1% /var/lock
tmpfs            6097252     1456   6095796   1% /var/run
tmpfs            6097252        0   6097252   0% /media
/dev/sda1          85528    57053     24059  71% /boot


24 MB > 4 MB, by the latest laws of mathematics.
Comment 2 Jan Engelhardt 2012-09-17 03:18:57 UTC
*** Bug 777390 has been marked as a duplicate of this bug. ***
Comment 3 Felix Miata 2015-10-03 09:35:56 UTC
This annoyance bites me routinely in TW. Please fix it.
Comment 4 Fabian Vogt 2015-10-05 13:13:39 UTC
I  found this while looking at the rpm's header using python3-rpm:

> python3
Python 3.4.3 (default, Mar 27 2015, 02:30:53) [GCC] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import os, rpm
>>> ts = rpm.TransactionSet()
>>> hdr = ts.hdrFromFdno(os.open("glibc-locale-2.22-3.1.x86_64.rpm", os.O_RDONLY))
>>> a = 0
>>> for i in hdr[rpm.RPMTAG_FILESIZES]:
...     a += max(4096, i) <-- RPM uses the fs block size as min. file size, for all types of files
... 
>>> str(int(a/1024/1024))+" MiB"
'485 MiB' <-- What RPM assumes as package size (and complains about)
>>> str(int(hdr[rpm.RPMTAG_SIZE]/1024/1024))+" MiB"
'115 MiB' <-- What Zypper uses as package size

So there is a inconsistency in the RPM package itself, probably caused by hardlinks(?) counted multiple times.

The increasing size demand is a rather simple issue, bug 947835.
Comment 5 Fabian Vogt 2015-10-05 16:16:53 UTC
Indeed, the issue is that rpm does not treat hardlinks in a special way when calculating the size.
In case of glibc-locale, there are some inodes with more than 100 links,
so RPMs size calculation is (NLINKS-1)*FILESIZE off, causing the discrepancy.
This is even more extreme on NFS, where the (apparent) blocksize is much larger (>1 MiB)
and each hardlink accounts for 1 block. making RPM believe that glibc-locale is almost 7 GiB big. (I have a workaround for that, I'm probably going to submit that soon)

The question is now, how should this be fixed correctly?
Comment 6 Jan Engelhardt 2015-10-05 17:58:47 UTC
foreach file in filelist
  ++seen{file}
  if seen{file} == 1
    spaceneeded += file.size

like that? In light that NFS may not report a useful block size, I think the rouding up to block size should be dropped altogether, also because filesystems may do different fancy stuff, including but not limited to, transparent compression.
Comment 7 Felix Miata 2015-11-20 05:35:28 UTC
I just ran into this again on TW host fi965. Glibc-locale will not install, because something thinks more space is required than is available on this EXT4 filesystem:

sda19
4843161 total blocks
4175485 91% blocks used
 417788 1K-blocks available

Kernel-default 4.3 installs, but dracut fails due to insufficient freespace after kernel extraction knocks freespace down to 4% free.

Maybe it's not actually rpm's fault. I thought I could compact directories by rsync -av the whole filesystem to an identically formatted empty filesystem, then back after deleting or reformatting, but this is what happened to the temporary target:
sda26
4843161 total blocks
4636185 100% blocks used
0 available
Comment 8 Jan Engelhardt 2015-11-20 09:57:11 UTC
Well if dracut fails, it's hardly rpm's fault.
Comment 9 Felix Miata 2016-03-05 08:04:54 UTC
This is preventing TW 20160303 from fully updating on host gx620. Download size of glibc-locale is 6.1MiB, requiring 115.9MiB unpacked. Freespace on / is >480M, yet zypper claims this package requires 46MiB on the / filesystem in order to install, so the rpm process for glibc-locale fails.
Comment 10 Jan Engelhardt 2016-03-05 09:21:00 UTC
Note that (part of) this was also reported upstream, and the answer was something like:

"""
"installing package glibc-locale-2.14-12.3.x86_64 needs 71MB on the / filesystem"
should indeed have read
"installing package glibc-blah needs the / filesystem to have an extra 71MB free (than it already does now)"
"""
Comment 11 Felix Miata 2016-03-05 09:57:10 UTC
(In reply to Jan Engelhardt from comment #10)
> Note that (part of) this was also reported upstream...

Please provide URL of that report.
Comment 12 Felix Miata 2016-12-03 10:14:56 UTC
Happening on another TW host (a-865):
The following package is going to be upgraded:
  glibc-locale

1 package to upgrade.
Overall download size: 6.2 MiB. Already cached: 0 B. No additional space will be used or freed after the
operation.
Continue? [y/n/? shows all options] (y): y
Checking for file conflicts: ............................................................................[done]
Warning: Checking for file conflicts requires not installed packages to be downloaded in advance in order to access their file lists. See option '--download-in-advance' in the zypper manual page for details.

The following package had to be excluded from file conflicts check because it is not yet downloaded:
  glibc-locale

Retrieving package glibc-locale-2.24-2.2.i586                             (1/1),   6.2 MiB (122.5 MiB unpacked)
Retrieving: glibc-locale-2.24-2.2.i586.rpm ................................................[done (978.6 KiB/s)]
(1/1) Installing: glibc-locale-2.24-2.2.i586 ...........................................................[error]
Installation of glibc-locale-2.24-2.2.i586 failed:
Error: Subprocess failed. Error: RPM failed:    installing package glibc-locale-2.24-2.2.i586 needs 178MB on the / filesystem

Abort, retry, ignore? [a/r/i] (a):     

# df /
Filesystem     1K-blocks    Used Available Use% Mounted on
/dev/sda21       4843161 4238296    359073  93% /
# blkid /dev/sda21
/dev/sda21: LABEL="os133SS25" UUID="f80d697e-9a56-4c94-9d33-a5b2dbde1d03" TYPE="ext3" PTTYPE="dos" PARTUUID="df5ee105-15"
Comment 13 Felix Miata 2017-01-27 09:42:36 UTC
On x86_64 TW host hpg33, attempt to upgrade glibc-locale from 2.24-2.2 to 6.2MiB 2.24-2.3 package failed because 408MB of freespace on EXT4 / filesystem was insufficient, while RPM subprocess reported needing 122MB of freespace. Uninstalling oldest kernel raised freespace to 622MB and allowed glibc-locale to install.
Comment 14 Michael Schröder 2017-05-11 15:23:31 UTC
*** Bug 1011022 has been marked as a duplicate of this bug. ***
Comment 15 Felix Miata 2017-06-19 08:28:40 UTC
This has apparently gotten worse. After freeing up space on / on host phg33, each subsequent attempt to update glibc-locale results in a larger reported shortfall than previously. If I don't free up space, but merely retry, the requirement nevertheless increases every time. The installed size of the glibc-locale package only about 22% of the total current freespace on the / filesystem.
#       installing package glibc-locale-2.25-2.3.x86_64 needs 48KB on the / filesystem
#       installing package glibc-locale-2.25-2.3.x86_64 needs 49KB on the / filesystem
#       installing package glibc-locale-2.25-2.3.x86_64 needs 50KB on the / filesystem
#       installing package glibc-locale-2.25-2.3.x86_64 needs 52KB on the / filesystem
#       installing package glibc-locale-2.25-2.3.x86_64 needs 54KB on the / filesystem
#       installing package glibc-locale-2.25-2.3.x86_64 needs 56KB on the / filesystem
#       installing package glibc-locale-2.25-2.3.x86_64 needs 57KB on the / filesystem
#       installing package glibc-locale-2.25-2.3.x86_64 needs 58KB on the / filesystem
#       installing package glibc-locale-2.25-2.3.x86_64 needs 60KB on the / filesystem
#       installing package glibc-locale-2.25-2.3.x86_64 needs 61KB on the / filesystem
#       installing package glibc-locale-2.25-2.3.x86_64 needs 63KB on the / filesystem
#       installing package glibc-locale-2.25-2.3.x86_64 needs 397KB on the / filesystem
#       installing package glibc-locale-2.25-2.3.x86_64 needs 962KB on the / filesystem
#       installing package glibc-locale-2.22-1.1.x86_64 needs 2MB on the / filesystem
I temporarily moved /boot/vmlinu* and /boot/initrd* off / and got glibc-locale to install, which changed the / filesystem freespace from 10% to 9% of total 5.6G.
Comment 16 Richard Biener 2017-09-14 08:52:26 UTC
I'm also routinely hitting this issue when trying to update a VM with 8GiB / and "only" around 300MB of free space (glibc-locale is now ancient there).
Comment 17 Felix Miata 2017-12-19 18:38:43 UTC
Host gx780 with 9% of 5655093 total 1K blocks free still cannot upgrade glibc-locale because its 122.3MiB unpacked cannot fit into the 4X as large 514691 1K blocks of available / freespace.
Comment 18 Felix Miata 2018-06-10 10:37:09 UTC
On my host hpg33 TW/KDE5 installation on blocksize 1024 EXT4, df / showed 93% in use. I manually deleted the content of:

	/usr/share/locale
	/usr/lib64/gconv
	/usr/lib/locale

This decreased space used to 91%.

Then I did 'zypper -v in -f rpm glibc-locale', 

which increased space in use to 94%, leaving 350,119 blocks available.

I repeated the manual deletion,

which decreased space used to 91%, leaving 492,661 blocks available.

Then I repeated 'zypper -v in -f rpm glibc-locale',

which increased space in use to 94%, leaving 349,665 blocks available, or

454 fewer blocks available than resulted from the previous iteration.

Given the high frequency of glibc-locale updates, this free blocks reduction pattern seems to explain the steady disappearance of freespace, and need to periodically format / and install fresh to prevent inability to update glibc-locale for insufficiency of freespace.
Comment 19 Jan Engelhardt 2020-10-09 17:30:01 UTC
Fixed by rpm commit 5584438180cb21af73aad3ee8ab4f9e83161af83 (4.16).
Comment 20 Felix Miata 2020-10-09 17:57:12 UTC
(In reply to Jan Engelhardt from comment #19)
> Fixed by rpm commit 5584438180cb21af73aad3ee8ab4f9e83161af83 (4.16).

No link. :( Where can I see this commit, or better yet, a bug report that gave birth to it?