Bugzilla – Bug 720150
rpm: free space calculation is botched
Last modified: 2021-07-24 20:21:33 UTC
Installing glibc-locale in my factory VM just failed, as rpm-4.9.1-29.1 thinks it has no space left: linux:/var/cache/zypp/packages/openSUSE-12.1-12.1-0/suse/x86_64 # rpm -ihv glibc-locale-2.14-12.3.x86_64.rpm Preparing... ########################################### [100%] installing package glibc-locale-2.14-12.3.x86_64 needs 71MB on the / filesystem linux:/var/cache/zypp/packages/openSUSE-12.1-12.1-0/suse/x86_64 # df / Filesystem 1K-blocks Used Available Use% Mounted on /dev/sda2 948408 464256 435976 52% / The funny thing: the disk space rpm wants increases linearly with the amount of used bytes on the disk: linux:/var/cache/zypp/packages/openSUSE-12.1-12.1-0/suse/x86_64 # dd if=/dev/zero of=test bs=1M count=20 20+0 records in 20+0 records out 20971520 bytes (21 MB) copied, 0.0156471 s, 1.3 GB/s linux:/var/cache/zypp/packages/openSUSE-12.1-12.1-0/suse/x86_64 # rpm -ihv glibc-locale-2.14-12.3.x86_64.rpm Preparing... ########################################### [100%] installing package glibc-locale-2.14-12.3.x86_64 needs 91MB on the / filesystem linux:/var/cache/zypp/packages/openSUSE-12.1-12.1-0/suse/x86_64 # dd if=/dev/zero of=test bs=1M count=40 40+0 records in 40+0 records out 41943040 bytes (42 MB) copied, 0.0359535 s, 1.2 GB/s linux:/var/cache/zypp/packages/openSUSE-12.1-12.1-0/suse/x86_64 # rpm -ihv glibc-locale-2.14-12.3.x86_64.rpm Preparing... ########################################### [100%] installing package glibc-locale-2.14-12.3.x86_64 needs 111MB on the / filesystem linux:/var/cache/zypp/packages/openSUSE-12.1-12.1-0/suse/x86_64 # dd if=/dev/zero of=test bs=1M count=80 80+0 records in 80+0 records out 83886080 bytes (84 MB) copied, 0.0763985 s, 1.1 GB/s linux:/var/cache/zypp/packages/openSUSE-12.1-12.1-0/suse/x86_64 # rpm -ihv glibc-locale-2.14-12.3.x86_64.rpm Preparing... ########################################### [100%] installing package glibc-locale-2.14-12.3.x86_64 needs 151MB on the / filesystem linux:/var/cache/zypp/packages/openSUSE-12.1-12.1-0/suse/x86_64 # dd if=/dev/zero of=test bs=1M count=800 dd: writing `test': No space left on device 473+0 records in 472+0 records out 495747072 bytes (496 MB) copied, 0.813001 s, 610 MB/s linux:/var/cache/zypp/packages/openSUSE-12.1-12.1-0/suse/x86_64 # rpm -ihv glibc-locale-2.14-12.3.x86_64.rpm Preparing... ########################################### [100%] installing package glibc-locale-2.14-12.3.x86_64 needs 496MB on the / filesystem
Please consider fixing this. The following packages are going to be upgraded: xen 4.1.2_05-1.1.1 -> 4.1.2_16-1.7.1 x86_64 update openSUSE xen-libs 4.1.2_05-1.1.1 -> 4.1.2_16-1.7.1 x86_64 update openSUSE xen-tools 4.1.2_05-1.1.1 -> 4.1.2_16-1.7.1 x86_64 update openSUSE 3 packages to upgrade. Overall download size: 13.1 MiB. After the operation, additional 695.0 KiB will be used. Continue? [y/n/?] (y): Retrieving package xen-4.1.2_16-1.7.1.x86_64 (1/3), 7.5 MiB (27.5 MiB unpacked) Retrieving: xen-4.1.2_16-1.7.1.x86_64.rpm [done (6.2 MiB/s)] Retrieving package xen-libs-4.1.2_16-1.7.1.x86_64 (2/3), 376.0 KiB (1000.0 KiB unpacked) Retrieving: xen-libs-4.1.2_16-1.7.1.x86_64.rpm [done (0 B/s)] Retrieving package xen-tools-4.1.2_16-1.7.1.x86_64 (3/3), 5.2 MiB (12.2 MiB unpacked) Retrieving: xen-tools-4.1.2_16-1.7.1.x86_64.rpm [done] Installing: xen-4.1.2_16-1.7.1 [error] Installation of xen-4.1.2_16-1.7.1 failed: (with --nodeps --force) Error: Subprocess failed. Error: RPM failed: installing package xen-4.1.2_16-1.7.1.x86_64 needs 4MB on the /boot filesystem Abort, retry, ignore? [a/r/i] (a): ^Z [1]+ Stopped zypper up xen'*' 06:10 xen35:~ # df Filesystem 1K-blocks Used Available Use% Mounted on rootfs 139737100 72592820 60051880 55% / devtmpfs 5879976 76 5879900 1% /dev tmpfs 6097252 6588 6090664 1% /dev/shm tmpfs 6097252 1456 6095796 1% /run /dev/sda3 139737100 72592820 60051880 55% / tmpfs 6097252 0 6097252 0% /sys/fs/cgroup tmpfs 6097252 1456 6095796 1% /var/lock tmpfs 6097252 1456 6095796 1% /var/run tmpfs 6097252 0 6097252 0% /media /dev/sda1 85528 57053 24059 71% /boot 24 MB > 4 MB, by the latest laws of mathematics.
*** Bug 777390 has been marked as a duplicate of this bug. ***
This annoyance bites me routinely in TW. Please fix it.
I found this while looking at the rpm's header using python3-rpm: > python3 Python 3.4.3 (default, Mar 27 2015, 02:30:53) [GCC] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import os, rpm >>> ts = rpm.TransactionSet() >>> hdr = ts.hdrFromFdno(os.open("glibc-locale-2.22-3.1.x86_64.rpm", os.O_RDONLY)) >>> a = 0 >>> for i in hdr[rpm.RPMTAG_FILESIZES]: ... a += max(4096, i) <-- RPM uses the fs block size as min. file size, for all types of files ... >>> str(int(a/1024/1024))+" MiB" '485 MiB' <-- What RPM assumes as package size (and complains about) >>> str(int(hdr[rpm.RPMTAG_SIZE]/1024/1024))+" MiB" '115 MiB' <-- What Zypper uses as package size So there is a inconsistency in the RPM package itself, probably caused by hardlinks(?) counted multiple times. The increasing size demand is a rather simple issue, bug 947835.
Indeed, the issue is that rpm does not treat hardlinks in a special way when calculating the size. In case of glibc-locale, there are some inodes with more than 100 links, so RPMs size calculation is (NLINKS-1)*FILESIZE off, causing the discrepancy. This is even more extreme on NFS, where the (apparent) blocksize is much larger (>1 MiB) and each hardlink accounts for 1 block. making RPM believe that glibc-locale is almost 7 GiB big. (I have a workaround for that, I'm probably going to submit that soon) The question is now, how should this be fixed correctly?
foreach file in filelist ++seen{file} if seen{file} == 1 spaceneeded += file.size like that? In light that NFS may not report a useful block size, I think the rouding up to block size should be dropped altogether, also because filesystems may do different fancy stuff, including but not limited to, transparent compression.
I just ran into this again on TW host fi965. Glibc-locale will not install, because something thinks more space is required than is available on this EXT4 filesystem: sda19 4843161 total blocks 4175485 91% blocks used 417788 1K-blocks available Kernel-default 4.3 installs, but dracut fails due to insufficient freespace after kernel extraction knocks freespace down to 4% free. Maybe it's not actually rpm's fault. I thought I could compact directories by rsync -av the whole filesystem to an identically formatted empty filesystem, then back after deleting or reformatting, but this is what happened to the temporary target: sda26 4843161 total blocks 4636185 100% blocks used 0 available
Well if dracut fails, it's hardly rpm's fault.
This is preventing TW 20160303 from fully updating on host gx620. Download size of glibc-locale is 6.1MiB, requiring 115.9MiB unpacked. Freespace on / is >480M, yet zypper claims this package requires 46MiB on the / filesystem in order to install, so the rpm process for glibc-locale fails.
Note that (part of) this was also reported upstream, and the answer was something like: """ "installing package glibc-locale-2.14-12.3.x86_64 needs 71MB on the / filesystem" should indeed have read "installing package glibc-blah needs the / filesystem to have an extra 71MB free (than it already does now)" """
(In reply to Jan Engelhardt from comment #10) > Note that (part of) this was also reported upstream... Please provide URL of that report.
Happening on another TW host (a-865): The following package is going to be upgraded: glibc-locale 1 package to upgrade. Overall download size: 6.2 MiB. Already cached: 0 B. No additional space will be used or freed after the operation. Continue? [y/n/? shows all options] (y): y Checking for file conflicts: ............................................................................[done] Warning: Checking for file conflicts requires not installed packages to be downloaded in advance in order to access their file lists. See option '--download-in-advance' in the zypper manual page for details. The following package had to be excluded from file conflicts check because it is not yet downloaded: glibc-locale Retrieving package glibc-locale-2.24-2.2.i586 (1/1), 6.2 MiB (122.5 MiB unpacked) Retrieving: glibc-locale-2.24-2.2.i586.rpm ................................................[done (978.6 KiB/s)] (1/1) Installing: glibc-locale-2.24-2.2.i586 ...........................................................[error] Installation of glibc-locale-2.24-2.2.i586 failed: Error: Subprocess failed. Error: RPM failed: installing package glibc-locale-2.24-2.2.i586 needs 178MB on the / filesystem Abort, retry, ignore? [a/r/i] (a): # df / Filesystem 1K-blocks Used Available Use% Mounted on /dev/sda21 4843161 4238296 359073 93% / # blkid /dev/sda21 /dev/sda21: LABEL="os133SS25" UUID="f80d697e-9a56-4c94-9d33-a5b2dbde1d03" TYPE="ext3" PTTYPE="dos" PARTUUID="df5ee105-15"
On x86_64 TW host hpg33, attempt to upgrade glibc-locale from 2.24-2.2 to 6.2MiB 2.24-2.3 package failed because 408MB of freespace on EXT4 / filesystem was insufficient, while RPM subprocess reported needing 122MB of freespace. Uninstalling oldest kernel raised freespace to 622MB and allowed glibc-locale to install.
*** Bug 1011022 has been marked as a duplicate of this bug. ***
This has apparently gotten worse. After freeing up space on / on host phg33, each subsequent attempt to update glibc-locale results in a larger reported shortfall than previously. If I don't free up space, but merely retry, the requirement nevertheless increases every time. The installed size of the glibc-locale package only about 22% of the total current freespace on the / filesystem. # installing package glibc-locale-2.25-2.3.x86_64 needs 48KB on the / filesystem # installing package glibc-locale-2.25-2.3.x86_64 needs 49KB on the / filesystem # installing package glibc-locale-2.25-2.3.x86_64 needs 50KB on the / filesystem # installing package glibc-locale-2.25-2.3.x86_64 needs 52KB on the / filesystem # installing package glibc-locale-2.25-2.3.x86_64 needs 54KB on the / filesystem # installing package glibc-locale-2.25-2.3.x86_64 needs 56KB on the / filesystem # installing package glibc-locale-2.25-2.3.x86_64 needs 57KB on the / filesystem # installing package glibc-locale-2.25-2.3.x86_64 needs 58KB on the / filesystem # installing package glibc-locale-2.25-2.3.x86_64 needs 60KB on the / filesystem # installing package glibc-locale-2.25-2.3.x86_64 needs 61KB on the / filesystem # installing package glibc-locale-2.25-2.3.x86_64 needs 63KB on the / filesystem # installing package glibc-locale-2.25-2.3.x86_64 needs 397KB on the / filesystem # installing package glibc-locale-2.25-2.3.x86_64 needs 962KB on the / filesystem # installing package glibc-locale-2.22-1.1.x86_64 needs 2MB on the / filesystem I temporarily moved /boot/vmlinu* and /boot/initrd* off / and got glibc-locale to install, which changed the / filesystem freespace from 10% to 9% of total 5.6G.
I'm also routinely hitting this issue when trying to update a VM with 8GiB / and "only" around 300MB of free space (glibc-locale is now ancient there).
Host gx780 with 9% of 5655093 total 1K blocks free still cannot upgrade glibc-locale because its 122.3MiB unpacked cannot fit into the 4X as large 514691 1K blocks of available / freespace.
On my host hpg33 TW/KDE5 installation on blocksize 1024 EXT4, df / showed 93% in use. I manually deleted the content of: /usr/share/locale /usr/lib64/gconv /usr/lib/locale This decreased space used to 91%. Then I did 'zypper -v in -f rpm glibc-locale', which increased space in use to 94%, leaving 350,119 blocks available. I repeated the manual deletion, which decreased space used to 91%, leaving 492,661 blocks available. Then I repeated 'zypper -v in -f rpm glibc-locale', which increased space in use to 94%, leaving 349,665 blocks available, or 454 fewer blocks available than resulted from the previous iteration. Given the high frequency of glibc-locale updates, this free blocks reduction pattern seems to explain the steady disappearance of freespace, and need to periodically format / and install fresh to prevent inability to update glibc-locale for insufficiency of freespace.
Fixed by rpm commit 5584438180cb21af73aad3ee8ab4f9e83161af83 (4.16).
(In reply to Jan Engelhardt from comment #19) > Fixed by rpm commit 5584438180cb21af73aad3ee8ab4f9e83161af83 (4.16). No link. :( Where can I see this commit, or better yet, a bug report that gave birth to it?
https://github.com/rpm-software-management/rpm/commit/5584438180cb21af73aad3ee8ab4f9e83161af83