|
Bugzilla – Full Text Bug Listing |
| Summary: | After Upgrade, at least some libs have zero bytes. | ||
|---|---|---|---|
| Product: | [openSUSE] openSUSE Distribution | Reporter: | Carlos Robinson <carlos.e.r> |
| Component: | Upgrade Problems | Assignee: | E-mail List <screening-team-bugs> |
| Status: | NEW --- | QA Contact: | Jiri Srain <jsrain> |
| Severity: | Normal | ||
| Priority: | P5 - None | CC: | carlos.e.r |
| Version: | Leap 15.5 | ||
| Target Milestone: | --- | ||
| Hardware: | Other | ||
| OS: | Other | ||
| Whiteboard: | |||
| Found By: | --- | Services Priority: | |
| Business Priority: | Blocker: | --- | |
| Marketing QA Status: | --- | IT Deployment: | --- |
| Attachments: |
zypper log
history file The detailed repository list that was active The filtered package list. All belongs to 15.5 terminal log zypper log |
||
(In reply to Carlos Robinson from comment #0) > After some routine update Which one exactly? Give a list. How was this update performed? Provide reliable steps to reproduce > [...] > I don't know how many more are bad or correct. I think I would have to > compare against the rpm database. Why don't you go ahead and do that. > I managed to overwrite those files with the corresponding ones from another > computer. The machine may be too borked to be useful in reproducing this issue, as your local changes will interfere. > I did not expect this problem to happen in the middle of a > minor update, it has worked fine with openSUSE 15.3 and 15.4, and 15.5 till > a week or two ago. On TW, maybe, but not on Leap < 16. I am not sure what you are trying to say here. Provide reliable steps to reproduce this issue. is your disk full perhaps? or any other disk corruption? if its ext3/4 can you run fsck?
you can force override installation with "zypper in -f packagename"
Use
rpm -qf /usr/lib64/libappstream.so.0.15.6
to find out which RPM it is.
rpm -Va
would show all invalid rpms too.
(In reply to Marcus Meissner from comment #2) > is your disk full perhaps? or any other disk corruption? if its ext3/4 can > you run fsck? No, disk is not full, there is ample margin. fsck? Ok, I will try that when I go back to that location, will take a day or two. But I thought that the boot process checked ext4 partitions. > you can force override installation with "zypper in -f packagename" > > Use > rpm -qf /usr/lib64/libappstream.so.0.15.6 > > to find out which RPM it is. I did that already. The file remained zero bytes. Maybe I can find something in the log about that single package update, but I noticed nothing interesting in the terminal. > > rpm -Va > > would show all invalid rpms too. Yes, that is my plan, to concoct a script using that to find all zero bytes files. There could be more trouble if some of those are supposed to be linked to another location in /lib, not sure how to find out if that is the case. (In reply to Andreas Stieger from comment #1) > (In reply to Carlos Robinson from comment #0) > > After some routine update > > Which one exactly? Give a list. If you wish, I can extract the update from the journal. The machine is at a remote location and I have to go there for any job. The failed update was done just before I left the location on June 14th, and I found the problem when I came back yesterday. > How was this update performed? Provide reliable steps to reproduce I am not at the location now, but yesterday I did: rpm -qf /usr/lib64/libappstream.so.0.15.6 Whatever.rpm (1) zypper install --force Whatever the result was a zero bytes file /usr/lib64/libappstream.so.0.15.6 I can repeat the process under trace. Please just tell me the exact command to use that will write your desired detailed log or trace and I will do it - unless the package is updated at the repository in the interval. (1) At my current location I don't have that file, I can't find the corresponding rpm here. > > > [...] > > I don't know how many more are bad or correct. I think I would have to > > compare against the rpm database. > > Why don't you go ahead and do that. Because I don't know, yet, how exactly to run that comparison to find zero byte files. I had other matters to attend to while I was at the location, I could not dedicate enough time to design a procedure. > > > I managed to overwrite those files with the corresponding ones from another > > computer. > > The machine may be too borked to be useful in reproducing this issue, as > your local changes will interfere. Both machines run 15.5; if a particular rpm is installed on both, the files will be the same. The number of files I replaced is minimal, and I have a log, which I posted. The ask was not to show how you see the problem of empty files, but how to recreate the problem on a blank system. (In reply to Andreas Stieger from comment #5) > The ask was not to show how you see the problem of empty files, but how to > recreate the problem on a blank system. How the heck am I to know? I tell you that I can get a zypper trace that proves how it happens and allows you to correct the problem. But I need you to tell me how to do that, and what logs you need. Right now, "zypper install --force certainpackage" produces, reliably, some empty files in my system. Is that not enough? Just tell me how to obtain a trace and I will give you proof. I have the suspicion, but only a suspicion, that if you install and update a 15.5 system in which /usr is a separate partition, it will break on some rpms. Can you try that? It will take me days to prepare and run that test. Does not trigger here as described. Continue to provide reliable steps to reproduce. Not just repeating the current situation, but all the steps. If you have a suspicion that a separate /usr triggers this you should run that test. I repeat that I do not have steps to reproduce. I can only run commands you tell me, and supply the results and logs. Alright, let us know if this changes at any time in the future. I booted from the XFCE rescue image on USB stick, forcd fsck of both "/" and "/usr". Nothing found. I have forced an update of every single package in the system, using yast single, repository tab, select system tab, then right click all on this list update unconditionally; the libraries I knew were zero are restored, but I don't know about all. reboot. However, there are still problems. The applications menu file is still empty. I reinstall it, and it remains zero bytes. rpm command, before and after, says the file is different size. How can this happen? Elesar:/home/cer # rpm -q --verify libgarcon-branding-openSUSE S.5...... c /etc/xdg/menus/xfce-applications.menu Elesar:/home/cer # zypper install --force libgarcon-branding-openSUSE System management is locked by the application with pid 6458 (/usr/bin/ruby.ruby2.5). Close this application before trying again. Elesar:/home/cer # zypper install --force libgarcon-branding-openSUSE Error building the cache: [X11_Pantheon_Apps|https://download.opensuse.org/repositories/X11:/Pantheon:/Apps/openSUSE_Leap_15.5/] Valid metadata not found at specified URL History: - [X11_Pantheon_Apps|https://download.opensuse.org/repositories/X11:/Pantheon:/Apps/openSUSE_Leap_15.5/] Repository type can't be determined. Warning: Skipping repository 'X11:Pantheon:Apps' because of the above error. Some of the repositories have not been refreshed because of an error. Loading repository data... Reading installed packages... Forcing installation of 'libgarcon-branding-openSUSE-4.18.0+git0.9a2f754-lp155.1.6.noarch' from repository 'openSUSE-Leap-15.5-1'. Resolving package dependencies... The following package is going to be reinstalled: libgarcon-branding-openSUSE 1 package to reinstall. Overall download size: 23.9 KiB. Already cached: 0 B. No additional space will be used or freed after the operation. Continue? [y/n/v/...? shows all options] (y): Retrieving: libgarcon-branding-openSUSE-4.18.0+git0.9a2f754-lp155.1.6.noarch (openSUSE-Leap-15.5-1) (1/1), 23.9 KiB Checking for file conflicts: ........................................................................................................[done] (1/1) Installing: libgarcon-branding-openSUSE-4.18.0+git0.9a2f754-lp155.1.6.noarch ..................................................[done] Elesar:/home/cer # rpm -q --verify libgarcon-branding-openSUSE S.5...... c /etc/xdg/menus/xfce-applications.menu Elesar:/home/cer # l /etc/xdg/menus/xfce-applications.menu -rw-r--r-- 1 root root 0 May 22 12:35 /etc/xdg/menus/xfce-applications.menu Elesar:/home/cer # ls -l /etc/xdg/menus/xfce-applications.menu -rw-r--r-- 1 root root 0 May 22 12:35 /etc/xdg/menus/xfce-applications.menu Elesar:/home/cer # I can not tell you how to reproduce this problem. I tried to reproduce on a virtual machine, and failed. But it does happen reliably in my machine. I have logs, and I am this minute running a ltrace. Created attachment 868080 [details] zypper log zypper log resulting of doing: Elesar:~/tmp # l /etc/xdg/menus/xfce-applications.menu -rw-r--r-- 1 root root 0 May 22 12:35 /etc/xdg/menus/xfce-applications.menu Elesar:~/tmp # rpm -qf /etc/xdg/menus/xfce-applications.menu libgarcon-branding-openSUSE-4.18.0+git0.9a2f754-lp155.1.6.noarch Elesar:~/tmp # rpm -qf --verify /etc/xdg/menus/xfce-applications.menu S.5...... c /etc/xdg/menus/xfce-applications.menu Elesar:~/tmp # date --iso=ns ; ltrace -f -o zypper_ltrace_libgarcon-branding-openSUSE -tt -S zypper --non-interactive install --force libgarcon-branding-openSUSE ; date --iso=ns 2023-07-06T22:38:32,407457434+02:00 Error building the cache: [X11_Pantheon_Apps|https://download.opensuse.org/repositories/X11:/Pantheon:/Apps/openSUSE_Leap_15.5/] Valid metadata not found at specified URL History: - [X11_Pantheon_Apps|https://download.opensuse.org/repositories/X11:/Pantheon:/Apps/openSUSE_Leap_15.5/] Repository type can't be determined. Warning: Skipping repository 'X11:Pantheon:Apps' because of the above error. Some of the repositories have not been refreshed because of an error. Loading repository data... Reading installed packages... Forcing installation of 'libgarcon-branding-openSUSE-4.18.0+git0.9a2f754-lp155.1.6.noarch' from repository 'openSUSE-Leap-15.5-1'. Resolving package dependencies... The following package is going to be reinstalled: libgarcon-branding-openSUSE 1 package to reinstall. Overall download size: 23.9 KiB. Already cached: 0 B. No additional space will be used or freed after the operation. Continue? [y/n/v/...? shows all options] (y): y Retrieving: libgarcon-branding-openSUSE-4.18.0+git0.9a2f754-lp155.1.6.noarch (openSUSE-Leap-15.5-1) (1/1), 23.9 KiB Checking for file conflicts: ................................................................................................................................................................[done] (1/1) Installing: libgarcon-branding-openSUSE-4.18.0+git0.9a2f754-lp155.1.6.noarch ..........................................................................................................[done] 2023-07-06T23:20:55,245602354+02:00 Elesar:~/tmp # Elesar:~/tmp # l /etc/xdg/menus/xfce-applications.menu -rw-r--r-- 1 root root 0 May 22 12:35 /etc/xdg/menus/xfce-applications.menu Elesar:~/tmp # rpm -qf /etc/xdg/menus/xfce-applications.menu libgarcon-branding-openSUSE-4.18.0+git0.9a2f754-lp155.1.6.noarch Elesar:~/tmp # rpm -qf --verify /etc/xdg/menus/xfce-applications.menu S.5...... c /etc/xdg/menus/xfce-applications.menu Elesar:~/tmp # The string "xfce-applications.menu" doesn't appear in the log, and something is "killed by signal 15 (Terminated)" at the end. I tried twice at least. As you can see, the file remains zero bytes and I don't have the menu in my XFCE desktop, any user. The ltrace is at your disposal. -rw-r--r-- 1 root root 111M Jul 6 23:20 zypper_ltrace_libgarcon-branding-openSUSE.xz Tell me where I can upload it. If not, perhaps I can share it on google drive. (In reply to Carlos Robinson from comment #0) > After some routine update, some libraries finished having files of zero > bytes and things broke: Checking my log, it was not a routine update. It was the update from 15.4 to 15.5 (DVD method). Sorry, it was half a month ago, I thought there were other updates in the middle. 2023-06-14T19:02:29 15.4 2023-06-14T21:54:27 15.5 But at the time, I did not notice a non working menu. History file attached. Created attachment 868081 [details]
history file
(In reply to Carlos Robinson from comment #10) > I can not tell you how to reproduce this problem. I tried to reproduce on a > virtual machine, and failed. Keep trying. Don't just keep pointing to your machine. (In reply to Carlos Robinson from comment #11) > Tell me where I can upload it. If not, perhaps I can share it on google > drive. The internet I guess. (In reply to Carlos Robinson from comment #12) > Checking my log, it was not a routine update. It was the update from 15.4 to > 15.5 (DVD method). Sorry, it was half a month ago, I thought there were > other updates in the middle. Right, why report it then if you don't know? Please update the bug summary to reflect that. But did you look at the zypper.log file? You keep ignoring that. Memory can fail, but not logs. Your logs tell that you have a completely messed up repository list. Remove invalid Leap 15.3/15.4, offline repos and Pantheon repos. Uninstall all packages from this no longer existing repos and vendor switch to official openSUSE Leap 15.5 repos. Do a clean dup to Leap 15.5 Only after this steps are done it makes sense to dig deeper.... Created attachment 868104 [details] The detailed repository list that was active (In reply to hui from comment #16) > Your logs tell that you have a completely messed up repository list. Remove > invalid Leap 15.3/15.4, offline repos and Pantheon repos. Uninstall all > packages from this no longer existing repos and vendor switch to official > openSUSE Leap 15.5 repos. Do a clean dup to Leap 15.5 > > Only after this steps are done it makes sense to dig deeper.... I was so surprised by your finding, that I have just made the 30 km to the location to find this out. Unfortunately, you are wrong. There are no 15.3 repositories in the list of repos, which is attached. You may have been confused by the existence of a plainder repo still named "dir:/root/LocalRPMs_15.3". That repo contains only this: Elesar:~ # l LocalRPMs_15.3/ total 416 drwxr-xr-x 2 root root 4096 Mar 30 2022 ./ drwx------ 34 root root 4096 Jul 8 20:15 ../ -rw-r--r-- 1 root root 175960 Mar 1 2022 multiload-ng-743885d.git-lp153.1.1.x86_64.rpm -rw-r--r-- 1 root root 148904 Mar 4 2022 multiload-ng-base-743885d.git-lp153.3.1.x86_64.rpm -rw-r--r-- 1 root root 89928 Mar 4 2022 multiload-ng-xfce4-743885d.git-lp153.3.1.x86_64.rpm Elesar:~ # which surely has no effect on things, but don't worry, I'm replacing that single package with the 15.5 one which is back on the distro (it was gone on 15.3), and deactivating the directory repo. About the Pantheon repo you need not worry, as it is not found. It has no effect. Created attachment 868105 [details]
The filtered package list. All belongs to 15.5
I run this detailed query to find out what installed packages were not for 15.5. Result: none, except the proprietary Chrome, and old PGP keys.
rpm -q -a --queryformat "%{INSTALLTIME}\t%{INSTALLTIME:day} \
%{BUILDTIME:day} %-30{NAME}\t%15{VERSION}-%-7{RELEASE}\t%{arch} \
%25{VENDOR}%25{PACKAGER} == %{DISTRIBUTION} %{DISTTAG}\n" \
| sort | cut --fields="2-" | tee rpmlist \
| egrep -v "openSUSE Leap 15\.5|openSUSE_Leap_15.5|\-lp153|SUSE Linux Enterprise 15|openSUSE\ Leap\ 15.5" > /tmp/rpmlist.txt
As you can see, all kosher.
Created attachment 868106 [details]
terminal log
My problem remains the same: zypper installs a zero byte /etc/xdg/menus/xfce-applications.menu file, everytime I try.
Elesar:~/tmp # ls -l /etc/xdg/menus/xfce-applications.menu
-rw-r--r-- 1 root root 0 May 22 12:35 /etc/xdg/menus/xfce-applications.menu
Elesar:~/tmp # rpm -q --verify libgarcon-branding-openSUSE
S.5...... c /etc/xdg/menus/xfce-applications.menu
Elesar:~/tmp #
Created attachment 868107 [details]
zypper log
Zypper log of attempt to install the package, again.
I will try to upload the 111 MB share (2 GB expanded) ltrace when I get back home, as my internet here on location is limited. Oh, in case you ask, zypper dup says "nothing to do". file zypper_ltrace_libgarcon-branding-openSUSE_2.xz, resulting of command: ltrace -f -o zypper_ltrace_libgarcon-branding-openSUSE_2 -tt -S zypper --non-interactive install --force libgarcon-branding-openSUSE https://drive.google.com/file/d/187slY-WveHtepI7cFXEscBozcyW5oqtD/view?usp=sharing |
After some routine update, some libraries finished having files of zero bytes and things broke: cer@Elesar:~> su - Password: flatpak: error while loading shared libraries: /usr/lib64/libappstream.so.4: file too short Elesar:~ # flatpak? What flatpak? I know nothing of such a thing. :-o file too short? Indeed! cer@Elesar:~> l /usr/lib64/libappstream.so.4 lrwxrwxrwx 1 root root 22 Mar 10 15:11 /usr/lib64/libappstream.so.4 -> libappstream.so.0.15.6* cer@Elesar:~> l /usr/lib64/libappstream.so.0.15.6 -rwxr-xr-x 1 root root 0 Mar 10 15:11 /usr/lib64/libappstream.so.0.15.6* cer@Elesar:~> Also, the contents of the Whisker menu (xfce) are totally gone. I only see "Favorites", "recently used", and "All applications", but empty. Look: Elesar:~ # ls -lS /usr/lib64/ | tail -25 lrwxrwxrwx 1 root root 11 Jul 28 2022 libelf.so -> libelf.so.1 lrwxrwxrwx 1 root root 11 Jun 28 2022 libowcrypt.so -> libcrypt.so lrwxrwxrwx 1 root root 11 Jun 28 2022 libxcrypt.so -> libcrypt.so lrwxrwxrwx 1 root root 10 May 2 2022 libss.so -> libss.so.2 lrwxrwxrwx 1 root root 9 Mar 6 19:07 python -> python2.7 -rw-r--r-- 1 root root 0 May 17 15:23 .libgcrypt.so.20.fips -rwxr-xr-x 1 root root 0 Mar 10 15:11 libappstream.so.0.15.6 -rwxr-xr-x 1 root root 0 May 22 12:00 libcfitsio.so.9.4.1.0 -rwxr-xr-x 1 root root 0 May 22 12:05 libgstplay-1.0.so.0.2200.0 -rwxr-xr-x 1 root root 0 Mar 10 14:09 libgstriff-1.0.so.0.2200.0 -rwxr-xr-x 1 root root 0 Mar 10 14:09 libgstrtp-1.0.so.0.2200.0 -rwxr-xr-x 1 root root 0 Mar 10 14:09 libgstrtsp-1.0.so.0.2200.0 -rwxr-xr-x 1 root root 0 May 22 12:05 libgstva-1.0.so.0.2200.0 -rwxr-xr-x 1 root root 0 May 22 11:52 libgstvalidate-1.0.so.0.2200.0 -rwxr-xr-x 1 root root 0 May 22 11:52 libgstvalidate-default-overrides-1.0.so.0.2200.0 -rwxr-xr-x 1 root root 0 May 22 12:05 libgstvulkan-1.0.so.0.2200.0 -rwxr-xr-x 1 root root 0 May 22 12:05 libgstwayland-1.0.so.0.2200.0 -rwxr-xr-x 1 root root 0 May 8 2022 libgupnp-dlna-2.0.so.4.0.0 -rwxr-xr-x 1 root root 0 May 8 2022 libgupnp-dlna-gst-2.0.so.4.0.0 -rwxr-xr-x 1 root root 0 Jan 25 22:55 libpoppler-cpp.so.0.11.0 -rwxr-xr-x 1 root root 0 Jan 25 22:55 libpoppler.so.126.0.0 -rwxr-xr-x 1 root root 0 May 22 11:56 libshotwell-authenticator.so.0.30.16 -rwxr-xr-x 1 root root 0 May 22 11:56 libshotwell-plugin-common.so.0.30.16 -rwxr-xr-x 1 root root 0 May 22 11:56 libshotwell-plugin-dev-1.0.so.0.30.16 -rwxr-xr-x 1 root root 0 Dec 2 2022 libva-glx.so.2.1600.0 Elesar:~ # I managed to overwrite those files with the corresponding ones from another computer. Then I wrote a script to copy all that I could find: !/bin/bash echo > I_dont_have_it.log echo > I_have_it.log echo > I_have_it_but_is_zero.log echo > zerousrfiles.log while read FILES ; do OTHER=/root/fusermount$FILES if test -f "$FILES" -a -f "$OTHER" ; then if test -s "$OTHER" ; then echo $FILES >> I_have_it.log cp "$OTHER" "$FILES" >> zerousrfiles.log else echo $FILES >> I_have_it_but_is_zero.log fi else echo $FILES >> I_dont_have_it.log fi done < list_of_zero_size_files_in_usr which managed to repair these: /usr/share/locale/hi/LC_MESSAGES/thunar-volman.mo /usr/share/man/man1/Xwayland.1.gz /usr/share/icons/hicolor/scalable/status/org.xfce.notification.unread-emblem-symbolic.svg /usr/share/licenses/libpoppler126/COPYING /usr/share/licenses/libpoppler126/COPYING3 /usr/share/licenses/libgupnp-dlna-2_0-4/COPYING /usr/share/licenses/libappstream4/AUTHORS /usr/share/licenses/libappstream4/COPYING /usr/share/doc/packages/xfce4-notifyd/README.md /usr/share/doc/packages/libpoppler126/NEWS /usr/share/doc/packages/libpoppler126/README.md /usr/share/doc/packages/libpoppler126/README-XPDF /usr/share/doc/packages/libgupnp-dlna-2_0-4/AUTHORS /usr/share/doc/packages/libgupnp-dlna-2_0-4/TODO /usr/lib64/gallium-pipe/pipe_i915.so /usr/lib64/gupnp-dlna/libgstreamer.so I don't know how many more are bad or correct. I think I would have to compare against the rpm database. My suspicion is that some updates goofed because this machine has a separate /usr partition. I did not expect this problem to happen in the middle of a minor update, it has worked fine with openSUSE 15.3 and 15.4, and 15.5 till a week or two ago. On TW, maybe, but not on Leap < 16.