Bugzilla – Bug 1212335
Provide a statically linked basic packaging stack (zypper/libzypp/rpm)
Last modified: 2023-06-19 11:00:58 UTC
Hi all, After fighting off another library dependency issue this week which inadvertently brought down rpm, zypper, and yast, I realized just how important it is for these tools to work no matter if their shared library dependencies are present. So to save everyone a lot of headaches, I propose new rpm-static, zypper-static, and yast2-static packages that would be statically linked and not rely on any shared objects on the system. For more background, here's when it happened before this week: https://www.reddit.com/r/openSUSE/comments/jm2vwj/zypper_broken_symbol_lookup_error_libzyppso1722/. And this week: https://twitter.com/ArtemR/status/1668450501208444928. Many other results: https://www.google.com/search?q=yast+broken+dependency&oq=yast+broken+dependency&aqs=chrome..69i57j33i10i160l2.3183j0j7&sourceid=chrome&ie=UTF-8. Thank you for your consideration.
YaST is not just a binary; it's a framework of scripts based on Ruby, Perl, libyui, libzypp and some hundred files based on those. They all need to work together. There is no way to just roll that into one single binary, even if that wasn't a violation of the LGPL that parts of it use.
What you didn't mention at all (or I must have overlooked it) is how you got into the situations that you describe in the threads that you linked. To get there, you must have read and deliberately overridden a lot of stern warnings that the tools on various levels (rpm, zypper, yast sw_single) gave you. Well, you can do that, but when you do, you are in uncharted territory; you are on your own. You'd better have a backup or a snapshot to get back to in case things go horribly wrong, such as what you experienced. A default SUSE installation proposes a storage setup with Btrfs and snapshots; that is one realistic way out of that. If you don't like that, better start tinkering on that dangerous level using virtual machines that have similar snapshot mechanisms, like VirtualBox or VmWare.
To underline the point of my comment #1: [sh @ balrog] ~ 11 % tree -d /usr/share/YaST2 /usr/share/YaST2 ├── clients ├── control ├── data │ ├── devtools │ │ ├── admin │ │ │ └── aminclude │ │ ├── bin │ │ └── data │ ├── docbook │ │ ├── css │ │ ├── images │ │ │ ├── admon │ │ │ └── navig │ │ └── stylesheets │ ├── languages │ ├── network │ ├── security │ ├── services-manager │ └── testsuite │ └── skel ├── images ├── include │ ├── add-on │ ├── autoinstall │ ├── checkmedia │ ├── hwinfo │ ├── installation │ ├── iscsi-client │ ├── kdump │ ├── ldap │ ├── network │ │ ├── lan │ │ └── services │ ├── nfs │ ├── ntp-client │ ├── online-update-configuration │ ├── packager │ ├── packages │ ├── printer │ ├── proxy │ ├── relocation-server │ ├── samba-client │ ├── samba-server │ ├── scanner │ ├── security │ ├── snapper │ ├── sound │ ├── sysconfig │ ├── timezone │ ├── update │ ├── users │ └── wizard ├── lib │ ├── add-on │ │ └── clients │ ├── autoinstall │ │ ├── ask │ │ ├── autoinst_profile │ │ ├── clients │ │ ├── dialogs │ │ ├── entries │ │ ├── presenters │ │ └── widgets │ │ ├── ask │ │ └── storage │ ├── bootloader │ │ └── autoinst_profile │ ├── cfa │ ├── cwm │ ├── installation │ │ ├── autoinst_issues │ │ ├── autoinst_profile │ │ ├── clients │ │ ├── console │ │ │ └── plugins │ │ ├── dialogs │ │ └── widgets │ ├── kdump │ │ └── clients │ ├── network │ │ └── clients │ ├── online-update-configuration │ ├── packager │ │ ├── cfa │ │ └── clients │ ├── packages │ ├── proxy │ ├── security │ │ └── clients │ ├── services-manager │ │ ├── clients │ │ ├── dialogs │ │ └── widgets │ ├── slp │ │ └── dialogs │ ├── transfer │ ├── tune │ ├── ui │ │ ├── examples │ │ └── wizards │ ├── update │ │ └── clients │ ├── users │ │ ├── clients │ │ ├── dialogs │ │ └── widgets │ ├── widget_demo │ │ └── pages │ ├── y2_alternatives │ │ ├── client │ │ ├── control │ │ └── dialog │ ├── y2country │ │ ├── clients │ │ └── widgets │ ├── y2firewall │ │ ├── autoinst_profile │ │ ├── clients │ │ ├── dialogs │ │ ├── firewalld │ │ │ └── api │ │ ├── helpers │ │ ├── importer_strategies │ │ └── widgets │ │ └── pages │ ├── y2iscsi_client │ ├── y2issues │ ├── y2journal │ ├── y2keyboard │ │ ├── clients │ │ ├── dialogs │ │ └── strategies │ ├── y2network │ │ ├── autoinst │ │ ├── autoinst_profile │ │ ├── backends │ │ ├── clients │ │ ├── config_writers │ │ ├── connection_config │ │ ├── dialogs │ │ ├── interface_config_builders │ │ ├── network_manager │ │ │ └── connection_config_writers │ │ ├── none │ │ ├── presenters │ │ ├── s390_device_activators │ │ ├── sequences │ │ ├── serializer │ │ ├── startmodes │ │ ├── wicked │ │ │ ├── connection_config_readers │ │ │ └── connection_config_writers │ │ └── widgets │ ├── y2nfs_client │ │ └── widgets │ ├── y2ntp_client │ │ ├── client │ │ ├── dialog │ │ └── widgets │ ├── y2packager │ │ ├── clients │ │ ├── dialogs │ │ ├── licenses_fetchers │ │ ├── licenses_handlers │ │ ├── product_spec_readers │ │ ├── release_notes_fetchers │ │ └── widgets │ ├── y2partitioner │ │ ├── actions │ │ │ └── controllers │ │ ├── clients │ │ ├── dialogs │ │ └── widgets │ │ ├── columns │ │ ├── description_section │ │ ├── menus │ │ └── pages │ ├── y2remote │ │ ├── clients │ │ ├── dialogs │ │ ├── modes │ │ └── widgets │ ├── y2security │ │ ├── autoinst │ │ ├── autoinst_profile │ │ ├── clients │ │ ├── lsm │ │ └── security_policies │ ├── y2storage │ │ ├── autoinst_issues │ │ ├── autoinst_profile │ │ ├── boot_requirements_strategies │ │ ├── callbacks │ │ ├── clients │ │ ├── dialogs │ │ │ ├── callbacks │ │ │ └── guided_setup │ │ │ ├── helpers │ │ │ └── widgets │ │ ├── encryption_method │ │ ├── encryption_processes │ │ ├── filesystems │ │ ├── inhibitors │ │ ├── partition_tables │ │ ├── planned │ │ ├── proposal │ │ │ ├── phys_vol_strategies │ │ │ ├── space_maker_actions │ │ │ └── space_maker_prospects │ │ ├── refinements │ │ └── widgets │ ├── y2users │ │ ├── autoinst │ │ ├── autoinst_profile │ │ ├── clients │ │ ├── linux │ │ ├── parsers │ │ └── users_module │ └── yast2 │ ├── clients │ ├── refinements │ └── systemd ├── locale │ ├── ar │ │ └── LC_MESSAGES │ ├── de │ │ └── LC_MESSAGES │ └── he │ └── LC_MESSAGES ├── modules │ └── YaPI ├── schema │ ├── autoyast │ │ ├── rnc │ │ └── rng │ └── control │ └── rnc ├── scrconf └── theme ├── current │ ├── animations │ ├── wmconfig │ └── worldmap ├── Leap │ ├── animations │ ├── wizard │ ├── wmconfig │ └── worldmap └── SLE ├── animations ├── wizard ├── wmconfig └── worldmap 253 directories [sh @ balrog] ~ 12 % find /usr/share/YaST2 -type f | wc -l 2990
Stefan spent a lot of time explaining what is not possible. Let's talk about what is needed, and what is possible: We don't actually need YaST if a broken YaST can be fixed via zypper. So let's talk about the plain packaging stack. I sometimes found myself in similar situations, and found a lack of a packaging stack in the rescue system. The rescue system does not have zypper to operate on the broken root from the outside, and the binary in the chroot can't be used. The installer itself does though, and I believe this is what I did. zypp maintainers, what do you think about the idea of deploying a rescue zypp stack into the rescue system?
Tools in the rescue system are barely, if ever, tested, and historically accumulated a lot of bugs. It's basically dead code. So why not do the reasonable thing and use snapshots, backups and VMs instead? What problem are we trying to fix here?
(In reply to Andreas Stieger from comment #4) > I sometimes found myself in similar situations, and found a lack of a > packaging stack in the rescue system. The rescue system does not have zypper > to operate on the broken root from the outside, and the binary in the chroot > can't be used. That's why we are using btrfs snapshots by default, you only need to boot into an older snapshot where zypper is still working. And we have MicroOS and variants with transactional-update, where this should not happen.
(In reply to Andreas Stieger from comment #4) > The installer itself does though, and I believe this is what I did. zypp > maintainers, what do you think about the idea of deploying a rescue zypp > stack into the rescue system? That's up to the team maintaining the rescue system. But basically Stefan is right, if it's not maintained, tested and updated, all kinds of bugs may hide inside. If you maintain the system with a too old version of the stack, you may easily damage it. And in extreme, with a too new version as well. Just think about relocation of the rpm database, changes in the rpm database layout, or changing .rpm default compression formats. And these are just the pitfalls on .rpm level. If the rescue system provides a full stack it is expected to operate safely with any underlying system it accepts. For complex tools this is not easy to judge. Frankly, if I had to maintain it, I would think twice before offering the full stack.
OP here. I wanted to provide some answers to the comments above. 1. Why not use snapshots and roll back? Well, depending on the system, rolling back is not always easy and could result in data loss. For example, the systems this happened on are Elasticsearch nodes, and restoring a node to an earlier backup will likely confuse the cluster and that'd be a whole other can of worms I'd rather not open. Furthermore, we are using ext4, not btrfs, so snapshots are not readily available. Regardless, even if this solution were available, it would be much more time-consuming and risky compared to having a tool that isn't reliant on shared libraries. For example, while trying to fix glibc on one of the machines, I copied a version from another machine, but it immediately made pretty much everything unusable due to a shared object error - including ls, cp, mv, rm, etc. Someone suggested a solution - Busybox https://busybox.net. Busybox has statically compiled replacement utilities and had I had it installed, it would have made getting out of this mess much simpler. I didn't, so I had to boot into rescue mode to swap the libc so back in. This is just an example of how valuable statically compiled tools could be. 2. I can appreciate how yast is a collection of tools and scripts and has a lot of dependencies, so it'd be hard to make a statically compiled version. What about zypper and at the very least rpm? Not having a working rpm made it really had to install packages that could potentially fix the issue, and not having zypper removed my ability to do so with conflict and dependency resolution. I would be very happy with just static zypper and rpm. 3. "It's your own fault, you must have screwed something up to arrive to a broken situation." It's a plausible scenario, but I've been running OpenSUSE in production for 13 years and feel pretty comfortable around it. Yast and zypper have failed me in the past, despite my efforts to be careful, and it's not always my fault. For example, https://www.reddit.com/r/openSUSE/comments/jm2vwj/zypper_broken_symbol_lookup_error_libzyppso1722/. The issue there was: zypper: symbol lookup error: /usr/lib64/libzypp.so.1722: undefined symbol: _ZN4YAML6detail9node_data12empty_scalarB5cxx11E Was this my fault? No, something broke upstream and OpenSUSE released updates that broke yast/zypper: https://bugzilla.opensuse.org/show_bug.cgi?id=1178332#c24 https://www.suse.com/support/kb/doc/?id=000020725 As for the issue from this week, it was probably some sort of a race condition and I can't fully explain it, but what happened was I had glibc and some other gcc/cpp/libc related dependencies installed from a gcc:next repo (something like https://download.opensuse.org/repositories/devel:/gcc:/next/), which I at some point realized was pushing out unstable glibc releases. This repo also one day disappeared for 15.4, the maintainers decided to delete it without warning, I guess, leaving me with unstable releases that were no longer updating. So I tried going back to glibc 2.31 from the regular stable repos, accepted related downgrading for several other dependencies from yast, and then something failed in that process (don't remember exactly what, but it was reproducible and happened on 2 machines I tried it on), and in the end I was left with broken zypper/rpm https://i.imgur.com/OWShyii.png. Was it my fault? Once again I don't think so, as I followed zypper/yast's own self-resolution steps, which then failed along the way. -------- So with these in mind, with the common end goal of making OpenSUSE better no matter whose fault it is breaking zypper/rpm, can we refocus the scope of this ticket to zypper/rpm and forget about yast for now? Are those tools still doable as static? Thank you.
Please remember that Btrfs snapshots are just ONE solution to that problem. If you don't like that, there is also the time-honored concept of backups. Even good old "rsync" goes a long way. And please don't insinuate that it's a common thing to break basic system tools like rpm or zypper. That never happened to me even once during all the time that I've been using openSUSE / SuSE / S.u.S.E. (since about 1997).
Artem, your report is actually a feature request. So, please, if you feel it's important also for others, file the feature request at the right place https://code.opensuse.org/leap/features/issues Thank you!