Bug 293428

Media handling is currently in progress, libzypp doesn't support multi-media repositories.

Will handling be ready for 10.3 or is it postponed?

*** Bug 303906 has been marked as a duplicate of this bug. ***

Nikolay, can you reproduce this with Beta2?

Yes.

Please attach logs. I need to see the Exception messages, there should be a
MediaNotDesired Exception.

Created attachment 161299 [details]
y2log

For reference:

2007-08-31 11:27:25 <0> linux(3271) [zypp] MediaManager.cc(checkDesired):113 checkDesired(14): desired (report by zypp::media::NoVerifier)
2007-08-31 11:27:25 <0> linux(3271) [zypp] MediaManager.cc(checkDesired):116 checkDesired(14): desired (cached)
2007-08-31 11:27:27 <5> linux(3271) [zypp] Exception.cc(log):119 MediaHandler.cc(getFile):1240 THROW:    MediaHandler.cc(getFile):1240: File ./suse/noarch/bundle-lang-common-en-10.3-6.noarch.rpm not found on media: cd:///?devices=/dev/sr0

And before that:

2007-08-31 11:27:25 <1> linux(3271) [zypp] RepoProvideFile.cc(provideFile):214 Providing file of repo 'openSUSE-10.3-OSS-KDE 10.3' from cd:///?devices=/dev/sr0
2007-08-31 11:27:25 <1> linux(3271) [zypp] MediaSetAccess.cc(MediaSetAccess):36 initializing..
2007-08-31 11:27:25 <2> linux(3271) [zypp] RepoProvideFile.cc(setVerifierForRepo):160 No media verifier for repo 'openSUSE-10.3-OSS-KDE 10.3
2007-08-31 11:27:25 <1> linux(3271) [zypp] MediaAccess.cc(open):109 Trying scheme 'cd'
2007-08-31 11:27:25 <1> linux(3271) [zypp] MediaCD.cc(MediaCD):129 MediaCD::MediaCD(cd:///?devices=/dev/sr0, )
2007-08-31 11:27:25 <0> linux(3271) [zypp] MediaCD.cc(MediaCD):149 parse /dev/sr0

So the verifier is not set, why?

Nikolay, if you go to console, in /var/cache/zypp/raw/$reponame is there a media.1 directory?

Yes. is does exist in both repository caches

Nikolay, can you attach the output of ls in /var/cache/zypp/raw to see the alias names of the repositories, and also, the content of the media/media.1 files there.

The debug logline:

2007-08-31 11:27:25 <2> linux(3271) [zypp]
RepoProvideFile.cc(setVerifierForRepo):160 No media verifier for repo
'openSUSE-10.3-OSS-KDE 10.3'

Is shown when /var/cache/zypp/raw/openSUSE-10.3-OSS-KDE 10.3/media.1/media is not found.

/var/cache/zypp/raw:

openSUSE-10.3-Addon-NonOSS 10.3
openSUSE-10.3-OSS-KDE 10.3
Repository

The contents of corresponding directories:

content
content.asc
content.key
gpg-pubkey-0dfb3188-41ed929b.asc
gpg-pubkey-307e3d54-44201d5d.asc
gpg-pubkey-3d25d3d9-36e12d04.asc
gpg-pubkey-7e2e3b05-44748aba.asc
gpg-pubkey-9c800aca-40d8063e.asc
gpg-pubkey-a1912208-446a0899.asc
media.1
suse

--

content
content.asc
content.key
gpg-pubkey-0dfb3188-41ed929b.asc
gpg-pubkey-307e3d54-44201d5d.asc
gpg-pubkey-3d25d3d9-36e12d04.asc
gpg-pubkey-7e2e3b05-44748aba.asc
gpg-pubkey-9c800aca-40d8063e.asc
gpg-pubkey-a1912208-446a0899.asc
media.1
suse

--

content
content.asc
content.key
gpg-pubkey-0dfb3188-41ed929b.asc
gpg-pubkey-307e3d54-44201d5d.asc
gpg-pubkey-3d25d3d9-36e12d04.asc
gpg-pubkey-7e2e3b05-44748aba.asc
gpg-pubkey-9c800aca-40d8063e.asc
gpg-pubkey-a1912208-446a0899.asc
media.1
suse

The contents of corresponding media files:

SUSE Linux Products GmbH
20070822172025
2

SUSE Linux Products GmbH
20070822123731
1

SUSE Linux Products GmbH
20070822172025
2

Created attachment 162117 [details]
Reproducing in Beta3

Testing internal beta3 candidate using qemu.

As you see in comment #12, I tried to reproduce it in internal beta3 candidate.

I added the lang addon CD, and selected a package from there to install.

Then installation started, and it asks me for CD1.

If you look carefully at your repositories, the 1st and 3rd CD (I suppose the 3rd is the language addon), they have the same identifiers in media.1, which is broken, so YaST think the CD is inserted, but the file not found. Definitely not a ZYpp bug, but a media problem I guess.

Stano, we either lower severity or close it (WORKSFORME)?

not really. this is a 1-second resolution timestamp.
For these to be identical I would need to start these CDs within
the same second, which is remotely possible, but not likely.

looking at the setup above there is:
openSUSE-10.3-Addon-NonOSS 10.3
openSUSE-10.3-OSS-KDE 10.3
Repository

what is "Repository" ?

1 and 3 are identical, and the one ending in with the line "2"
is the Addon CD (the KDE and GNOME are pure 1 cd solutions,
while the Addon CDs have a second CD, with the sources)

If the user says he added a lang addon, and nonOSS != lang addon, then it means either that Repository is the lang addon, and it is broken, or the user never added the lang addon cd and ended adding the main media twice.

The fact that package bundle-lang-common-en-10.3-6.noarch.rpm is in the list means this has to come from somewhere.

This looks like the old bug with filechecker, where repomd.xml from another repository was taken erroneously from other repository raw cache. Could be the same be happening with /media.1/media file?

2007-08-31 11:25:41 <0> linux(3271) [zypp] ExternalProgram.cc(start_program):172 copy /var/adm/mount/AP_0x00000001/media.1/media -> /var/cache/zypp/raw/Repository0lPOic/media.1/media Executing '/bin/cp' '--' '/var/adm/mount/AP_0x00000001/media.1/media' '/var/cache/zypp/raw/Repository0lPOic/media.1/media'

[1]./suse/noarch/bundle-lang-common-en-10.3-6.noarch.rpm{398.7 K|sha1-118b4fdb0274d6e2b1d7bdc3e35427c18e3cd14f}
2007-08-31 11:27:25 <1> linux(3271) [zypp] RepoProvideFile.cc(provideFile):214 Providing file of repo 'openSUSE-10.3-OSS-KDE 10.3' from cd:///?devices=/dev/sr0

Anyway that file is from the main media.

2007-08-31 11:24:49 <0> linux(3271) [zypp] ExternalProgram.cc(start_program):172 copy /var/adm/mount/AP_0x00000001/media.1/media -> /var/cache/zypp/raw/openSUSE-10.3-OSS-KDE 10.3342JpN/media.1/media Executing '/bin/cp' '--' '/var/adm/mount/AP_0x00000001/media.1/media' '/var/cache/zypp/raw/openSUSE-10.3-OSS-KDE 10.3342JpN/media.1/media'

The point of the bug, is why the code insists there is no verifier for that repository later when getting the file.

It seems the critical point is the isFileExist for that media file. But as I said, I can't reproduce it on beta3

(In reply to comment #15 from Duncan Mac-Vicar Prett)
> If the user says he added a lang addon, and nonOSS != lang addon, then it means
> either that Repository is the lang addon, and it is broken, or the user never
> added the lang addon cd and ended adding the main media twice.

I can reproduce it on both Lang and NonOSS CDs. But the comment #11 refers to setup with KDE CD plus NonOSS CD.

*** Bug 308237 has been marked as a duplicate of this bug. ***

(In reply to comment #16 from Duncan Mac-Vicar)
> The point of the bug, is why the code insists there is no verifier for that
> repository later when getting the file.
> 
> It seems the critical point is the isFileExist for that media file. But as I
> said, I can't reproduce it on beta3

Indeed, RepoInfo::isFileExist() returns false if the repo alias happens to
contain white space: /var/cache/zypp/raw/openSUSE-10.3-OSS-KDE
10.3342JpN/media.1/media

/var/cache/zypp/raw/openSUSE-10.3-OSS-KDE (which is what ::stat() probably
sees) really does not exist.

The bug can be fixed in PathInfo::operator()() by escaping the special
characters in the path string given to ::stat() or ::lstat() or in
RepoManager::rawcache_path_for_repoinfo() by filtering the special characters
in the alias when creating the raw cache path.

Beta3 behaviour has slightly changed. Now after package installation started I get one more "Eject" button in addition to the previous three.
I'm still unable to eject CD manually by pushing tray button after adding Addon CD.
And what's the most annoying is that after clicking "Eject" in that window installation just crashes. YaST logs will follow.

Created attachment 163443 [details]
y2log

I also think I would be usefull to ask user after Addon CDs added to change back to the CD from which an installation was started (main CD or DVD). Just the same behaviour as "Media Check" has.

Jano, ok. I remember I tested in qemu for Beta3 and the problem of fileExist was gone.

The bug duplicate of this one had beta3 in it so I got confused.

It seems all the problem has now to do with eject.

OK, so if we consider the "No media verifier" problem gone. Then we have the failure of eject. This may be caused by dbus not running (bug #303799), i'll check the code.

But the failure obviously results in an uncaught Exception (thrown out of RepoProvideFile::provideFile after 'Trying next URL' fails. Lada, can you check why is the exception not caught byt pkg-bindings?.

As for the eject, it is done in MediaCD.cc::forceEject(), and yes, it is probably caused by the dbus not running (which should be fixed by now) since the code relies on hal/dbus for the device detection. I can't tell for sure without debugging logs.

So i suggest to close this bug as duplicate of bug #303799. Nikolay, please retest with RC1, and if the bug occurs again, turn on the debug log (by pressing Shift+F7) before clicking on the Eject button, and attach it here.

However, we still may fix the problem of YaST not catching the exception and/or not displaying the error message instead of exiting.

Should I open a separate bug for comment #22?

(In reply to comment #26 from Nikolay Derkach)
> Should I open a separate bug for comment #22?

Well, it should not be needed, if the other stuff worked correctly, but that's something for the YaST folks to decide. So yes, please open a new bug report (severity: enhancement). Thanx!

To comment #25:

There is a global catch(...) in pkg-binding evaluation code:

    YCPValue Y2PkgFunction::evaluateCall ()
    {
        ycpmilestone ("Pkg Builtin called: %s", name().c_str() );
        
        try
        {
            switch (m_position) {
#include "PkgBuiltinCalls.h"
            }
        }
        catch (const std::exception& excpt)
        {
            y2internal("Caught an unhandled exception: %s", excpt.what());
        }
        catch (...)
        {
            y2internal("Caught an unhandled exception");
        }

        return YCPNull ();
    }

"PkgBuiltinCalls.h" is a switch which evaluates the set pkg call.

So yast should not abort due to an uncaught exception because all exceptions are either caught in the respective binding or caught by this fallback (if the they are not handled properly).

Then, the logs suggest that the failure is not handled in yast2-packager, inst_rpmcopy.ycp:726-730 (abort is returned without an error pop-up). That should be the last thing to fix regarding this bug.

Error handling has been fixed in yast2-packager-2.15.80.

I reused error message "Installation of Some Packages Failed" due to text freeze. After 10.3 a better message should be used, resolving as LATER.

I do have same problem as in bug #308237 with RC1.

When adding Language AddOn as CD and it is still in drive when installation starts, opensuse10.3 DVD will be requested but I cannot open the CD drive manually. When clicking on eject a the CD drive opens but I get a warning: there was an error installing one package. When I click OK -> Linuxrc

Created attachment 173271 [details]
YaST2.tgz

So reopening

So the MediaCD.cc::forceEject() still fails. I'd say it's because the CD is still mounted (not released). And that's probably because no media verifier is set for the 10.3 CD.

Can you please reproduce with debug logging turned on (right before the installation starts)?

Holgi is on vacation, please burn yourself two CDs and reproduce in prague office

OK, i got it. It all goes like this:

1) start installation with DVD
2) add an Add-On product from a CD/DVD (tried with Language Add-on CD1)
   *the Add-on CD/DVD stays mounted*
3) go through the proposals, proceed with installation
4) zypp wants to provide rpm file from the CD/DVD, sees it is mounted,
   *increases its use-count*
5) now zypp fails to mount the rpm file, releases the accessId (which effectively
   just decreases the use-count, but does not umount the CD/DVD, and ask for
   another CD.
6) because of the fact that the CD/DVD is still mounted:
   - eject ejects another drive, if available, otherwise the eject probably fails
   - manual eject (drive button) does not work either

How to solve it? Three options come to my mind - first of all, somebody please try to recall how we did it in 10.2 :O)) The other two:

a) Call SourceReleaseAll binding (MediaSetAccess::release()) in YaST whenever the
   module/section ceases to needed it (in this case the Add-on product manager
   screen right after adding the add-on or right before adding a new one, and on
   quitting the screen.

b) Try to unmount all, even shared mounts in the MediaSetAccess before asking
   for a CD (or reporting an error).

Attaching the logs in a minute.

Created attachment 173891 [details]
yast logs

OK i know how we've been doing it in 10.2 but that won't help us: we've had this sourceManager()->releaseAllSources() in zypp before. This released the media of all known repositories. We can't do it anymore within libzypp.

See http://svn.opensuse.org/svn/zypp/trunk/libzypp/zypp/MediaSetAccess.cc provideFileInternal, MediaException handling.

Bug 326336 is probably a duplicate of this one. The missing releaseAll() together with the missing CD verifier are capable of causing it.

*** Bug 326336 has been marked as a duplicate of this bug. ***

ok, I just verified: the bug is really reproducible on every installation with two media (two DVD9s in my case). Even worse: pressing eject still crashes yast.

We have a prepared fallback bugfix for yast2-add-on however (ReleaseAll call) nevertheless this may appear anywhere else in the code and should be solved internally by libzypp later.

OK here comes the solution from YaST side:

   * yast2-add-on: Calling Pkg::SourceReleaseAll(); just after adding
                   new Add-On in 'Add' dialog (also AutoYaST covered)
                   Calling Pkg::SourceReleaseAll(); after all Add-Ons are
                   merged with their installation.xml and y2update.tgz files
                   (locilka)

   * yast2-packager: Calling Pkg::SourceReleaseAll(); in (repositories.ycp,
                     AKA inst_source.ycp) just after new repository is added.
                     Calling Pkg::SourceReleaseAll(); when finishing 
                     repositories.ycp
                     (lslezak)

From libzypp side, Jano will propose a solution and discuss the patch with other libzypp guys.

- Calling Pkg::SourceReleaseAll(); after a new Add-On is used and
  completely merged (#293428 c#44).
- yast2-add-on-2.15.16

Created attachment 174167 [details]
proposed patch

This should serve as a replacement for the old SourceManager::releaseAllSources()

A simple test of the patch with zypper works, but it is not possible to simulate the situation from the installation with zypper so a new build has to be tested. If something will go wrong, we can remove the patch and rely on the workarounds in YaST (the ones mentioned by Lukas).

Comment #44, yast2-packager - fixed in yast2-packager-2.15.81 (Pkg::SourceReleaseAll() is called after adding a source, the call is not needed at the end.)

I debugged to see why we get these lines:

[zypp] RepoProvideFile.cc(setVerifierForRepo):162 No media verifier for repo 'openSUSE-10.3-DVD 10.3'

The reason was package-bindings not adding the repo when starting using it, so metadataPath in the repository info was not set, so later RepoProvideFile could not set the verifier.

The only way to fix it at this point was: expose a way in RepoManager to get the metadataPath and let pkg-bindings to set it manually.

libzypp 3.25.0, assigning to lslezak for package-bindings part.

Fixed in yast2-pkg-bindings-2.15.51

Tested with the inst-sys from the latest KDE CD build (Build20070924), patched with the new libzypp, pkg-bindings, qt, perl-bindings, yast2-addon. Installing from RC1 DVD, added CD language addon. After registering the addon (in the installation proposal) I was able to eject the addon medium in the drive.

Resolving as FIXED.

ok, all packages are in and it's still crashing when you try to change from DVD1 to DVD2 in the retail medium.

Created attachment 174527 [details]
latests y2logs from coolo's box.

(In reply to comment #52 from Stephan Kulow)
> ok, all packages are in and it's still crashing when you try to change from
Is the patch proposed in comment #46 also in? From the logs (comment #53) it does not seem so. Note that the patch has not been submitted into svn yet!

However, Lada did his test with the patch included, so this should really be fixed. Will check that with Lada.

submitted in libzypp 3.26.0