Bug 157535

Summary:	query-system takes ages when installing packages with yast2
Product:	[openSUSE] SUSE Linux 10.1	Reporter:	Matthias Hopf <mhopf>
Component:	Zenworks	Assignee:	Dan Mills <thunder>
Status:	RESOLVED DUPLICATE	QA Contact:	Nat Budin <nbudin>
Severity:	Blocker
Priority:	P5 - None	CC:	kkaempf, suse-beta
Version:	Beta 7
Target Milestone:	---
Hardware:	Other
OS:	Other
Whiteboard:
Found By:	Other	Services Priority:
Business Priority:		Blocker:	---
Marketing QA Status:	---	IT Deployment:	---
Attachments:	Requested Yast2 logs. Yast2 Logs ZMD logs

Description Matthias Hopf 2006-03-13 11:11:17 UTC

Freshly installed SL10.1b7 (no update). KDE, C/C++ development, advanced user as selection.

After installation I started Yast2, Software Management. I want to install libsvg, libsvg-cairo, gconf-editor, control-center2. This results (AFAIR) in 11 packages from CD2 and 2 from CD5 to be installed. Yast2 starts the download from CD2 and takes ages for a single package. top shows that approx. 80% of the time is consumed by query-system, and a substantially large part by mono. Between package 2 and 3 it took several minutes, a typical user would have interrupted the system already. This is critical, or even blocker.

Machine is gkar, ssh is open. I have to do some tests on the machine, though, so it could crash on rare occasions.

Comment 1 Michael Gross 2006-03-13 11:50:31 UTC

Please attach /var/log/YaST2/* as tarball.

Comment 2 Klaus Kämpf 2006-03-13 12:37:06 UTC

How much main memory does the machine have ?

Comment 3 Matthias Hopf 2006-03-13 17:42:23 UTC

(In reply to comment #2)
> How much main memory does the machine have ?

Not very much. 256MB. It's also only a 1GHz Celeron.

However, during installation things worked pretty well. I only have issues *after* installation when installing additional packages. I also noticed that query-system seems to be used as well when I'm directly invoking rpm. And I already hit the 'waiting for rpm lock' message several times on a 1.8GHz PentiumM machine with 1GB of memory. Though this definitvely does not take 'ages' as on the older system.

Comment 4 Matthias Hopf 2006-03-13 17:44:57 UTC

Created attachment 72622 [details]
Requested Yast2 logs.

Comment 5 Michael Gross 2006-03-14 11:13:03 UTC

Reassigning to Klaus.

Comment 6 Marcel Hilzinger 2006-03-17 10:39:15 UTC

Still there in Beta8. It also affects the command line. So installing a 400k RPM takes 2 minutes on a 1,4 GHz machine with 256 MB memory. Installing Gnome took 2 hours...

top

shows me query-system and zmd all the time, when installing rpm packages

Set priority to blocker

Comment 8 Klaus Kämpf 2006-03-18 23:56:13 UTC

Please provide YaST logfiles from Beta8.
And also provide /var/log/zmd-*.log

Comment 9 Matthias Hopf 2006-03-20 15:36:56 UTC

My machine is currently installing... Maybe Marcel can provide his logfiles?

Comment 10 Matthias Hopf 2006-03-20 17:06:36 UTC

Installing rpms on Beta 8 is dead slow as well. I'm currently installing  packages at a rate of approximately 1 package per minute. Interestingly, packages 9-13 installed very well without additional waiting time. But the query-system calls just seemed to be delayed to the end of package 13, which was delayed by 5 minutes.

Just rpm -Uhv. It always complains about 'warning: waiting to reestablish exclusive database lock', and either zmd or query-system is running all the time (mostly query-system). Most of the time they just seem to do I/O (typically 20-40% wait), and they are resource hogs as well (up to 60MB residential or 97MB virtual for query-system).

Logs will follow.

Comment 11 Matthias Hopf 2006-03-22 10:52:26 UTC

Created attachment 74402 [details]
Yast2 Logs

Oops. Sorry, forgot.

Comment 12 Matthias Hopf 2006-03-22 10:52:48 UTC

Created attachment 74403 [details]
ZMD logs

Comment 13 Klaus Kämpf 2006-03-24 05:43:58 UTC

So it looks like zmd has a watch on the rpm database and runs "rpm -qa" (== query-system) as soon as it detects a change.

For zmd/yast, this can be circumvented by introducing some 'transaction start/stop' callbacks from yast to zmd. -> Assign to Naresh.

However, for normal rpm operation (e.g. running a couple of rpm commands), this issue must be solved within ZMD

Comment 15 Matthias Hopf 2006-03-24 18:36:07 UTC

1) I don't think that invoking 'rpm -qa' after each and every rpm call can be a justified solution. This call is much too expensive.
2) query-system cannot just do 'rpm -qa' - that doesn't use 97MB main memory (for a very small package selection) and take over a minute on a 1GHz machine.
3) Any package manager that doesn't use the rpm database must be inherently broken. As soon as the databases get out-of-sync, anything can happen. Keeping the databases in sync would need rpm to be extended.

Which brings me to the base question: why is a different database used in the first place?

Comment 17 Naresh Wignarajah 2006-03-26 19:52:15 UTC

Regarding the rest assigning to Dan (thunder) to review/explain/fix.

Comment 18 Dan Mills 2006-03-26 20:08:54 UTC

I recently fixed a separate bug #159240 that is the same thing.

zmd now starts a 30-second timer whenever the rpm db changes, which is reset if it changes again, and only then does it re-scan the system packages.

About #15, please talk to Klaus Kaempf about questions about the general architecture of zypp and how it ties into zmd.

*** This bug has been marked as a duplicate of 159240 ***