Bug 577790

Summary:	aria2 makes libzypp slow till unusable
Product:	[openSUSE] openSUSE 11.3	Reporter:	Lars Müller <lmuelle>
Component:	libzypp	Assignee:	Pascal Bleser <pascal.bleser>
Status:	RESOLVED FIXED	QA Contact:	E-mail List <qa-bugs>
Severity:	Major
Priority:	P2 - High	CC:	albryan, dmacvicar, forgotten_4yKfwBnymp, jnelson-suse, ma, tatsuhiro.t
Version:	Factory
Target Milestone:	---
Hardware:	All
OS:	openSUSE 11.3
Whiteboard:
Found By:	Development	Services Priority:
Business Priority:		Blocker:	---
Marketing QA Status:	---	IT Deployment:	---
Attachments:	/var/cache/zypp/aria2.stats

Description Lars Müller 2010-02-07 14:30:30 UTC

While calling zypper dup (after a zypper refresh) one system fetched packages with a noticeable delay.

While a second system failed to download at all.

Using ZYPP_ARIA2C=0 zypper refresh|up zypper|dup works as expected.

Comment 1 Duncan Mac-Vicar 2010-02-08 09:59:25 UTC

Can you attach /var/cache/zypp/aria2.stats ?

Comment 2 Lars Müller 2010-02-08 10:17:12 UTC

Created attachment 341219 [details]
/var/cache/zypp/aria2.stats

Comment 3 Lars Müller 2010-02-08 10:22:35 UTC

I see dist.suse.de and download.suse.de in the list.

Both hosts are reachable from my side of the network.

Comment 4 Michael Andres 2010-02-08 15:50:59 UTC

Sounds like bug #552013 or bug #555803.

Is it possible to login to one of the machines and do some investigation?

Comment 5 Lars Müller 2010-02-09 10:02:01 UTC

It's possible.  Please call me for the details.

Comment 6 Lars Müller 2010-02-09 11:21:28 UTC

You might also catch me on IRC or Novell IM.

Comment 7 Jon Nelson 2010-08-02 21:19:53 UTC

I may have found one of the issues.

aria2c *never exits* after successfully downloading a file, under certain conditions.

I took an strace of a zypper command that was never returning (even after 10's of minutes) and constructed the following aria2c invocation:


/usr/bin/aria2c --log-level=debug --user-agent="ZYpp 7.7.5 (aria2 version 1.9.1) openSUSE-11.3-x86_64" "--summary-interval=1" "--follow-metalink=mem" "--check-integrity=true" "--file-allocation=none" "--server-stat-of=/var/cache/zypp/aria2.stats" "--server-stat-if=/var/cache/zypp/aria2.stats" "--uri-selector=adaptive" "--use-head=false" "--max-tries=5" "--max-concurrent-downloads=2" "--header=X-ZYpp-AnonymousId: 1462ba10-d059-4720-9686-375f085c19c5" "--header=X-ZYpp-DistributionFlavor: dvd" "--header=Pragma:" "--connect-timeout=180" "--http-proxy=http://192.168.2.1:3128/" "--dir=/tmp/AP_0x00000002/repodata" "http://packman.inode.at/suse/11.3/repodata/repomd.xml" 


When *I* invoke that, the download completes almost immediately, and then aria2c just... sits there.

Like this:

+ /usr/bin/aria2c --log-level=debug '--user-agent=ZYpp 7.7.5 (aria2 version 1.9.1) openSUSE-11.3-x86_64' --summary-interval=1 --follow-metalink=mem --check-integrity=true --file-allocation=none --server-stat-of=/var/cache/zypp/aria2.stats --server-stat-if=/var/cache/zypp/aria2.stats --uri-selector=adaptive --use-head=false --max-tries=5 --max-concurrent-downloads=2 '--header=X-ZYpp-AnonymousId: 1462ba10-d059-4720-9686-375f085c19c5' '--header=X-ZYpp-DistributionFlavor: dvd' --header=Pragma: --connect-timeout=180 --http-proxy=http://192.168.2.1:3128/ --dir=/tmp/AP_0x00000002/repodata http://packman.inode.at/suse/11.3/repodata/repomd.xml

2010-08-02 16:17:35.911093 WARN - You may encounter the certificate verification error with HTTPS server. See --ca-certificate and --check-certificate option.

2010-08-02 16:17:35.911337 NOTICE - ServerStat file /var/cache/zypp/aria2.stats loaded successfully.

2010-08-02 16:17:36.251824 NOTICE - GID#1 - Download has already completed: /tmp/AP_0x00000002/repodata/repomd.xml

2010-08-02 16:17:36.251975 NOTICE - Download complete: /tmp/AP_0x00000002/repodata/repomd.xml

^^^^ Nothing ever again.



Control-c will cancel it.


This seems to happen only on machines which go through squid!

If this is not the ultimate cause of your bad behavior. it's still a major issue with aria2c.

Comment 8 Michael Andres 2010-08-03 10:27:28 UTC

JFYI: You can try out Factory libzypp-8.0.1 (http://download.opensuse.org/repositories/zypp:/Head/openSUSE_11.3/). It has builtin metalink and zsync support based on libcurl, thus does not call aria anymore.

Comment 9 Jon Nelson 2010-08-15 16:34:09 UTC

Unfortunately, using that repo doesn't seem very practical, due to problems like this:


Problem: zypper-1.4.7-3.2.x86_64 requires libzypp.so.800()(64bit), but this requirement cannot be provided
  uninstallable providers: libzypp-8.1.1-2.1.x86_64[zypp_Head]
 Solution 1: Following actions will be done:
  deinstallation of yast2-qt-pkg-2.19.2-1.10.x86_64
  deinstallation of yast2-ncurses-pkg-2.19.3-1.4.x86_64




primarily because 1 jillion packages require a specific version of libzypp. :-(

Comment 10 Tatsuhiro Tsujikawa 2010-09-07 15:42:58 UTC

Could you attach aria2 log file of following command?

/usr/bin/aria2c --log-level=debug --user-agent="ZYpp 7.7.5 (aria2 version1.9.1) openSUSE-11.3-x86_64"
"--summary-interval=1" "--follow-metalink=mem"
"--check-integrity=true" "--file-allocation=none"
"--server-stat-of=/var/cache/zypp/aria2.stats"
"--server-stat-if=/var/cache/zypp/aria2.stats" "--uri-selector=adaptive"
"--use-head=false" "--max-tries=5" "--max-concurrent-downloads=2"
"--header=X-ZYpp-AnonymousId: 1462ba10-d059-4720-9686-375f085c19c5"
"--header=X-ZYpp-DistributionFlavor: dvd" "--header=Pragma:"
"--connect-timeout=180" "--http-proxy=http://192.168.2.1:3128/"
"--dir=/tmp/AP_0x00000002/repodata"
"--log=/tmp/log.txt"
"http://packman.inode.at/suse/11.3/repodata/repomd.xml"

The log file is written into /tmp/log.txt

I tested this on my machine, but it did not reproduce the failure.

Comment 11 Jon Nelson 2013-01-24 02:00:26 UTC

Does zypper still use aria2?  If not, feel free to close this bug.

Comment 12 Michael Andres 2013-01-24 09:48:20 UTC

No, since 11.4 the multicurl backend is the default.