|
Bugzilla – Full Text Bug Listing |
| Summary: | zypp does not resume partial downloads | ||
|---|---|---|---|
| Product: | [openSUSE] openSUSE Tumbleweed | Reporter: | Christopher Yeleighton <giecrilj> |
| Component: | libzypp | Assignee: | E-mail List <zypp-maintainers> |
| Status: | NEW --- | QA Contact: | E-mail List <qa-bugs> |
| Severity: | Enhancement | ||
| Priority: | P5 - None | CC: | bzeller, giecrilj, mls |
| Version: | Current | ||
| Target Milestone: | --- | ||
| Hardware: | x86-64 | ||
| OS: | openSUSE Tumbleweed | ||
| URL: | http://download.opensuse.org/tumbleweed/repo/oss/x86_64/kernel-default-6.3.9-1.1.x86_64.rpm | ||
| See Also: | https://bugzilla.opensuse.org/show_bug.cgi?id=458924 | ||
| Whiteboard: | |||
| Found By: | Community User | Services Priority: | |
| Business Priority: | Blocker: | --- | |
| Marketing QA Status: | --- | IT Deployment: | --- |
Files that already have been downloaded into the cache ( e.g. download was finished and signature/checksums were verified ) should not be downloaded again unless the repo changes and never version numbers become available. However we do not support resuming files that have not been downloaded completely. (In reply to Benjamin Zeller from comment #1) > However we do not support resuming files that have not been downloaded > completely. I understand that zypp uses curl and curl does not support resuming, so fixing this would require improving curl or using another mechanism. In either case, it looks like a lot of work to do. (Note that the MultiCurl code did reuse blocks from interrupted downloads if the metalink file contains block checksums) @Christopher, is the log snippet consecutive? The kernel-default download takes about an hour (14:01:17 - 14:57:11)? If so, please attach the full log. (In reply to Michael Schröder from comment #3) > (Note that the MultiCurl code did reuse blocks from interrupted downloads if > the metalink file contains block checksums) This download was using the MultiCurl backend, however IIRC this feature never was used, because files are first downloaded into a temporary location, and only if the checksum/signatures are valid are copied into the cache location. Which results in the partial files being destroyed when the temporary location is cleaned up. It used to save the tmp location to a "MultiCurl.failed" file so that it can be reused. (In reply to Michael Schröder from comment #6) > It used to save the tmp location to a "MultiCurl.failed" file so that it can > be reused. It still does. Here is a little work-around that works for me: for N in kernel-default-6.5.4-1.1.x86_64.rpm do { <"$N" || pkexec cat '/var/cache/zypp/MultiCurl.failed' >"$N" } && wget -vc "http://download.opensuse.org/tumbleweed/repo/oss/x86_64/${N}" done > Łączenie się z ftp.vectranet.pl (ftp.vectranet.pl)|88.156.222.90|:80... połączono. > Żądanie HTTP wysłano, oczekiwanie na odpowiedź... 206 Partial Content > Długość: 169174424 (161M), 56403854 (54M) pozostało [application/x-redhat-package-manager] > Zapis do: `kernel-default-6.5.4-1.1.x86_64.rpm' I used to download big packages using a download manager but this wastes the effort of the first hour :-( (In reply to Michael Andres from comment #4) > @Christopher, is the log snippet consecutive? The kernel-default download > takes about an hour (14:01:17 - 14:57:11)? If so, please attach the full log. The kernel download takes at least 90 minutes. It may take much longer, depending on weather conditions and brainless tiktokers in the neighbourhood. This gap here is shorter because zypper was downloading other packages before, using up a part of the allotted time slot. > 2023-09-23 17:48:58 <1> localhost.localdomain(8370) [zypp::media++] MediaCurl.cc(doGetFileCopyFile):1200 {T:140331427215040} URL: http://download.opensuse.org/tumbleweed/repo/oss/x86_64/kernel-default-6.5.4-1.1.x86_64.rpm > 2023-09-23 17:48:58 <1> localhost.localdomain(8370) [zypp-curl++] curlhelper.cc(log_redirects_curl):153 {T:140331427215040} redirecting to Location: http://ftp.vectranet.pl/opensuse/tumbleweed/repo/oss/x86_64/kernel-default-6.5.4-1.1.x86_64.rpm > 2023-09-23 18:23:05 <3> localhost.localdomain(8370) [zypp::media] MediaCurl.cc(doGetFileCopyFile):1265 {T:140331427215040} curl error: 42: Callback aborted, temp file size 112770570 bytes. > 2023-09-23 18:23:05 <5> localhost.localdomain(8370) [zypp-core] Exception.cc(log):186 {T:140331427215040} MediaCurl.cc(evaluateCurlCode):939 THROW: Timeout exceeded when accessing 'http://download.opensuse.org/tumbleweed/repo/oss/x86_64/kernel-default-6.5.4-1.1.x86_64.rpm'. > 2023-09-23 18:23:05 <5> localhost.localdomain(8370) [zypp-core] Exception.cc(log):186 {T:140331427215040} MediaCurl.cc(doGetFileCopyFile):1281 RETHROW: Timeout exceeded when accessing 'http://download.opensuse.org/tumbleweed/repo/oss/x86_64/kernel-default-6.5.4-1.1.x86_64.rpm'. > 2023-09-23 18:23:05 <1> localhost.localdomain(8370) [zypp-core] PathInfo.cc(unlink):706 {T:140331427215040} unlink /var/tmp/AP_0xCriglu/x86_64/kernel-default-6.5.4-1.1.x86_64.rpm.new.zypp.1wAra2 > 2023-09-23 18:23:05 <5> localhost.localdomain(8370) [zypp-core] Exception.cc(log):186 {T:140331427215040} MediaCurl.cc(getFileCopy):783 RETHROW: Timeout exceeded when accessing 'http://download.opensuse.org/tumbleweed/repo/oss/x86_64/kernel-default-6.5.4-1.1.x86_64.rpm'. > 2023-09-23 18:23:05 <5> localhost.localdomain(8370) [zypp-core] Exception.cc(log):186 {T:140331427215040} MediaSetAccess.cc(provide):288 CAUGHT: Timeout exceeded when accessing 'http://download.opensuse.org/tumbleweed/repo/oss/x86_64/kernel-default-6.5.4-1.1.x86_64.rpm'. > 2023-09-23 18:23:05 <1> localhost.localdomain(8370) [zypp::media++] MediaHandler.cc(getDetectedDevices):1347 {T:140331427215040} No devices for this medium > 2023-09-23 18:23:05 <1> localhost.localdomain(8370) [zypp::media] MediaManager.cc(releaseAll):546 {T:140331427215040} Releasing all attached media > 2023-09-23 18:23:05 <1> localhost.localdomain(8370) [zypp::media++] MediaManager.cc(operator()):552 {T:140331427215040} Releasing media id 5 > 2023-09-23 18:23:05 <1> localhost.localdomain(8370) [zypp::media++] MediaHandler.cc(release):713 {T:140331427215040} Request to release attached media http<http://download.opensuse.org/tumbleweed/repo/oss/>, use count=1 > 2023-09-23 18:23:05 <1> localhost.localdomain(8370) [zypp::media++] MediaHandler.cc(release):720 {T:140331427215040} Releasing media http<http://download.opensuse.org/tumbleweed/repo/oss/> > 2023-09-23 18:23:05 <1> localhost.localdomain(8370) [zypp::media] MediaHandler.cc(disconnect):692 {T:140331427215040} Disconnected: http://download.opensuse.org/tumbleweed/repo/oss/ attached; localRoot "/var/tmp/AP_0xCriglu" > 2023-09-23 18:23:05 <1> localhost.localdomain(8370) [zypp::media++] MediaHandler.cc(removeAttachPoint):149 {T:140331427215040} MediaHandler - checking if to remove attach point > 2023-09-23 18:23:05 <1> localhost.localdomain(8370) [zypp-core] PathInfo.cc(recursive_rmdir):434 {T:140331427215040} recursive_rmdir /var/tmp/AP_0xCriglu > 2023-09-23 18:23:05 <1> localhost.localdomain(8370) [zypp::media] MediaHandler.cc(removeAttachPoint):161 {T:140331427215040} Deleted default attach point /var/tmp/AP_0xCriglu > 2023-09-23 18:23:05 <1> localhost.localdomain(8370) [zypp::media] MediaHandler.cc(release):774 {T:140331427215040} Released: http://download.opensuse.org/tumbleweed/repo/oss/ not attached; localRoot "" > 2023-09-23 18:23:05 <1> localhost.localdomain(8370) [zypp::media] MediaManager.cc(releaseAll):576 {T:140331427215040} Exit > 2023-09-23 18:23:05 <1> localhost.localdomain(8370) [zypp::fetcher] MediaSetAccess.cc(provide):339 {T:140331427215040} ProvideFile exception caught, callback answer: 0 > 2023-09-23 18:23:05 <1> localhost.localdomain(8370) [zypp::fetcher++] MediaSetAccess.cc(provide):343 {T:140331427215040} Aborting > 2023-09-23 18:23:05 <5> localhost.localdomain(8370) [zypp-core] Exception.cc(log):186 {T:140331427215040} MediaSetAccess.cc(provide):346 THROW: ABORT request: Aborting requested by user > 2023-09-23 18:23:05 <5> localhost.localdomain(8370) [zypp-core] Exception.cc(log):186 {T:140331427215040} History: > 2023-09-23 18:23:05 <5> localhost.localdomain(8370) [zypp-core] Exception.cc(log):186 {T:140331427215040} - Timeout exceeded when accessing 'http://download.opensuse.org/tumbleweed/repo/oss/x86_64/kernel-default-6.5.4-1.1.x86_64.rpm'. > 2023-09-23 18:23:05 <5> localhost.localdomain(8370) [zypp-core] Exception.cc(log):186 {T:140331427215040} > 2023-09-23 18:23:05 <5> localhost.localdomain(8370) [zypp-core] Exception.cc(log):186 {T:140331427215040} Fetcher.cc(provideToDest):571 RETHROW: ABORT request: Aborting requested by user > 2023-09-23 18:23:05 <5> localhost.localdomain(8370) [zypp-core] Exception.cc(log):186 {T:140331427215040} History: > 2023-09-23 18:23:05 <5> localhost.localdomain(8370) [zypp-core] Exception.cc(log):186 {T:140331427215040} - Can't provide ./x86_64/kernel-default-6.5.4-1.1.x86_64.rpm > 2023-09-23 18:23:05 <5> localhost.localdomain(8370) [zypp-core] Exception.cc(log):186 {T:140331427215040} - Timeout exceeded when accessing 'http://download.opensuse.org/tumbleweed/repo/oss/x86_64/kernel-default-6.5.4-1.1.x86_64.rpm'. > 2023-09-23 18:23:05 <5> localhost.localdomain(8370) [zypp-core] Exception.cc(log):186 {T:140331427215040} > 2023-09-23 18:23:05 <5> localhost.localdomain(8370) [zypp-core] Exception.cc(log):186 {T:140331427215040} RepoProvideFile.cc(provideFile):312 RETHROW: ABORT request: Aborting requested by user > 2023-09-23 18:23:05 <5> localhost.localdomain(8370) [zypp-core] Exception.cc(log):186 {T:140331427215040} History: > 2023-09-23 18:23:05 <5> localhost.localdomain(8370) [zypp-core] Exception.cc(log):186 {T:140331427215040} - Can't provide ./x86_64/kernel-default-6.5.4-1.1.x86_64.rpm > 2023-09-23 18:23:05 <5> localhost.localdomain(8370) [zypp-core] Exception.cc(log):186 {T:140331427215040} - Timeout exceeded when accessing 'http://download.opensuse.org/tumbleweed/repo/oss/x86_64/kernel-default-6.5.4-1.1.x86_64.rpm'. > 2023-09-23 18:23:05 <5> localhost.localdomain(8370) [zypp-core] Exception.cc(log):186 {T:140331427215040} > 2023-09-23 18:23:05 <3> localhost.localdomain(8370) [zypp] PackageProvider.cc(providePackage):429 {T:140331427215040} Failed to provide Package (68532)kernel-default-6.5.4-1.1.x86_64(download.opensuse.org-oss) > 2023-09-23 18:23:05 <5> localhost.localdomain(8370) [zypp-core] Exception.cc(log):186 {T:140331427215040} PackageProvider.cc(providePackage):431 RETHROW: ABORT request: Aborting requested by user > 2023-09-23 18:23:05 <5> localhost.localdomain(8370) [zypp-core] Exception.cc(log):186 {T:140331427215040} History: > 2023-09-23 18:23:05 <5> localhost.localdomain(8370) [zypp-core] Exception.cc(log):186 {T:140331427215040} - Can't provide ./x86_64/kernel-default-6.5.4-1.1.x86_64.rpm > 2023-09-23 18:23:05 <5> localhost.localdomain(8370) [zypp-core] Exception.cc(log):186 {T:140331427215040} - Timeout exceeded when accessing 'http://download.opensuse.org/tumbleweed/repo/oss/x86_64/kernel-default-6.5.4-1.1.x86_64.rpm'. > 2023-09-23 18:23:05 <5> localhost.localdomain(8370) [zypp-core] Exception.cc(log):186 {T:140331427215040} > 2023-09-23 18:23:05 <2> localhost.localdomain(8370) [zypp] TargetImpl.cc(commit):1485 {T:140331427215040} commit cache preload aborted by the user > 2023-09-23 18:23:05 <5> localhost.localdomain(8370) [zypp-core] Exception.cc(log):186 {T:140331427215040} TargetImpl.cc(commit):1486 THROW: Installation has been aborted as directed. > 2023-09-23 18:23:05 <1> localhost.localdomain(8370) [zypp::fetcher++] MediaSetAccess.cc(release):490 {T:140331427215040} Releasing all media IDs held by this MediaSetAccess |
I cannot stay connected longer than for 1h at a time. When I get disconnected, zypp just fails and starts downloading again from the beginning. This is an endless loop. Work-around: download the package and install using Discover. However, the work-around does not work for large packages that require other large packages (like kernel-firmware). 2023-06-27 14:01:17 <1> localhost.localdomain(2080) [zypp-core] PathInfo.cc(chmod):1098 {T:140616997660352} assert_file_mode 00600 /var/lib/YaST2/cookieschmod /var/lib/YaST2/cookies 00600 2023-06-27 14:57:11 <5> localhost.localdomain(2080) [zypp-core] Exception.cc(log):186 {T:140616997660352} MediaMultiCurl.cc(run):1155 THROW: Timeout exceeded when accessing 'http://download.opensuse.org/tumbleweed/repo/oss/x86_64/kernel-default-6.3.9-1.1.x86_64.rpm'. 2023-06-27 14:57:11 <1> localhost.localdomain(2080) [zypp::media++] MediaHandler.cc(removeAttachPoint):149 {T:140616997660352} MediaHandler - checking if to remove attach point 2023-06-27 14:57:11 <1> localhost.localdomain(2080) [zypp-core] PathInfo.cc(hardlinkCopy):916 {T:140616997660352} hardlinkCopy /var/tmp/AP_0xBzFTkr/x86_64/kernel-default-6.3.9-1.1.x86_64.rpm.new.zypp.xxiKoy -> /var/cache/zypp/MultiCurl.failed 2023-06-27 14:57:11 <1> localhost.localdomain(2080) [zypp::media++] MediaCurl.cc(doGetFileCopyFile):1193 {T:140616997660352} ./x86_64/kernel-default-6.3.9-1.1.x86_64.rpm 2023-06-27 14:57:11 <1> localhost.localdomain(2080) [zypp::media++] MediaCurl.cc(doGetFileCopyFile):1203 {T:140616997660352} URL: http://download.opensuse.org/tumbleweed/repo/oss/x86_64/kernel-default-6.3.9-1.1.x86_64.rpm 2023-06-27 14:57:33 <1> localhost.localdomain(2080) [zypp-curl++] curlhelper.cc(log_redirects_curl):153 {T:140616997660352} redirecting to Location: http://ftp.vectranet.pl/opensuse/tumbleweed/repo/oss/x86_64/kernel-default-6.3.9-1.1.x86_64.rpm