Bug 1194196 - Inconsistent mirrors
Summary: Inconsistent mirrors
Status: NEW
Alias: None
Product: openSUSE.org
Classification: openSUSE
Component: BuildService (show other bugs)
Version: unspecified
Hardware: Other Other
: P5 - None : Normal with 1 vote (vote)
Target Milestone: ---
Assignee: Michael Schröder
QA Contact: Adrian Schröter
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2021-12-31 13:36 UTC by Vincent Bernat
Modified: 2022-01-19 12:45 UTC (History)
2 users (show)

See Also:
Found By: ---
Services Priority:
Business Priority:
Blocker: ---
Marketing QA Status: ---
IT Deployment: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Vincent Bernat 2021-12-31 13:36:19 UTC
Hello,

Looking at https://mirrorcache.opensuse.org/repositories/home:/vbernat/CentOS_7/repodata/primary.xml.gz.mirrorlist, it mentions `widehat.opensuse.org` which does not resolve. Some mirrors also seem late. For example https://ftp.lysator.liu.se/pub/opensuse/repositories/home:/vbernat/CentOS_7/repodata/primary.xml.gz has 1.0.11, https://ftp.gwdg.de/pub/opensuse/repositories/home:/vbernat/CentOS_7/repodata/primary.xml.gz has 1.0.12 as well as some others. https://mirror.firstyear.id.au/repositories/home:/vbernat/CentOS_7/repodata/primary.xml.gz seems up-to-date.

See https://github.com/lldpd/lldpd/issues/495 for the original report. It also mentions that yum sometimes fail with:

```
https://download.opensuse.org/repositories/home:/vbernat/CentOS_7/repodata/primary.xml.gz:
[Errno -1] Metadata file does not match checksum
```

Dunno if download.opensuse.org relies on the mirrorlist or if it is another different problem.
Comment 1 Andreas Stieger 2021-12-31 15:14:26 UTC
Adding some clarification so we know what we are speaking about...

Mirrors not being up to date is expected under normal operations, and the download.opensuse.org interface is build to compensate and offer the correct user experience here. The mirrorbrain redirector on download.opensuse.org will only redirect to mirrors are known to have the most recent files, and will serve most repository meta files directly. So if one mirror stops syncing your particular project it is a non-issue as far as the user-visible interface is concerned - as long as the recommended download.o.o entry point is used.

Further, each mirror can independently determine their replication frequency and scope. Repositories under home: are simply less interesting for most mirrors seeking to offer bandwidth (actually often also campus mirrors seeking to serve directly from their own networks) Repositories for CentOS even less so. Ultimately this is a matter of space. If you check your other repositories you will see that some of them are not mirrored anywhere at all.

So in this case we have a repository path which is not generally interesting for public mirrors, so fewer mirrors of which one has stopped syncing. So lack of mirrors means a lower likelihood that all of them will be current.

With that being said, what is the user-visible problem or problem that you are attempting to solve with this report?

a) widehat being listed despite no longer resolving - that is a given
b) any particular mirror NOT accessed through download.opensuse.org ... that is not an issue unless you see redirects that should not happen
c) ... lack of mirrors? You could attempt to request a top-level project outside of the home: namespace or serve the files/packages through github releases / github packages

Anything else?
Comment 2 Tim Meusel 2021-12-31 15:33:05 UTC
Hi,
I raised the issue at https://github.com/lldpd/lldpd/issues/495. The error message I get from github.com CI runners is:

https://download.opensuse.org/repositories/home:/vbernat/CentOS_7/repodata/primary.xml.gz:
[Errno -1] Metadata file does not match checksum

I took a look at the mirrorlist site and found the mirrors listed in the original issue description where widehat.opensuse.org is dead and ftp.lysator.liu.se/mirror.firstyear.id.au provide a different primary.xml.gz compared to the others. My guess is that the error message is raised because the used mirror is inconsistent/outdated. The GitHub CI infrastructure is a bit locked down so I don't know which mirror is actually used/accessed.
Comment 3 Tim Meusel 2021-12-31 15:38:14 UTC
If that helps, the used yumrepo is the one provided: https://download.opensuse.org/repositories/home:/vbernat/CentOS_7/home:vbernat.repo

I didn't notice any issues on CentOS 8  or Debian 9/10/11 or Ubuntu 16/18/20. Those mirrors seem to be in sync or I've more luck with the mirror selection.