Bug 1227207 - Failure to fetch ext_pillar git
Summary: Failure to fetch ext_pillar git
Status: CONFIRMED
Alias: None
Product: openSUSE Distribution
Classification: openSUSE
Component: Salt (show other bugs)
Version: Leap 15.6
Hardware: x86-64 Other
: P2 - High : Normal (vote)
Target Milestone: Leap 15.6
Assignee: E-Mail List
QA Contact: E-mail List
URL: https://github.com/SUSE/spacewalk/iss...
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2024-06-30 10:26 UTC by Olav Reinert
Modified: 2024-07-11 14:57 UTC (History)
3 users (show)

See Also:
Found By: ---
Services Priority:
Business Priority:
Blocker: ---
Marketing QA Status: ---
IT Deployment: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Olav Reinert 2024-06-30 10:26:22 UTC
I have a salt-master configured to have an external pillar, as follows:


ext_pillar:
  - git:
    - master pki@git:pki-pillar.git

git_pillar_update_interval: 3600

git_pillar_privkey: /var/lib/salt/.ssh/id_ed25519
git_pillar_pubkey: /var/lib/salt/.ssh/id_ed25519.pub


When the salt-master starts, it is unable to load the pillar configuration, failing with these log entries:


2024-06-30 12:15:06,712 [salt.utils.gitfs :295 ][DEBUG   ][24319] Set update lock for git_pillar remote 'master pki@tresor.plexor.net:pki-pillar.git'
2024-06-30 12:15:06,712 [salt.utils.gitfs :295 ][DEBUG   ][24319] Fetching git_pillar remote 'master pki@tresor.plexor.net:pki-pillar.git'
2024-06-30 12:15:06,725 [salt.utils.gitfs :295 ][ERROR   ][24319] Error occurred fetching git_pillar remote 'master pki@tresor.plexor.net:pki-pillar.git': error loading known_hosts: 
Traceback (most recent call last):
  File "/usr/lib/python3.11/site-packages/salt/utils/gitfs.py", line 1989, in _fetch
    fetch_results = origin.fetch(**fetch_kwargs)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/site-packages/pygit2/remotes.py", line 160, in fetch
    payload.check_error(err)
  File "/usr/lib64/python3.11/site-packages/pygit2/callbacks.py", line 99, in check_error
    check_error(error_code)
  File "/usr/lib64/python3.11/site-packages/pygit2/errors.py", line 66, in check_error
    raise GitError(message)
_pygit2.GitError: error loading known_hosts: 
2024-06-30 12:15:06,726 [salt.utils.gitfs :295 ][DEBUG   ][24319] Removed update lock for git_pillar remote 'master pki@tresor.plexor.net:pki-pillar.git'


This issue is described in the following bug report:

    https://github.com/saltstack/salt/issues/64121

As suggested in the bug report, adding the following override to the salt-master.service unit file solves the problem for me:

    [Service]
    Environment=HOME=/var/lib/salt

Perhaps consider amending the unit file provided by the salt-master RPM accordingly?
Comment 1 Olav Reinert 2024-06-30 10:35:16 UTC
NAME="openSUSE Tumbleweed"
VERSION_ID="20240628"
Comment 2 Olav Reinert 2024-07-10 16:07:17 UTC
Modified bug to apply to Leap 15.6, where I have also found it.
Comment 3 Pablo Suárez Hernández 2024-07-11 09:32:25 UTC
Hmmm, this is strange. This issue should be already fixed in our Salt package by: https://github.com/openSUSE/salt/pull/588

I just tested this in a Leap 15.6 container and it works fine to me, without any errors. Using an "ext_pillar" configuration as you did (with ssh).

Are you using the "salt-master" and "python3-salt" packages coming from the official Leap repositories?

Could you please run:

rpm -q python3-salt --changes | grep 1210994

to double check the package contains the fix?

Thanks in advance!
Comment 4 Olav Reinert 2024-07-11 13:04:04 UTC
(In reply to Pablo Suárez Hernández from comment #3)
> Hmmm, this is strange. This issue should be already fixed in our Salt
> package by: https://github.com/openSUSE/salt/pull/588
> 
> I just tested this in a Leap 15.6 container and it works fine to me, without
> any errors. Using an "ext_pillar" configuration as you did (with ssh).

FYI: My master runs in a KVM (managed by libvirt) running a default install of Leap 15.6 using the "Server" system role.

> Are you using the "salt-master" and "python3-salt" packages coming from the
> official Leap repositories?

Yes - and I have not added any repositories.

> Could you please run:
> 
> rpm -q python3-salt --changes | grep 1210994
> 
> to double check the package contains the fix?

master:~ # rpm -q python3-salt --changes | grep 1210994
- Make sure configured user is properly set by Salt (bsc#1210994)
- Prevent _pygit2.GitError: error loading known_hosts when $HOME is not set (bsc#1210994)


Just retested - when I remove my override for salt-master.service, the error occurs again; and it disappears after I reinstate the override.