Bug 1212591

I don't quite understand. No matter if you are doing a normal installation at the machine's console, no matter if using the Qt user interface (UI) or the NCurses (text based) one, no matter if you are doing the same via SSH or VNC, the installation workflow is the same.

So what exactly did you do in either case? Are you sure the scenarios are really comparable?

Also, I am not sure how far that is supported in the first place. There is the option to import existing user accounts from a previous installation. That's not what you mean, right?

During an upgrade to a newer release, the machine's SSH key should be preserved, of course. But for a new installation, I am not so sure. It might not even be desired to retain a previous SSH key from an old Red Hat or Debian or Ubuntu installation; that would be more questions the user would have to answer during installation, and there are way too many already.

I'll ask my team mates how this is supposed to behave.

It's possible that a SSH-based installation (inst-sys) preserves SSH-keys from the installation itself rather than from some other systems found on disks(s). SSH has to be started before YaST or hardware probing including disk scanning. That's done in Linuxrc / inst-sys.

I received answers that this is indeed an officially supported feature: It should import the SSH keys from the "most recently accesses Linux system" on that machine, whatever exactly that means.

So please attach y2logs generated with the supplied "save_y2logs" script for one of the problematic cases.

See also

  https://en.opensuse.org/openSUSE:Report_a_YaST_bug#Attachments_-_y2logs,_hwinfo_etc.

The feature of preserving previous installation ssh key has been there since many many years. Starting with Leap 15.1 during the installation process in the screen where you enter the root password it is possible to select from which partition to import the ssh key (in case there are several installation on the disks).
My installation workflow is exactly the same with SSH or other methods.
I have verified (as comment two suggested) that indeed the ssh key generated during the installation process in case of a ssh installation are the one copied on the installed system rather then the one from a previous installation, therefore ignoring what is selected in the Authentication for "root" screen.

A workaround I found is to delete the ssh keys from /etc/ssh just before starting yast.ssh. In this way no key from the installation process are copied to the installed system and the previous installation keys are preserved. Looking at the yast log I found that the copy process happens in the file: usr/share/YaST2/lib/installation/clients/ssh_settings_finish.rb.
The command executed is:
/bin/cp -a /etc/ssh/*key* 

At the time such copy command is executed, in the installed system there are already some keys in /etc/ssh: either the key of the previous installation or some new generated keys.
There is no need to execute the key's copy in ssh_settings_finish.rb
I think the fix should be to remove completely the copy command from ssh_settings_finish.rb

If you still need the full yast logs let me know.

Yes, we do. Thanks in advance.

Created attachment 867772 [details]
y2 logs: ssh key not preserved

logs for the case where the ssh host keys are not preserved.
I could not spot any relevant message about the issue.

Created attachment 867774 [details]
y2 logs: ssh key preserved

logs for the case where the ssh host keys are preserved
by executing: rm /etc/ssh/*key* and then yast.ssh
while executing the hook: before_ssh_settings_finish
there is a message about the failed attempt to overwrite the keys

Giacomo,

thanks for the y2logs for both the good and the bad case. That will make debugging this a lot easier.

I think you are right: That copying in ssh_settings_finish.rb destroys all the work done shortly before.

In both cases, the good one and the bad one, the ssh keys from the previous installation are faithfully imported, stored and finally written to the newly installed system.

But in the bad case, they are overwritten shortly afterwards with the (probably freshly generated for the installation) ssh keys from the inst-sys with that extra /bin/cp in ssh_settings_finish.rb.



Good Case: SSH Key Preserved
============================

2023-06-23 08:19:28 <1> install(4697) [Ruby]
  clients/copy_files_finish.rb(copy_ssh_files):176
  Copying SSH keys and config files

2023-06-23 08:19:28 <1> install(4697) [Ruby]
  installation/ssh_config.rb(write_files):141
  Writing SSH keys and/or configuration: keys: true, config: false

2023-06-23 08:19:28 <1> install(4697) [Ruby]
  installation/ssh_key.rb(write_files):46
  Write SSH keys to /mnt/etc/ssh:

ssh_host_rsa_key:
  ssh_host_rsa_key
  ssh_host_rsa_key.pub

ssh_host_dsa_key:
  ssh_host_dsa_key
  ssh_host_dsa_key.pub

ssh_host_ecdsa_key:
  ssh_host_ecdsa_key
  ssh_host_ecdsa_key.pub
  
ssh_host_ed25519_key:
  ssh_host_ed25519_key
  ssh_host_ed25519_key.pub
  
2023-06-23 08:19:28 <1> install(4697) [Interpreter]
  clients/inst_finish.rb:117
  Called YaST client returned.

---

2023-06-23 08:19:36 <1> install(4697) [Ruby]
  clients/ssh_settings_finish.rb(main):54
  starting ssh_settings_finish

2023-06-23 08:19:36 <3> install(4697) [bash]
  ShellCommand.cc(shellcommand):78
  /bin/cp: cannot stat '/etc/ssh/*key*':
  No such file or directory
  ^^^^^^^^^^^^^^^^^^^^^^^^^

  This is what prevents overwriting the keys.


2023-06-23 08:19:36 <1> install(4697) [Ruby]
  clients/ssh_settings_finish.rb(main):99
  ssh_settings_finish finished



Bad Case: SSH Key Not Preserved
===============================

2023-06-23 08:48:14 <1> install(4514) [Ruby]
  clients/copy_files_finish.rb(copy_ssh_files):176
  Copying SSH keys and config files

2023-06-23 08:48:14 <1> install(4514) [Ruby]
  installation/ssh_config.rb(write_files):141
  Writing SSH keys and/or configuration: keys: true, config: false

2023-06-23 08:48:14 <1> install(4514) [Ruby]
  installation/ssh_key.rb(write_files):46
  Write SSH keys to /mnt/etc/ssh:

ssh_host_rsa_key:
  ssh_host_rsa_key
  ssh_host_rsa_key.pub
  
ssh_host_dsa_key:
  ssh_host_dsa_key
  ssh_host_dsa_key.pub
  
ssh_host_ecdsa_key:
  ssh_host_ecdsa_key
  ssh_host_ecdsa_key.pub
  
ssh_host_ed25519_key:
  ssh_host_ed25519_key
  ssh_host_ed25519_key.pub
  
2023-06-23 08:48:14 <1> install(4514) [Interpreter]
  clients/inst_finish.rb:117
  Called YaST client returned.

---

2023-06-23 08:48:21 <1> install(4514) [Ruby]
  clients/ssh_settings_finish.rb(main):54
  starting ssh_settings_finish

(No error, so the copy operation was successful,
 overwriting the ssh keys with those from the inst-sys)

2023-06-23 08:48:21 <1> install(4514) [Ruby]
  clients/ssh_settings_finish.rb(main):99
  ssh_settings_finish finished

So the obvious fix is to remove that extra copying step.

But over the years, I became wary of such seemingly simple solutions; there might be a reason for that copying. I'll have to dig a bit in the sources and ask around if there still is one; maybe in one of the more arcane installation or system upgrade modes.

At the very least, the copy should be prevented if there already are keys in the installation target's /etc/ssh directory.

That copying operation goes back 18 years to April 2005 when the history of our Git comes to its earliest limit: It was imported from SVN some time in those years from CVS where we obviously didn't or couldn't import any previous history; which should have gone back to mid-1999. 

https://github.com/yast/yast-installation/blob/SLE-10/src/clients/ssh_settings_finish.ycp#L51

> WFM::Execute (.local.bash, "/bin/cp -a /etc/ssh/*key* " + Installation::destdir + "/etc/ssh/");


That might mean that it's so old that it has become obsolete, and it might be pure coincidence that in most cases it doesn't cause any problems because /etc/ssh in the inst-sys does not contain any keys. I'll check.

Created attachment 867834 [details]
Screenshot: Leap 15.5 installation started with "ssh=1"

As suspected, without using ssh for the installation, there is no host key in /etc/ssh.

But when starting the installation with "ssh=1" at the boot prompt, it generates a new ssh host key (see screenshot) while setting up the network for ssh, and from that time on, there are host keys in /etc/ssh:

0:install:/etc/ssh # ls *key*

ssh_host_dsa_key	ssh_host_ed25519_key
ssh_host_dsa_key.pub	ssh_host_ed25519_key.pub
ssh_host_ecdsa_key	ssh_host_rsa_key
ssh_host_ecdsa_key.pub	ssh_host_rsa_key.pub


And those are copied over at the end of the installation so any later ssh connection doesn't complain about a changed host key.

And that is what overwrites the host key that were imported earlier in the installation workflow from a previous installation.

So that copying of the freshly generated host key from the inst-sys into the target system needs to check if the target now already contains a key before it attempts to copy the one from the inst-sys.

Moved to our Trello task queue (non-public, sorry for that).

> So that copying of the freshly generated host key from the inst-sys into the  > target system needs to check if the target now already contains a key before  > it attempts to copy the one from the inst-sys.

When the hook: before_ssh_settings_finish is executed and the key (if any) from the inst-sys are copied into the target system, there are always already host keys in the target system:
if there was a previous installation, such keys were preserved,
if there was no previous installation a new set of keys has already been created. So simply checking if the target already contains a key will never allow to copy the key from inst-sys

If there is no previous installation and the keys created for the ssh installation should be copied from inst-sys to the target system then I think there are two possible ways to proceed:

1) yast should not generate new keys for the target system because such step will be carried later by the hook before_ssh_settings_finish

or

2) the hook before_ssh_settings_finish should able to know if the key present in the target system should be overwritten (no previous installation) or not (they came from a previous installation).