Bug 52104 (suse37104)

<!-- SBZ_reproduce  -->
Install Beta3 from text mode.
Choose czech language
When asked to enter a user name, try to use accented characters (especially
combined ones).

This is because of bug 48617 that keeps coming and going for years. However 
we do it, somebody will demand we do it the other way round. People complained 
about those characters being written to /etc/passwd in UTF-8, others 
complained when we used the current locale.  
 
The common conclusion was that non-ASCII characters there are simply undefined 
- see "man 5 passwd", so we turned it off. 

<!-- SBZ_reopen -->Reopened by kkaempf@suse.de at Fri Mar 26 15:49:23 2004, took initial reporter lsafar@suse.cz to cc

Hmm, the loginame cannot contain non-ascii characters. 
However, the user name (gecos field in passwd) can be UTF-8 encoded. 

It IS possible to set accentef gecos, I think this is not Ludek's problem.

Well, the problem is exactly as Klaus said - I'm sorry to not making me clearer
before. 

I understand you can't use non 7bASCII chars in login name. But YaST gets
confused while filling user's real name (geco field) in. Non-composed (single
stroke) chars looks OK, composed ones are causing problem described above.

The problem is with entering the characters, not with proccessing them via
yast2-users module.

As this works fine in Qt, I find it as some YaST ncurses problem.

The problem only occurs with uppercase combined characters? Is it possible to
enter other czech characters correctly?
How can I reproduce the bug? I don't know how to enter an uppercase accented
combined czech character.

I've just retried - it doesn't accept non ISO8859-1 (not sure) accented
characters in ncurses - it works for á,í combined or not. Doesn't work for ščřž
etc... QT interface is OK.

You don't have to reinstall to repro, just run users module.

If you set keyboard to Czech, then you can use numeric characters, our specials
are mapped there.
Mapping:
       Characters                    Accents
key:   1 2 3 4 5 6 7 8 9 0 ;         = + ~
char:  + ě š č ř ž ý á í é ů         ´ ˇ ˚

Obviously it's not only a ncurses problem.
It is also not possible to type correct czech characters on console nor to enter
them in linuxrc text input fields.


 

Well, localized console behavior during installation is broken - see Bug 52107.
It's because of lack of unicode support in other than yast consoles.

You're right composed characters don't work in unicode consoles in common. This
is a bigger issue. It looks it should be assigned to someone else, I don't know
who, sorry.

I have just tested with beta4. Seems to work now, I could enter czech characters
in first and second part of the installation. On console it is also possible.

Setting resolution to fixed.

Maybe it works now because kbd_mode and /etc/init.d/kbd have been fixed
to handle not only the first console correctly but the others as well?
See 

http://bugzilla.suse.de/show_bug.cgi?id=37367

By the way, we *do* accept non-ASCII UTF-8 login names now.
mingetty was changed to allow this.

<!-- SBZ_reopen -->Reopened by lsafar@suse.cz at Wed Apr  7 13:10:43 2004, took initial reporter kkaempf@suse.de to cc

Just reproduced on beta7. It behaves quite weirdly right now:

During installation:
- console is broken for non-composed accented characters (i.e. š, mapped on 3),
see  above. Composed ones are OK (!).
- impossible to enter composed accented characters to YaST

Installed system:
- console is broken for composed characters (i.e. Š, which you can't write other
way than ˇ (mapped to +) and S)
- same behaviour in YaST.

Is this console problem reproducable without having a czech keyboard? 

Well, I suppose with any keyboard which combines accented letter from dead keys
and the rest in unicode environment. I don't know how many of these keyboard
mappings we have. 

Maybe there are some on german keyboard as well - how do you write Ü? I suppose
you combine it from ¨ and U. Hopefully. I don't really know and don't have any
german / croatian / other keyboard :-). My LANG is set to cs_CZ.UTF-8.

I have just tested with RC1:
I can type german umlauts on console and enter them into YaST text fields.
The german umlauts are not combined from ¨ and o, a, u.
And I can enter czech characters  ě š č ř ž ý á í é ů in YaST. But I don't know
how to test with combined czech characters.
   

You're lucky not to have dead umlaut. I can enter ěščř as well (note original
text of the bug, especially the part "especially combined ones"). 

Problem is with these letters in CAPITALS. You can't write them with Shift as
numbers are mapped there. You have to combine them.

There is one key on english keyboard which writes "=", with shift it writes
"+".This is key we have dead "´" and with shift "ˇ". I wrote it twice already in
bug description above.


You can very easily reproduce this bug by these 4 steps (in czech console
environment):
1) enter "sometext"
output:~>sometext
2) hit "=" english key - it produces nothing as it's mapped to dead "´"
output:~>sometext
3) hit "a" key - it should produce á, combined from "´" and "a"
output:~>sometext
!!! here it is broken already- nothing appeared.
4) hit "a" key five times more - it should produce more "a" characters, without
accent
output:~>someaaaaa
!!! it's broken even more, whole command line starts to act madly, deleting
previously entered characters

* This comment was added by mail.
You can reproduce it like this on a standard German installation.
You don't need YaST2 to reproduce, just try the following on the
commandline of a console:

First verify that it works in non-Unicode mode:

    export LANG=de_DE@euro
    unicode_stop
    loadkeys de-latin1

Then type the dead accent left of the "Backspace" of a German
keyboard followed by an 'a', i.e. '`' followed by 'a'.

You get 'à'. OK, it works in non-Unicode mode.

Now switch back to Unicode again:

    export LANG=de_DE.UTF-8
    unicode_start
    loadkeys de-latin1
    dumpkeys | loadkeys --unicode

And try the same key sequence. It doesn't work anymore :-(

Apparently this is a kernel problem. See also:

    From: Radovan Garabik (garabik@melkor.dnp.fmph.uniba.sk)
    Date: Mon Oct 08 2001 - 14:53:13 EST
    http://www.ussg.iu.edu/hypermail/linux/kernel/0110.1/0148.html

and followups. 

Looks like it still has not been fixed properly.

Still present in 9.1-final. The prompt is completely broken after
an attempt to use dead chars.

*** Bug 52379 has been marked as a duplicate of this bug. ***

Any news?

Created attachment 18921 [details]
This patch (for 2.6 kernels) should enable UTF8 characters via dead-key

This patch (for 2.6 kernels) should enable UTF8 characters via dead-key.
Is that enough for you?

It depends if it's functional :-)
Could you provide an RPM for testing?

You can find it in my Export directory.

Well, I have to say, it's not working for us. It does not compose characters at
all (local output is `a and ^s instead of á and š), terminal connected from ssh
does not work as well (no output from composed chars, broken prompt).

test@hedgehog:~> cat /etc/SuSE-release 
SuSE Linux 9.1 (i586)
test@hedgehog:~> uname -a
Linux hedgehog 2.6.5-10-default #1 Mon May 10 12:06:21 UTC 2004 i686 athlon i386
GNU/Linux

Well, it needs updated loadkeys that can upload unicode compose tables.

And regarding ssh terminals, that's a different matter altogether.

I see. Sorry I didn't know that. What else do we need apart of updated kernel to
test it then?
I expected loadkeys shipped with unicode enabled SL9.1 (kbd-1.12-26 package)
beeing able of whatever desired functionality, uploading unicode compose tables
included.

And regarding the ssh terminals :-) - I'm glad it's not connected but it's
broken and that is what matters. Broken functionality should be either fixed or
disabled as whole. Is it worth to file another bug?

I have discussed this problem with Radovan Garabik in 2002. I have tested his
patch and it works. But he says, that his simple patch makes kernel binary
incompatible.

The patch attached above implements the functionality Radovan's patch did,
however it keeps binary compatibility and thus loadkeys needs to be patched
to use the new Unicode IOCTL variants.

Created attachment 20465 [details]
Patch for kbd-1.12 to be able to load UTF dead keys into kernel

Both patches sent to Andries Brouwer, maintainer of kbd, and the kernel
patch to LKML for approval.

Juergen, can you please prepare a test RPM with the above patch,
so that he can test whether entering accented UTF8 chars via
dead-key works with it?

find the latest rpm in ~jw/Export/kbd/kbd-1.12-28.i586.rpm 
 
sorry for the delay! 

Ok, we now have a kernel patch and updated kbd. Can you test it?

Should it be functional on 9.2-beta5?

Probably not, the patch wasn't applied, as far as I know.

Please reassign to me, if the patch is effective. 

I'm just in second stage of the installation. Czech installation.

Small czech characters (ě š č ř ž ý á í é ů ú) entered on the /Alt+F2 screen/
using /one key/ on keyboard doesn't work (they are printed as two-character
strings).

Small characters entered using the dead keys (ˇ and ´) works. Also big
characters entered using dead keys plus SHIFT also works. Also with CapsLock.

Testing in textual YaST (second installation) stopped it was impossible to enter
any character in the yast even to do anything else. So I killed y2base and the
system booted to the "login:" dialog.

After login:
Small czech characters entered withoud dead keys works. Small characters using
dead keys doesn't work (are not printed at all). Big characters using dead keys
doesn't work either (also are not printed at all).

Sorry for spamming (added into Cc:).

Can you please check with the attached patch and updated kbd?

Created attachment 25923 [details]
patch for the kernel that enables unicode composed characters

Created attachment 25924 [details]
Patch for the kbd package that correctly translates and loads unicode composed key definitions to the (patched) kernel

Above, you can find new versions of the patches mentioned before. I modified the
kernel patch to apply to a recent SUSE kernel and fixed a bug in the patch. I
also have a new version of the kbd patch, that applies to kbd-1.12. The previous 
versions could not work.

I tested with the czech keyboard (cz) in both unicode and non-unicode modes, and
it seems to work well for both composed and uncomposed diacritical characters.

LANG=cs_CZ
unicode_stop
loadkeys cz
-> input works

LANG=cs_CZ
unicode_start
loadkeys -u cz
-> input works

Created attachment 25926 [details]
patch for the kernel that enables unicode composed characters

in #17924 one bug was not fixed for s390, fixed now ... #17924 worked well on
i386 (none of the above had been tested on a s390 anyway)

oops ... #48 (attachment id=17926) is a new version of the kernel patch
(originally id=17923) ... NOT the kbd patch. Sorry for confusion.

So... has anyone tried the the patches out so far? 
(patches id 17926 and 17924)
Do they fix the problem for you?

I'm not sure who is the question mentioned for, but I'm original submitter, so 
it looks like it's me (no one else is responding though :-)) 
To be honest, I'm not able to test kernel and kbd patches - it's quite an 
overhead for anyone (not mentioning I'm personally probably not technically 
capable of such an task :-) ). 
 
If the patch meets sanity quickchecks, would it be possible to make RPMs out 
of patched vesions for wider ability of testing? It would help a lot. 
 
Thanks 

With a perspective of year and half it looks like this bug is not as critical as
I thought :-)

The bug is still present in last-updated SUSE LINUX Professional 9.3. It's
impossible to enter composed czech chatecters to yast users module nor to console.

Hello!

Due to the proposal from Andreas Jaeger, this bug will be closed as WONTFIX,
because there was no change to this bug for more than 2 months by now which
indicates a very low activity. 

If this bug is of relevance for the current release or should be kept open due
to another (not obvious reason), please reopen it ans state a reason for your
decision.

Please also check if the status-information for this bug is correct at all
correct if you reopen it. Generally the product-version should be elevated to
the current release in this case. Sorry if this causes you any inconvenience.

Kind regards,
the BNC-Screening-Team

Good. I just thought that we have a "needinfo" state for cases when asignee
needs an action from reporter. Inactivity in bugzilla could mean someone is
trying to fix the problem (which is apparently not the case). I don't think that
this is the right way of bug-triaging.

However, the bug is still present in SUSE LINUX 10 Final. Text console is not
able to write composed characters. I've updated bug attributes (hope it is what
I've been asked for), and status as well of course (back to open).

Reopen

Jiri, did you submit the patch already?

No. I know of another patch that might as well fix this and would not require kbd to be modified. I will test it and build a test kernel/kbd rpm next week.

I submited a patch to the kernel CVS. It works well on an installed system. I'll check if it works during installation as soon as the new kernel appears in our install mirror.

Fixed in 10.1 beta 7; thanks for the patience!