Bug 4411

Sorry, I must be blind - I see no ca(enhanced). 
Are you talking about some odd old layout with 2 groups?

hrm, sorry, I'm on crack; setxkbmap -layout ca -variant enhanced succeeded, so I
didn't check that carefully.

http://cvs.freedesktop.org/xorg/xc/programs/xkbcomp/symbols/ca_enhanced?rev=HEAD
seems to be a two-group + three-level layout, yeah.

So, what should we do about this one?

hmm.  arguably we should preserve it and make something that at least acts like
it (ca(enhanced) with a compat map from ca_enhanced), but it's a pretty ugly
map, and will probably need a fair bit of work.

Well, it was really ugly map. So I would prefer to bury it, unless someone would
come forward to make it single-group and compatible...

The ca layout already contains "multi" and "multi-2gr" variants to
implement CAN/CSA-Z243.200-92 national standard; "fr-legacy" is
the mapping used by some obscure OS.  Thus my understanding is
that Ivan Pascal did not want to clutter "ca" with more variants,
he only took the first group of ca_enhanced (renamed into "fr"),
the 2nd group being pretty useless.
So normally people want ca(fr); if they need some symbols from
this 2nd group, they should tell which ones so that we can
consider adding another variant.

So, will we close this one as NOTABUG or WONTFIX?

comment from the original reporter:
'Now, I use this configuration:
   Option      "XkbRules"   "xorg"
   Option      "XkbModel"   "pc105"
   Option      "XkbLayout"   "ca"
   Option      "XkbVariant"   "fr"

With this, I have all my french accents and keys, but this is not my real
keyboard configuration and I don't like this config.'

Hmmm, your reporter is right, the fr XKBVariant is ignored because
of the rule for 'ca' layout in the layout+model=symbols section of
rules/xorg.
You can add specific rules for fr and fr-legacy variants in the
layout+model+variant=symbols section of rules/xorg to avoid this
problem.  I will send a patch tonight.
The bug reporter may also need to add grp or lv3 options to change
modifiers.

Would something like just removing the special-casing for model=ca in the rules
and just adding a different default section in symbols/ca that includes all that
stuff, be sufficient?

Still broken, people will complain that it cannot
be combined with other layouts (say "ca,ru"), this
is exactly what happened in the past when layouts
did contain several groups.
IMO layouts should always contain only one group,
and combinations can be embedded into base.xml
(current DTD needs of course to be extended) so
that layout switcher applets can transform these
combinations into valid XKB rules.
Until this is done, your solution may indeed be
the best one from the user's point of view.

Right, I'm not proposing that the multi-group monstrosity be fixed at the moment.

(as an addendum, my proposal is exactly how the layout behaves at the moment,
with the translation in rules; with the exception that variants actually *work*.)

(In reply to comment #13)
> (as an addendum, my proposal is exactly how the layout behaves at the moment,
But current behavior is very inconsistent - the semantic of 'ca' depends on
whether it is alone or combined with other layouts. I suspect some people may be
frustrated...

Sure, I just figure that having arguably broken ca_enhanced + ca + ca(fr) is
better than an arguably broken ca, and a non-existent ca_enhanced and ca(fr).

> But current behavior is very inconsistent - the semantic of 'ca' depends on
> whether it is alone or combined with other layouts. I suspect some people
> may be frustrated...

Daniel suggests (roughly) to replace
  $pcmodels ca = pc(%m)+ca(multi)+ca(multi-2gr):2+group(rctrl_switch)
in rules/base by
  default partial
  xkb_symbols "standard" {
    name[Group1] = "Canada - Standard";
    include "ca(multi)+ca(multi-2gr):2+group(rctrl_switch)"
  };
in symbols/ca.  This is indeed much better because variants can
be used without any trouble, but all problems are of course not gone.
IMO such a change could be committed until a better solution is found.

>     include "ca(multi)+ca(multi-2gr):2+group(rctrl_switch)"
This still would leave us with 2-group variant - which suxx and breaks
everything (and will cause more bug reports, once users will try to combine it).
I am still in favor or removing ANY traces of canadian 2-group layouts/variants.
I'd prefer just to have 2 variants with the names giving the hint that they are
recommended to be used together.

well sure it sucks, but 'canadian doesn't work with multi-group layouts' beats
'canadian doesn't work with multi-group layouts, canadian french doesn't work at
all, neither does canadian french dvorak, nor the old canadian french "enhanced"
keymap'.  i'll be including that fix in the next ubuntu release, f.e., because
it certainly doesn't make current behaviour any *worse*, only far better.

> it certainly doesn't make current behaviour any *worse*, only far better.
What if we totally remove the "special" rules for Canadian - to make it
consistent and allowing people to combine Canadian variants with whatever they
want? Would it be inacceptable?

er, yeah, dude, that's what I'm proposing: remove all special-casing of ca in
the rules code, and moving the default stuff down into symbols/ca.

(In reply to comment #20)
> er, yeah, dude, that's what I'm proposing: remove all special-casing of ca in
> the rules code, and moving the default stuff down into symbols/ca.
OK. What I am doing in CVS is removing 'ca' from rules. If you want to create
2-group variant in symbols/ca - suit yourself, I don't think we should have it
in the main repository. At least not now.

i understand and agree with your quest for purity, but while we're coming up
with a single-group solution, arguably the best thing to do is to preserve
current behaviour.

Created attachment 3358 [details] [review]
add ca_enhanced, remove ca special-casing

this is the patch I'm running with

Well, let's keep this patch here (and bug open) for some while. 
The only REAL solution I see is to create ca_enhanced using shift levels 5 - 8.
Any volunteers? If noone finds time, I'll try to do it in a week or so...

Not exactly, ca_enhanced is "ca(fr)+group(rctrl_switch)"
and thus do not need to be defined (except for backward
compatibility purpose, of course).  You are talking here
about the expected behavior of the ca layout when no
variant is selected.

> Not exactly, ca_enhanced is "ca(fr)+group(rctrl_switch)"
> and thus do not need to be defined (except for backward
> compatibility purpose, of course).  You are talking here
> about the expected behavior of the ca layout when no
> variant is selected.
Probably I do no understand correctly. The ugly 8-level variant I propose (let's
call it ca(huge)) could be made default, if necessary. The main point is that it
would be unsplittable (which is good, I'd say) and don't break
"group-per-layout" convention

Anyone cared to try ca(multix) - it contains merged canadian multilingual
layout. If it works, I would close this bug (and remove partial multilingual
layouts)

Unfortunately ' setxkbmap "ca(multix)" ' does not really work for me.

I can't get working the "Selection de Groupe" key (RCTRL) - it beeps and 
returns a "~" when it is pressed. So characters in "groupe 2a" cannot be 
entered.

Some details about the keyboard layout:

http://externe.net/clavier-normalise/touche.gif
http://externe.net/clavier-normalise/

What has been the alternative to do instead of "setxkmap "ca(multix)"?

Stefan, could you please save your keyboard layout into textual format (using
xkbcomp) and attach it here?

Well, I forgot how to do this. :-(

# xkbcomp 
Error:            No input file specified

NP:)
xkbcomp :0 -xkb aa.xkb

> Well, I forgot how to do this. :-(

Created attachment 5792 [details]
xkbcomp output

Sergey, anything strange in my keyboard layout I attached?

Well, looking at it I cannot really understand which options did you use to get
this configuration. Could you please also give me your
model/layouts/variants/options?

Hmm. Does this help?

# setxkbmap "ca(multix)" -v
Warning! Multiple definitions of keyboard layout
         Using command line, ignoring X server
Trying to build keymap using the following components:
keycodes:   xfree86+aliases(qwerty)
types:      complete
compat:     complete
symbols:    pc(pc105)+ca(multix)
geometry:   pc(pc105)

Yes, thanks you.

I was dumb - this info is actuall available as names in the xkbcomp output.
Digging into the problem. ca(multix) will wreck my head, I know...:)

First thing I found - it was not wise to use F33-F35 as ISO_Level5_* keysyms.
Since they are used in the mousekeys. So I am switching to F21-F23. Actually,
xorg needs to add ISO_Level5_* keys to keysymdef.h.
But this not the whole story. The investigation is in progress...

Stephan, I committed some fixes to xkeyboard-config CVS - could you please
confirm that pressing RCTL + some keys gives you symbols from the high levels...
(which earlier were in the second group)?

Yes, this works now, but pressing <RCTL> gives you immediately a "~", which 
you need to delete first. :-(

Yes I know about "~" thingie, that is why I am not closing this bug yet:)

OK, here is the story.
This "~" character appears because I am using F21 instead of not-existing
XK_ISO_Level5_Shift. xterm/gnome-terminal/others interpret F21 so that the "~"
character appears. Now, keysymdef.h has these XK_ISO_Level5_* characters (it is
already in CVS:
http://webcvs.freedesktop.org/xorg/proto/X11/keysymdef.h?r1=1.9&r2=1.10), and I
am planning to replace F21 as soon as get the build with those characters. For
older xorg releases ... only patching of xterm/gnome-terminal(gtk?) could help.

Thanks, Sergey. That's great news! 
Will this make compat/level5 obsolete or is still required and needs to be 
adjusted? 
Could you apply a patch (probably for symbols/level5 and possibly 
compat/level5) for now?

> Will this make compat/level5 obsolete or is still required and needs to be 
> adjusted? 
It will have to be adjusted.

> Could you apply a patch (probably for symbols/level5 and possibly 
> compat/level5) for now?
Do you mean - using the symbols which nearly noone have in their xorg distro? It
would cause massive stream of bug reports, I'm afraid...

> > Could you apply a patch (probably for symbols/level5 and possibly 
> > compat/level5) for now?
> Do you mean - using the symbols which nearly noone have in their xorg
> distro? It would cause massive stream of bug reports, I'm afraid...
Sorry, I've written something stupid. I've meant attaching it to the bugreport 
for now. It might be useful for users/distributors, which alreay apply the 
patch for keysymdef.h ...

Created attachment 5868 [details] [review]
ISO_Level5.diff

Something like this?

This is exactly the patch I would apply:)

(In reply to comment #45)
> Created an attachment (id=5868) [edit]
> ISO_Level5.diff
> 
> Something like this?

JFYI, using this patch after recompiling X.Org with the keysymdef.h patch 
works fine including using the "multix" variant. :-)

OK, so actually this bug is fixed except that I'll wait a bit till new
keysymdef.h is settled.

Yes, the question is how long you need to wait. The problem is that using 
ISO_Level_* symbols with a non-patched keysmdef.h Xserver seems to break the 
keyboard input completely. :-( At least it did for me, also with a keyboard 
layout, which does not make use of these symbols ("de"). Maybe you can check 
during build, if the symbols are available? Otherwise I'm afraid you need to 
wait some years before you can apply this patch.

It is a very good question. Unfortunately there is no way to check this thing
reliably (grep on /usr/include/X11/keysymdef.h sounds a bit hackish)? Also, I'd
hate this kind of hacks, really.
What I think I'll do is wait till major distros have new keysymdef.h - and then
commit this patch. Would it be good enough?
You are right, without new keysymdef.h the entire configuration is broken,
unfortunately...

(In reply to comment #49)
> Yes, the question is how long you need to wait. The problem is that using 
> ISO_Level_* symbols with a non-patched keysmdef.h Xserver seems to break the 
> keyboard input completely. :-( At least it did for me, also with a keyboard 
> layout, which does not make use of these symbols ("de"). Maybe you can check 
> during build, if the symbols are available? Otherwise I'm afraid you need to 
> wait some years before you can apply this patch.

Well, I think for this you need to wait untile X.Org 7.2 has been released and 
the major distributions have switched to it. X.Org 7.2 is planned to be 
released in November 2006 and given that many distributions only are released 
1-2 times a year you might need to wait one year or even longer before you can 
apply it.

If I manage to convince major distros to patch their X as updates to current
stable version (which I think is perfectly safe), I'd be inclined to commit this
even before 7.2..

Well, I've done this now for SLES10/SLED10 and will provide an update for SUSE 
10.1.

Thanks Stefan. If same change would be propageted to Ubuntu 6.06 (+Debian) and
FC5 - I think I could "safely" commit...

Sergey, you can apply this patch now and add a configure option to
enable/disable it.

> Sergey, you can apply this patch now and add a configure option to
> enable/disable it.
It would be rather hackish enable/disable option, involving some sed machinery
to replace the new symbols with the old ones (in two files). And, as usual, the
question is what would be the default behaviour...

I am inclined to commit the fix just before the next release of xkeyboard-config
(around august-septemper) - and attach the extensive explanation along with the
patch to xorg (and explanation on how to make it compatible with old xorg - and
BUGGY). I know, I am kind of pushing things - but otherwise the process would be
inacceptably long...

Actually, there is another option to try - instead of XK_ISO_Level5* put
(temporary) literal hexadecimal values 0xFE1? - I'll check it, and if it works,
this could be a temporary solution for, say, a year...

The idea with explicit hexadecimal values worked out quite well. So I committed
it. In a year or two I'll replace hex values with XK_ISO_Level5*. Is everybody
happy?

I doublechecked that using explicit hexadecimal values works. Thanks a lot, 
Sergey! You're amazing!

Firstly, where do I add/apply this "hexadecimal values 0xFE1" hack?

Secondly, I'm copying over a comment (rant) I posted on my original bug report 
at http://bugs.kde.org/show_bug.cgi?id=128704. My intention is not to urge, 
annoy or "piss off" devs (svu) but rather to put some emphasis on the 
(philosophical/sociological) implications of the ca_enhanced issue. All in all, 
keyboard maps shouldn't just be "dropped" arbitrarily.

...
     There are two layouts used in Canada for french keyboard. The first one 
(-layout ca -variant fr) will generate the accented characters on a single 
keypress, meaning that specific keys are assigned a specific character. I don't 
know who came up with this layout but I suspect that the person did no write 
French or had a very bad grammar since they seem to have dropped one important 
character: ù (` + u). I have yet to find how to generate this character with 
the ca(fr) layout. 
 
     The second layout (traditionally ca_enhanced), was much more versatile 
since the generation of the _most_ accented character is the result of a 
sequence of two keypresses. For example: 
 à = (` + a) 
 ù = (` + u) 
 è = (` + e) 
 î = (^ + i) 
 ...and so on. There is one exception to this which is the é. This is logical 
since this accent, in French, is only applicable to the letter "e". 
 
 <RANT> 
     I've been pushing Linux as a valid desktop OS for many years now and this 
is one of the most frustrating elements I have to go through. It's happened 2 
or 3 times in the past 3-4 years that ca_enhanced either "disappeared" or "was 
renamed to... and all downstream apps have to be hacked to recognize it". 
 
     Some will dismiss this as a minor issue, but it is NOT. The keyboard is 
one of the primary interfaces to the computer and can become a very high source 
of frustrations for any user (from the total computer neophyte the most 
advanced power user, believe me, my patience wears thin when I am _forced_ to 
use a french keyboard and keep typing é in stead of / in a terminal...with all 
the implied command line corruptions). 
 
     I know I'm being a utopist but it would be damned nice if the X(org) 
people would just STOP re-defining/re-naming/eliminating this keyboard layout 
since it has a _very_ negative impact on users and is throwing a dark shadow 
over the entire pro-Linux community. This type of problem will be used and 
abused of (with with some right IMHO) by the computer-illiterate MBAs when it 
comes to bashing on Linux as an inappropriate OS for Schools, Universities, 
Governments, etc... 
 
     So, for the sake of the evolution and acceptance of the Linux/GNU in the 
wide community (well...I guess this only applies to Québec/Canada, 
insignificant, right?), STOP fscking around with the ca_enhanced keyboard 
layouts! 
 </RANT>

The hexvalues hack is in xkeyboard-config CVS now.

Original ca_enhanced was ugly. It contained several groups so xkeyboard-config
would never contain it in its original form, period. As I said earlier, if
someone (you?) would hack the single-group version of it, I would happily accept
it as ca(enhanced) and add the compatibility rule mapping ca_enhanced to it.

Yeah, I was told about the horrority of ca_enhanced (but it worked!). My 
background is Electrical Engineering with a good grasp of state machines, low 
level (ANSI) C programming, and some C++ programming. I'd be glad to hack it 
but I'm afraid setting up the environment and the learning curve would be 
prohibitive unless you prove me wrong :/ (note that I can easily switch to 
modular being under Gentoo ;). Could this be played with/modified in a "small" 
environment, is there a good document that would explain this keyboard 
leveling? I do have some interest in the keyboard part of X for interfacing 
purposes (re-assigning keys and all, trying to get it to "act" exactly like 
a "Windows keyboard").

> Yeah, I was told about the horrority of ca_enhanced (but it worked!).
In most circumstances - yes it worked. But not everywhere.

> modular being under Gentoo ;). Could this be played with/modified in a "small" 
> environment, is there a good document that would explain this keyboard 
> leveling? 
Sure you can do play with these things locally (since xkbcomp can work with
local XKB repository).

I would recommend starting with http://pascal.tsu.ru/en/xkb/, more links
available on xkeyboard-config page
(http://www.freedesktop.org/wiki/Software_2fXKeyboardConfig). Also, there is
http://community.livejournal.com/xkbconfig/

Well, I won't have to go through with "fixing the layout" finally. As of xkeyboard-config-0.8, 
the CA layout with the FR variant is equivalent to the ca_enhanced. Tested with:

setxkbmap -model pc104 -layout ca -variant fr

So this is ok IMHO.

1.I succeeded in applying this hack to make ca(multix) work and used this method
for fixing my german keyboard but why can`t I just map noSymbol onto <LCTL>
(left control) and map the LevelFive modifier used in types directly onto <LCTL>
 ? I don`t need this key to ever produce a symbol. I think this would be a great
simplification.

Well, it can be done this way (if I understand you right) - just do not like the
idea of putting the modifiers directly into symbol files (though I know it is
allowed). Just to keep the things separate, you know...

(In reply to comment #65)
> 1.I succeeded in applying this hack to make ca(multix) work and used this method
> for fixing my german keyboard but why can`t I just map noSymbol onto <LCTL>
> (left control) and map the LevelFive modifier used in types directly onto <LCTL>
>  ? I don`t need this key to ever produce a symbol. I think this would be a great
> simplification.
> 

Is it really possible? 
If you want to keep it seperate, what about putting "key <RCTL> {[NoSymbol]};" 
in the symbols file and "interpret <RCTL>+Any {...}" in the compat file?
By the way, I don`t understand why the line "modifier_map Mod3 {...};" is 
needed.

> If you want to keep it seperate, what about putting "key <RCTL> {[NoSymbol]};" 
> in the symbols file and "interpret <RCTL>+Any {...}" in the compat file?
> By the way, I don`t understand why the line "modifier_map Mod3 {...};" is 
> needed.
RCTL is bound to 0xfe11 and 0xfe11 is mapped to "real" modifier Mod3 because
Mod3 is used for keeping the state (level5) in X server.
In the compat file, they use keysyms (0xfe11), not keycodes (RCTL), so I am not
sure where "interpret <RCTL>" would work..

Mass reopen.  The "LATER" resolution is lame, I'm deleting it.  Consider LATER to have arrived.

Now, level5 seems to be ok, as long as ca(multix). I guess the issue is closed.