(In reply to comment #0)
> Using shift:both_capslock does not have the desired result.
> Functionality of the left Shift key is lost and Caps Lock cannot be ativated by
> pressing both Shift keys (in any order).
> 
> Adding
> ---
>     actions [Group1] = [
>       SetMods(modifiers=Shift),
>       LockMods(modifiers=Lock)
>     ]
> ---
> to both shift key mappings in symbols/shift gives close to the desired result.
> One remaining problem is that Caps Lock is only activated when the left Shift
> key is pressed first, in other words: the left Shift key is still not properly
> mapped.
> 
> Relevant parts of the xkbcomp output:
> ---
>     key <LFSH> {
>         type= "TWO_LEVEL",
>         symbols[Group1]= [         Shift_L,       Caps_Lock ],
>         actions[Group1]= [ SetMods(modifiers=Shift), SetMods(modifiers=Shift) ]
>     };
> --
>     key <RTSH> {
>         type= "TWO_LEVEL",
>         symbols[Group1]= [         Shift_R,       Caps_Lock ],
>         actions[Group1]= [ SetMods(modifiers=Shift), LockMods(modifiers=Lock) ]
>     };
> --
>     modifier_map Shift { <LFSH> };
>     modifier_map Lock { <LFSH> };
>     modifier_map Shift { <RTSH> };
> ---
> 
> The behaviour is found in two different systems, both using the dvorak-alt-intl
> layout with minor modifications and the capslock:backspace option.

The above xkbcomp output was generated using the modified symbols/shift. With an unmodified version, this is the relevant output:
---
    key <LFSH> {
        type= "TWO_LEVEL",
        symbols[Group1]= [         Shift_L,       Caps_Lock ]
    };
--
    key <RTSH> {
        type= "TWO_LEVEL",
        symbols[Group1]= [         Shift_R,       Caps_Lock ]
    };
--
    modifier_map Shift { <LFSH> };
    modifier_map Lock { <LFSH> };
    modifier_map Shift { <RTSH> };
---
(as expected)

Thanks a bunch, your actions seem to fix the problem. Please check in git.

Part of the problem remains, part of the solution is wrong.
First a correction of the solution, next the remaining problem.


The symbols should be:
---
      symbols[Group1] = [ Shift_*, Shift_* ],
----
because having Caps_Lock in the second place raises a key-release for Caps Lock, every time the shift key is released (it puts itself into the second level), instead of the correct shift key release event.


The remaining problem is that pressing the right shift key first, still does not trigger Caps Lock. This is because the left shift key actions are not set correctly. The relevant output of xkbcomp is the actions line in:
---
    key <LFSH> {
        type= "TWO_LEVEL",
        symbols[Group1]= [         Shift_L,       Caps_Lock ],
        actions[Group1]= [ SetMods(modifiers=Shift), SetMods(modifiers=Shift) ]
    };
---
Note the SetMods(modifiers=Shift) instead of LockMods(modifiers=Lock).
The original commit for the shift_both_capslock code contained the following compatibility code:
---
  partial xkb_compatibility "assign_shift_left_action"  {
      // Because of the irrevertable modifier mapping in symbols/pc <LFSH> is 
      // getting bound to the Lock modifier when using 
      // symbols/shift(both_capslock), creating unwanted behaviour. 
      // This is a quirk, to circumvent the problem.
      interpret Shift_L {
          action = SetMods(modifiers = Shift);
      };
  };
---
Perhaps this is key to the solution.

> because having Caps_Lock in the second place raises a key-release for Caps
> Lock, every time the shift key is released (it puts itself into the second
> level), instead of the correct shift key release event.
I noticed KeyRelease for CapsLock - but I do not have any issues with that side-effect. Do you? Traditionally X11 uses keypress, I do not remember any app that would depend on keyrelease. I will check your version though.

> The remaining problem is that pressing the right shift key first, still does
> not trigger Caps Lock.
Ghm. That's strange. I checked the code - pressing right shift first - works for me. Are you sure you took the stuff from git? I changed LFSH:
    actions[Group1] = [ SetMods(modifiers=Shift), LockMods(modifiers=Lock) ]

That way it is in sync with RTSH - and both scenarios work: LFSH first and RTSH first.

shift:both_capslock was originally intended to be used together with the compat module caps(caps_lock) which explicitly defines the desired action and therefore allows the use of the keysym Caps_Lock on other levels than level1.

Generally you don't want to define the actions in a symbols-file, but instead define them on the desired keysym.

I suggest to include the compat module caps(caps_lock) by default and revert 85157ad78fd4075d91bbd9bb6fc4ed65de8fb2fe.

If there are concerns about including it by default you could as well add a rule including it when the option shift:both_capslock is used.

Stephan, could you provide a patch that would be the best solution (in your opinion), so other ppl could test?

Created attachment 41565 [details] [review]
suggestion1: revert previous fix and include caps(caps_lock) by default

Created attachment 41566 [details] [review]
suggestion2: revert previous fix and include caps(caps_lock) when option shift:both_capslock is specified

I remember compat/misc(assign_shift_left_action) being needed together with shift:both_capslock, otherwise the left shift key would lose it's normal shift functionality.
Probably this is also needed. I will post an additional patch for this later on.

Created attachment 41569 [details] [review]
left shift-key fix

Thanks for your patches

I would really like j.witteveen@gmail.com to test them and comment (sorry I do not see the name in bugzilla). Since he/she was the person who actually opened the bug...

I tested with the patches 'suggestion1' and 'left shift-key fix'.
I did not test the patch for 'suggestion2', because I could not find an easy way to apply it to my in-use system. It is, however, the suggestion I would prefer.

RESULTS
It works beautifully now. Of course, the key-release of the Shift keys still emits the key-release event of Caps Lock, but as far as I could see, there are no direct side-effects of this noticeable.

Thanks a bunch! I am looking forward to see the patches pushed.

Ok, so now that everybody's happy, pushing it.
Thanks for cooperation, lads

*** Bug 34318 has been marked as a duplicate of this bug. ***

The bug #34318 says the current fix is broken. Lads, could you please settle the solution that would make both sides happy?

I've explained my position in the duplicate blog post.
I don't think this patch is at fault.

(In reply to comment #16)
> I've explained my position in the duplicate blog post.
> I don't think this patch is at fault.

The patch is at fault because it introduced a regression.  It broke working
configurations.

Assigning the Control_L keysym to the caps lock key (which is what the
ctrl:nocaps option does) is not a suitable option: it means that applications
can no longer distinguish between the caps lock and left control keys.  Since they are different keys, applications need to be able to distinguish between them.

For a specific example, assigning keycode 66 = Control_L makes it impossible to
configure separate bindings for caps lock and control in openarena.

Created attachment 43531 [details] [review]
Patch to disable default include of caps(caps_lock)

The patch reallows the rebinding of capslock.
But I would recommend using some other keysym (there are plenty which are generally not used e.g. Meta_L, Hyper_L).

Best thing would be to add a xkb option to do the thing you desire (then we could even keep the capslock-keysym because xkb can overwrite defined actions what xmodmap doesn't seem to be able to).

I am totally against changing the default action for Caps_Lock:

LockMods(modifiers = Lock)

This is reasonable default that should be kept and should not require any extra xkb options.

So I think I support Stephan's idea: map CAPS keycode to some different keysym (not Caps_Lock) and then use ctrl:nocaps.

Either you want Caps to behave like Control - or not. Nick, it seems you want some strange scenario "behave like Control, but be slightly different". If application really wants to distinguish (why?) - it should check keycodes (which are going to be different).

(In reply to comment #20)
> I am totally against changing the default action for Caps_Lock:
> 
> LockMods(modifiers = Lock)

I have no idea what the above means.

> This is reasonable default that should be kept and should not require any
> extra xkb options.
>
> So I think I support Stephan's idea: map CAPS keycode to some different
> keysym (not Caps_Lock) and then use ctrl:nocaps.
>
> Either you want Caps to behave like Control - or not. Nick, it seems you want
> some strange scenario "behave like Control, but be slightly different".

I want three keys assigned to the 'control' modifier.  Those three keys are:
Caps_Lock, Control_L and Control_R.  This has worked perfectly fine for as long
as I've used X, and it wasn't broken until the current release of
xkeyboard-config.

What is the point of having configurable 'lock' and 'control' modifiers if they
can't actually be configured?

> If application really wants to distinguish (why?) - it should check keycodes
> (which are going to be different).

Keycodes don't have any meaning from the point of view of the user.  No
X application that I'm aware of uses the keycodes for key bindings: they always
use symbolic names.  Proposing that I try and fix every app because of a
regression in xkeyboard-config seems a little ridiculous.

> > LockMods(modifiers = Lock)
> I have no idea what the above means.
That's what in compat/caps file - assigning LockMods action to Caps_Lock keysym.

> I want three keys assigned to the 'control' modifier.  Those three keys are:
> Caps_Lock, Control_L and Control_R.  This has worked perfectly fine for as long
> as I've used X, and it wasn't broken until the current release of
> xkeyboard-config.
Well, it was not really fine - it was a side-effect of the bug that was reported and fixed here.

> What is the point of having configurable 'lock' and 'control' modifiers if they
> can't actually be configured?
They can. Here we have traditional conflict between usability and configurability. If we disable caps(caps_lock) by default, as Stefan proposed, - we get "nothing by default" scenario - and then you can do with Caps_Lock whatever you want. Simple and consistent, right - but very bad from usability POV. By default, people want CapsLock to work properly, without necessity to provide extra XKB options. And that sane default config sets some limit on what you can (and cannot) do with Caps_Lock keysym.

> use symbolic names.  Proposing that I try and fix every app because of a
> regression in xkeyboard-config seems a little ridiculous.
I agree. But IMHO equally ridiculous (or at least unclear) is the requirement to have Caps_Lock act as Control - but keep it different.

Perhaps, here is my major understanding. Could you please elaborate - why do you want Caps to behave like Control, but at the same time be distinguishable. I cannot think of any use case.

(In reply to comment #20)
> So I think I support Stephan's idea: map CAPS keycode to some different keysym
> (not Caps_Lock) and then use ctrl:nocaps.

Let's see what happens if I assign the Hyper_L keysym to the caps lock key:

  (1) I can add it to the 'control' modifier group and have it actually work
      as a control modifier (yay).

  (2) The key does not perform its expected function in DOSBox.

  (3) The key is not bindable at all in openarena.

I stopped testing at this point -- I assume Meta_L has similar problems.

(In reply to comment #22)
> Perhaps, here is my major understanding. Could you please elaborate - why do
> you want Caps to behave like Control, but at the same time be distinguishable.
> I cannot think of any use case.

I already gave you an example in comment #17:

    For a specific example, assigning keycode 66 = Control_L makes it
    impossible to configure separate bindings for caps lock and control in
    openarena.

> Let's see what happens if I assign the Hyper_L keysym to the caps lock key:
> 
>   (1) I can add it to the 'control' modifier group and have it actually work
>       as a control modifier (yay).
> 
>   (2) The key does not perform its expected function in DOSBox.
> 
>   (3) The key is not bindable at all in openarena.
> 
> I stopped testing at this point -- I assume Meta_L has similar problems.
Thanks! That explanation makes sense to me. But I think what you're trying to achieve is somewhat "unfair" - you want "two behaviors" from one key, two functions: to be caps lock and control at the same time! You want it to be Control modifier for all applications except DOSBox ("expected function" = Capslock, I guess?) and openarena. That dualistic behavior I would see as inconsistency (if it was allowed earlier).

You can assign some other key (well, for example, SCLK=scroll lock) to work as Caps_Lock (for all intents and purposes, including DOSBox, openarena etc) and CAPS key work as Control_L. That would be consistent and logical, that would follow "one key - one function" principle.

(In reply to comment #25)
> But I think what you're trying to achieve is somewhat "unfair" - you want
> "two behaviors" from one key, two functions: to be caps lock and control at
> the same time!

What I want is a third control key that is distinguishable from the other two
in applications.  I do not want the caps lock key to do anything other than
be a control modifier (I don't want it to actually cause capital letters to
appear).

> You want it to be Control modifier for all applications except DOSBox
> ("expected function" = Capslock, I guess?)

Yes.  As an emulator, it is important for the keys in DOSBox to behave exactly
as they would on the original system.  It seems that, to this end, DOSBox
expects the caps lock key to use the Caps_Lock keysym.

> and openarena. That dualistic behavior I would see as inconsistency (if it
> was allowed earlier).

Openarena doesn't care about modifiers, and neither do its players.  All the
keys are simply buttons.  I expect to be able to bind every key on my keyboard
to distinct functions.

> You can assign some other key (well, for example, SCLK=scroll lock) to work
> as Caps_Lock (for all intents and purposes, including DOSBox, openarena etc)

Then what do I do with my scroll lock key?

> and CAPS key work as Control_L. That would be consistent and logical, that
> would follow "one key - one function" principle.

Modifiers, by their very design, give two functions to keys.  They behave as
keys (applications receive keydown/up events) and modifiers (applications
receive the modifier state along with other keydown/up events).

(In reply to comment #26)
> (In reply to comment #25)
> > But I think what you're trying to achieve is somewhat "unfair" - you want
> > "two behaviors" from one key, two functions: to be caps lock and control at
> > the same time!
> 
> What I want is a third control key that is distinguishable from the other two
> in applications.  I do not want the caps lock key to do anything other than
> be a control modifier (I don't want it to actually cause capital letters to
> appear).

Note that the left and right control keys are distinguishable because they
use different keysyms.

> I do not want the caps lock key to do anything other than
> be a control modifier (I don't want it to actually cause capital letters to
> appear).
In that case - why bother about this particular key being caps lock in DOSBox?

> Yes.  As an emulator, it is important for the keys in DOSBox to behave exactly
> as they would on the original system.  It seems that, to this end, DOSBox
> expects the caps lock key to use the Caps_Lock keysym.
If they are looking for Caps_Lock keysym - you can map it to the other key. As I said, it is wrong - expect the key to have 2 functions depending on the app.

> Openarena doesn't care about modifiers, and neither do its players.  All the
> keys are simply buttons.  I expect to be able to bind every key on my keyboard
> to distinct functions.
Why cannot it bind Hyper_L? From XKB POV all keysyms are equal.

> Then what do I do with my scroll lock key?
Do you actually use it? If so, find some key that you don't use. But do not expect two functions from one key.
 
> Modifiers, by their very design, give two functions to keys.  They behave as
> keys (applications receive keydown/up events) and modifiers (applications
> receive the modifier state along with other keydown/up events).
That is very true, in general. But I already explained why in this particular case the modifier is bound to the keysym by default - because otherwise we'll have to force people to use xkb option just to get reasonable Caps_Lock behaviour.

> Note that the left and right control keys are distinguishable because they
> use different keysyms.
Yes. And if people want different functions - they have no problem mapping other keysyms to those keycodes. See, for example, symbols/ctrl(swapcaps)

(In reply to comment #28)
> > I do not want the caps lock key to do anything other than
> > be a control modifier (I don't want it to actually cause capital letters to
> > appear).
> In that case - why bother about this particular key being caps lock in DOSBox?
> 
> > Yes.  As an emulator, it is important for the keys in DOSBox to behave exactly
> > as they would on the original system.  It seems that, to this end, DOSBox
> > expects the caps lock key to use the Caps_Lock keysym.
>
> If they are looking for Caps_Lock keysym - you can map it to the other key.
> As I said, it is wrong - expect the key to have 2 functions depending on the
> app.

I don't expect it to depend on the app.  I expect the caps lock key to do the
same thing in every app:

  (1) Emit the Caps_Lock keysym (I think I've found enough apps which break
      to show that changing this is a bad idea...)
  (2) Cause the control modifier bit to be set for key events while it is held.

Until now, this has always been possible.

What's important is not that it's possible to create Caps_Lock keypresses with
some key on the keyboard.  What's important is that the key to the left of the
one labeled 'A' does so.

> > Openarena doesn't care about modifiers, and neither do its players.  All the
> > keys are simply buttons.  I expect to be able to bind every key on my keyboard
> > to distinct functions.
>
> Why cannot it bind Hyper_L? From XKB POV all keysyms are equal.

I have no idea.  You'd have to ask the people who wrote openarena.

> > Then what do I do with my scroll lock key?
>
> Do you actually use it? If so, find some key that you don't use. But do not
> expect two functions from one key.

There is not a single key on the keyboard that I do not use.

> > Modifiers, by their very design, give two functions to keys.  They behave as
> > keys (applications receive keydown/up events) and modifiers (applications
> > receive the modifier state along with other keydown/up events).
>
> That is very true, in general. But I already explained why in this particular
> case the modifier is bound to the keysym by default - because otherwise we'll
> have to force people to use xkb option just to get reasonable Caps_Lock
> behaviour.

It always seemed to work fine by default for me.  From the description of this
bug, it seems like only people who use xkb options (specifically,
shift:both_capslock) were affected by it...

Created attachment 43552 [details] [review]
Add option to bind Control modifier to <CAPS> key while preserving the keysym

The new option in the patch should do what you want. It makes the caps key behave as a Control modifier but keeps the CapsLock keysym.

Accomplishing the same with xmodmap is unfortunately not possible, as xmodmap cannot overwrite keysym actions.

Well, that might be the solution. but instead of new option - perhaps we could just change ctrl(nocaps)? What do you think, Nick? It would return the functionality of the original behavior.

(In reply to comment #32)
> The new option in the patch should do what you want. It makes the caps key
> behave as a Control modifier but keeps the CapsLock keysym.
>
> Accomplishing the same with xmodmap is unfortunately not possible, as xmodmap
> cannot overwrite keysym actions.

OK.  So I tried this patch, and this new xkb option seems to at least *allow*
things to be set up the way I want (which is good).  However, this option
behaves rather strangely in a number of ways:

 * First, this doesn't actually allow the modifier to be changed with xmodmap.
   This is not too important to me personally (since I _want_ it to be a
   control modifier), but it is rather disconcerting to see that I can do
   xmodmap -e 'remove control = Caps_Lock' -e 'add mod3 = Caps_Lock' and I
   still get the control modifier instead of mod3...

 * Second, after enabling this option, the Caps_Lock keysym is listed under
   *both* the "lock" and "control" modifiers in xmodmap.  Nevertheless, this
   seems to not actually mean anything (presumably because the xmodmap settings
   are ignored as per above).

 * Third, when using the colemak variant, this option ends up making the caps
   lock key both a backspace and a control modifier, which is rather strange.
   This is fixable by re-assigning the keysym after the fact with xmodmap.

Can't we just have an option which removes the special Caps_Lock magic?
An option like caps:no_magic, which makes everything normal again (and doesn't
touch the modifier mappings...)

(In reply to comment #33)
> instead of new option - perhaps we could just change ctrl(nocaps)

Changing the behaviour of ctrl(nocaps) is disingenuous, as it will likely
break people's setups.

>  * First, this doesn't actually allow the modifier to be changed with xmodmap.
Well, that's just because xmodmap is not flexible enough. Not an issue.

>  * Second, after enabling this option, the Caps_Lock keysym is listed under
>    *both* the "lock" and "control" modifiers in xmodmap.  
Again, xmodmap is not a worry.
 
>  * Third, when using the colemak variant, this option ends up making the caps
>    lock key both a backspace and a control modifier, which is rather strange.
>    This is fixable by re-assigning the keysym after the fact with xmodmap.
Colemak is a special case, it explicitly specifies CAPS, which really conflicts with any other caps-related option. Not a worry.

If these are the only concerns, I would be happy to commit the last patch

(In reply to comment #35)
> >  * First, this doesn't actually allow the modifier to be changed with xmodmap.
> Well, that's just because xmodmap is not flexible enough. Not an issue.
[...]
> Again, xmodmap is not a worry.

Deliberately breaking xmodmap is not a worry?  If that is the case, please
don't do it half-assedly: go all the way and remove xmodmap from X.org.

> >  * Third, when using the colemak variant, this option ends up making the
> >  caps lock key both a backspace and a control modifier, which is rather
> >  strange.  This is fixable by re-assigning the keysym after the fact with
> >  xmodmap.
>
> Colemak is a special case, it explicitly specifies CAPS, which really
> conflicts with any other caps-related option. Not a worry.

It's a worry for me, because I use colemak.  Using xmodmap to reassign the
keysym (I do this currently) still works with this patch, but judging from
previous comments, it seems like using xmodmap is not a good strategy if I
want my keyboard settings to not break in the future.

> Deliberately breaking xmodmap is not a worry?  If that is the case, please
> don't do it half-assedly: go all the way and remove xmodmap from X.org.
The word "Deliberately" is excessive. I'd say I never promised xmodmap compatibility for xk-c changes. And if fixing things from xkb POV means breaking them from xmodmap POV - so be it. Having correct XKB codebase is more important for me than keeping xmodmap tweaks working.
 
> > Colemak is a special case, it explicitly specifies CAPS, which really
> > conflicts with any other caps-related option. Not a worry.
> It's a worry for me, because I use colemak.  
Ugh... You're tough...:) But at this point I lost you - if you want CAPS to be used as Backspace (as a part of colemac layout), why all these troubles?

> Using xmodmap to reassign the
> keysym (I do this currently) still works with this patch, but judging from
> previous comments, it seems like using xmodmap is not a good strategy if I
> want my keyboard settings to not break in the future.
Well, you can use xmodmap as you did before - but I officially warn you that xmodmap compatibility across releases is not promised.

(In reply to comment #37)
> > > Colemak is a special case, it explicitly specifies CAPS, which really
> > > conflicts with any other caps-related option. Not a worry.
> > It's a worry for me, because I use colemak.  
>
> Ugh... You're tough...:) But at this point I lost you - if you want CAPS to
> be used as Backspace (as a part of colemac layout), why all these troubles?

No, no: I want caps lock to work as a third control key on my colemak (as well
as on my qwerty).  I don't like the caps-lock-as-backspace feature of colemak.
Hence why I reassigned the keysym back to Caps_Lock (using xmodmap...).

> > Using xmodmap to reassign the
> > keysym (I do this currently) still works with this patch, but judging from
> > previous comments, it seems like using xmodmap is not a good strategy if I
> > want my keyboard settings to not break in the future.
>
> Well, you can use xmodmap as you did before - but I officially warn you that
> xmodmap compatibility across releases is not promised.

I'd be happy to not use xmodmap, but this this xkb stuff is voodoo to me, and
the man page for setxkbmap doesn't help me too much.  Xmodmap is at least easy
to use...

How can I make things work without xmodmap?

You can reset the Caps_Lock keysym by using the caps:capslock option.
Having the patch applied and specifying both options works for me:

setxkbmap us colemak -option caps:capslock -option caps:ctrl_modifier

The first option resets the keysym to Caps_Lock and the second changes the modifier behavior.
But we could as well change the option in the patch (caps:ctrl_modifier) to also set the keysym.

(In reply to comment #39)
> You can reset the Caps_Lock keysym by using the caps:capslock option.
> Having the patch applied and specifying both options works for me:
> 
> setxkbmap us colemak -option caps:capslock -option caps:ctrl_modifier

Yes, this seems to work, thanks.

> The first option resets the keysym to Caps_Lock and the second changes the
> modifier behavior.
>
> But we could as well change the option in the patch (caps:ctrl_modifier) to
> also set the keysym.

Doesn't matter to me.

> > But we could as well change the option in the patch (caps:ctrl_modifier) to
> > also set the keysym.
Yes, let's do it. It would look nicer. After that, I'll commit the patch and we'll consider the issue closed (hopefully)

Created attachment 43869 [details] [review]
Add option to bind Control modifier to <CAPS> key while preserving the Caps_Lock keysym

Patch updated:
Changed option to explicitly set the Caps_Lock keysym
Added an entry for option in base.xml.in

Thanks a bunch!

For some reason the patch did not apply cleanly, but anyway it is there now. Hope everybody is happy.

Stephan, what do you think about bug #35317. I do not see any way to incorporate what they're talking about - but may be you do?