Re: mined - a utf-8 editor

[Nadim Shaikli <shaikli at yahoo dot com>]

On Thursday - 17.October.2002 [17:19:50 +0200 (MEST)],
 Thomas Wolff (mined at towo dot net) wrote:
> Nadim Shaikli wrote:
> > Mined works fine under mlterm except for the curses
> > menu and scroll-bar since there is bidi interaction and the curses
> > ctrl characters gets mirrored when they shouldn't be if a line starts
> > with an Arabic character (I'm not sure if unicode's UAX#9 talks about
> > handling this complex scenario -- in reality curses should not be
> > treated as normal characters and thus should not be under the bidi
> > algorithm). 
> > 
> > You can see what I mean,
> > 
> >   http://www.arabeyes.org/~nadim/mined_view.jpg    (normal view)
> >   http://www.arabeyes.org/~nadim/mined_curses.jpg  (curses bidi problem)
>
> The pictures look terrible with respect to the scrollbar and menu.

Well, they are a bit confusing :-)

> > For those Bidi experts on the list -- any ideas on what's supposed to
> > happen when curses/control-characters are involved with regard to Bidi ?
>
> First: mined does not use curses, so the question should rather be 
> about interpretation of xterm/vt100 escape control sequences. 
> And I would definitely assume that a positioning sequences should always 
> be interpreted absolutely, not affected by any right-to-left state.
> I consider this a design bug in mlterm. If that gets fixed, at least 
> the menus should display correctly.
> The scrollbar would perhaps still interfere with the terminals 
> assumption about left and right borders, so until a solution is 
> conceived, it might be switched off for bidi mode (option -o).

I'm not familiar with those escape sequences and whether they are unique
to the drawing of those curses-like borders or not.  I'm CC'ing this email
to mlterm's author to see what his thoughts on this subject are.  But
hypothetically speaking, if those escape sequences are only present to
represent a menu (or a scroll-bar) and they are unique in that regard
in all applications (ie. not just in mined), then I think it would be
safe to expect their location to be absolute.  Otherwise, if these
escape sequences were part of a more generic world in which these
sequences reflect and integrate very deeply within the application
(ie. the same sequences could be programmed-in with absolute and/or
relative positioning), then I don't really think it would be fair to
expect mlterm (or any other terminal emulator) to "know" the intent of
the application apriori.  Furthermore, if these escape sequences
(btw: is there a website or a resource that enumerates them ?) are
position agnostic, does unicode offer anything in their specs to
address these scenarios (don't recall that they did) ?  Is the
inclusion/creation of one more escape sequence possible (who controls
their definitions/settings) ?  I was thinking more in terms of making
a distinguish between something that is absolute and something that
is relative (all legacy sequences would be absolute unless preceded
by a "what-follows-is-relative" sequence or something).

> About keymaps:
>
> Thanks for your links. I see from the pictures that arabic keymaps 
> are a one-to-one keyboard translation, nothing more complex like 
> Chinese multi-key character composition. That should be easy 
> but I cannot derive from the images of Arabic letters (which I'm 
> not familiar with) which Unicode characters they correspond to.
> If you send me or point me to a mapping (key -> Unicode value), 
> I'll implement that.

Sure, this is rather simple.  The only catch here is that "some"
keys (and the mapping is positional and not actuall ASCII
characters - to address your question below) will require the
insertion of two characters (such as position 'b' which inserts
both a 'LAM+ALEF').  The map can be viewed and cross-referenced
here,

 
http://www.arabeyes.org/cvsweb/projects/external/vim/arabic_iso-10646-1.vim?rev=1.2

The 2-character keys noted above can be easily seen in the file noted
and are 'b', 'T', 'G' and 'B'

> One problem is: How is the mapping supposed to be on non-US keyboards?
> For example, on a German keyboard, letters "y" and "z" are swapped, 
> and some special characters as well. Would the mapping rather apply to 
> ASCII characters or to actual key positions?

The idea (and this applies to any language really) is that a person sits
down to use a foreign keyboard he/she will revert to what they are used
to in terms of the position of the characters and not what's printed/noted
on them.  So in short its positional.

> > As for switching, its completely upto you how you'd like to define that.
>
> OK, I can assign it to a function key, and perhaps to an ESC command.

Yup, set it to anything you like that makes sense to your application.
 
> > > Also, please not that the default mode of mined is a sort of 
> > > "poor man's right-to-left", meaning that it produces right-to-left 
> > > appearance on terminals that do not know anything about right-to-left.
> > > This can only work together with internal right-to-left ordering of 
> > > the text in the file (it's basically input support) which is not 
> > > the preferred way of handling text files in regular right-to-left 
> > > applications if I understood various comments correctly - please tell 
> > > me what you think about it. Mined can be switched to work with a 
> > > right-to-left terminal with the -b option. I would expect it to 
> > > work with mlterm then - if only mlterm would work at all...
> > 
> > Mlterm takes care of this completely since it offers Bidi
> > (di-directionality).  Right-to-left and Left-to-right only simply
> > isn't sufficient when files containing both encodings (such as
> > English and Arabic) are viewed.
>
> As I told, mined's own right-to-left mode is of course not
> compatible with any right-to-left mode run by the terminal.
> So you have to switch it off (option -b). Did you do that? 
> If you didn't, how does it look if you do?

It made no difference in the example at hand (noted screenshots).
In any regard without native application Bidi support, the wisest
thing for Arabic-script users is to simply disable mined's support
and let mlterm take care of things.

Regards,

 - Nadim


__________________________________________________
Do you Yahoo!?
Faith Hill - Exclusive Performances, Videos & More
http://faith.yahoo.com