[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: iconv (linux vs solaris)



Dear Nadim,

> I attempted to mail you at your "haible at clisp dot cons dot org" address which
> bounced

Sorry, my old email address stopped working, the new one is <bruno at clisp dot org>.

> One of the guys helping with this effort is on a Solaris box
> 
>   uname -a: SunOS mipo 5.7 Generic_106541-15 sun4u
> 
> notes that he's able to see certain characters that we (some on linux
> others on FreeBSD) aren't.  The only thing we could attribute our
> short-comings to is that we are using different iconv utilities (we're
> all configuring with the same options, etc) the solaris guy does NOT
> have liconv (GNU's version of iconv) installed though.  The guy on
> Solaris is able to see 'shaped' letters (Unicode FE70 - FEFF) if the
> keymap encodings are simply set to those values.  We all can see
> (Unicode 0600 - 06FF) without a hitch. So why can't linux display
> Unicode FE70 - FEFF like solaris can (same VIM code) ?
> 
> To reproduce the problem, please do the following (use VIM-6.0)
> 
> 1. download the font and keymap files (add the .bdf file to your fp path)
>    http://www.arabeyes.org/~nadim/vim/arab_shape_bug.tar.gz
> 
> 2. start vim6 (vim -g) & enable the various arabic settings
>    :set keymap=arabic
>    :set guifont=-misc-fixed-medium-r-normal--20-200-75-75-c-100-iso10646-1
>    :set encoding=utf-8
> 
> 3. Press 'a' (note visible non-shaped letter)
>    Press 'q' (note BOX for shaped letter -- WRONG)
>    Press 'w' (note BOX for shaped letter -- WRONG)
> 
> So why is solaris able to display those shaped letters where-as
> linux/FreeBSD are unable ? How to fix it ?

It is not frequent that I must say, I am _glad_ to _not_ be able to
reproduce a problem reported to me. Indeed I installed vim 6.0 on my
Linux box, explicitly linked with GNU libiconv 1.7, followed your
instructions, and am able to view files containing Unicode characters
in the mentioned areas. You find attached three screenshots

  - right after typing 'a' 'q' 'w'
  - an UTF-8 encoded alphabetically sorted list of Unicode characters,
  - a hebrew text with punctuation characters, it uses characters
    from the range U+FB20..U+FB4F.

It works equally well with GNU libiconv 1.7 and with glibc 2.2.4's
iconv.

The way I did it:

  * building vim:
    - unpack vim-6.0
    - add to the src/Makefile the line "CONF_ARGS = --prefix=/packages/gnu --with-features=big"
    - for building with libiconv:
      export CFLAGS="-I/packages/gnu/include"
      export LDFLAGS="-L/packages/gnu/lib -Wl,-rpath,/packages/gnu/lib"
    - for building without libiconv:
      remove a few obnoxious paragraphs from src/auto/configure
    - in the src directory type "make" and "make install"

  * installing the keymap:
    copied arabic_iso-10646-1.vim to
    /packages/gnu/share/vim/vim60/keymap/arabic.vim

  * installing the font:
    - Choose a directory for it:
    - Looked at the font path my X server uses. (In /etc/X11/XF86Config)
      You can install new fonts either in an existing directory that
      is already known to the X server (note: many X servers remove
      nonexisting directories from their fontpath at startup!), or
      into a fresh directory which you have to add using e.g.
      "xset +fp myfontdir".
    - bdftopcf -o 10x20.pcf 10x20.bdf
    - gzip -9 10x20.pcf
    - cp 10x20.pcf.gz myfontdir
    - cd myfontdir ; mkfontdir
    - xset fp rehash

  * The rest of my config is: SuSE Linux, glibc-2.2.4, XFree86 4.1.0
    (according to "xdpyinfo").

> So why is solaris able to display those shaped letters where-as
> linux/FreeBSD are unable ? How to fix it ?

My guess is that it is related to the way you installed the 10x20
font. The difference between Linux/FreeBSD and Solaris is that the
former systems ship with XFree86 4.x, which already comes with a 10x20
font. This font ends at U+3000 because Markus Kuhn judged that a memory
saving of 600 KB per open Unicode font is more important than the
ability to display characters >= 0x3000. When you install your 10x20
font, you must ensure that it is in a directory that is listed in the
X server's fontpath before /usr/X11R6/lib/X11/fonts/misc/ (which
contains Markus Kuhn's font). One way to ensure this is to use "xset +fp".

> Bruno, we didn't know whom else to turn to regarding this potential issue.
> But we believe we are seeing some discrepancies between libiconv and solaris'
> implementation of iconv

I cannot imagine how iconv could be involved here.

Bruno

Attachment: vim60ar1.gif
Description: vim60ar1.gif

Attachment: vim60ar2.gif
Description: vim60ar2.gif

Attachment: vim60ar3.gif
Description: vim60ar3.gif