Re: [I18n]Re: U+3000 limit

[Keith Packard <keithp at keithp dot com>]

Around 11 o'clock on Jan 8, Mohammed Elzubeir wrote:

> It seems to me that XFree86's #1 priority is optimization for the English
> language, and not i18n and then optimization. On the other hand, our
> #1 priority is the availability of full Arabic fixed-width fonts, and time
> is a big factor.

The problem is multi-faceted, and truely not limited to any latin-centric
text bias.  As X was designed before Unicode was developed, the encoding of
font data was modeled around existing language encodings -- the only
widespread 16 bit encodings from that era were for asian languages which
split the font into "pages" -- each page held a similar range of indices
for various glyphs. X exposes these fonts as a simple 2-dimensional array
and manages reasonable storage density as the fonts are relatively compact
with that simple encoding.

Unicode breaks this assumption -- irrespective of non-BMP glyphs, a typical
XFree86 font has glyphs spread throughout the 16-bit space with large holes
in various places.  Attempting to use these sparse fonts places tremendous
burdens on the X server, the network and the application.

The kludge for XFree86 was to discard glyphs above 0x3000 as beyond that
range the typical XFree86 font becomes increasingly sparse.  

One possible solution is to use a different encoding for the glyph you're
interested in; your application would have to translate from Unicode to
this alternate encoding, but you'd gain significant improvements from being
able to represent the most common glyphs using 8-bit indices, rather than
always needing to use 16 bits.

The alternative is to stop using server-side fonts altogether and switch 
to Xft.  That library provides a uniform Unicode API with client-side 
fonts that avoids all of the problems mentioned above.  It has the added 
benefit of providing anti-aliasing where possible which can dramatically 
improve the appearance of your text.

The current version of Xft requires the Render extension and outline fonts
to take advantage of client-side fonts, but I'm busy fixing those problems so
that older servers and bitmap fonts can be used.  A version of Xft that 
supports PCF fonts but still requires Render can be found in my CVS 
repository at http://keithp.com.

Keith Packard        XFree86 Core Team        Compaq Cambridge Research Lab