Re: Quran data and issues in encoding the Quran in unicode

[Meor Ridzuan Meor Yahaya <meor dot ridzuan at gmail dot com>]

It's good that a discussion is going on. However, I think someone need
to lead the discussion in a detailed manner so that a proposal to
unicode can be made ASAP. I made the quran data public with the
intention of getting the issues resolve. I've made my point the
regarding the issues, and seems that others agree that those are the
issues need to resolve, but they might differeces in opinion on how to
solve it, which is good.

Let me made clear a few points. First, my goal with the project. What
I would like to have is some thing like this. We should have the quran
encoded in full with the ability to extract it in 3 forms: the
original Uthmani script (without the marks, dot is ok to encure the
character), the current visual representation (according to madinah
mushaf), and the current spelling to the words, which is more suitable
for searching (esp for meaning). I think I manage to achive the 2nd
goal with the workaround found in the file.

Second, I'm not an arabic speaker, thus not very well versed in arabic
grammar etc (I did took a course in elementary arabic though, and
quite well versed in Quranic tajweed rules) So , it will be a bit
difficult for me to accomplish the 3rd goal. This is where I really
need help from others. That's why I post this issues to arabeyes, and
hope that the encoding issues can be resolve asap. I don't really have
a strong opinion on how it should have been done, it does not really
matter to me as long as the standard is define.

To accomplish the 3rd goal, we need a rasm scholar. I think in my
town, we don't have any. There might be a few in my country, but not
sure. Some might argue the need of this,but I personally think that it
would be the best if we can do this, so that the quran will be
preserved in it's original state  (in digital form, or course)

On top of that, there is one more goal, that would be great if we can
accomplish also. That is, the different reading style of the quran.
The files from the site is based on the madinah mushaf, which is based
on Hafs reading. I think the Madinah Quran complex print at least 2
edition, which is the Hafs and Warsh. A friend of mine, who is from
Mali, said that people from his country use Qaloon style. I really
would like to have a copy of each to see the diffrerent style.  Anyone
would like to send me a copy (other than the Hafs style) are most
welcome.

Final note, I will release a totally new font which I created from
scratch (digitally). The font was created from the actual mushaf, and
I think this will pleased most people. The font will not have any
truetype hint at all, but the quality rendered under linux is awesome
(thanks to freetype autohinter). Under windows, it renders ok, but not
that great. One of the issues that prevented me from realeasing it
earlier was the license. ATM, I don't see any advantage of releasing
it under GPL, since there is no source code involve here. The font is
the source. So, I've decided to release it Free for non commercial use
(use and distribution). Hope you will enjoy it.

Regards.




On 6/21/05, Mete Kural <metek at touchtonecorp dot com> wrote:
> >1. Mohamed Zakariya told me sequential tanween was practised in the North
> >African tradition.
> 
> Hmm.. interesting information. Do you know since what century it has been used? By the way when you say North Africa are you also referring to Egypt or do you mean that it's a Maghribi tradition?
> 
> 
> >2. I concur that a clean encoding should leave tanween as such intact
> >(whether it is encoded as fatha-fatha of fathatan etc would be immaterial)
> >so that the phonetic variants need a separate code point.
> 
> I agree with you that the best way to handle this is to keep the tanween intact and use a special codepoint that comes after the tanween codepoint in order to trigger the variant sequential tanween glyph.
> 
> Regards,
> Mete
> 
> >
> >Regards,
> >
> >t
> >
> >
> >
> >Mete Kural wrote:
> >> Salaam Abdulhaq,
> >>
> >> On the point of so-called "sequential tanween" I am a little bit
> >> undecided. First of all, it is perfectly clear that what we call a
> >> sequential fathatan, sequential dammatan, and sequential kasratan in
> >> fact without doubt is a fathatan, a dammatan and a kasratan
> >> respectively. As far as I know (please confirm this those who know)
> >> the 1924 Egyptian printing of the Quran was the first to use variant
> >> glyphs for fathatan, dammatan and kasratan for the cases when the
> >> noon is not pronounced. So these sequential tanweens have not been a
> >> part of Arabic until the 20th century. Regardless since most of the
> >> Qurans printed in the world today employ these sequential tanweens
> >> Unicode has to accomodate for them somehow and these sequential
> >> tanweens need to be listed in the Arabic code page. But the thing is
> >> that since these sequential tanweens are essentially no more than
> >> just tanweens, I would be inclined to encode them as regular
> >> tanweens, fathatan, dammatan, or kasratan, in order to preserve the
> >> graphemic integrity of the text. To trigger the sequential behaviour,
> >> a special codepoint could be added that would be placed right after
> >> the respective tanween codepoint. Well this would be the ideal state
> >> of things.
> >>
> >> Although if three seperate codepoints were added for sequential
> >> fathatan, sequential dammatan and sequential kasratan, this would not
> >> be totally inconsistent with how the Arabic codepage has been
> >> evolving since to me the Unicode Arabic codepage as a whole is a
> >> hybrid of grapheme (character) based encoding and some glyph based
> >> encoding, which is ugly. So I guess adding the three seperate
> >> sequential tanween codepoints would not be inconsistent with the
> >> current ugly state of things in the Arabic codepage, but I would
> >> prefer a cleaner method such as a special codepoint that triggers
> >> sequential behaviour.
> >>
> >> Eventually the Arabic codepage needs to evolve to at least allow a
> >> clean encoding of the Quran, although it seems like even if that
> >> happens, the uglier method of encoding will always be available to
> >> whoever chooses to anyways.
> >>
> >> Kind regards,
> >> Mete
> >>
> >> ---------- Original Message ----------------------------------
> >> From: Abdulhaq Lynch <al-arabeyes at alinsyria dot fsnet dot co dot uk>
> >> Reply-To: Development Discussions <developer at arabeyes dot org>
> >> Date:  Mon, 20 Jun 2005 22:49:29 +0100
> >>
> >>> I don't agree with some basic points about all this. As I understand
> >>> it Unicode wants to move from what has become a glyph-based coding
> >>> over to a semantic-based encoding, and allow the font technology to
> >>> worry about the glyphs. Fine.
> >>>
> >>> However, Thomas et al. seem to be determined to pursue the opposite
> >>> in terms of tanween and tajweed related marks. Tanween is
> >>> semantically totally different to a fatha or one fatha followed by
> >>> another. It is a semantic character of its own and deserves codes of
> >>> its own. Iqlaab, ikhfaa, madd etc are tajweed marks that govern the
> >>> pronounciation of the arabic and each carries a full semantic load.
> >>> It should be possible to encode these semantically loaded objects in
> >>> any textual representation of quran. If the Unicode consortium is
> >>> not interested in encoding one of the most common books in the world
> >>> then a further code standard must be developed on top of the unicode
> >>> one. Hacks like placing two characters next to each other to
> >>> 'inspire' the font renderer to display a third semantically
> >>> different character just don't cut it.
> >>>
> >>> If Unicode really is about semantics and not glyphs, then let's have
> >>> that then please and give us a code-point per semantic load. If
> >>> instead we have to hack around with glued-together glyphs to try and
> >>> indicate missing meaning, then we should look elsewhere.
> >>>
> >>> If anyone is interested (on the safe assumption that Unicode is not
> >>> interested in that) then perhaps we could discuss such a code
> >>> extension here.
> >>>
> >>> Abdulhaq
> >>>
> >>> On Friday 17 June 2005 09:29, Thomas Milo wrote:
> >>>> Hi Mete, Meor,
> >>>>
> >>>> Just a quick reaction: U+0641 TATWEEL does not represent a
> >>>> character (or rather, grapheme) but a unit of typography (i.e., a
> >>>> glyph). It should never have been part of the Arabic code block in
> >>>> the first place. If you prefer the sequence like
> >>>> Fatha-SmalAlifAbove (in regular Unicode) to print with a upporting
> >>>> Tatweel, consider building a substitution in your OTF.
> >>>>
> >>>> A second point is the use of tanween followed by SmalMeemAbove and
> >>>> SmalMeemBelow. This is non-standard use of the small meems, plain
> >>>> and simple. The fact that the obvious encoding with sequential
> >>>> single harakat and single harakat+small meem is not supported
> >>>> correctly by Microsofts Uniscribe does not justify the use of
> >>>> illegal encoding. It would be better to report a bug to MS
> >>>> typography or, if you don't like to be the prisoner of third
> >>>> party's prorietary solutions, develop your own OTF parser (what we
> >>>> do).
> >>>>
> >>>> BTW, Mete did not mention Decotype's Naskh as a font that handles
> >>>> Qur'anic Arabic, because it is not yet published. In this project
> >>>> we consider the Uncode points SmalMeemAbove and SmalMeemBelow a
> >>>> mistake: they are contextual variants of SmallMeem (were - a
> >>>> single! -kasra pulls it below the script line). Therefore we treat
> >>>> them as identical. However, for compatibility's sake we could add a
> >>>> few front end substitutions to convert your private encoding to our
> >>>> (private?) encoding (which I believe you could have done to bypass
> >>>> the MS Unicsribe constraints)
> >>>>
> >>>> Regards,
> >>>>
> >>>> t
> >>>>
> >>>> Mete Kural wrote:
> >>>>> Hello Meor,
> >>>>>
> >>>>> Please find my suggested encodings and explanations below.
> >>>>>
> >>>>>> About the small alef, personally I would like to encode it using a
> >>>>>> tatweel + superscipt alef  for medial position, and a space +
> >>>>>> superscript alef for isolated position. The reason being is that
> >>>>>> the sequence will work on most, if not all existing font. You
> >>>>>> might argue that we don't need a tatweel for medial position, but
> >>>>>> without it, you will encounter another problem under windows. The
> >>>>>> same goes for small noon and yeh, which i thnk beter encode it
> >>>>>> with a tatweel. For small waw, I agree with Mr Milo.
> >>>>>
> >>>>> First of all, I would suggest to you not to steer the project in a
> >>>>> way to accomodate the variety of Arabic fonts that are available
> >>>>> today which do not implement the Unicode Arabic spec adequately.
> >>>>> More than 90% of the Arabic fonts out there ignore implementing
> >>>>> considerable sections of the Unicode Arabic specs. Only a handful
> >>>>> of fonts come close to rendering the Quran correctly (at least
> >>>>> rendering what can be encoded of the Quran with the current
> >>>>> Unicode spec, excepting the missing needed Quranic characters).
> >>>>> Take notice that I say come close to rendering the Quran
> >>>>> correctly. I wouldn't be surprised if there are only a handful
> >>>>> fonts in the world today that in fact do render correctly what can
> >>>>> be encoded of the Quran with the current Unicode spec. In fact the
> >>>>> only four that I know are Microsoft's Arabic Typesetting, your
> >>>>> Arabeyes.org Meor font, and SIL's Scheherazade (even these two
> >>>>> have problems with small alef I think) and Lateef fonts
> >>>>
> >>>>
> >(http://scripts.sil.org/cms/scripts/page.php?site_id=nrsi&item_id=ArabicFon
> >>>> t s).
> >>>>
> >>>>> There may be other solutions that are not yet delivered to the
> >>>>> market or I haven't heard of. So I think trying to accomodate the
> >>>>> encoding of the text to render the small alef correctly with other
> >>>>> fonts that aren't suitable for rendering the Quran is unnecessary.
> >>>>> The compromise made on the consistency of the encoding is not
> >>>>> worth it to try to accomodate these unsuitable fonts. You already
> >>>>> have a challenge to accomodate for the Gnome and Uniscribe
> >>>>> rendering engines; trying to accomodate for some incomplete fonts
> >>>>> in addition to that would leave you with a not so desired encoding
> >>>>> quality. This is why I would recommend you not to use a tatweel
> >>>>> before the small alef.
> >>>
> >>>
> >>
> >> --
> >> Mete Kural
> >> Touchtone Corporation
> >> 714-755-2810
> >
> >
> 
> --
> Mete Kural
> Touchtone Corporation
> 714-755-2810
> --
> 
> 
> _______________________________________________
> Developer mailing list
> Developer at arabeyes dot org
> http://lists.arabeyes.org/mailman/listinfo/developer
>