Re: [developers] Offline Edition of Wikipedia

["Muayyad AlSadi" <alsadi at gmail dot com>]

itar is both a tool and a library,
it's implemented in C and there is another pure python implementation
and I think it's very easy to make a pure java implementation if all
bz functions are there in java

it uses bz2, if you want another compression this is very simple if
you know how to hack that compression alg

the code is in
thwab-lib-1.1.2/src/libitar

add libitar_7z.h and .c

and add it to
enum itar_type {ITAR_GZ, ITAR_BZ};

your write function should be able to know how many bytes have been
output so far in the middle of compression and put that in

typedef struct
{
  char typ,*ptr;
  int size;
  /* private */
  int min_size,done; /* with mmap min_size = size if no yet done+=size
after that it's 0  */
  char *once; /* with mmap used as mmaped ptr, while ptr=once+offset*/
} itar_w;

the alg works like this:
if what is done so far is larger than some value, then create a new chunk
it it's not put several files in the same block to be globally compressed
for example if the chunk size is 64KB

and you have sent a 1MB file to be compressed which is compressed to
30KB then next file will be with it in the same chunk
to gain better compression and it won't slow extraction because
extracting a 30KB is done in almost no time

when it exceeds the limit, a new chunk is created which will not
degrade compression much because those bytes are already sent to
output

On Thu, Jul 10, 2008 at 11:22 AM, Ahmed Al-Obaidy
<ahmad_alobaidy at yahoo dot com> wrote:
> is itar a tool or a library?
> how can I use it in java?
> I think it will be very useful because I am looking to use .tar.z7 archive.
>
>
> --- On Fri, 7/4/08, Muayyad AlSadi <alsadi at gmail dot com> wrote:
>
> From: Muayyad AlSadi <alsadi at gmail dot com>
> Subject: Re: [developers] Offline Edition of Wikipedia
> To: ahmad_alobaidy at yahoo dot com, "Development Discussions"
> <developer at arabeyes dot org>
> Date: Friday, July 4, 2008, 10:46 PM
>
> in http://code.google.com/p/eclipsedia/wiki/Roadmap
>> 3. It would be great if we manage to use 7zip archives directly.
>
> I already may a tool called itar, it generate .tar.bz2 that can be
> randomly accessed very fast
> please see www.thwab.net
> feel free to use it
>
> On Fri,
>  Jul 4, 2008 at 6:58 PM, Ahmed Al-Obaidy
> <ahmad_alobaidy at yahoo dot com> wrote:
>> Dear All,
>>
>> I am working on an offline edition of wikipedia, to allow people in less
>> connected places to access the content of wikipedia from DVD without the
>> need of Internet access.
>>
>> The project is released under GPLv2. You can have more information about
> the
>> project in this site http://code.google.com/p/eclipsedia/
>>
>> Everyone is welcomed to help and make feedback.
>>
>> Best regards,
>> Ahmad
>>
>>
>> _______________________________________________
>> Developer mailing list
>> Developer at arabeyes dot org
>> http://lists.arabeyes.org/mailman/listinfo/developer
>>
>
> _______________________________________________
> Developer mailing list
> Developer at arabeyes dot org
> http://lists.arabeyes.org/mailman/listinfo/developer
>