Text 24034, 105 rader
Skriven 2011-12-27 23:33:43 av Michiel van der Vlist (2:280/5555)
     Kommentar till en text av Alexey Vissarionov (2:5020/545)
Ärende: utf-8
=============
Hello Alexey,

On Saturday December 24 2011 23:58, you wrote to me:

 MV>> In FidoNet it is only CP866 in what I see of it.

 AV> Obviously, it was the first codepage which worked in DOS out-of-box
 AV> with only minor configuration.

Makes sense.

 MV>> that in the early days one thought that 7 bits -with the luxury
 MV>> of upper/lower case - would be enough. After all, computers were
 MV>> for big companies and universities, not for the man in the
 MV>> street. Scientists and engineers can deal with ASCII only
 MV>> computers.

 AV> When the Soviet scientists faced the same problem, they used the whole
 AV> machine word per symbol, as it could be a Latin character, a Cyrillic
 AV> character, some special character like Greek letter, a digit, a
 AV> punctuation mark, etc.

An early attempt to solve the charavter problem.

 AV>  That produced a large overhead, but those early
 AV> "codes for information exchange" (literally translated "Ko oMeHa
 AV> HopMae", which gives the abbreviation "KOI"), having, for
 AV> example, 28 bits (0 000 000 000 to 1 777 777 777 in octal) allowed
 AV> encoding millions of different characters used by all world languages,
 AV> including Chinese and Japanese hieroglyphs.

 AV> Alas, this did not survive up to our days...

The time probably was not ripe for it. When I started to do some serious
programming in the mid seventees, the problem simply did not exist. The ASCII
set is enough for programming a computer. COBOL. FORTRAN, ALGOL and assembler
does not have anything other than what can be found in ASCII. It is enough to
control a soldering machine or a robot assembling cars.

It is only when I started using computers to process natural languages that I
needed more than ASCII. That was much later.

 MV>> So don't be too harsh on those 7-bit brains.

 AV> I will, for the reason described above. Of course, I'll not be pissing
 AV> at their graves, but I'll poke at them with finger and say "they
 AV> didn't think".

You are too harsh on them. There error was that they did not properly forsee
the future. The share that inability with almost everyone else.

 MV>> What nobody foresaw in the 60ties was that computers would be
 MV>> come so small and so cheap that individuals could affort to have
 MV>> one all for themselves in their own homes. It was then rhat ASCII
 MV>> only was no longer enough.

 AV> Once that happened, why no one said "now we'll review everything"?

Isn't that exactly what happened with the introdcution of Unicode in the early
90ties?

 MV>> Considering that Windows form XP on supports UTF-8 in the console
 MV>> (code page 65001), I think full UTF-8 support wil be possible
 MV>> without a major rewrite. At least for the 32 bit Windows version.
 MV>> Changing the screen I/O routines and wrapping mechanism might do
 MV>> it.

 AV> There's too much ad-hockery in that code...

Has anyone seriously looked at it?

 MV>> Also, support for the full universal character set is not needed.
 MV>> (Un)fortunately fidonet has shrunk to the point that support for
 MV>> LATIN-1 and Cyrrilic would be enough...

 AV> Hmmm... AFAIR, the CP850 keeps the letters in the 0x80-0xBF range.
 AV> Filling the 0xC0-0xFF range with KOI8-R characters and using the
 AV> resulting codepage could make a good trick...

We will be making the same error as all those others that tried to squeeze a
usable character set into 8 bits.

 AV> I don't like the very idea of using multi-byte encoding for the
 AV> language having native (locally developed) single-byte encoding for
 AV> its' own 33 letters and 26 of the only language I know.

Then you are making the same error you accuse those of "not thinking". You do
not look beyond your own needs and that is what got us into the ASCII only
stranglehold and the subsequent mess with codepages and co.

http://czyborra.com/

Trying to squeeze it into one byte made sense when memory was scarce and
expensive. It makes no sense any more. And regearding the transport layer, if
you mostly use characters in the same "block", any standard compression
algorith we see in Fidonet, will take care of almost all of the extra overhead.

The world of the InterNet has de facto switched to UTF-8. FidoNet that once was
a club of pioneers, is now lagging behind...


Cheers, Michiel

--- GoldED+/W32-MINGW 1.1.5-b20070503
 * Origin: http://www.vlist.org (2:280/5555)