Text 15747, 84 rader
Skriven 2012-05-07 12:51:16 av Markus Reschke (2:244/1661)
  Kommentar till text 15744 av Maurice Kinal (1:261/38.9)
Ärende: silence is golden
=========================
Hi Maurice!

May 06 20:04 2012, Maurice Kinal wrote to Markus Reschke:

 MK> The message I am replying to unpacked was 1485 bytes.  Simply 
 MK> removing the pktHeader and the binary bits in the msgHeader, as well 
 MK> as the 16 bit pktHeader delimeter, reduces it by 74 bytes without any 
 MK> additional jiggery-pokery.  The mixed binary bytes simply bloat 
 MK> messages as well as increase the amount of CPU cycles in order to 
 MK> heave-ho them, which shouldn't have needed to be done in the first 
 MK> place had they not existed.  How do you figure they reduce file size?

Here we go: (but don't complain, you asked for it :-)

There are several address fields in the packet header and in each message
header, like origNode, destNode, origNet, destNet, origZone, destZone,
origPoint and destPoint. Each field has a size of 16 bits (2 bytes).

Take the node number for example. Its valid range is 0 up to 65535. If you want
to encode the node number in plain ASCII you'll need to provide 5 bytes (one
byte for each digit) any maybe another byte as delimiter (strings end with a
trailing 0). The binary encoding needs 2 bytes, so it saves 3 bytes or even 4
if a delimiter is used. Now do the maths :-)

And now to the CPU cycles. When processing packet/message headers one has to
access all the address fields. In binary encoding I can treat an address field
as a raw binary value and access it directly by a variable. In most cases you
would define a structure with all the fixed-sized header fields, create a
structure pointer and set the pointer to the beginning of the packet/message.
Bingo! Now you can access the fields directly via the structures elements.

With ASCII encoding you have to separate or extract the field to get a string
(terminated by trailing 0), then convert the string into an integer value while
checking if the conversion was successful. That needs a lot more CPU cycles
than just accessing binary values.

Back to your assumption about bloat and CPU cycles. If you try to process
packets/messages with tools designed for dealing with ASCII data you'll have a
hell of a lot of fun :-)

 MK> Having said the above your message in the new offline stored message 
 MK> scheme is exactly 941 bytes and didn't require the processes to deal 
 MK> with anything other than 8 bit characters, and without any 
 MK> compression whatsoever.  That in itself is a 40% savings in file 
 MK> size.

Have you stored all fields from the message header too?

 MK> Hard to say for sure how much faster this scheme is compared to the 
 MK> usual Fidonet software but it is faster in {,un}packing and tossing.  

Todays CPUs are so fast one doesn't see a difference between 0.01s and 0.005s.
In the past it was 2s vs. 1s. In order to get real numbers you should measure
the time used by both schemes.

 MK> Further savings in storage size is gained by gzipping the stored 
 MK> msg's (another 40%) and can be readily accessed by any half decent 
 MK> Unix text/file applications.  A simular strategy could be followed on 
 MK> an 8 bit CPU given the lossiness of all the 16 bit binary.

Compression is a very old technology :-)

 MK> How were/are you planning to deal with 16 bit binary data on an 8 bit 
 MK> CPU, nevermind run the usual 16 bit DOS-think Fidonet apps on it?

The same way as one deals with 64 bit data on a 32 bit CPU. Same procedure :-)
BTW, some 8 bit CPUs got (pseudo) 16 bit registers.

 MK> Also the new stored msg format lends itself well to htmlization or 
 MK> ansification on the fly.

Sure, it's ASCII too.

 MK> Lean and mean *ALWAYS* wins ... nevermind stinkin' 8 bit CPUs that 
 MK> cannot run GNU-Linux apps in the first place.  :::evil grin:::

You're totaly wrong ;-) Google for Dmitry Grinbergs ARMv5 emulator for the
Atmel ATmega1284p. He booted linux on a 8 bit CPU running at 24MHz!

Regards,
Markus

--- 
 * Origin: *** theca tabellaria *** (2:244/1661)