icon splitter

[miker1264]

@magorium

Thank you for the update. I have made some progress with my AROS test program that contains icon read/write functions that I need for IconPress and another icon app.

I will provide my test program in C code. It may help you to write your Pascal icon & ILBM handling code. So far I can read the IFF header data (FORM & ICON) & can find the FACE chunk & IMAG chunks. I can access all the data in the IMAG chunks & I can write ILBM files that are "almost" correct. I'm writing the valid IFF header for the files (FORM & ILBM) & the BMHD is correct. I can also write the correct color palette. But the image data is somehow incorrect.

I'll put the test program and a lengthy text document in an archive for you. In the text file I will provide all the technical information about ILBM files and Glow Icon files that I've learned over the past few years. 

[miker1264]

As you can see in the attached screenshot I can successfully access the data from each IMAG chunk which also means I can read the icon file to get to the IFF data.

The datatypes system recognizes the output files as valid ILBM picture files. But that's because the IFF header data and the BMHD are correct. The CMAP (colormap chunk) is also correct. I verified in a hex Editor. But the image data is wacky! The datatype tried to read it but the image is scattered and fuzzy. I assumed the binary data in the icon for the image was compressed planar data so I just copied it to the ILBM. Maybe that's incorrect. It needs processing.

In my experience dealing with saving ILBM files and HAM6/8 conversion there are a few reasons the image data is skewed. Either the colormap is wacky which it isn't or the image data is in the wrong format. For example if comp in BMHD is set to zero but we are trying to display RLE data the datatype gets confused so the image is scattered. Or if comp is one (compressed) but the data is actually uncompressed it's the same result. There is one other possible reason if the compression values are actually correct.

In the documentation for the palette the palfmt (compression) is usually always 0 and each byte represents a pixel. So it's 8 bits per pixel. So we can easily just copy the colormap byte array to the ILBM. But the information is vague concerning the image data. It says it can be compressed or uncompressed (imgfmt = 0 or 1). But it doesn't specify the underlying pixel format. Is it planar arranged in the correct number of bitplanes. Or is it chunky?

 I suspect the image data is the same as the palette. One byte represents one pixel (8 bitplanes). But it is also RLE compressed. That explains why the datatype displays it as scatted and fuzzy. The BMHD says depth = 4 but it has 8 bitplanes or it's chunky not planar. So it needs conversion.

@magorium
If you discover the mysterious format of the image data in the IMAG chunk please let me know. I'm not sure about it.

[miker1264]

Ah. I made a possible discovery.

In ilbmtoicon there is a clue in write icon in the function writeimagchunk.

when it gets to the point where it compresses the image data it does this:

gfx = createrle (img->bmh.bmh_Depth, &gfxpacked, &gfxsize, img->bmh.bmh_Width * img->bmh.bmh_Height, img->chunkybuffer);

It's an rle compressed chunky buffer most likely with one byte for each pixel (8bits per pixel). That's why my image was scattered and fuzzy! I was trying to display chunky as planar but it wasn't planar data.

That's easy to fix. Use unpackbyterun to decode RLE them convert chunky to planar with the correct number of planes.

Decode35 in icon library likely does both at the same time but I'll do it in two parts to verify my data. After the rle is unpacked I have a chunky to planar that reduces bitplanes that I used with HAM conversion. I will try to use that. 

Then if all goes well the uncompressed planar data can be written to the ILBM file.

[miker1264]

I haven't decided what the format is for the image data in the IMAG chunk. It can be RLE compressed but what are the data underneath the compression? Chunky bytes or planar data?

[magorium]

@miker:
I have the same question but I assume (might be completely wrong) that it is stored in the same way as the non 24/32 bit formats, except that the number of bytes per pixel is not a single byte anymore (but 3 or 4). Or in short: I assume in planar mode and without a palette but using true RGB values.

I was unable to find official documentation yet. But I have to search a little deeper/harder in the OS4 SDK. My initial searches did not reveal anything related to the IMAG chunk.

What do the existing 24/32 bit glow icons use ? or are you not able to determine that just yet ?

[miker1264]

The format for the existing 32bit Glow Icons is easier. Immediately following the IFF chunks there are two ARGB chunks with zlib compressed 32bit pixel data.

[magorium]

aaargh. My bad.

That is what you get when reading about picture storage formats when the topic is about icon storage   

The best documentation I have found so far seems to be that of evillabs but imho there is still some ambiguity how exactly things are stored. e.g. i have a difficult time interpretating what it actually means. In that regards I should perhaps better focus first on the imag chunk decryption to see what is about exactly

Ofc i can take a peek at an existing library and/or code that loads these icons but I would like to figure it out myself first before looking at an actual implementation from someone else.

[miker1264]

aaargh. My bad.

That is what you get when reading about picture storage formats when the topic is about icon storage   

The best documentation I have found so far seems to be that of evillabs but imho there is still some ambiguity how exactly things are stored. e.g. i have a difficult time interpretating what it actually means. In that regards I should perhaps better focus first on the imag chunk decryption to see what is about exactly

Ofc i can take a peek at an existing library and/or code that loads these icons but I would like to figure it out myself first before looking at an actual implementation from someone else.

I have some information that may help. I can post it tomorrow.

I also found this when looking at the Amiga Icon Converter that is written in Java.
But the note was interesting concerning Reading IFF Icon Data.

// note: this is BIT aligned, not byte aligned ...
// -> RLE control chars are 8 bits, but the data elements are n bits, determined by depth

Which may or may not indicate that the image data is RLE compressed but the data itself is planar by depth.

To test the image data I borrowed an image decoder from another program. It reads the icon image data. It's very similar to the decoder as used in icon library to read OS35 icons. I finally got the decoder to output data for my test program but it only works at full size ,(width * height).

Curiously I couldn't figure out why my ILBM images were still scattered and fuzzy. The images were scrambled. But then I realized that I may be expecting the wrong out put format. I will do some tests to verify.

When all else fails I will put a --v switch in my test program to optionally view the images so I can draw them on the screen. If that works then the decoder output format is 8bit chunky bytes not planar data as I had expected.

For example the decoder was using a strange formula if the image data in the IMAG chunk is uncompressed the output size is (imgsize * 8/depth) which if the image has 4 bitplanes is (imgsize * 8/4). It's twice as large. That's very puzzling!

[magorium]

Just a quick note with respect to the latter remark concerning uncompressed data.

The evillabs entry has a clue on that. No matter the bitplane depth the data for a pixel is stored in a byte. The upper (or lower depending on how you look at it) bits are
not used and need to be ignored.

That is what confuses me (at least initially) as the text on evillabs wrt RLE compressed data does mention the exact bitdepth. I just haven't come to those parts yet and have not investigated it. I'll come to it 

[miker1264]

Just a quick note with respect to the latter remark concerning uncompressed data.

The evillabs entry has a clue on that. No matter the bitplane depth the data for a pixel is stored in a byte. The upper (or lower depending on how you look at it) bits are
not used and need to be ignored.

That is what confuses me (at least initially) as the text on evillabs wrt RLE compressed data does mention the exact bitdepth. I just haven't come to those parts yet and have not investigated it. I'll come to it 

I do remember seeing that also. It makes sense that the planar data is stored in bytes. That's how the planar bitmap in picture datatypes store planar data.

No matter the bitdepth (8bitplanes or below) the planar bitmap always has 8bitplanes. Maybe not all of them have planar data though.

The Image Decoder seems to confirm that at least it is performing bit masking & it is making bytes from the bits. Maybe it converts planar data to chunky bytes?

The good thing about all this is that my IconEdit program which uses a variation of the Icon Image Decoder works on AROS One x86.

It kept crashing on IcarosDesktop. I thought it was broken! I will use it for testing now to get it to save ILBM images.

This is a good place to find ILBM information:
https://www.fileformat.info/format/iff/egff.htm

[miker1264]

I took a while to write the SaveImage function for IconEdit but it is actually saving an ILBM picture file!

But it contains uncompressed chunky bytes instead of planar data so it isn't quite right. It's a good start though.

It seems that I'm just missing a chunky to planar conversion algorithm to make it all work correctly.

Thank goodness for Dopus4 & ZuneHexEditor! Nice programming tools.

[miker1264]

@magorium

Here is some imformation that may be helpful for you to write your Pascal Icon Splitter.

It also contains the SaveImage function that contains native methods to save an ILBM picture file.

I included the various functions and sample code for the Icon Image Decoders and some other information.

Also include in the archive you will find my SaveHAMPic code that converts 24bit image data to HAM6 or HAM8. 

At the end of DO_HAM_BODY is my custom Chunky to Planar covversion routine used with SaveHAMPic.

[miker1264]

I'm pleased that this has happened the way it did. I'm not entirely sure that the image data in the icon IMAG chunk is well suited to be copied directly to an ILBM.

Even if it is planar data or bits of planar data contained in bytes with or without RLE compression we can't just copy a byte array from the icon to an ILBM picture file.

It's good that the Image Decoder used masking and bit shifting to convert the data directly to chunky bytes (8bits per index value).

The ILBM file format requires that each scanline of bytes must be modulus 16 (evenly divisible) so that it ends on an even bitplane. We can't have partial bitplanes!

So at the end of each chunky scanline padding bytes (empty bytes or junk bytes) are added as needed. For example 46 is not evenly divisible by 16. It needs to be 48 so we add two bytes.

After the padding bytes are added to each scanline then the chunky data is converted to the proper number of bitplanes. We process each scanline independently then write to file.

But first I had to be 100% sure I was looking at 8bit chunky bytes for the index values that is the output of the Image Decoder. So I used a png screenshot, and a paint program to identify each rgb pixel.

I matched each pixel (it's only 46x46 pixels) to colors in the colormap, verifying the colormap index making sure it matched the index values in the image data. It all matches perfectly!

Now we just have to implement chunky to planar conversion and bitplane reduction which has already been done.

[miker1264]

Maybe I have had too much caffeine! But the mystery of the image data format is beginning to reveal itself.

What does the mystery formula (numimg * 8/depth) mean?

Here is my theory:
For uncompressed data numimg refers to the total number of storage bytes present in the data. In that case if the depth is 5 bitplanes for example (numimg * 8/5) is the "planar capacity" or "c.apacity o.f p.lanar' (cop). It's the total size of the planar bits stored inside the bytes. You would need this number if you wanted to extract planar by depth.

For RLE compressed data numimg is the size of compressed data. The Image Decoder gives us a clue here. For compressed data the total uncompressed size is the total "entries" (each entry is index or pixel) is (width * height). I don't have a Glow Icon with uncompressed image data to test because 99% of the time it will be compressed. On Amiga computers size is important, the smaller the better. But in theory for uncompressed data numimg should equal (width * height).

So the image data in the IMAG chunk is multi-faceted. It is arranged by bitplanes in bytes. If you want planar data it is byte aligned by depth. Simply extract the correct number of bits from each byte and copy them to a planar bitbuffer. If you want the bytes arranged by bitplanes simply copy the storage bytes to your new byte buffer. If the bytes are in bitplanes if you think about it when you inflate for example 5 bitplanes of data to 8 bitplanes then three planes contain zeros. In each storage byte only 5 bits are valid and three are zeros. Since I suspect they are by bitplane it is already 8 bitplanes. Just copy the storage bytes to get 8 bitplanes of planar data. To get chunky bytes of index values the Image Decoder must convert 8 bit planar data to 8bit chunky bytes by masking and bit shifting.

If the storage bytes are not arranged by bitplanes then they are just 8bit chunky bytes that represent the index values.

[miker1264]

@magorium

This is the chunky to planar conversion with bitplane reduction.