It is currently Tue Oct 17, 2017 2:49 am

All times are UTC - 7 hours





Post new topic Reply to topic  [ 3 posts ] 
Author Message
 Post subject: PBJ compression format
PostPosted: Thu Dec 31, 2015 10:33 am 
Offline
Formerly 43110
User avatar

Joined: Wed Feb 05, 2014 7:01 am
Posts: 311
Location: us-east
Attachment:
pbj-image.png
pbj-image.png [ 8.49 KiB | Viewed 1572 times ]

In August I spent a lot of time crunching numbers to try to find a way to better compress chr data. Ultimately that effort failed because the LZMPi like bit-streams I was coming up with would bloat the decoder too much for my tastes, but I did find out that the core PB8 routine of PB53 was surprisingly effective. So late November I decided to upgrade my nametable compression format by having a PB8 mode. After I posted that quick hack I then proceeded to rearrange the codespace to make a much more efficient decoder.

Code:
## PBJ Stream format:
  Initially starts in PB8 mode.
  In PB8 mode:
    00000000        : 0x00 8 times.
    01111111        : 0xff 8 times.
    0nnnnnnn ...    : For each bit in the pb8 control byte,
                      0 is a new byte, 1 is the previous byte.
  In BG mode:
    0nnnnnnn        : 128-N Run of BG
  In both modes:
    10011111 xx     : Switch to BG mode and set BG to X.
    100nnnnn xx     : 32-N incrementing run starting at X.
    10111111        : Switch to PB8 mode.
    10111110 xx yy  : Set PPU_ADDR to yyxx
    101nnnnn xx yy  : For 32-N times, emit alternately X and Y.
    110nnnnn ...    : 32-N literal bytes.
    11111111        : End stream.
    111nnnnn xx     : 32-N run of X.


PB8 is the peanut butter, RLEINC2 the jelly, and my old nametable compression the bread that combines it all together into .pbj files.

The decoder assembles to 159 bytes of ROM and uses 5 bytes of RAM.
The current encoder is a bit dumb as it's only able to use either pb8 planes or everything else but not both.

Example file sizes:
The pbj picture in this post: 4513 bytes
The gus portrait from 240p test suite: 2297 bytes
The title screen from Zooming sectary: 1423 bytes

EDIT: oops, I uploaded the wrong decoder.


Attachments:
File comment: The 6502 decoder
pbj-decode.s [4.65 KiB]
Downloaded 67 times
File comment: The encoder written in python.
pbj-encode.py [14.26 KiB]
Downloaded 71 times


Last edited by JRoatch on Thu Dec 31, 2015 11:20 am, edited 1 time in total.
Top
 Profile  
 
PostPosted: Thu Dec 31, 2015 10:56 am 
Offline

Joined: Sun Sep 19, 2004 11:12 pm
Posts: 19084
Location: NE Indiana, USA (NTSC)
I assume "BG" is a common byte, used for runs of long lengths without needing to re-specify the run byte.

I see you're including a "seek in output" command (%10111110). That complicates an offline decoder, as it has to buffer the entire output in case the stream seeks backwards. What was the reasoning behind that command?

Can your decoder break the RLEINC2 runs into fixed-size output packets? Say I wanted to decode 128 bytes, send that to VRAM, decode the next 128 bytes, send that to VRAM, etc. That was my original motivation behind PB8: to be able to decode while rendering is on.


Top
 Profile  
 
PostPosted: Thu Dec 31, 2015 11:30 am 
Offline
Formerly 43110
User avatar

Joined: Wed Feb 05, 2014 7:01 am
Posts: 311
Location: us-east
tepples wrote:
I assume "BG" is a common byte, used for runs of long lengths without needing to re-specify the run byte.

That is correct.

tepples wrote:
Can your decoder break the RLEINC2 runs into fixed-size output packets? Say I wanted to decode 128 bytes, send that to VRAM, decode the next 128 bytes, send that to VRAM, etc. That was my original motivation behind PB8: to be able to decode while rendering is on.

I say continue to use PB53 for that. PBJ was design for only blank screen bulk uploads. In fact PB53 beats PBJ in chr compression (PBJ has to code a duplicate 8 byte plane twice). I suppose a decoder can be made to keep partial RLEINC2 run commands in memory for the next partial decode, but the size of such a decoder will probably be larger then just including PB53 as well.

tepples wrote:
I see you're including a "seek in output" command (%10111110). What was the reasoning behind that command?

So that a complete PPU picture can be rendered. Sometimes I also want to be able write a ASCII tile set to a different offset then $0000, or maybe I need to write just the bottom of the screen for a status bar. The double byte command already reads in two bytes, so it's not that far away from writing those two bytes to PPU_ADDR instead of PPU_DATA (12 bytes of code in the decoder).

tepples wrote:
That complicates an offline decoder, as it has to buffer the entire output in case the stream seeks backwards.

It does complicate things, but I plan to have some options to read/write a list of ppu address with corresponding file offsets, and only process files like a bag of bytes when the "seek in output" command is not present. Also I'll include a different option to write out a sav file for your editor as well.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 3 posts ] 

All times are UTC - 7 hours


Who is online

Users browsing this forum: Google [Bot] and 3 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB® Forum Software © phpBB Group