It is currently Fri Dec 15, 2017 3:23 am

All times are UTC - 7 hours





Post new topic Reply to topic  [ 84 posts ]  Go to page Previous  1, 2, 3, 4, 5, 6  Next
Author Message
 Post subject:
PostPosted: Mon Jan 04, 2010 5:01 pm 
Offline
User avatar

Joined: Sat Feb 12, 2005 9:43 pm
Posts: 10164
Location: Rio de Janeiro - Brazil
What can I say? I guess I'm not the type that releases user-friendly tools. For the record, I never intended to be, I just wanted to share something I though others could possibly find interesting.

To me it really feels like a waste of time putting a lot of effort into something like this while I could be working on my game. I'll probably stick to my game from now, if I want it to ever be released.

With tools it's hard to please everybody: one is worried about licenses, the other is bothered about speed, someone else wants it to run while rendering is on. This is probably why I hate coding tools for other people. With games it's easier, people just want to play them, or not play them.

BTW, I wouldn't worry about those warnings. I have no use for the value returned by fread, if the file suddenly becomes inaccessible as the encoder runs you probably have worse problems to worry about than not being able to compress your NES tiles, such as your hard disk frying. About the two variables, they are set inside a conditional block that always executes at least once, so your smart ass compiler can complain all it wants but the variables aren't used uninitialized.


Top
 Profile  
 
 Post subject:
PostPosted: Tue Jan 05, 2010 11:49 am 
Offline

Joined: Sat Nov 17, 2007 8:44 pm
Posts: 385
In my opinion, user friendly isn't really necessary, as long as you're here to answer questions, which you appear to be. :)

This is a really cool compression scheme. I don't fully understand the method behind the tables but I can see what is happening enough to trust it to handle my graphics. I'll have to plan on implementing CHR-RAM now.

I think a buffering option would be great, although as you said, you have other things to work on.

tepples wrote:
Quote:
4bit packed pixel tiles compress much better than planar (composite planar like the snes), but not sure how much this applies to the NES planar format.

Have you tried compressing 2-bit packed pixel tiles? That's what CMM does. Even on PCE and SNES, 2-bit tiles often show up in things like fonts.

I was just thinking about this. Will this compressor handle these just as efficiently as other graphics, or could a modified routine improve speed/compression further? Theoretically, I mean - we don't usually need to load and unload fonts rapidly in the middle of a level.


Top
 Profile  
 
 Post subject:
PostPosted: Tue Jan 05, 2010 12:58 pm 
Offline
User avatar

Joined: Fri Nov 19, 2004 7:35 pm
Posts: 3968
1-bit tiles suck under this compression scheme.

_________________
Here come the fortune cookies! Here come the fortune cookies! They're wearing paper hats!


Top
 Profile  
 
 Post subject:
PostPosted: Tue Jan 05, 2010 1:26 pm 
Offline

Joined: Sun Sep 19, 2004 11:12 pm
Posts: 19346
Location: NE Indiana, USA (NTSC)
A run of tiles with only 2 colors would get compressed with 1 bit to indicate the start of a run with new tables, 8 bits to set modes 1, 1, 0, 0, 2 to 4 bits to specify the colors, 8 bits per tile for repeats, and then 8 more bits for each row of pixels that isn't repeated. So you're right that 1-bit tiles get expanded by 12.5%. Perhaps the table definition needs an additional bit to say whether or not to use row repeating in a run.

But without arithmetic coding, is there a good way to encode 1-bit tiles at all?
What schemes have you seen in commercial NES/GB/SNES/GBA games to compress 1-bit tiles?


Top
 Profile  
 
 Post subject:
PostPosted: Tue Jan 05, 2010 3:36 pm 
Offline
User avatar

Joined: Sat Feb 12, 2005 9:43 pm
Posts: 10164
Location: Rio de Janeiro - Brazil
tepples wrote:
What schemes have you seen in commercial NES/GB/SNES/GBA games to compress 1-bit tiles?

The quadtree method I was experimenting with before this one worked on the individual planes of the tiles (i.e. 1-bit tiles). It was slightly worse than this scheme when used on generic tiles, and the decompressor was pretty complex/large.


Top
 Profile  
 
 Post subject:
PostPosted: Tue Jan 12, 2010 8:32 pm 
Offline

Joined: Sat Nov 17, 2007 8:44 pm
Posts: 385
So TileCount is hard-coded at compression time, and if I want to be able to load in smaller chunks I will need to compress each group individually and incbin them together, with a lookup table for each starting address?

It would be nice if the compressor could do this automatically: take an argument for the size of the groups, and begin output with a table of relative pointers to each group. It makes me wonder how much that might affect the compression ratio, though.


Top
 Profile  
 
 Post subject:
PostPosted: Tue Jan 12, 2010 9:15 pm 
Offline
User avatar

Joined: Sat Feb 12, 2005 9:43 pm
Posts: 10164
Location: Rio de Janeiro - Brazil
Well, I don't know how other people usually handle their graphics, but I usually don't have a single CHR file. I have separate files for each character, item, font, and so on. Depending on which tiles will be used at the same time I merge the files to create blocks of tiles that I can compress.

I don't know if implementing a feature like this is worth the trouble. If you have a large CHR file but want to encode it to separate blocks, doesn't that mean that there is a reason you think they shouldn't be packed together, which means it makes sense to have them in separate CHR files to begin with?

About the pointers, I wouldn't make that automatic because I like to have options. You can incbin a compressed block anywhere in your source under a label and you have it's address. However, you might want to handle them differently, you may want to have a bank index along with each pointer (in case you have graphics scattered across different program banks). You might not need a table at all, if you have just a couple of graphic blocks and the code to read from the small table would be larger than using immediate values.

I don't like to work with tools that are too specific and limit what you can do with them. As it is now I don't think it's hard at all to include the files by hand and build your own table of pointers. Like I said before (and you agreed! :wink:) I don't care too much for user friendliness, and this qualifies as it IMO. When I'm programming I just want the tools to be usable, the main project is the game, and that's where I'll put my efforts in order to make it a polished product.


Top
 Profile  
 
 Post subject:
PostPosted: Wed Jan 13, 2010 3:52 am 
Offline

Joined: Sat Nov 17, 2007 8:44 pm
Posts: 385
Alright, that's fine. :) I know you have lots of other things to work on.

The only reason I asked was because of this:

Quote:
I don't like to work with tools that are too specific and limit what you can do with them.


When I first saw TileCount, I thought, oh cool, I can tell it to replace 16 tiles starting here. When I found out it was encoded as part of the compression it just seemed limiting to me.

My natural inclination is to want to be able to switch out any number of tiles at any location, considering that's the primary advantage of CHR-RAM. Whether you want to change the boss monster while keeping regular enemies around, or just change one type of ground tile, it seems useful to me. But I suppose that could just as easily stay uncompressed and be inserted like normal.


Top
 Profile  
 
 Post subject:
PostPosted: Wed Jan 13, 2010 11:58 am 
Offline
User avatar

Joined: Sat Feb 12, 2005 9:43 pm
Posts: 10164
Location: Rio de Janeiro - Brazil
UncleSporky wrote:
My natural inclination is to want to be able to switch out any number of tiles at any location, considering that's the primary advantage of CHR-RAM.

I'm not sure I get what you mean here, but you can't just start decompressing from arbitrary positions in the stream, because of the "blocky" nature of the scheme. Are you saying you'd like to force the start of a new block at certain locations so that you could start decompressing from that point on?

Quote:
Whether you want to change the boss monster while keeping regular enemies around, or just change one type of ground tile, it seems useful to me.

To me, the natural way to do those things is to compress the basic tileset to a single file and put the boss and new ground tiles in different files. You decompress the basic tileset before the level starts, and later you "patch" the basic tileset with whatever you want, replacing whatever you want.

Quote:
But I suppose that could just as easily stay uncompressed and be inserted like normal.

I haven't coded a buffered version of the decompressor yet, so you can't really decompress with rendering on yet. In my own game I don't use compression for tiles that are changed during the level, because I want them to be updated as quickly as possible. I believe the original Sonic games are like this as well.


Top
 Profile  
 
 Post subject:
PostPosted: Wed Jan 13, 2010 3:24 pm 
Offline

Joined: Sat Nov 17, 2007 8:44 pm
Posts: 385
tokumaru wrote:
UncleSporky wrote:
My natural inclination is to want to be able to switch out any number of tiles at any location, considering that's the primary advantage of CHR-RAM.

I'm not sure I get what you mean here, but you can't just start decompressing from arbitrary positions in the stream, because of the "blocky" nature of the scheme. Are you saying you'd like to force the start of a new block at certain locations so that you could start decompressing from that point on?

Yes, that's what I've been saying. I figured that out about the blocky nature, and since I can't just load in as many as I want, I at least wanted the compressor to be able to stop and start a new segment at a specific length.

Right now it compresses as many tiles as it finds - in your example it's 256 tiles. I want to be able to tell it to compress 32 tiles at a time so that I can "patch in" more tiles, as you say.

Don't worry about it, I'll just make seperate chr files in 32 tile chunks and compress them individually.


Top
 Profile  
 
 Post subject:
PostPosted: Wed Jan 13, 2010 3:34 pm 
Offline

Joined: Sun Sep 19, 2004 11:12 pm
Posts: 19346
Location: NE Indiana, USA (NTSC)
UncleSporky wrote:
Don't worry about it, I'll just make seperate chr files in 32 tile chunks and compress them individually.

That shouldn't be a problem. Given that runs of the same tables are only a couple tiles long anyway, the only significant overhead of each compressed segment is 1 byte for the length of the segment and 2 bytes for the starting address in your directory.


Top
 Profile  
 
 Post subject:
PostPosted: Thu Jan 21, 2010 10:31 pm 
Offline

Joined: Mon Oct 30, 2006 8:32 pm
Posts: 122
I made a collection of CHRs ripped from demos and homebrews. I call it the NESdev Corpus. Here's the download link:

http://www.mediafire.com/?tdilw2ghk1g

It has nearly 250k of graphics data spread across 14 CHR files. Large (>8k) CHR files have been split into 8k chunks. It also includes a text file with the original file sizes and the sizes they've been compressed to using Tokumaru's latest CHR compressor. Hope this helps with testing.


Attachments:
NESdev Corpus.zip [202.02 KiB]
Downloaded 80 times
Top
 Profile  
 
 Post subject:
PostPosted: Sat Mar 13, 2010 1:28 pm 
Offline

Joined: Wed Feb 17, 2010 5:42 pm
Posts: 359
Location: Denine's Devil Mansion
SO..,thanks to this it's possible to compress tiles into very small KB size?
But..it is possible to DECOMPRESS tiles from ROM,edit it,compress it again,and finally,Put compressed tiles into rom?
I'm pretty interested sice I'm hacking GDG(GO Dizzy GO),Which is Codemasters game.


Top
 Profile  
 
 Post subject:
PostPosted: Sat Mar 13, 2010 4:51 pm 
Offline
User avatar

Joined: Sat Feb 12, 2005 9:43 pm
Posts: 10164
Location: Rio de Janeiro - Brazil
No, you can't use my application for hacking, at least not in a straightforward way. First because my format is slightly different than Codemasters'. And Codemasters used different compression algorithms, so the one used in GO Dizzy GO might not even be the same one as Bee52.

But if you're really up to the the task, you could debug the game and find the routine that decompresses tiles (not caring if it's the same one as Bee52 or not) and set up a breakpoint on calls to it, so that you can take note of where in the ROM the compressed graphics are and dump the tiles after they are decompressed.

Then you edit the tiles at will, and once you're done you can compress them with my application. Then you could replace the games decompression routine with mine (it's smaller than the one used in Bee52 at least) and replace the graphics with your own.

Not a simple task by any means, but unless there are tools specifically written for the games you hack you have to do this sort of old-school hardcore hacking.


Top
 Profile  
 
 Post subject:
PostPosted: Sat Mar 13, 2010 5:06 pm 
Offline

Joined: Wed Feb 17, 2010 5:42 pm
Posts: 359
Location: Denine's Devil Mansion
As I didn't understand anything,I guess i'll drop the subject untill i learn more about NES,and 6502.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 84 posts ]  Go to page Previous  1, 2, 3, 4, 5, 6  Next

All times are UTC - 7 hours


Who is online

Users browsing this forum: No registered users and 9 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB® Forum Software © phpBB Group