It is currently Tue Feb 19, 2019 12:42 pm

All times are UTC - 7 hours



Forum rules





Post new topic Reply to topic  [ 5 posts ] 
Author Message
PostPosted: Tue Jan 24, 2012 4:39 pm 
Offline
User avatar

Joined: Sat Oct 29, 2005 2:09 am
Posts: 504
Location: Indianapolis
Welll, I finished my FPGA SPC player and needed some way to play SPC files. Therein was the problem: The current SPC collection format is .RSN, which is a RAR set containing multiple SPC files.

The problem with this is that nothing short of a powerful computer (i.e. PC) can decompress them to play them. The SNES cannot unrar, my FPGA cannot do it either, so I decided to come up with a new format for storing multiple SPCs to make playing the SPC sets on real hardware a reality.

My method is to take all the SPCs of a collection, extract the 64K RAM portion, break it up into 256 256 byte pieces and remove redundant pieces from the SPCs, then store a header for each included SPC which contains a map of the parts used to assemble its RAM area, the register values, song name, etc. Then finally the RAM data, which has all redundant blocks removed.

A real SNES and my FPGA (or other hardware players like CaitSith2's) can now easily play the complete SPC set!

As a bonus, this new format wrangles the multiple SPC sub-formats into a single uniform format that is easier to process and 100% nonambiguous.

The specification is located here:

http://blog.kevtris.org/blogfiles/spc2_ ... ion_v1.txt

Edit: here's the link to CaitSith2's packer:

http://dl.dropbox.com/u/20737085/spc2_pack.zip


marshallh was nice enough to make a converter program for this new specification, and then CaitSith2 took this code and added all of the SPC field reading and massaging voodoo to round out the format, and helped greatly with defining the specification.

Much thanks to them for making this a reality!

I will leave it up to them to post their converter code, or if they want I can host it and post it too.


Now, onto some details and numbers:

Using the conversion program, all 1458 RSN sets from snesmusic.org were converted into SPC2 files.

* The uncompressed SPCs took up 2.04GB in 34689 files.
* The original RSNs occupied 152MB.
* The SPC2's take up 578MB.

SPC2 doesn't compress the data as much as RAR does, but compressing as good or better than RAR is a pretty tall order, especially when a host system like SNES has to decompress it.

Also, storage space is so stupidly cheap these days that a little more space isn't going to hurt. I was shopping at Fry's last week and 2GB SD cards are the smallest you can buy, and even these were being phased out. 4GB cards were stupidly cheap (under 10 bucks).

I believe this is a decent tradeoff between space and convenience.

I honestly don't see everyone jumping to use this new format, but it exists now and works excellent for our purposes, and a SNES player could be made that can turn SPC sets into a single ROM file that can be played easier. The only downside on SNES is you'd have to hit the reset button to start the next song, but by keeping a signature and song count in RAM, it'd be easy to jump to the next song every time the reset button is pressed.

_________________
/* this is a comment */


Top
 Profile  
 
 Post subject:
PostPosted: Wed Jan 25, 2012 8:37 am 
Online

Joined: Sun Sep 19, 2004 11:12 pm
Posts: 21105
Location: NE Indiana, USA (NTSC)
Great idea.

But I predict that some people would still want to use RSN on a PC, PDA, or smartphone because it's smaller, even though it's non-free. Any idea how much space the set of .spc2.gz, .spc2.zip, .spc2.lzma, or .spc2.7z takes, even if a Super NES can't decompress it on its own?

Does the DSP allow samples to begin on non-256-byte boundaries? Or do they have to start on 256-byte boundaries the way NES samples have to start on 64-byte boundaries in $C000-$FFFF?


Top
 Profile  
 
 Post subject:
PostPosted: Wed Jan 25, 2012 8:57 am 
Offline

Joined: Sat Jan 23, 2010 11:41 pm
Posts: 1161
Just for information, there is an actual RAR decompressor for ZX Spectrum that works well (even with 48K and 3.5 Mhz), and had some practical use for years. So it could be done on SNES too, it just would require someone really dedicated to write it.

Anyway, I like the simpler format better, because RAR would be not free anyway, and it is certainly slower. Also, 150 and 600 MB is not that big difference these days to bother with some very complex decompression code.

DSP does not require to align samples at all. Loops are aligned to 16 samples (9 bytes), though.


Top
 Profile  
 
 Post subject:
PostPosted: Wed Jan 25, 2012 9:46 am 
Offline

Joined: Thu Oct 05, 2006 6:29 am
Posts: 911
Quote:
Does the DSP allow samples to begin on non-256-byte boundaries?

The only limitation is that the sample "directory" has to start at a 256-byte boundary, but the actual samples don't have to (and usually don't).


Top
 Profile  
 
 Post subject:
PostPosted: Wed Jan 25, 2012 12:29 pm 
Offline

Joined: Mon Apr 16, 2007 10:07 am
Posts: 119
Nice. I always thought using a renamed archive container for SPC sets was a bit of a hack solution.

However there are a couple of small niggles with your proposed spec that I would like to bring up:

1. The use of the .SP2 extension. AFAIK, some emulators can still emit SPC files with the .SP2 extension. They start at .SPC and then go .SP0, .SP1, .SP2, etc. Whats wrong with using .KSPC to match the new spec's header ID tag? Using .SP2 isn't much of an issue for players that check the header ID first but there is always going to be one that will try to open a .SP2 set thinking that its a regular SPC dump.

2. The way overflowing text fields are handled seems a little overcomplicated. If we are going to have a chunk at the end of the file to store the overflow, why not just store the entire text there? Let me explain my idea:

If the text is >32 bytes then set the first byte to NULL followed by a pointer pointing to a NULL or a BSTR style string stored in the metadata block. (I cant see why both can't be supported at the same time) So when the file is read just check each text field for a NULL byte at the beginning and act accordingly. If you have a blank field then just have a NULL pointer follow the NULL byte, simples.

Beyond those two points the spec seems decently laid out. I might just have a go at implementing the KSPC spec in the new version of JCOM-SPC I'm slowly working on. (Alpha-II's SNESAPU engine, I got to have my individually interpolated channels @ 96KHz ^_^)


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 5 posts ] 

All times are UTC - 7 hours


Who is online

Users browsing this forum: No registered users and 11 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB® Forum Software © phpBB Group