New SNES SPC Music Collection Format Specification

Discussion of hardware and software development for Super NES and Super Famicom.

Moderator: Moderators

Forum rules
  • For making cartridges of your Super NES games, see Reproduction.
Post Reply
User avatar
kevtris
Posts: 504
Joined: Sat Oct 29, 2005 2:09 am
Location: Indianapolis
Contact:

New SNES SPC Music Collection Format Specification

Post by kevtris » Tue Jan 24, 2012 4:39 pm

Welll, I finished my FPGA SPC player and needed some way to play SPC files. Therein was the problem: The current SPC collection format is .RSN, which is a RAR set containing multiple SPC files.

The problem with this is that nothing short of a powerful computer (i.e. PC) can decompress them to play them. The SNES cannot unrar, my FPGA cannot do it either, so I decided to come up with a new format for storing multiple SPCs to make playing the SPC sets on real hardware a reality.

My method is to take all the SPCs of a collection, extract the 64K RAM portion, break it up into 256 256 byte pieces and remove redundant pieces from the SPCs, then store a header for each included SPC which contains a map of the parts used to assemble its RAM area, the register values, song name, etc. Then finally the RAM data, which has all redundant blocks removed.

A real SNES and my FPGA (or other hardware players like CaitSith2's) can now easily play the complete SPC set!

As a bonus, this new format wrangles the multiple SPC sub-formats into a single uniform format that is easier to process and 100% nonambiguous.

The specification is located here:

http://blog.kevtris.org/blogfiles/spc2_ ... ion_v1.txt

Edit: here's the link to CaitSith2's packer:

http://dl.dropbox.com/u/20737085/spc2_pack.zip


marshallh was nice enough to make a converter program for this new specification, and then CaitSith2 took this code and added all of the SPC field reading and massaging voodoo to round out the format, and helped greatly with defining the specification.

Much thanks to them for making this a reality!

I will leave it up to them to post their converter code, or if they want I can host it and post it too.


Now, onto some details and numbers:

Using the conversion program, all 1458 RSN sets from snesmusic.org were converted into SPC2 files.

* The uncompressed SPCs took up 2.04GB in 34689 files.
* The original RSNs occupied 152MB.
* The SPC2's take up 578MB.

SPC2 doesn't compress the data as much as RAR does, but compressing as good or better than RAR is a pretty tall order, especially when a host system like SNES has to decompress it.

Also, storage space is so stupidly cheap these days that a little more space isn't going to hurt. I was shopping at Fry's last week and 2GB SD cards are the smallest you can buy, and even these were being phased out. 4GB cards were stupidly cheap (under 10 bucks).

I believe this is a decent tradeoff between space and convenience.

I honestly don't see everyone jumping to use this new format, but it exists now and works excellent for our purposes, and a SNES player could be made that can turn SPC sets into a single ROM file that can be played easier. The only downside on SNES is you'd have to hit the reset button to start the next song, but by keeping a signature and song count in RAM, it'd be easy to jump to the next song every time the reset button is pressed.
/* this is a comment */

tepples
Posts: 21883
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Post by tepples » Wed Jan 25, 2012 8:37 am

Great idea.

But I predict that some people would still want to use RSN on a PC, PDA, or smartphone because it's smaller, even though it's non-free. Any idea how much space the set of .spc2.gz, .spc2.zip, .spc2.lzma, or .spc2.7z takes, even if a Super NES can't decompress it on its own?

Does the DSP allow samples to begin on non-256-byte boundaries? Or do they have to start on 256-byte boundaries the way NES samples have to start on 64-byte boundaries in $C000-$FFFF?

Shiru
Posts: 1161
Joined: Sat Jan 23, 2010 11:41 pm

Post by Shiru » Wed Jan 25, 2012 8:57 am

Just for information, there is an actual RAR decompressor for ZX Spectrum that works well (even with 48K and 3.5 Mhz), and had some practical use for years. So it could be done on SNES too, it just would require someone really dedicated to write it.

Anyway, I like the simpler format better, because RAR would be not free anyway, and it is certainly slower. Also, 150 and 600 MB is not that big difference these days to bother with some very complex decompression code.

DSP does not require to align samples at all. Loops are aligned to 16 samples (9 bytes), though.

mic_
Posts: 922
Joined: Thu Oct 05, 2006 6:29 am

Post by mic_ » Wed Jan 25, 2012 9:46 am

Does the DSP allow samples to begin on non-256-byte boundaries?
The only limitation is that the sample "directory" has to start at a 256-byte boundary, but the actual samples don't have to (and usually don't).

Hojo_Norem
Posts: 125
Joined: Mon Apr 16, 2007 10:07 am
Contact:

Post by Hojo_Norem » Wed Jan 25, 2012 12:29 pm

Nice. I always thought using a renamed archive container for SPC sets was a bit of a hack solution.

However there are a couple of small niggles with your proposed spec that I would like to bring up:

1. The use of the .SP2 extension. AFAIK, some emulators can still emit SPC files with the .SP2 extension. They start at .SPC and then go .SP0, .SP1, .SP2, etc. Whats wrong with using .KSPC to match the new spec's header ID tag? Using .SP2 isn't much of an issue for players that check the header ID first but there is always going to be one that will try to open a .SP2 set thinking that its a regular SPC dump.

2. The way overflowing text fields are handled seems a little overcomplicated. If we are going to have a chunk at the end of the file to store the overflow, why not just store the entire text there? Let me explain my idea:

If the text is >32 bytes then set the first byte to NULL followed by a pointer pointing to a NULL or a BSTR style string stored in the metadata block. (I cant see why both can't be supported at the same time) So when the file is read just check each text field for a NULL byte at the beginning and act accordingly. If you have a blank field then just have a NULL pointer follow the NULL byte, simples.

Beyond those two points the spec seems decently laid out. I might just have a go at implementing the KSPC spec in the new version of JCOM-SPC I'm slowly working on. (Alpha-II's SNESAPU engine, I got to have my individually interpolated channels @ 96KHz ^_^)

Post Reply