Opinon needed: Data structure for my sound engine

Discuss NSF files, FamiTracker, MML tools, or anything else related to NES music.

Moderator: Moderators

User avatar
rainwarrior
Posts: 8731
Joined: Sun Jan 22, 2012 12:03 pm
Location: Canada
Contact:

Re: Opinon needed: Data structure for my sound engine

Post by rainwarrior »

dougeff wrote:I think this whole conversation points out that Famitracker could use a better tool (like Famitone) to specifically use Famitracker songs in a game. The easiest approach (IMHO) would be to modify Famitone to accept more/all Famitracker features.
As someone who recently did that, I'd say it is really not the easiest approach. The "easiest" approach would be to just use Famitracker's existing driver, and add a sound effect override on top of it. If you don't need sound effects, it's actually very easy to use Famitracker's driver to play Famitracker tunes (why wouldn't it be?).

In a game, though, there are lots of reasons why supporting every Famitracker feature might not be the best idea:

1. The Famitracker driver is about 5k of code. A more restrained engine like Famitone is more like 1k. This can be huge if space is a problem.
2. A lot of the extra features require extra RAM usage to track their state.
3. Some of the extra features increase CPU usage.
User avatar
Bregalad
Posts: 8055
Joined: Fri Nov 12, 2004 2:49 pm
Location: Divonne-les-bains, France

Re: Opinon needed: Data structure for my sound engine

Post by Bregalad »

But now the tutorial includes all the really advanced features, like volume envelopes, opcodes etc.
Those are actually very simple and are not advanced features at all.

The only remotely efficient way to store music is using some sort of byte code. I did not look in the details you proposed, but it looks like at least two bytes per note is required. With a properly deisgned byte code, you have only one byte per note, and this is capital if you're going to have many music in the game.
User avatar
DRW
Posts: 2225
Joined: Sat Sep 07, 2013 2:59 pm

Re: Opinon needed: Data structure for my sound engine

Post by DRW »

Bregalad wrote:Those are actually very simple and are not advanced features at all.
Advanced or not, but what I wanted to say:
You have 11 bits of frequency values + volume + duty cycle in the APU.
And then you have 26 Famitracker effects that do different stuff to these bits.
So, transforming a Famitracker song into a compressed version of the APU values, you're really better off using the debug log file instead of the text file with the Famitracker syntax.
Bregalad wrote:The only remotely efficient way to store music is using some sort of byte code. I did not look in the details you proposed, but it looks like at least two bytes per note is required. With a properly deisgned byte code, you have only one byte per note, and this is capital if you're going to have many music in the game.
Storing the music is not the problem for me.
And yes, one byte is enough as long as you restrict yourself to just using the frequencies that correspond to actual keys on a keyboard. (Which is what Famitracker does.) Because then you have 96 values.
As shown above, I designed my own little data format, so that's not the issue. The issue is: We have a composer who knows almost nothing about 6502 or the NES APU. And we have me, the programmer, who knows almost nothing about Famitracker.
And I want to convert his songs into my format.

That's why I was hoping that there's a tool that converts Famitracker data into APU data. But not just all APU data per frame. But instead:
1. All five channels treated separately.
2. Only the values that have changed since the last frame.

So, the file would tell you:
Square wave
Frame 1: Volume 5, Duty cycle 1, Frequency 385
Number of frames for these values: 12
Frame 13: Volume 5, Duty cycle 1, Frequency 209
Number of frames for these values: 20

It's still not my own format, of course. But it would help. I could do some search and replace and then use copy and paste.

But it looks like I have to write my own program that checks the log file and then transforms the whole song directly into my data format.

What I also don't understand: You have to fiddle with Famitracker's source code and have to uncomment a #define to enable the logging. Why didn't they include that feature a least as a command line option if they didn't want to include it into the GUI?
My game "City Trouble":
Gameplay video: https://youtu.be/Eee0yurkIW4
Download (ROM, manual, artworks): http://www.denny-r-walter.de/city.html
User avatar
rainwarrior
Posts: 8731
Joined: Sun Jan 22, 2012 12:03 pm
Location: Canada
Contact:

Re: Opinon needed: Data structure for my sound engine

Post by rainwarrior »

If you're going to just be playing back an APU write stream, I don't see why you'd bother arguing about whether a note takes 1 or 2 bytes. Logged formats are huuuuuuge by comparison.

The idea of an "instrument" is actually a compression method. For example, having a volume macro says "you're going to see this pattern of volume writes '10 6 4 3 2 2 1 1 1 0' many times in this song, so let's just store it once". This is the problem instruments are intended to solve. They make the data smaller.
User avatar
rainwarrior
Posts: 8731
Joined: Sun Jan 22, 2012 12:03 pm
Location: Canada
Contact:

Re: Opinon needed: Data structure for my sound engine

Post by rainwarrior »

DRW wrote:As shown above, I designed my own little data format, so that's not the issue. The issue is: We have a composer who knows almost nothing about 6502 or the NES APU. And we have me, the programmer, who knows almost nothing about Famitracker.
And I want to convert his songs into my format.
If you refuse to get to know the tool your composer is using, you're probably going to find it a huge source of frustration trying to implement their songs in your game.

My own personal limited subset of Famitracker is:
No effects except Dxx/Bxx/Fxx (skip/loop/speed control). Volume column allowed. Volume, pitch, arpeggio, and duty macros supported (relative only).

Most other things that effects do can already be duplicated somehow with instrument macros (e.g. vibrato 4xx can be replaced by a pitch macro).

If you can tell your composer not to use any effects, you can cut out a large amount of implementation, and it's easy for the exporter to check if any forbidden effects are used and warn the composer. Give them an export tool that will tell them if they're doing something bad, and will make them an NSF/NES they can test their music in, and they'll be able to validate their music as they go along.
User avatar
dougeff
Posts: 3078
Joined: Fri May 08, 2015 7:17 pm

Re: Opinon needed: Data structure for my sound engine

Post by dougeff »

Famitracker driver is about 5k of code
Well, you don't need EVERY feature. Simply adding volume column and tempo shifts and maybe a few more things (to Famitone2) should suffice for DRW's needs. And just ban the use of the other effects. If his composer is using Famitracker, than it might be the best direction for him.
nesdoug.com -- blog/tutorial on programming for the NES
User avatar
DRW
Posts: 2225
Joined: Sat Sep 07, 2013 2:59 pm

Re: Opinon needed: Data structure for my sound engine

Post by DRW »

rainwarrior wrote:If you're going to just be playing back an APU write stream, I don't see why you'd bother arguing about whether a note takes 1 or 2 bytes.
I don't play back the stream 1:1. I have a minimalistic format:

I store each note as a one byte value which serves as the index for a lookup array that contains all the frequency values for the 96 possible notes.
Furthermore, I store the length of the following notes whenever it changes.
This is what I call a tune. One tune is basically an array of notes, lengths and an end character ($FF).

And then I have an array of addresses to a bunch of tunes. This array is the song.

So, in each frame, the inner loop iterates through the tune and plays the current note and counts down the length counter. When the length counter is 0, the next note (or note and length) is played.
And when the tune has ended, then the outer loop increments the index in the song array and the inner loop can start anew with the new tune. If the outer array ends, the music either ends or it jumps back to the beginning, depending on the end character.

So, every repeating part of the song only needs to be saved once.

I asked my composer not to change the volume and the duty cycle in the middle of the song. I could implement this as well, but I want the music function to be quick and small, so in my game, the volume and duty cycle for each square wave is only set once in the beginning of the song. Saves me a few ifs.
rainwarrior wrote:If you refuse to get to know the tool your composer is using
Well, what does refusing mean? It's simply that I don't have the time to get to know a completely new tool (Famitracker) for a task that I don't know very much about anyway (making music).
I want to create a little game, but I don't want to become an expert in music because of that.
rainwarrior wrote:you're probably going to find it a huge source of frustration trying to implement their songs in your game.
I can understand that many effects might make the songs bigger. But why should the implementation be a source of frustration?
Every effect that you can use in the Famitracker has a direct equivalent in the NES APU. So, what kind of frustration could arise?

From my point of view, I simply take the APU writes of every frame.
Then I calculate for how many frames a note was played and store this as a length/note combination.
Then I see how many consecutive notes have the same length to further compress it into a length/notes combination.
Then maybe checking for repititions, so that I can put them into separate tune arrays.
And that's it.

So, yeah, the size issue is something that I definitely have to communicate to my composer. But why should anything be difficult when it comes to the actual implementation of my engine?
rainwarrior wrote:If you can tell your composer not to use any effects, you can cut out a large amount of implementation, and it's easy for the exporter to check if any forbidden effects are used and warn the composer. Give them an export tool that will tell them if they're doing something bad, and will make them an NSF/NES they can test their music in, and they'll be able to validate their music as they go along.
Not really possible in my case. As I said, my implementation is nothing but a collection of notes (or rather, note indices for a lookup table), lengths per frame and an outer array that stores these collections to avoid redundancies.

I know that you can implement a lot of muscial stuff. The Nerdy Nights tutorials had volume envelopes and opcodes that trigger certain specific functions. But I consciously decided against that. My function shall be quick and simple.
My game "City Trouble":
Gameplay video: https://youtu.be/Eee0yurkIW4
Download (ROM, manual, artworks): http://www.denny-r-walter.de/city.html
User avatar
rainwarrior
Posts: 8731
Joined: Sun Jan 22, 2012 12:03 pm
Location: Canada
Contact:

Re: Opinon needed: Data structure for my sound engine

Post by rainwarrior »

The format you're describing is not even complex enough to play the music of Donkey Kong. A talented composer might be able to make interesting music nonetheless, but it's going to sound strangely minimal compared to most NES music.
DRW wrote:...what does refusing mean? .... I don't have the time to ... I don't want to ...
I literally meant refuse: v. indicate or show that one is not willing to do something.

Anyhow, good luck with however you want to try to go about this. I highly suspect you won't be happy with what you've proposed, but you can decide that for yourself once you have it working. Maybe I'm wrong.

DRW wrote:What I also don't understand: You have to fiddle with Famitracker's source code and have to uncomment a #define to enable the logging. Why didn't they include that feature a least as a command line option if they didn't want to include it into the GUI?
Because it's a feature jsr added entirely for debugging. He didn't think it would be useful to anybody. You are probably the first person besides him to ever use it.

Alternatively you could export to NSF first, and then use something else. NSFPlay has an NSF logging option you can turn on in the .ini file, and shiru's famitone SFX exporter has code that emulates and logs an NSF to do something similar to what you're describing.
User avatar
DRW
Posts: 2225
Joined: Sat Sep 07, 2013 2:59 pm

Re: Opinon needed: Data structure for my sound engine

Post by DRW »

rainwarrior wrote:The format you're describing is not even complex enough to play the music of Donkey Kong.
Why not? It should be complex enough to play any music because it saves any note.
Or do you mean it takes too much space for each song?

What is my intended engine missing?

And what kind of music engine do the more simple commercial games use?
Does "Super Mario Bros." have 2000 lines of code just for the music functions? Don't they just put the notes from ROM to the APU, maybe having some functions to jump between tunes for repetition and maybe something to change the volume?

I really want to know this. Because I want to create a working sound engine. So, if my idea is crap, I'd like people to tell me what to do better.
rainwarrior wrote:Because it's a feature jsr added entirely for debugging. He didn't think it would be useful to anybody. You are probably the first person besides him to ever use it.
Alright, so the ability to transform Famitracker stuff into the actual NES APU values is something that nobody would ever need? Sounds a bit strange to me. Isn't this the first thing that should be featured? The ability to see what the program actually does to create NES sounds?
My game "City Trouble":
Gameplay video: https://youtu.be/Eee0yurkIW4
Download (ROM, manual, artworks): http://www.denny-r-walter.de/city.html
User avatar
rainwarrior
Posts: 8731
Joined: Sun Jan 22, 2012 12:03 pm
Location: Canada
Contact:

Re: Opinon needed: Data structure for my sound engine

Post by rainwarrior »

DRW wrote:Why not? It should be complex enough to play any music because it saves any note.
The most important thing you are missing is volume envelopes. Without these, the only sound you can produce is a solid beep with a sharp start and stop. Just about every NES game implements volume envelopes so that a note can have a shape to it.

In decreasing importance you have things like vibrato or pitch slides. Not every engine has these, but they transform a flat sound into one with some pitch shape. You need to be able to represent pitches between the notes of your scale to do this; a pitch envelope is effective enough.

Being able to change duty in the middle of the song, or middle of a note is also pretty useful. Then there's the idea of arpeggios, where you can change the note on every frame; this lets you make sounds of large chords despite having only a few channels to work with.

These things are often implemented as "macro" envelopes, which are just a series of values that get ticked through each frame. When a note starts, all of its macro envelopes reset to their starting position, and each frame it just reads the next value from the envelope for each of these parameters. Conveniently, you can reuse the same code for updating all of the envelopes.

In general, the idea of an "instrument" is a set of envelopes to be applied. When you change instruments, any notes played after the instrument change will use one set of envelopes until the instrument is changed. Being able to change instrument mid-song is very helpful for composing.

Anyhow, as I said, the biggest omission here is volume envelopes. This is the one thing from that list that will have the most impact. Which gets to...
DRW wrote:Or do you mean it takes too much space for each song?
APU logs take a lot of space because of their inability to repeat patterns effectively (see what I said about instruments above). If your playback is truly limited to just turning the channels on and off at a regimented set of pitches, maybe that's not much overall.

However, as soon as you track volume changes, or any kind of envelope, your data will get huge.

We haven't even touched effects, though as I said above, once you have envelopes, you can do pretty much anything famitracker effects do already with just envelope macros, so they're not really important, IMO.

DRW wrote:Alright, so the ability to transform Famitracker stuff into the actual NES APU values is something that nobody would ever need? Sounds a bit strange to me. Isn't this the first thing that should be featured? The ability to see what the program actually does to create NES sounds?
Famitracker is a program for composing. That's its first thing, to be able to make and hear music that sounds like the NES. APU writes are a detail that are entirely hidden from the user throughout the program. They're important to a programmer maybe, but not the composer, usually. The majority of people using Famitracker just want to be able to trust that the program is accurately producing the NES sound; knowing what values are written to what register is precisely the kind of thing they are using Famitracker to be sheltered from.
User avatar
DRW
Posts: 2225
Joined: Sat Sep 07, 2013 2:59 pm

Re: Opinon needed: Data structure for my sound engine

Post by DRW »

rainwarrior wrote:The most important thing you are missing is volume envelopes. Without these, the only sound you can produce is a solid beep with a sharp start and stop. Just about every NES game implements volume envelopes so that a note can have a shape to it.
Is a volume envelope supposed to have the same sequence of volume values for each note throughout the whole song? So, when I declare the volume values
1, 2, 4, 5, 6, 7, 6, 5, 4, 2, 1
does every note use them and do these values never change throughout the song?
What happens if a note is shorter than this sequence? Does the next note's volume start from the beginning or does it take over where the previous note left off?
And if a note is too long? Does it simply set the first volume after reaching the last?
rainwarrior wrote:Then there's the idea of arpeggios, where you can change the note on every frame; this lets you make sounds of large chords despite having only a few channels to work with.
Why isn't this possible with my attempt? Simply set the length to 1, then store the notes.
rainwarrior wrote:These things are often implemented as "macro" envelopes, which are just a series of values that get ticked through each frame.
Isn't it a huge performance hit when your music engine has to go through 20 envelopes and adjust each note according to these values?
Also, what happens when two values clash?
Say you have a volume envelope, but you can also change the volume mid-song. What happens then? Which value does the program take?
Or take an envelope that might change actual notes instead of just the volume or the duty cycle.
rainwarrior wrote:We haven't even touched effects, though as I said above, once you have envelopes, you can do pretty much anything famitracker effects do already with just envelope macros, so they're not really important, IMO.
But this would also require to go through the whole song manually and painstakingly check what could be created with a macro, right? Or intensively studying the Famitracker to get to know each and every feature.


So, how about just using the Famitracker library in my program and letting the composer do whatever he wants? Would this be an option?
Especially: Is that code quick enough? I don't need a sound library that takes one third of my frame update time to do the background music.
And how much space would this plus the soundtrack occupy? (My ROM is supposed to be 32 + 8 KB.)
My game "City Trouble":
Gameplay video: https://youtu.be/Eee0yurkIW4
Download (ROM, manual, artworks): http://www.denny-r-walter.de/city.html
User avatar
rainwarrior
Posts: 8731
Joined: Sun Jan 22, 2012 12:03 pm
Location: Canada
Contact:

Re: Opinon needed: Data structure for my sound engine

Post by rainwarrior »

DRW wrote:Is a volume envelope supposed to have the same sequence of volume values for each note throughout the whole song?
In Famitracker you can change instruments as often as you like. You can tell your composer they can only use one instrument, if you want, but that's pretty restrictive.
DRW wrote:Does the next note's volume start from the beginning?
Yes.
DRW wrote:And if a note is too long? Does it simply set the first volume after reaching the last?
Macros envelopes usually have a loop point. If no loop is specified, you can just loop on the end of it.
DRW wrote:Why isn't ( arpeggio ) possible with my attempt? Simply set the length to 1, then store the notes.
The data for doing this becomes very large very quickly. (This principle applies to all use of envelopes.) An instrument would store the arpeggio in a couple of bytes, and it be applied to a thousand notes at no extra cost.
DRW wrote:Isn't it a huge performance hit when your music engine has to go through 20 envelopes and adjust each note according to these values?
It takes CPU time, but it doesn't necessarily take a lot of it.
DRW wrote:Also, what happens when two values clash?
You'd have to be specific, as I think each type of envelope is applied differently.
DRW wrote:Say you have a volume envelope, but you can also change the volume mid-song. What happens then? Which value does the program take?
Specifically Famitracker multiplies the volume column value with the volume envelope value (rounding up to 1 if nonzero). (Table lookup implementation.)

You're asking a lot of questions about how Famitracker works. This dialogue is really not a very good way for you to get this information, though. A lot of this would probably very quickly become clear if you read Famitracker's documentation or followed a tutorial, or read its wiki. Actually using the program will explain this better than asking me sparse questions about its implementation.
DRW wrote:But this would also require to go through the whole song manually and painstakingly check what could be created with a macro, right? Or intensively studying the Famitracker to get to know each and every feature.
No, if you want to use Famitracker's features, you should not try to reverse detect them from a register write log. You should just take the exported data as Famitracker produces it. You can check right there if any effects are used, and warn the composer if anything you don't support it used. You don't have to try and detect envelope macros from register logs, instrument definitions are right there in the exported data.
DRW wrote:So, how about just using the Famitracker library in my program and letting the composer do whatever he wants? Would this be an option?
Especially: Is that code quick enough? I don't need a sound library that takes one third of my frame update time to do the background music.
And how much space would this plus the soundtrack occupy? (My ROM is supposed to be 32 + 8 KB.)
Its base size is about 5k. If you went through it carefully you could probably trim a lot of stuff you don't need. I can't possibly tell you how much space your soundtrack would occupy. Depends on how much music you make for it.

I don't know what your performance requirements are, here's what it says in its readme.txt:
Famitracker NSF driver 0.4.6 readme.txt wrote:Average CPU usage is somewhere between 1200-2000 cycles (4-6% of a video frame)
depending on the complexity of the song. Peak usage is higher, usually between
10-15% when switching patterns. This might be improved in future versions.
The main thing that Famitracker is lacking for use in games is sound effect overrides. You'll need to handle that yourself.

Alternatively, you can just use Famitone2, which takes Famitracker music files and turns them into its own format. Famitone2 only implements a subset of Famitracker features and your composer would need to know exactly what's missing. Its biggest missing feature is the volume column; it only has volume envelopes. (It's missing a bunch of other stuff, but that's the one that stands out for most people.)

There's also Muse Tracker. I don't know much about it, but I believe it is well featured and does have a sound effects feature. Famitone2 seems to be more popular, but that might have more to do with all the tutorial stuff Shiru has written than its feature set.


Stepping back a bit, there's other ways to implement some of these concepts; you're asking specifically about Famitracker, and if your composer is using Famitracker you might as well just implement things that work the same way. That's one of Famitone2's problems; it's not really based on a subset of Famitracker, it's just that a lot of it was similar enough to what Famitracker does that Shiru could translate Famitracker tunes into Famitone2 data (as long as the composer doesn't try stuff that would expose the differences); many things about it are fundamentally different than Famitracker's implementation though (e.g. how it handles pitch).

If you look at other NES music engines, you'll find all sorts of ways of doing things. As I mentioned, almost all of them have something that works like a volume envelope macro, but other than that, all bets are off, pretty much. Famitracker has a very versatile set of features, and just the 4 envelope types I mentioned are extremely powerful together, but lots of engines don't have pitch/arpeggio/duty envelopes.

My own approach was to write my own engine that worked fundamentally the same as Famitracker, just as a strict subset. That way there aren't surprises when I am exporting, everything that can be exported sounds and works the same, and everything that can't gives me a warning/error on export.

One of the things that you can't do well with envelopes is sliding pitches between notes; I can't think of an commercial NES games that do this, but Famitracker's pitch slide effects (e.g. Qxx/Rxx/3xx) are very expressive, and well liked by Famitracker composers.
User avatar
DRW
Posts: 2225
Joined: Sat Sep 07, 2013 2:59 pm

Re: Opinon needed: Data structure for my sound engine

Post by DRW »

O.k., I can watch that tutorial.
Reading the wiki would be a bit too much, though. As I said, I didn't intend to become a music expert.
And it wouldn't even stop with the Famitracker. Not only do I not know the Famitracker effects. I don't even know what the regular musical terms mean. What is a pitch, what is a vibrato? I would have to study music 101 just to understand the words.

After having a look at it, I'm playing with the thought of simply using Famitone.
I actually wanted to do everything myself, but if I really need to implement the equivalent of a music player instead of just a function for simple data copying, I guess using an established third party library doesn't go against my principles.
My game "City Trouble":
Gameplay video: https://youtu.be/Eee0yurkIW4
Download (ROM, manual, artworks): http://www.denny-r-walter.de/city.html
User avatar
dougeff
Posts: 3078
Joined: Fri May 08, 2015 7:17 pm

Re: Opinon needed: Data structure for my sound engine

Post by dougeff »

regular musical terms mean. What is a pitch, what is a vibrato?
Pitch refers to the frequency of the sound wave. Higher pitches have more rapid frequency. We call certain frequencies by name (middle C, A1, etc.)

Vibrato - imagine a violinist. He has to press his finger on the fretboard to shorten the length of the string (and increase its frequency). Now, imagine he's shaking his finger rapidly... which effectively lengthens and shortens the string rapidly. The frequency shifts up and down rapidly.

Tremolo - the volume is now rapidly shifting up and down.

Portamento - similar to a pitch slide, but with a specific ending pitch.

Arpeggio - quickly alternating frequencies in a pattern, usually low, middle, high, low, middle, high, etc.
nesdoug.com -- blog/tutorial on programming for the NES
User avatar
DRW
Posts: 2225
Joined: Sat Sep 07, 2013 2:59 pm

Re: Opinon needed: Data structure for my sound engine

Post by DRW »

Thanks, but the question wasn't really intended to be answered. I just asked it to demonstrate that learning about the Famitracker would require learning about another general topic as well.

By the way, I will definitely use Famitone now.
My game "City Trouble":
Gameplay video: https://youtu.be/Eee0yurkIW4
Download (ROM, manual, artworks): http://www.denny-r-walter.de/city.html
Post Reply