Instruments vs. effects?

Discuss NSF files, FamiTracker, MML tools, or anything else related to NES music.

Moderator: Moderators

tepples
Posts: 22708
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Re: Instruments vs. effects?

Post by tepples »

WheelInventor wrote:Regarding file format, is famitone or any or all of the other engines that have been released to the community assuming that unless an instruction states otherwise, the track or frame stays the same? (for example, track 3 played frame 5, so it will play frame 5 the next frame switch, too, unless there's new data) Or is it more like 'do this for that long time'?
NerdTracker II stores the order list as 5-tuples: (pattern for pulse 1, pattern for pulse 2, pattern for triangle, pattern for noise, pattern for DMC). So does FamiTracker, and I assume that engines based on FamiTracker text export do likewise. Thus if two consecutive frames play pattern 5 on track 3, the 5 will be stored twice.

Pently operates differently, not being quite as closely tied to tracker heritage. The order list is a list of "At X rows after song start, start playing and looping pattern X". Individual patterns can have different loop lengths, making an ostinato or short drum loop efficient. A pattern can be interrupted by the start of another pattern or by a stop command, making drum fills efficient. Patterns can also be reused and transposed across channels with different default instruments, making parallel motion of two channels efficient. And because pattern start commands have row granularity, not frame switch granularity, 2-channel echo is also efficient. But the price it pays for this flexibility is that frame switch commands take more bytes.
User avatar
tokumaru
Posts: 12427
Joined: Sat Feb 12, 2005 9:43 pm
Location: Rio de Janeiro - Brazil

Re: Instruments vs. effects?

Post by tokumaru »

rainwarrior wrote:I was the coder for the music
I didn't know that!
Only some of Famitracker's effects are equivalent to instrument macros (e.g. vibrato, or fixed duty). Some aren't, like targeted pitch slides, which have to stop based on a comparison with the current pitch.
Couldn't I just write some slide macros with the few combinations of offsets and speeds I need? I know that's not nearly as efficient space-wise, but as long as I don't need many combinations of offsets and speeds, I think this could work.
I don't think you're saving much by trying to think of an effect as another envelope, if anything.
The playback code is simpler, I guess, since the code for processing envelopes is already there for instruments, and could be reused for these effects.
I'd guess it would be about the same CPU cost as FT's existing vibrato, and take up more ROM space.
Yes, songs may take more space, depending on how varied the effects are.
In mine specifically: an "instrument" is 4 bytes, and a "macro" varies but the average is 15 bytes, and macros can be shared across many instruments.
How exactly do 4-byte instruments work? Can an instrument do anything without a macro? The "macros" you're talking about are what Famitracker calls "sequences"?
tepples wrote:(Comparison)
I do like your version much better... it's cleaner and more fluid.
RushJet1 wrote:Few effect commands:
https://www.youtube.com/watch?v=WluHxbGQkBk
This one is awesome!
Only effect commands (no instruments, note that this gets super cluttered around 2:55):
https://www.youtube.com/watch?v=D6EDwXayQY4
This... is pretty insane!
One instrument and no effect commands (except volume):
https://www.youtube.com/watch?v=bDoIIUBmGg4
Sounds pretty good, specially considering the harsh restrictions!
User avatar
rainwarrior
Posts: 8734
Joined: Sun Jan 22, 2012 12:03 pm
Location: Canada
Contact:

Re: Instruments vs. effects?

Post by rainwarrior »

tokumaru wrote:How exactly do 4-byte instruments work? Can an instrument do anything without a macro? The "macros" you're talking about are what Famitracker calls "sequences"?
I guess "macro" was a term picked up from PPMCK/MML, but yes it is synonymous with "sequence" in Famitracker. I also often call them envelopes. Sorry if this is confusing. ;)

Yes, an instrument is just a selection of macros. One for each of the type of envelopes you can assign. (Famitracker actually has 5 envelope types, but "hi pitch" is not very useful; other data can be associated with an instrument, too, but it's not very essential.)
tokumaru wrote:Couldn't I just write some slide macros with the few combinations of offsets and speeds I need? I know that's not nearly as efficient space-wise, but as long as I don't need many combinations of offsets and speeds, I think this could work.
Depends on how much you want to use it. A few real problems:
1. Every interval is a different distance for each starting pitch, so if you use an envelope to make a slide from C-3 to E-3, you can't really use it for any other pair of pitches.
2. Determining the precise values needed for a targeted slide envelope is tedious.
3. Slow pitch slides require very large envelopes.
4. The pitch slide effects all have configurable speed; each envelope only supports one speed.

If you want to do it rarely, sure, it's fine to create a couple of special instruments for specific slides. If you want to do it often, no, it's not really feasible without dedicated code.

Again, though, if you don't have this feature you just compose around it. You can still do plenty of pitch effects, it's just difficult to do a targeted slide from one precise pitch to another without implementing the effect.

There are other ways to handle pitch, too. You could create an expanded pitch table with 16 divisions of each semitone (like how MODs do fine pitch), allowing pitch envelopes to apply logarithmically so you could make a "5 semitone slide" envelope that could be used on any note of the scale. This is feasible from an engine perspective, but it's not compatible with Famitracker's linear model of pitch.
tokumaru wrote:The playback code is simpler, I guess, since the code for processing envelopes is already there for instruments, and could be reused for these effects.
I think the biggest problem with effects, is that each class of related effects requires RAM to maintain its state; this is true whether or not you try to implement it as an envelope. The actual code complexity of an individual effect tends to be pretty small, the inefficiency is usually just due to having to implement a collection of them. Each effect is one more bit of state you have to track and update every frame, multiplied by the number of channels that effect may apply to.

I was saying that it's true that a few of the effects could in theory be implemented by reusing other envelope code, but in general I think the extra envelope data would end up being bigger than the code you saved, and RAM usage would be the same or worse. CPU for any given effect is probably similar or better than an envelope. In some cases it could just be an approximation of the effect too. I understand the train of thought but I don't think there's much to gain here. The complexity of the effect code is not really the problem with effects, and probably not worth trying to "simplify" by replacing with extra envelopes.


The thing is, though, the set of 4 base envelopes that an instrument has (volume, pitch, arpeggio, duty) are very versatile as-is, easy to use, and they can all operate with the same code. As I said, you can just make vibrato as an envelope that's part of an instrument; you don't require an effect feature to add it. There's a lot of functional redundancy in what Famitracker's effects do; they're just there to support different workflows and make it easier to produce music. This is why my answer to your OP question was that you don't really need effects; you've got tons of functionality already with just the 4 envelopes.

You could do something like let a vibrato effect replace an instrument's pitch envelope if it's blank, rather than acting as an overlaid thing. That way the effect isn't actually doing more runtime work, just a 1-time setup thing, but if you do stuff like that it starts to get really divergent from how something works/sounds in Famitracker. I dunno how much you want to go down the road of either modifying Famitracker, or giving your composer features they can only hear if they do an export and run in an emulator, but in general I'd avoid this.

My approach was just to pick a flexible/versatile subset of features from Famitracker; I didn't really want to do stuff that Famitracker couldn't, even if that seemed convenient, because I wanted it to sound exactly the same in the tool as in the game, and to have no confusion about what's going to happen. Anything that's not supported throws an error during export, and everything that is supported should work identically, as much as it can. (I did make a few subtle differences.)
User avatar
tokumaru
Posts: 12427
Joined: Sat Feb 12, 2005 9:43 pm
Location: Rio de Janeiro - Brazil

Re: Instruments vs. effects?

Post by tokumaru »

rainwarrior wrote:Sorry if this is confusing. ;)
That's OK, thanks for clearing that up.
1. Every interval is a different distance for each starting pitch, so if you use an envelope to make a slide from C-3 to E-3, you can't really use it for any other pair of pitches.
Well, I do plan on using tables to create a fixed number of steps between notes, which will allow me to address them linearly... That should take care of this problem, right?
2. Determining the precise values needed for a targeted slide envelope is tedious.
This too shouldn't be such a big problem if notes can be addressed linearly, I guess.
3. Slow pitch slides require very large envelopes.
That's true.
4. The pitch slide effects all have configurable speed; each envelope only supports one speed.
Again, true.
If you want to do it rarely, sure
That will probably be the case.
There are other ways to handle pitch, too. You could create an expanded pitch table with 16 divisions of each semitone (like how MODs do fine pitch), allowing pitch envelopes to apply logarithmically so you could make a "5 semitone slide" envelope that could be used on any note of the scale.
Yes, that's exactly what I plan to do (I'm replying as I read, so I mentioned it before knowing you'd suggest it).
I think the biggest problem with effects, is that each class of related effects requires RAM to maintain its state;
Yeah, I also though about this after my last post. If I can have 2 or 3 effects per channel, that could add up to a significant amount of RAM, and I really am trying to keep RAM requirements low.
this is true whether or not you try to implement it as an envelope.
Well, I'm not so sure about that. For envelopes, all I have is a pointer to the current envelope step. If I add a second envelope I'll just need another pointer. My idea is to get the note from the song and the channel's volume, apply all the instrument modifiers, and on top of that apply all the effect modifiers, and then write the final result to the APU. I don't really need any state because each step of an envelope has all the info I need, so I can construct the final APU data from scratch every frame.
I was saying that it's true that a few of the effects could in theory be implemented by reusing other envelope code, but in general I think the extra envelope data would end up being bigger than the code you saved, and RAM usage would be the same or worse.
I agree about the data being bigger, but I guess my approach here is like "I don't want to make the engine overly complex, and I probably won't need effects very often, but I also don't want to completely get rid of them", so this is a quick and simple way I found to allow effects if they are really necessary, but when I use them I'll know that the price is precious ROM space. If I don't use them, I don't lose anything.

As for RAM usage, 2 bytes for an extra pointer per channel sounds MUCH better than whatever I'd need for the states of combined effects (I'm guessing it would be at least 2 bytes per effect, but maybe some need more?). Unless I'm missing something here, but I'm fairly sure I can create the APU data from scratch every frame like I described above.
This is why my answer to your OP question was that you don't really need effects; you've got tons of functionality already with just the 4 envelopes.
And I've been listening to some good examples of that.
I dunno how much you want to go down the road of either modifying Famitracker, or giving your composer features they can only hear if they do an export and run in an emulator, but in general I'd avoid this.
I want to keep things simple on all fronts. :wink: I'm not sure Famitracker will be involved at all, though.

Please note that I'm not trying to defend my choices or anything like that, I'm honestly looking for some good feedback so I'll end up with a sound engine that isn't too crippled but isn't so heavyweight either, and so far the replies have been very useful. If you think that things I say are stupid, or if you disagree with me on anything, please keep the replies coming. If you can validate my ideas too, that'd be great, so I know I'm not going forward with something completely weird. Audio is the thing I'm least experienced with when it comes to game programming, so I really need to be told that I'm going in the wrong direction if that's the case. Hopefully that'll make me better at this.

EDIT: I just realized I completely neglected to mention that the RAM requirements I mentioned (2 bytes per envelope) is only possible if the different types of envolopes aren't in separate reusable sequences, but instead pitch, duty and volume sequences are all packed together in the same sequence. I do realize that, again, this needs more ROM than separate combinable sequences each with their own loop points. This is not final though.
User avatar
rainwarrior
Posts: 8734
Joined: Sun Jan 22, 2012 12:03 pm
Location: Canada
Contact:

Re: Instruments vs. effects?

Post by rainwarrior »

Lots of engines combine volume + duty into a single macro (for obvious reasons). In Famitracker arpeggio and pitch macros are kind of exclusive anyway, so you could combine them as well and have some sort of mode flag to control it. Again, if you do this kind of thing, you're divergent from Famitracker, but that's your call. This would save RAM and CPU and instrument size all at once.

Famitone, for example, forbids duty envelopes and just keeps a single duty state per instrument instead, I think. For HH, I believe I added duty envelope functionality by combining it with volume, because it was sufficient for the music I had to work with. (Combining them puts restrictions on looped envelopes, sometimes you can't sensibly combine them.)

Famitone also "simplifies" pitch macros by removing their ability to accumulate. i.e. in FT you can slide pitch down with a 1-byte looping envelope of "1", but in Famitone you can't loop like that, and I think it stops at 127, or something, so it becomes useless for long falloff slides. He did this presumably because he thought it was simpler/smaller to implement a single byte pitch offset than a cumulative pitch state, but it's one of many decisions that results in a significant loss of functionality from the FT counterpart.
tokumaru wrote:Well, I'm not so sure about that. For envelopes, all I have is a pointer to the current envelope step. If I add a second envelope I'll just need another pointer. My idea is to get the note from the song and the channel's volume, apply all the instrument modifiers, and on top of that apply all the effect modifiers, and then write the final result to the APU. I don't really need any state because each step of an envelope has all the info I need, so I can construct the final APU data from scratch every frame.
No, you need 2 bytes of state for your envelope. Any effect that can be sufficiently implemented as an envelope has no more than 2 bytes of state. (Per channel.) Take a look at FT's NSF Driver source if you want to see how it organizes RAM, and how small individual effect implementations are.

The other thing is all the effects that can be implemented as an extra envelope are already kinda doable as instrument envelopes anyway. They're a redundancy. The effects that can't be envelopes are the ones that matter more, functionally.

The suggestion that effects could be implemented as more envelopes to me is analogous to suggesting that addition could be implemented as a lookup table instead. Sure it could, but the reasons why you wouldn't are very similar.

Sorry, I should just drop the argument. I think if you start to look into what specific effects do you'll understand why it's not really worthwhile. If you really want to simplify something, just don't implement them at all, or don't implement them as overlaid effects, or make them temporarily replace instrument envelopes instead, etc.
tokumaru wrote:I want to keep things simple on all fronts. :wink: I'm not sure Famitracker will be involved at all, though.
The reason I intentionally maintained a strict subset of Famitracker, instead of making convenient/interesting implementation decisions to make an engine divergent from it is that Famitracker is a great composing tool. The kind of feedback and quick iteration you get from it is fantastic. It's easier to use and easier to write better music with it than, for example, PPMCK/MML's compile-and-check process. If you make something that works with FT, you gain all the power of that tool, and it has a large community of skilled users who already know how to work with it.

There's lots of ways you can implement a music engine for NES, and there are a bunch of possibilities that I think would be interesting for an engine to have (e.g. the logarithmic pitch tables I suggested earlier), but personally I would only pursue them for a one song demo, or some proof of concept, where the amount of music work needed is more limited.

In a commercial project, if you can make good music faster with one tool than another, you should use it. If I was paying someone by the hour, I definitely wouldn't give them PPMCK/MML instead of FT to work with, if you understand what I'm saying, but even in a non-commercial project your time is still limited by fatigue and availability. If you can iterate faster, you can make better music with the time you've got.

Whatever you wanna use is up to you, but this is the reason why I stick with FT. There's lots of things about FT that I would/could have done differently if I was starting from scratch, but I'd rather not start from scratch if my real goal is just to make a good game soundtrack I can use in my game.
tepples
Posts: 22708
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Re: Instruments vs. effects?

Post by tepples »

Say there's an MML-style engine that produces smaller bytecode than FamiTone, let's say because of more efficient looping and transposition. And further say someone's composing for an NROM or CNROM game. Would the following process be practical?
  1. Compose in FT
  2. Convert FT text to MML once using engine's conversion tool
  3. Optimize MML by hand
  4. Compile-and-check optimized MML
User avatar
rainwarrior
Posts: 8734
Joined: Sun Jan 22, 2012 12:03 pm
Location: Canada
Contact:

Re: Instruments vs. effects?

Post by rainwarrior »

I can't seriously entertain a hypothetical question like that. There's nothing concrete to make a decision about. How many hand optimizations are you making in step 3? How many would be too much? How much ROM space do you need to save? Get a real project, and you'll know the answer.

Practical is anything that works when you're actually making something. Jeroen Tel got it done by typing hex values into a text file.

My answers above are an explanation of my own motivations for the decisions I've made, not laws of practicality.
User avatar
tokumaru
Posts: 12427
Joined: Sat Feb 12, 2005 9:43 pm
Location: Rio de Janeiro - Brazil

Re: Instruments vs. effects?

Post by tokumaru »

rainwarrior wrote:Lots of engines combine volume + duty into a single macro (for obvious reasons). In Famitracker arpeggio and pitch macros are kind of exclusive anyway, so you could combine them as well and have some sort of mode flag to control it. Again, if you do this kind of thing, you're divergent from Famitracker, but that's your call. This would save RAM and CPU and instrument size all at once.
It's good to know I'm not doing something crazy in this respect.
(Combining them puts restrictions on looped envelopes, sometimes you can't sensibly combine them.)
Yeah, I get that.
Famitone also "simplifies" pitch macros by removing their ability to accumulate. i.e. in FT you can slide pitch down with a 1-byte looping envelope of "1", but in Famitone you can't loop like that, and I think it stops at 127, or something, so it becomes useless for long falloff slides.
I see... I wouldn't support this either, since it'd need more state (a dynamic pitch).
No, you need 2 bytes of state for your envelope.
True, but that's just wording it differently. :wink:
Any effect that can be sufficiently implemented as an envelope has no more than 2 bytes of state. (Per channel.)
But each effect only affects one parameter, while an envelope can deal with duty, volume and pitch all at once.
Take a look at FT's NSF Driver source if you want to see how it organizes RAM, and how small individual effect implementations are.
Will do, but I'm already quite set on not implementing effects the way FT does. Whether I'll do it in some other way is another issue.
The other thing is all the effects that can be implemented as an extra envelope are already kinda doable as instrument envelopes anyway. They're a redundancy.
I agree, I'm just hoping I can save a little space by not duplicating entire instruments, when a little modifier would do. I honestly don't know how often these situations will come up, though.
The effects that can't be envelopes are the ones that matter more, functionally.
Guess I'll probably have to live without them.
The suggestion that effects could be implemented as more envelopes to me is analogous to suggesting that addition could be implemented as a lookup table instead.
I actually do that too!
Sorry, I should just drop the argument. I think if you start to look into what specific effects do you'll understand why it's not really worthwhile.
I'm seeing this almost as the equivalent of not implementing anything at all. If I don't ever use these hacked-in effects, all I'm losing is 2 bytes of RAM pet channel, which I can claim back at a later time if necessary. BTW, I'm not arguing. :)
Post Reply