Sound update timing problems

Discuss NSF files, FamiTracker, MML tools, or anything else related to NES music.

Moderator: Moderators

Shiru
Posts: 1161
Joined: Sat Jan 23, 2010 11:41 pm

Sound update timing problems

Post by Shiru » Fri Jan 28, 2011 11:29 am

There is a set of problems related to sound update routine calling in a more complex project than a simple demo. I still can't find nice NES-specific solutions to these, so maybe someone can give me some ideas or input how it was solved in existing games.


Main problem is that you need to call sound update routine with constant rate. Usually it is TV frame rate. There are two common ways to sync code with TV frames on NES - NMI or some flag polling (either PPU status or one changed in NMI handler). However, you can't just call sound update right when new frame starts if you need to make some things with VRAM first, otherwise you just miss the VRAM access time.

So I see few ways to call the sound update, all of them has drawbacks:

- Flag polling, then VRAM update, then sound update. Easy to work with, however, it is not acceptable if there is lag possibility (i.e. more than one TV frame between the polls).

- NMI handler, all the code that updates VRAM should be there, and after it sound update routine called. If it is a game, it could need many different pieces of code that updates VRAM depending from situation, that could make a lot of headache to organize and debug (compared to previous case).

- IRQ from frame counter. Interrupts disabled before polling flag, then VRAM update, then interrupts enabled, so IRQ handler never called in VRAM access time. This works, but because frame counter rate is a bit slower than a TV frame, and VRAM updates delays IRQ handler call, the update rate is not stable, and it is noticeable, at least in tests I've done in emulators.

- IRQ from DMC probably could be used too, and even could allow to compensate problems of previous way, but it limits or disables DPCM samples use. I guess, it is also the only way that allows to compensate NTSC/PAL frame rate difference without sound artefacts and without additional hardware (second problem).


Second problem is the NTSC/PAL frame rate difference. Pitch difference is easy to compensate, and not that important anyway (most of people don't care, they only notice it when compare side-by-side). Speed difference is very noticeable, though.

Simple way to compensate it is 'frame skip', either make 50hz music and call the update once per frame for PAL, but once per frame with skipping every sixth frame for NTSC; or make 60hz music and call the update once per frame for NTSC and once per frame but twice every fifth frame for PAL. However, this produces noticeable sound artifacts, and only works fine with simple music and sounds.

Better way is to have interrupt with better resolution than 50/60 hz (scanline counter, for example), but it is only available with some mappers.

User avatar
tokumaru
Posts: 11991
Joined: Sat Feb 12, 2005 9:43 pm
Location: Rio de Janeiro - Brazil

Re: Sound update timing problems

Post by tokumaru » Fri Jan 28, 2011 12:00 pm

Shiru wrote:- NMI handler, all the code that updates VRAM should be there, and after it sound update routine called. If it is a game, it could need many different pieces of code that updates VRAM depending from situation, that could make a lot of headache to organize and debug (compared to previous case).
I don't understand how the complications you mentioned affect the sound in any way. The variation of the amount of VRAM updates will be at most 2000 or so cycles, not really enough to cause any audible distortion I think.

This is the method I plan to use in my game (I don't have any sound yet), and since I in fact extend the vertical blank past the start of the frame (using timed code) the sound routine will actually be called at the same time every frame. Not that this matters much, because even the sound routine itself will write to the sound routine at varying times, depending on what calculations it has to perform first.
Second problem is the NTSC/PAL frame rate difference.
Yeah, this is something I'm not really sure how to fix yet. I'm thinking about using different tables for the note durations depending on whether the console is PAL or NTSC, but apply effects and such every frame regardless (I'm not sure how different the instruments will sound depending on the region because of this, though).

User avatar
thefox
Posts: 3141
Joined: Mon Jan 03, 2005 10:36 am
Location: Tampere, Finland
Contact:

Re: Sound update timing problems

Post by thefox » Fri Jan 28, 2011 12:00 pm

Shiru wrote:Simple way to compensate it is 'frame skip', either make 50hz music and call the update once per frame for PAL, but once per frame with skipping every sixth frame for NTSC; or make 60hz music and call the update once per frame for NTSC and once per frame but twice every fifth frame for PAL. However, this produces noticeable sound artifacts, and only works fine with simple music and sounds.
Pornotracker (and I believe Famitracker too) uses a 16-bit counter to achieve variable tempos. So you add some value to the counter each frame, and when it overflows/underflows, you increase the row count (new row means a new note can be started). Effects/envelopes are always updated every frame, so they don't run at exactly the same rate. This can also be used to compensate for PAL/NTSC timing differences. Of course it's not perfect, but it distributes the error more evenly than skipping updates or updating twice per frame.

User avatar
tokumaru
Posts: 11991
Joined: Sat Feb 12, 2005 9:43 pm
Location: Rio de Janeiro - Brazil

Re: Sound update timing problems

Post by tokumaru » Fri Jan 28, 2011 12:07 pm

thefox wrote:Effects/envelopes are always updated every frame
Cool, I was afraid my solution wasn't good. Good to know that it's been tested (and approved I guess?).

Shiru
Posts: 1161
Joined: Sat Jan 23, 2010 11:41 pm

Re: Sound update timing problems

Post by Shiru » Fri Jan 28, 2011 12:13 pm

tokumaru wrote:
Shiru wrote:- NMI handler, all the code that updates VRAM should be there, and after it sound update routine called. If it is a game, it could need many different pieces of code that updates VRAM depending from situation, that could make a lot of headache to organize and debug (compared to previous case).
I don't understand how the complications you mentioned affect the sound in any way. The variation of the amount of VRAM updates will be at most 2000 or so cycles, not really enough to cause any audible distortion I think.
This does not affects to the sound, but it affects to the overall project design, makes it more complex. That's the drawback.

thefox, thanks for reminder, somehow I've completely forgot about the alternating row length method (although I use it often). It is less noticeable than frame skips, indeed.
Last edited by Shiru on Fri Jan 28, 2011 12:19 pm, edited 1 time in total.

User avatar
thefox
Posts: 3141
Joined: Mon Jan 03, 2005 10:36 am
Location: Tampere, Finland
Contact:

Re: Sound update timing problems

Post by thefox » Fri Jan 28, 2011 12:18 pm

tokumaru wrote:
thefox wrote:Effects/envelopes are always updated every frame
Cool, I was afraid my solution wasn't good. Good to know that it's been tested (and approved I guess?).
Yeah I don't think it makes too much of a difference. Like Shiru said, people will notice if the tempo is different, but I doubt they'd notice if volume envelope update speed changed by 15% or whatever (without side-by-side comparison). I know I don't. :)

Besides, I really don't see any other solutions for this. While a scanline IRQ could be used to get a more stable update rate, I don't think it's a viable option if one wants to distribute the game on carts some day.

User avatar
tokumaru
Posts: 11991
Joined: Sat Feb 12, 2005 9:43 pm
Location: Rio de Janeiro - Brazil

Re: Sound update timing problems

Post by tokumaru » Fri Jan 28, 2011 12:21 pm

Shiru wrote:This does not affects to the sound, but it affects to the overall project design, makes it more complex.
Just a little bit, IMO. But the advantages of using this structure are tremendous, as you get to pick what lags and what doesn't in case the CPU gets overloaded. Things like music and status bars should never suffer from lag, because the results are very unpleasant.

Shiru
Posts: 1161
Joined: Sat Jan 23, 2010 11:41 pm

Post by Shiru » Fri Jan 28, 2011 12:31 pm

I don't think that putting all the code that access to VRAM to NMI affects to project design just 'a little bit'. I mean, let's say you have a game of few screens. For one screen you need to update part of nametable when user press keys; for other you need to animate part of nametable; for other you need to update text sometimes (not every frame). You either need to make a general VRAM update system (not going to be fast because it is general), or make set of switchable NMI handlers. I suppose it is much more complex than separate loops for different parts of a game with VRAM-related code after a flag polling (nothing to switch, no need to move parts of related code in other place).

Which advantages you see with moving all the VRAM updating code to NMI?

tepples
Posts: 22278
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Post by tepples » Fri Jan 28, 2011 12:37 pm

This wiki page should answer a few of your questions.

Shiru
Posts: 1161
Joined: Sat Jan 23, 2010 11:41 pm

Post by Shiru » Fri Jan 28, 2011 1:03 pm

It basically describes that I've meant under words 'a general VRAM update system'. Is there actual games that uses this method?

User avatar
tokumaru
Posts: 11991
Joined: Sat Feb 12, 2005 9:43 pm
Location: Rio de Janeiro - Brazil

Post by tokumaru » Fri Jan 28, 2011 1:06 pm

Shiru wrote:You either need to make a general VRAM update system (not going to be fast because it is general), or make set of switchable NMI handlers.
I use both actually. My NMI first checks if the is a specialized routine defined, in which case it jumps to it. If there isn't, it just sets the good old flag to let the program know VBlank started (which means I can use the old "wait for VBlank" technique if I want to). In the specialized NMI I use for the main game, I used a slot system. So in addition to performing wat has to be done every frame (sprites, scrolling, etc), it also calls a number of updating routines defined in a list (the list was populated by the main game engine during the previous frame). Each of these routines performs a different task, like updating metatiles, patterns, palette, and so on.

What I have is a bit more complex, but the typical NMI setup is pretty simple. I mean, this:

Code: Select all

GameLoop:

	;RUN GAME LOGIC

	;WAIT FOR THE NMI HANDLER TO FINISH

	jmp GameLoop

NMI:

	;PERFORM PPU UPDATES

	rti
is hardly any more complex than this:

Code: Select all

GameLoop:

	;RUN GAME LOGIC

	;WAIT FOR VBLABK

	;PERFORM PPU UPDATES

	jmp GameLoop

NMI:

	;INDICATE THAT VBLANK STARTED

	rti
Anf if you simply make JMP (NMIHandler) the first instruction in your NMI routine you can have as many different handlers as you like.
Which advantages you see with moving all the VRAM updating code to NMI?
If your program doesn't lag at all, ever, there's little advantage. But as programmers get better and start working on more complex games, chances are there will be lag at some points. In this case, the biggest advantage is that you get to pick what lags and what doesn't, because there is a piece of code that will run 50 or 60 times per second regardless of what happens.

You mentioned how weird the music sounds when you miss updates... And what happens to a status bar when you miss the split point because of lag is catastrophic, it can even lock the game up in certain cases. So it's important that you have a way to perform these critical operations no matter what, and the NMI gives you that.

Shiru
Posts: 1161
Joined: Sat Jan 23, 2010 11:41 pm

Post by Shiru » Fri Jan 28, 2011 1:19 pm

In your example there is just GameLoop, but actual game could have many loops (one for menu, other for game, other for pause, other for a cutscene). I think that this is where things turns to be more complex with VRAM update code in NMI, with switchable handlers or general update system you need more code to do the same.

Raster effects like splits are special case, I think - if game could lag, and no scanline IRQ available, there is just no other way than use NMI.

User avatar
tokumaru
Posts: 11991
Joined: Sat Feb 12, 2005 9:43 pm
Location: Rio de Janeiro - Brazil

Post by tokumaru » Fri Jan 28, 2011 1:57 pm

Shiru wrote:In your example there is just GameLoop, but actual game could have many loops (one for menu, other for game, other for pause, other for a cutscene).
But just like you might need specialized NMI handlers, wouldn't you have all these different kinds of updates next to their respective loops if you used the "newbie-friendly" method? I mean, if inside every loop you have a different code for PPU updates, how's that any different from having all of that as separate NMI handlers? It's practically the same amount of code, it's just ordered differently.
with switchable handlers or general update system you need more code to do the same.
That might be the case with a general update system, but for switchable handlers, I fail to see how this:

Code: Select all

Loop1:
	;RUN FRAME LOGIC FOR LOOP 1;
	;WAIT FOR VBLANK;
	;PPU UPDATES EXCLUSIVE TO LOOP 1;
	jmp Loop1

Loop2:
	;RUN FRAME LOGIC FOR LOOP 2;
	;WAIT FOR VBLANK;
	;PPU UPDATES EXCLUSIVE TO LOOP 2;
	jmp Loop2

NMI:
	;INDICATE THAT VBLANK STARTED
	;rti
is significantly simpler or any more efficient than this:

Code: Select all

Loop1:
	;RUN FRAME LOGIC FOR LOOP 1;
	;WAIT FOR THE NMI HANDLER TO FINISH;
	jmp Loop1

Loop1NMI:
	;PPU UPDATES EXCLUSIVE TO LOOP 1;
	rti

Loop2:
	;RUN FRAME LOGIC FOR LOOP 2;
	;WAIT FOR THE NMI HANDLER TO FINISH;
	jmp Loop2

Loop2NMI:
	;PPU UPDATES EXCLUSIVE TO LOOP 2;
	rti

NMI:
	;JUMP TO CURRENT NMI HANDLER;
It's practically the same code, it's just ordered differently. The only real overhead to the switchable NMIs is that you have to setup the pointer to the NMI handler, but that's something you can do once and really quickly during the initialization phase of each loop, and the stability you get more than makes up for that IMO. You can even reuse NMI handlers if different loops need similar kinds of updates. And like I said, you don't even have to sacrifice the other technique, you can use both. Here's the NMI routine I use in my game:

Code: Select all

NMI:

	;branch if the program is just waiting for VBlank
	bit WaitingVBlank
	bmi +FlipFlag

	;branch if there is no custom interrupt routine defined
	bit CustomNMI+1
	bpl +Return

	;transfer control to the the custom routine
	jmp (CustomNMI)

+FlipFlag:

	;indicate that the interrupt happened
	inc WaitingVBlank

+Return:

	;return from the interrupt
	rti
I can either define custom NMI handlers for each section of my game, or wait for VBlank with this:

Code: Select all

	dec WaitingVBlank
-	bit WaitingVBlank
	bmi -
And I can even do that while a custom handler is still defined, as this flag is checked first, so it takes precedence over the custom handler.

Shiru
Posts: 1161
Joined: Sat Jan 23, 2010 11:41 pm

Post by Shiru » Fri Jan 28, 2011 2:17 pm

tokumaru wrote:if inside every loop you have a different code for PPU updates, how's that any different from having all of that as separate NMI handlers? It's practically the same amount of code, it's just ordered differently.
That's it, code located in other place. Drawback is that it makes code more difficult to mantain when you have a lot of t - look here for one part of a single thing, look there for other part, try to find related code in thousands of lines in few files, etc.

If you use general VRAM update code, instead of straightforward VRAM update code pieces you have update setup code and (that's the overhead) the update code in NMI. Drawback here is that you need to design that system and always remember how it works, it is additional layer of complexity.

If you have switching handlers or pointers, there is not only some code to switch them (not much), but also need to switch them safely. Otherwise you can have wrong handler for a moment, or even non-existant handler address (updated LSB, NMI happened, crash). It could be source of weird bugs.

What I'm telling is not that this approach, with different handlers or general update code, is wrong, or bad, etc. I just think it is more complex.

Anyway, I got idea that you prefer NMI way. I would like to hear what others use and what commercial games used (if someone examined them) - to get idea what is common, what is not, etc.

User avatar
tokumaru
Posts: 11991
Joined: Sat Feb 12, 2005 9:43 pm
Location: Rio de Janeiro - Brazil

Post by tokumaru » Fri Jan 28, 2011 2:38 pm

Shiru wrote:Anyway, I got idea that you prefer NMI way.
Sure do! 8)
I would like to hear what others use
I believe most just wait for VBlank, as that's what all beginner tutorials seem to use (and many beginners end up polling $2002 instead of using NMIs to wait for VBlank, because a lot of old tutorials were made before we knew that frames can be missed that way). Most homebrew projects released so far don't even need very complex game logic, so many developers don't even realize that bad things could happen if there was any lag.
and what commercial games used (if someone examined them) - to get idea what is common, what is not, etc.
I think most commercial games use the NMI for PPU updates and critical things that can't lag, with a few earlier games (like, before 1989) having everything inside the NMI (which has the same shortcomings of waiting for VBlank in case there is lag). I'm not aware of any commercial games "waiting for VBlank" like many homebrewers do, but there probably are some. Let's see what others have to say.

Post Reply