Audio sync strategies

Discuss emulation of the Nintendo Entertainment System and Famicom.

Moderator: Moderators

Post Reply
drewying
Posts: 7
Joined: Fri Mar 02, 2018 12:22 am

Audio sync strategies

Post by drewying »

Hello friends!

My emulator, Nintendoish, is progressing along nicely. I have scan line accurate PPU emulation, mapper support for all of the official US cartridge boards. Currently working on APU emulation. So far it has been an amazing leaning experience!

I have a quick question on audio sync.

Right now my emulator targets ~60 fps. I can hit that on my desktop no problem. On my laptop however, when running in dev mode, it will drop to about 40 fps.

When fps drops, my strategy to prevent slowdown is to simply drop render frames. Keep clocking the NES, but just don't render to the GPU.

This works great with video. Barely noticeable. However, it doesn't seem to work great with audio. I get this weird cascading loop. The NES is running behind and can't fill the audio buffer fast enough, so I output silence. The NES notices it's behind and so skips video frames to catch up. Audio buffer fills back up. But now the audio and video are out of sync.

I've tried just skipping audio frames when I skip a video frame but that creates very noticeable distortion in the audio. Especially when running at 40 fps. While dropping 1/3 of your video frames isn't super noticeable, dropping 1/3rd of your audio very much is.

So I was just wondering, what are some strategies with dealing with this? How do emulators running on low powered devices manage to drop their frame rate without creating audio distortion or getting their video/audio out of sync? Any suggestions?

Thank you!

PS
Source code is here: https://github.com/drewying/Nintendoish. As you will see it still is very much a work in progress.
Last edited by drewying on Sat Apr 21, 2018 2:41 am, edited 1 time in total.
User avatar
Dwedit
Posts: 4924
Joined: Fri Nov 19, 2004 7:35 pm
Contact:

Re: Audio sync strategies

Post by Dwedit »

Briefly peeked at the source code.

You should probably redo your graphics engine, since all pixels are being done with OpenGL draw calls to set the coordinates then set the color.

The "right" way to do it is with a framebuffer and glTexSubImage2D.
Here come the fortune cookies! Here come the fortune cookies! They're wearing paper hats!
drewying
Posts: 7
Joined: Fri Mar 02, 2018 12:22 am

Re: Audio sync strategies

Post by drewying »

Thanks so much for the tip Dwedit!

I switched to glTexSubImage2D and that gave me a huge speed bump. Enough where it's a lot more reasonable to develop this in dev mode.

I still have work to do on the graphics engine. It's still very bare bones. Eventually I'll be moving everything to a shader where I have some plans for some graphics effects I hope to do. :)
User avatar
thefox
Posts: 3134
Joined: Mon Jan 03, 2005 10:36 am
Location: 🇫🇮
Contact:

Re: Audio sync strategies

Post by thefox »

Code: Select all

    //OpenGL demands we flip
    y = 240 - y;
    x = 256 - x;
Nope, it doesn't. Simply flip the texture coordinates or the vertex coordinates of the polygon you're drawing. Also there's an off-by-1 bug in this code leading to writes outside the buffer (should be 239-y and 255-x).
Download STREEMERZ for NES from fauxgame.com! — Some other stuff I've done: fo.aspekt.fi
drewying
Posts: 7
Joined: Fri Mar 02, 2018 12:22 am

Re: Audio sync strategies

Post by drewying »

Simply flip the texture coordinates or the vertex coordinates of the polygon you're drawing.
Great suggestion. Done and it looks great! Thank you!

I am curious though to my original question. Emulators which can't run at ~60 and have to drop frames... how do they manage to keep audio from staying in sync... or at least not sounding distorted if they drop a lot of frames?

Or do you just need to make sure that your emulator is fast enough where that isn't a serious concern?
User avatar
rainwarrior
Posts: 8732
Joined: Sun Jan 22, 2012 12:03 pm
Location: Canada
Contact:

Re: Audio sync strategies

Post by rainwarrior »

If the emulator has "frameskip" it skips rendering and tries to do every second (or more) frame faster, but generally in this case audio is never skipped.

With no frameskip, every emulator I can think of just has gaps in the audio when it can't keep up to 60fps. (Holding the last sample value across the gap makes a slightly less annoying pop than just inserting 0s.)

There are other ways to handle this, but those are the two that are common.
User avatar
koitsu
Posts: 4201
Joined: Sun Sep 19, 2004 9:28 pm
Location: A world gone mad

Re: Audio sync strategies

Post by koitsu »

Talked about in great detail: viewtopic.php?f=3&t=15405
User avatar
Dwedit
Posts: 4924
Joined: Fri Nov 19, 2004 7:35 pm
Contact:

Re: Audio sync strategies

Post by Dwedit »

rainwarrior wrote:With no frameskip, every emulator I can think of just has gaps in the audio when it can't keep up to 60fps.
The really old emulators ran the sound channels (square, triangle, noise, DMC) asynchronously, so at the end of the frame, it could generate sound for as long as it needed to, or generate sound for whatever speed it needed to generate it at. Usually the time to generate for would be about 1/60s.
Here come the fortune cookies! Here come the fortune cookies! They're wearing paper hats!
tepples
Posts: 22708
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Re: Audio sync strategies

Post by tepples »

Logging APU writes during the frame and then running them in a batch at the end isn't even inaccurate, so long as all logged APU writes are timestamped with what CPU cycle they occurred on, and rendering reads those timestamps. It might even help with speed, as the audio stuff will fit more cleanly into the host CPU's cache. What got inaccurate in old emulators was that they in effect rounded timestamps to the nearest frame.
shawnyadav
Posts: 1
Joined: Thu Apr 26, 2018 2:26 am

Re: Audio sync strategies

Post by shawnyadav »

Dwedit wrote:Briefly peeked at the source code.

You should probably redo your graphics engine, since all pixels are being done with OpenGL draw calls to set the coordinates then set the color.

The "right" way to do it is with a framebuffer and glTexSubImage2D.
Swtiching to glTexSubImage2D actually works real good! Thanks man.
Post Reply