Audio sync strategies
Moderator: Moderators
Audio sync strategies
Hello friends!
My emulator, Nintendoish, is progressing along nicely. I have scan line accurate PPU emulation, mapper support for all of the official US cartridge boards. Currently working on APU emulation. So far it has been an amazing leaning experience!
I have a quick question on audio sync.
Right now my emulator targets ~60 fps. I can hit that on my desktop no problem. On my laptop however, when running in dev mode, it will drop to about 40 fps.
When fps drops, my strategy to prevent slowdown is to simply drop render frames. Keep clocking the NES, but just don't render to the GPU.
This works great with video. Barely noticeable. However, it doesn't seem to work great with audio. I get this weird cascading loop. The NES is running behind and can't fill the audio buffer fast enough, so I output silence. The NES notices it's behind and so skips video frames to catch up. Audio buffer fills back up. But now the audio and video are out of sync.
I've tried just skipping audio frames when I skip a video frame but that creates very noticeable distortion in the audio. Especially when running at 40 fps. While dropping 1/3 of your video frames isn't super noticeable, dropping 1/3rd of your audio very much is.
So I was just wondering, what are some strategies with dealing with this? How do emulators running on low powered devices manage to drop their frame rate without creating audio distortion or getting their video/audio out of sync? Any suggestions?
Thank you!
PS
Source code is here: https://github.com/drewying/Nintendoish. As you will see it still is very much a work in progress.
My emulator, Nintendoish, is progressing along nicely. I have scan line accurate PPU emulation, mapper support for all of the official US cartridge boards. Currently working on APU emulation. So far it has been an amazing leaning experience!
I have a quick question on audio sync.
Right now my emulator targets ~60 fps. I can hit that on my desktop no problem. On my laptop however, when running in dev mode, it will drop to about 40 fps.
When fps drops, my strategy to prevent slowdown is to simply drop render frames. Keep clocking the NES, but just don't render to the GPU.
This works great with video. Barely noticeable. However, it doesn't seem to work great with audio. I get this weird cascading loop. The NES is running behind and can't fill the audio buffer fast enough, so I output silence. The NES notices it's behind and so skips video frames to catch up. Audio buffer fills back up. But now the audio and video are out of sync.
I've tried just skipping audio frames when I skip a video frame but that creates very noticeable distortion in the audio. Especially when running at 40 fps. While dropping 1/3 of your video frames isn't super noticeable, dropping 1/3rd of your audio very much is.
So I was just wondering, what are some strategies with dealing with this? How do emulators running on low powered devices manage to drop their frame rate without creating audio distortion or getting their video/audio out of sync? Any suggestions?
Thank you!
PS
Source code is here: https://github.com/drewying/Nintendoish. As you will see it still is very much a work in progress.
Last edited by drewying on Sat Apr 21, 2018 2:41 am, edited 1 time in total.
Re: Audio sync strategies
Briefly peeked at the source code.
You should probably redo your graphics engine, since all pixels are being done with OpenGL draw calls to set the coordinates then set the color.
The "right" way to do it is with a framebuffer and glTexSubImage2D.
You should probably redo your graphics engine, since all pixels are being done with OpenGL draw calls to set the coordinates then set the color.
The "right" way to do it is with a framebuffer and glTexSubImage2D.
Here come the fortune cookies! Here come the fortune cookies! They're wearing paper hats!
Re: Audio sync strategies
Thanks so much for the tip Dwedit!
I switched to glTexSubImage2D and that gave me a huge speed bump. Enough where it's a lot more reasonable to develop this in dev mode.
I still have work to do on the graphics engine. It's still very bare bones. Eventually I'll be moving everything to a shader where I have some plans for some graphics effects I hope to do.
I switched to glTexSubImage2D and that gave me a huge speed bump. Enough where it's a lot more reasonable to develop this in dev mode.
I still have work to do on the graphics engine. It's still very bare bones. Eventually I'll be moving everything to a shader where I have some plans for some graphics effects I hope to do.
Re: Audio sync strategies
Code: Select all
//OpenGL demands we flip
y = 240 - y;
x = 256 - x;
Download STREEMERZ for NES from fauxgame.com! — Some other stuff I've done: fo.aspekt.fi
Re: Audio sync strategies
Great suggestion. Done and it looks great! Thank you!Simply flip the texture coordinates or the vertex coordinates of the polygon you're drawing.
I am curious though to my original question. Emulators which can't run at ~60 and have to drop frames... how do they manage to keep audio from staying in sync... or at least not sounding distorted if they drop a lot of frames?
Or do you just need to make sure that your emulator is fast enough where that isn't a serious concern?
- rainwarrior
- Posts: 8732
- Joined: Sun Jan 22, 2012 12:03 pm
- Location: Canada
- Contact:
Re: Audio sync strategies
If the emulator has "frameskip" it skips rendering and tries to do every second (or more) frame faster, but generally in this case audio is never skipped.
With no frameskip, every emulator I can think of just has gaps in the audio when it can't keep up to 60fps. (Holding the last sample value across the gap makes a slightly less annoying pop than just inserting 0s.)
There are other ways to handle this, but those are the two that are common.
With no frameskip, every emulator I can think of just has gaps in the audio when it can't keep up to 60fps. (Holding the last sample value across the gap makes a slightly less annoying pop than just inserting 0s.)
There are other ways to handle this, but those are the two that are common.
Re: Audio sync strategies
Talked about in great detail: viewtopic.php?f=3&t=15405
Re: Audio sync strategies
The really old emulators ran the sound channels (square, triangle, noise, DMC) asynchronously, so at the end of the frame, it could generate sound for as long as it needed to, or generate sound for whatever speed it needed to generate it at. Usually the time to generate for would be about 1/60s.rainwarrior wrote:With no frameskip, every emulator I can think of just has gaps in the audio when it can't keep up to 60fps.
Here come the fortune cookies! Here come the fortune cookies! They're wearing paper hats!
Re: Audio sync strategies
Logging APU writes during the frame and then running them in a batch at the end isn't even inaccurate, so long as all logged APU writes are timestamped with what CPU cycle they occurred on, and rendering reads those timestamps. It might even help with speed, as the audio stuff will fit more cleanly into the host CPU's cache. What got inaccurate in old emulators was that they in effect rounded timestamps to the nearest frame.
-
- Posts: 1
- Joined: Thu Apr 26, 2018 2:26 am
Re: Audio sync strategies
Swtiching to glTexSubImage2D actually works real good! Thanks man.Dwedit wrote:Briefly peeked at the source code.
You should probably redo your graphics engine, since all pixels are being done with OpenGL draw calls to set the coordinates then set the color.
The "right" way to do it is with a framebuffer and glTexSubImage2D.