It is currently Mon Oct 15, 2018 2:09 pm

All times are UTC - 7 hours





Post new topic Reply to topic  [ 10 posts ] 
Author Message
 Post subject: Audio sync strategies
PostPosted: Fri Apr 20, 2018 1:33 pm 
Offline

Joined: Fri Mar 02, 2018 12:22 am
Posts: 7
Hello friends!

My emulator, Nintendoish, is progressing along nicely. I have scan line accurate PPU emulation, mapper support for all of the official US cartridge boards. Currently working on APU emulation. So far it has been an amazing leaning experience!

I have a quick question on audio sync.

Right now my emulator targets ~60 fps. I can hit that on my desktop no problem. On my laptop however, when running in dev mode, it will drop to about 40 fps.

When fps drops, my strategy to prevent slowdown is to simply drop render frames. Keep clocking the NES, but just don't render to the GPU.

This works great with video. Barely noticeable. However, it doesn't seem to work great with audio. I get this weird cascading loop. The NES is running behind and can't fill the audio buffer fast enough, so I output silence. The NES notices it's behind and so skips video frames to catch up. Audio buffer fills back up. But now the audio and video are out of sync.

I've tried just skipping audio frames when I skip a video frame but that creates very noticeable distortion in the audio. Especially when running at 40 fps. While dropping 1/3 of your video frames isn't super noticeable, dropping 1/3rd of your audio very much is.

So I was just wondering, what are some strategies with dealing with this? How do emulators running on low powered devices manage to drop their frame rate without creating audio distortion or getting their video/audio out of sync? Any suggestions?

Thank you!

PS
Source code is here: https://github.com/drewying/Nintendoish. As you will see it still is very much a work in progress.


Last edited by drewying on Sat Apr 21, 2018 2:41 am, edited 1 time in total.

Top
 Profile  
 
PostPosted: Fri Apr 20, 2018 10:24 pm 
Offline
User avatar

Joined: Fri Nov 19, 2004 7:35 pm
Posts: 4093
Briefly peeked at the source code.

You should probably redo your graphics engine, since all pixels are being done with OpenGL draw calls to set the coordinates then set the color.

The "right" way to do it is with a framebuffer and glTexSubImage2D.

_________________
Here come the fortune cookies! Here come the fortune cookies! They're wearing paper hats!


Top
 Profile  
 
PostPosted: Sat Apr 21, 2018 2:44 am 
Offline

Joined: Fri Mar 02, 2018 12:22 am
Posts: 7
Thanks so much for the tip Dwedit!

I switched to glTexSubImage2D and that gave me a huge speed bump. Enough where it's a lot more reasonable to develop this in dev mode.

I still have work to do on the graphics engine. It's still very bare bones. Eventually I'll be moving everything to a shader where I have some plans for some graphics effects I hope to do. :)


Top
 Profile  
 
PostPosted: Sat Apr 21, 2018 3:20 am 
Offline
User avatar

Joined: Mon Jan 03, 2005 10:36 am
Posts: 3132
Location: Tampere, Finland
Code:
    //OpenGL demands we flip
    y = 240 - y;
    x = 256 - x;

Nope, it doesn't. Simply flip the texture coordinates or the vertex coordinates of the polygon you're drawing. Also there's an off-by-1 bug in this code leading to writes outside the buffer (should be 239-y and 255-x).

_________________
Download STREEMERZ for NES from fauxgame.com! — Some other stuff I've done: fo.aspekt.fi


Top
 Profile  
 
PostPosted: Sat Apr 21, 2018 12:15 pm 
Offline

Joined: Fri Mar 02, 2018 12:22 am
Posts: 7
Quote:
Simply flip the texture coordinates or the vertex coordinates of the polygon you're drawing.


Great suggestion. Done and it looks great! Thank you!

I am curious though to my original question. Emulators which can't run at ~60 and have to drop frames... how do they manage to keep audio from staying in sync... or at least not sounding distorted if they drop a lot of frames?

Or do you just need to make sure that your emulator is fast enough where that isn't a serious concern?


Top
 Profile  
 
PostPosted: Sat Apr 21, 2018 12:44 pm 
Offline
User avatar

Joined: Sun Jan 22, 2012 12:03 pm
Posts: 6874
Location: Canada
If the emulator has "frameskip" it skips rendering and tries to do every second (or more) frame faster, but generally in this case audio is never skipped.

With no frameskip, every emulator I can think of just has gaps in the audio when it can't keep up to 60fps. (Holding the last sample value across the gap makes a slightly less annoying pop than just inserting 0s.)

There are other ways to handle this, but those are the two that are common.


Top
 Profile  
 
PostPosted: Sat Apr 21, 2018 12:51 pm 
Offline
User avatar

Joined: Sun Sep 19, 2004 9:28 pm
Posts: 3629
Location: Mountain View, CA
Talked about in great detail: viewtopic.php?f=3&t=15405


Top
 Profile  
 
PostPosted: Sat Apr 21, 2018 3:17 pm 
Offline
User avatar

Joined: Fri Nov 19, 2004 7:35 pm
Posts: 4093
rainwarrior wrote:
With no frameskip, every emulator I can think of just has gaps in the audio when it can't keep up to 60fps.

The really old emulators ran the sound channels (square, triangle, noise, DMC) asynchronously, so at the end of the frame, it could generate sound for as long as it needed to, or generate sound for whatever speed it needed to generate it at. Usually the time to generate for would be about 1/60s.

_________________
Here come the fortune cookies! Here come the fortune cookies! They're wearing paper hats!


Top
 Profile  
 
PostPosted: Sat Apr 21, 2018 3:28 pm 
Offline

Joined: Sun Sep 19, 2004 11:12 pm
Posts: 20656
Location: NE Indiana, USA (NTSC)
Logging APU writes during the frame and then running them in a batch at the end isn't even inaccurate, so long as all logged APU writes are timestamped with what CPU cycle they occurred on, and rendering reads those timestamps. It might even help with speed, as the audio stuff will fit more cleanly into the host CPU's cache. What got inaccurate in old emulators was that they in effect rounded timestamps to the nearest frame.


Top
 Profile  
 
PostPosted: Thu Apr 26, 2018 2:30 am 
Offline

Joined: Thu Apr 26, 2018 2:26 am
Posts: 1
Dwedit wrote:
Briefly peeked at the source code.

You should probably redo your graphics engine, since all pixels are being done with OpenGL draw calls to set the coordinates then set the color.

The "right" way to do it is with a framebuffer and glTexSubImage2D.


Swtiching to glTexSubImage2D actually works real good! Thanks man.

_________________
Shawn Yadav
showbox.onl mobdro.onl dltutuapp.com


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 10 posts ] 

All times are UTC - 7 hours


Who is online

Users browsing this forum: No registered users and 3 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB® Forum Software © phpBB Group