A/V synchronization

Discuss emulation of the Nintendo Entertainment System and Famicom.

Moderator: Moderators

tepples
Posts: 22708
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Re: A/V synchronization

Post by tepples »

rainwarrior wrote:Have you really encountered this with modern monitors or sound cards? (e.g. I've never found a computer sound device to be measurably out of tune, at least not in any of the many times I've tuned a musical device to one.)
The integrated sound in the desktop PC on my desk at my previous job (2007 through 2013) was noticeably flat. I seem to remember it being about half a semitone.
User avatar
rainwarrior
Posts: 8732
Joined: Sun Jan 22, 2012 12:03 pm
Location: Canada
Contact:

Re: A/V synchronization

Post by rainwarrior »

tepples wrote:
rainwarrior wrote:Have you really encountered this with modern monitors or sound cards? (e.g. I've never found a computer sound device to be measurably out of tune, at least not in any of the many times I've tuned a musical device to one.)
The integrated sound in the desktop PC on my desk at my previous job (2007 through 2013) was noticeably flat. I seem to remember it being about half a semitone.
Oh, yeah now that you've reminded me I've seen one or two sound cards that seem to have the wrong samplerate (always thought it was probably a bad driver issue), but never one one I've owned, so I've never investigated how to solve that problem. (It's not something I could live with.)

Does it drift, though? I'd expect it to be pretty consistent.

I've seen really cheap capture devices with a severely drifting pitch, but that's kind of a different category of device...
User avatar
James
Posts: 431
Joined: Sat Jan 22, 2005 8:51 am
Location: Chicago, IL
Contact:

Re: A/V synchronization

Post by James »

rainwarrior wrote:Here, I made a recording. Does this sound significantly different than it does in your own tests?
It does. I hear the oscillation on your sample as well. Is this from a Windows box or something else?
get nemulator
http://nemulator.com
User avatar
rainwarrior
Posts: 8732
Joined: Sun Jan 22, 2012 12:03 pm
Location: Canada
Contact:

Re: A/V synchronization

Post by rainwarrior »

Windows 7
VIA HD Audio driver, 48kHz native samplerate
Intel HD 4600 / Nvidia GTX 765M
User avatar
James
Posts: 431
Joined: Sat Jan 22, 2005 8:51 am
Location: Chicago, IL
Contact:

Re: A/V synchronization

Post by James »

I don't have much experience with SDL audio, so perhaps it has something to do with how different audio devices deal with the callback mechanism. I've tested the DirectSound version on a wide range of hardware, and haven't run into any issues. If you get a chance, and don't mind taking a few minutes, can you try nemulator and see how it performs? You can see similar displays there by pressing F4 (line graphs) and F9 (info).
get nemulator
http://nemulator.com
User avatar
rainwarrior
Posts: 8732
Joined: Sun Jan 22, 2012 12:03 pm
Location: Canada
Contact:

Re: A/V synchronization

Post by rainwarrior »

24bit, 48kHz stereo

The device seems incapable of really doing 44.1kHz, i dunno if it resamples at the driver level or what but I can hear the resampling if I don't use 48kHz with it.

I googled nemulator which took me here: http://nemulator.com/
The downloads page link took me here: http://nemulator.com/downloads.php

That page gave me a 404.
404 Not Found

Code: NoSuchKey
Message: The specified key does not exist.
Key: downloads.php
RequestId: 452040F5F3A0D845
HostId: QwPIYVJQypDiHYD7/9tLz40O7gOlUqN/I9FQi2Iez7dLVLedVG65lErMwBd7g9RWJOA6e2kYDHw=p
Edit: never mind, I found the other downloads link in the footer bar.

Trying to figure out the default keyboard mappings...

Your ini file lists them only as hex numbers, and then cites this as reference: http://msdn.microsoft.com/en-us/library ... S.85).aspx
(that reference has apparently been removed by microsoft, so I can't reverse look them up either)

AHA! finally found SELECT and START over on [ and ]
now i can start a game
User avatar
rainwarrior
Posts: 8732
Joined: Sun Jan 22, 2012 12:03 pm
Location: Canada
Contact:

Re: A/V synchronization

Post by rainwarrior »

So, trying nemulator, the effective rate seems to be a lot more stable than in your test program. It's not always stable, but mostly fairly stable (seems to settle for long periods of time), when it moves I do hear the difference, at least on simple tests of sound, but not as pronounced a difference as in the test program. Is there filtering to reduce the aliasing? That probably helps a lot, here.

When it shifts significantly, the pitch drift does bother me, but I don't think it'd be a dealbreaker when playing a game. Truthfully, I'd much prefer stable pitch if it were an option, though. Maybe it's because I'm musically oriented, I dunno. I do think it's an interesting approach to the sync problem.

What we're trading off against here, 60.1Hz dropping 1 frame every few seconds bothers me, but it's like, 1 dropped frame in 1000 is a bother, 1 dropped frame in 100,000 isn't at all, you know? That's why I think my most preferred approach would be a stable rate that closely matches my monitor's 60Hz refresh.

Disch's approach that fakes a little extra audio to take up the slack seems acceptable too-- it's similar to what a non-emulating game would typically do, i.e. just run audio generation in its own thread and respond to the CPU as signals come in. Can't fully do that with emulation where the music has to be played by the CPU, but the principle of letting the audio continue on its own is similar.


As an aside, nemulator's keyboard controls seem to be broken for me, like they seem to be doing typical text entry press once, pause, auto-repeat behaviour. In metroid I can't jump to my full height because of this. The A button cuts off instead of letting me hold it down, then rapidly repeats.
User avatar
James
Posts: 431
Joined: Sat Jan 22, 2005 8:51 am
Location: Chicago, IL
Contact:

Re: A/V synchronization

Post by James »

Thanks for testing it out. I've uploaded a short video of how nemulator behaves on my system (and this is typical of how it behaves on other systems I've tested on): https://youtu.be/3pmTXkYb1_s. I just don't see (or hear) the issues you're describing, so I guess I'm at a loss as to why you're experiencing problems.

BTW, re: keyboard controls -- sounds like you enabled turbo (mapped to A and S keys, for the B and A buttons, respectively).
get nemulator
http://nemulator.com
User avatar
koitsu
Posts: 4201
Joined: Sun Sep 19, 2004 9:28 pm
Location: A world gone mad

Re: A/V synchronization

Post by koitsu »

The situation is particularly awkward due to things like this (other reference), where some LCD monitors EDID report weird shit like 59.94Hz and Windows 7, in its infinite wisdom, chooses to *round down* to 59.00Hz which affects internal timers. Anything that's programmed to "key off of" a common 60Hz value will be affected.

Microsoft documented this in KB 2006076, which they have since removed from their site circa 2016 sometime (reasons unknown), but it basically said "this is normal". Windows 7 SP1 x64 has pretty much pissed in my Cheerios for some time now regarding this; shit that worked fine on XP doesn't in 7 (NestopiaUE is a good example -- the first thing I noticed was "cyclic stuttering" at set intervals that wasn't present on XP). I deal with this in other things too, including MAME. My Dell 2407WFP "suffers" from this phenomenon (and it'll be a cold day in hell before I give it up; PVA panels are great, and aren't made any more, unlike IPS and the several horrible traits it comes with (IPS glow ranks at the top of my list)).
Rahsennor
Posts: 479
Joined: Thu Aug 20, 2015 3:09 am

Re: A/V synchronization

Post by Rahsennor »

More framerate fun: my ASUS laptop's panel is locked to some weird-ass not-quite-NTSC rate regardless of what modeline I set, but pretends to be doing what it's told. I can make the OS think it's running at whatever rate I like, while timing actual draw calls always says ~59.9 hz.
User avatar
rainwarrior
Posts: 8732
Joined: Sun Jan 22, 2012 12:03 pm
Location: Canada
Contact:

Re: A/V synchronization

Post by rainwarrior »

Hrm, looking it up, at least in Windows a true refresh rate is not exposed, and unless you can go through the driver somehow, the only only Windows API for refresh rate query returns only an integer, so... asking Windows seems to be unhelpful. I don't quite understand what's new about Windows 7 from the stuff you linked, this API has been there for ages, but maybe the display settings utility now uses its own API and displays only the rounded-down integer approximation? (I don't remember what the settings looked like in XP).

Anyhow, what I'd take away from that is that if you're going to do anything based on a fixed refresh rate, do this:

1. Measure it directly.
2. Provide the user with a manual override, and make sure they can use a decimal point.

I believe, though, that monitor refresh rates do tend to be quite stable. Once you've chosen it accurately you should be able to get reliable behaviour from it.

Tepples asked how long you'd have to measure it to get reliable results. The HTML5 test I linked earlier seems to reach 4 digit accuracy in under 5 seconds, and settling on a 5th digit takes another 5 or 10 seconds. Because this kind of test tends to be subject to sudden spikes and interruptions, I believe it's important to filter outliers from your average when measuring; once removed the remaining points of data should be pretty consistent?
James wrote:Thanks for testing it out. I've uploaded a short video of how nemulator behaves on my system (and this is typical of how it behaves on other systems I've tested on): https://youtu.be/3pmTXkYb1_s. I just don't see (or hear) the issues you're describing, so I guess I'm at a loss as to why you're experiencing problems.

BTW, re: keyboard controls -- sounds like you enabled turbo (mapped to A and S keys, for the B and A buttons, respectively).
Ah, okay I must have pressed A or S by accident when I was looking for select/start. Seems better with that turned off.

As for the problems, it's stable most of the time but if there's a sudden spike it shifts a lot at once and I can hear the pitch drift.

I've attached a ROM that might help demonstrate the problem. Start the test tone in the ROM, and then play an A440 Hz reference pitch from some other device. The NES tone isn't exactly 440 Hz (there are accuracy constraints) but it's a constant pitch, and it will have a consistent beating according to how many Hz it's out of tune with your reference pitch.

Once doing this test, cause a spike in execution to happen. I guess it sometimes happens due to background processes, but it seems like I can reliably make it happen by switching to another windows for a moment, and then switching back. After the momentary lag listen to the rate of beating change as it suddenly drifts to compensate. I can easily measure differences of several Hz this way.

Like, probably wouldn't be very noticeable in to most people, especially when playing a game, but for me personally it's really objectionable to have the pitch drifting like this. Some people might like this way of syncing though, like it does at least produce "seamless" audio to match its target framerate, and that much is good. (I probably have a strong bias because I have an interest in musical instrument tuning.)

If you had a way to filter out lag spikes, it might make it possible to eliminate significant drift, too? Dunno how much you can do this without losing your 1:1 audio:video goal.
Attachments
a440.nes
Test ROM that plays a steady tone (not quite 440 Hz)
(40.02 KiB) Downloaded 195 times
User avatar
James
Posts: 431
Joined: Sat Jan 22, 2005 8:51 am
Location: Chicago, IL
Contact:

Re: A/V synchronization

Post by James »

The test rom is useful -- thanks for passing that along.
rainwarrior wrote:Once doing this test, cause a spike in execution to happen. I guess it sometimes happens due to background processes, but it seems like I can reliably make it happen by switching to another windows for a moment, and then switching back. After the momentary lag listen to the rate of beating change as it suddenly drifts to compensate. I can easily measure differences of several Hz this way.
Switching windows essentially causes the calibration cycle to start all over, so a drift in frequency there is expected. I think I'll add some logic to record the stable frequency and reset to that instead of 48kHz.

You mention beating again. I don't hear that on my system, but I do hear it in the flac file you posted. I wonder why this is...
rainwarrior wrote:If you had a way to filter out lag spikes, it might make it possible to eliminate significant drift, too?
I've added an additional filter that should smooth out any random spikes. It won't help with the 'move the window' issue, though. You can grab it at here if you care to try it.

Thanks for all of the feedback.
get nemulator
http://nemulator.com
User avatar
rainwarrior
Posts: 8732
Joined: Sun Jan 22, 2012 12:03 pm
Location: Canada
Contact:

Re: A/V synchronization

Post by rainwarrior »

By beating I meant in the sense of tuning. It's from this trigonometric identity:

sin(x) + sin(y) = 2 * sin(​​(x+y)/2) * cos((x-y)/2)

This means if you play two sine tones that are close in pitch, what you will here is one sine tone at their average pitch that "beats", i.e. fades in and out at their difference.

For example if you play 440 Hz and 442 Hz together, it is the same sound as 441 Hz beating at 2 hz. You can use this to accurately measure the difference between two tones by listening and timing the beats.


So... when I was referring to beating in this last post, I was not referring to a sound heard from nemulator, but the combined sound of a reference tone I was playing externally with the tone coming from the test ROM in nemulator.
User avatar
James
Posts: 431
Joined: Sat Jan 22, 2005 8:51 am
Location: Chicago, IL
Contact:

Re: A/V synchronization

Post by James »

rainwarrior wrote:the combined sound of a reference tone I was playing externally with the tone coming from the test ROM in nemulator.
Well that certainly makes sense.

I still hear something in the flac file though between about :29 and :36.
get nemulator
http://nemulator.com
User avatar
rainwarrior
Posts: 8732
Joined: Sun Jan 22, 2012 12:03 pm
Location: Canada
Contact:

Re: A/V synchronization

Post by rainwarrior »

The flac file does have a beating sound in it, but it's caused by something else. I was not comparing a reference pitch to anything else, the FLAC file was simply the direct output from your program to my sound device.

In that case I think it's basically just aliasing caused by your not-bandlimited input being highly sensitive to minute differences in your resampling rate. Some harmonic relationship of the samplerate vs. some harmonic (or aliased harmonic) of the tone generated where they are close to in tune with each other.
Post Reply