It is currently Sun Oct 22, 2017 12:18 pm

All times are UTC - 7 hours





Post new topic Reply to topic  [ 34 posts ]  Go to page 1, 2, 3  Next
Author Message
 Post subject: Audio rendering?
PostPosted: Sun Jan 17, 2016 1:23 pm 
Offline
User avatar

Joined: Sun Mar 19, 2006 3:06 am
Posts: 583
Location: Gothenburg/Sweden
I'm pretty much a newbie to "advanced" programming on Windows. I was better on the Amiga back in the days. :)
I'm curious, how are you guys generating the audio in your NES emulators on Windows? The Amiga had 4 soundchannels that could be manipulated in realtime (which made NES audio emulation, well, not that hard anyway).
But things work differently on my PC obviously. Audio has to be rendered to a buffer and then played?
I am no soundexpert at all, but doesn't that seems a bit difficult when generating NES audio? You really have to know what you're doing concerning samplerates/frequencies etc.
Or is there a shortcut?
I've checked out some blip_buf.dll but that was obviously not for mortals like myself.

_________________
http://nes.goondocks.se/


Top
 Profile  
 
 Post subject: Re: Audio rendering?
PostPosted: Sun Jan 17, 2016 4:53 pm 
Offline
User avatar

Joined: Sun Jan 22, 2012 12:03 pm
Posts: 5732
Location: Canada
Yes, you generate the audio yourself as a raw PCM stream, it gets copied to a buffer, usually something like 4096 samples at a time.

There's an inherent delay in the buffered approach, though, but this is to be expected on most modern platforms. The delay is usually small enough not to be a really noticable problem, and you can try decreasing the buffer size to make the delay smaller, at the risk of skipping if the thread ever gets starved for too long. (There is also a special interface called ASIO for very low latency audio on PCs, but it's usually reserved for musical applications.)


Top
 Profile  
 
 Post subject: Re: Audio rendering?
PostPosted: Sun Jan 17, 2016 6:49 pm 
Offline

Joined: Sun Sep 19, 2004 11:12 pm
Posts: 19115
Location: NE Indiana, USA (NTSC)
Would D-Pad Hero be a "musical application" in this sense?


Top
 Profile  
 
 Post subject: Re: Audio rendering?
PostPosted: Sun Jan 17, 2016 11:31 pm 
Offline
User avatar

Joined: Sun Mar 19, 2006 3:06 am
Posts: 583
Location: Gothenburg/Sweden
To me it seems difficult to calculate those waves in realtime (more or less), applying sweeps and such..

_________________
http://nes.goondocks.se/


Top
 Profile  
 
 Post subject: Re: Audio rendering?
PostPosted: Mon Jan 18, 2016 2:56 am 
Offline
User avatar

Joined: Sat Jan 22, 2005 8:51 am
Posts: 427
Location: Chicago, IL
oRBIT2002 wrote:
To me it seems difficult to calculate those waves in realtime (more or less), applying sweeps and such..

Read through Blargg's APU reference if you haven't already (http://nesdev.com/apu_ref.txt). The block diagrams are really helpful.

_________________
get nemulator
http://nemulator.com


Top
 Profile  
 
 Post subject: Re: Audio rendering?
PostPosted: Mon Jan 18, 2016 6:26 am 
Offline
User avatar

Joined: Sun Mar 19, 2006 3:06 am
Posts: 583
Location: Gothenburg/Sweden
It's all pretty complex to be honest. To render audio and how the waveforms should look at a certain time, well, it's a alot to think about (AND understand :)).
If there's any friendly english/swedish soul in here with skills in this area and good knowledge of C#, feel free to drop me a pm.

_________________
http://nes.goondocks.se/


Top
 Profile  
 
 Post subject: Re: Audio rendering?
PostPosted: Mon Jan 18, 2016 9:32 am 
Offline
User avatar

Joined: Wed Nov 10, 2004 6:47 pm
Posts: 1845
Why PM? Why not keep the conversation public so that others can benefit from it?

PCM data is fairly simple to understand. You can think of a waveform as a digital recording of how the cone in the speaker is going to vibrate. Several "samples" at taken...each of which are effectively the cone's position at a particular point in time.

This image may help:
https://upload.wikimedia.org/wikipedia/ ... cm.svg.png

The Y axis here would be the value of the sample (position of the cone), and the X axis would be time. The "taller" the wave, the louder it is. The "wider" the wave, the lower pitch it is.

The NES produces its waveforms by rapidly changing output values for a channel. For example, the pulse channels form a wave by rapidly alternating between '0' output and 'V' output (their volume). Effectively, the channel is clocked every cycle*, and that clock is divided by a down-counter (the "period"). When the period loops, the channel changes its output**. The period then acts are sort of a "delay" which dictates how much of a gap there is between the change in output. Smaller period = smaller delay.

So with a small period, (and with volume=8), the pulse channel might output the following:
00008888000088880000888800008888

But a longer period might output this:
00000000888888880000000088888888

If you were to plot that output out in a method similar to the above image (Y axis=output value, X axis=time), you'd see that it forms a sort of "square" shaped wave. The wave is basically how the speaker cone moves, generating your output.

The interesting thing to note here is that the 'height' of the wave is dictated by V. Higher values for V will produce a taller wave, resulting in a louder sound. And the 'width' of the wave is dictated by the period. So a longer period will create a longer delay which results in a wider wave, and therefore a lower pitch. So the period is effectively the pitch control.



You can think of the NES as outputting one sample every CPU cycle... effectively having a samplerate of 1.79 MHz. The tricky part is scaling that data down... because PCs typically only want to output 44100 samples every second. The easiest way to do this (and the way I recommend starting) is just to do a "nearest neighbor" approach:

1789772.7272 / 44100 = ~40.5

So you can downsample the audio by ignoring APU output for 39 cycles, then using the 40th cycle... then ignoring 40, using 41st... then ignoring 39, using 40th... etc.
This will more or less work, and is VERY easy to implement, but produces low quality sound (it'll sound grainy, particularly with higher tones).

There are higher quality approaches that can be done here -- but I wouldn't concern myself with that yet. Just get it outputting something, and worry about making it sound good later.



*Technically I think the squares are clocked every APU cycle, which is every OTHER CPU cycle.
**The period divider actually clocks the duty cycle generator, which is responsible for changing the output.





I'm happy to help with questions / clarifications.


Top
 Profile  
 
 Post subject: Re: Audio rendering?
PostPosted: Mon Jan 18, 2016 12:24 pm 
Offline
User avatar

Joined: Sun Mar 19, 2006 3:06 am
Posts: 583
Location: Gothenburg/Sweden
I understand the theory of waveforms, but the idea of generating them in realtime and dealing with it's effects, well that's a different story.
Ok, so you can render them byte per byte after one clockcycle, I get that. But you still have to keep track of the waveform when it should "shift shape" (sorry If I use bad terminology here, but perhaps you get what I mean anyway).
Assume you start playing a normal squarewave. How do you keep track of how it "should look"? I mean, init the square wave, and if you don't touch any of it's controls, how does the waveform look a few ms later?
I had some hopes this could be done on vertical blank basis instead of after each clockcycle (since most ROMs afaik do their music-stuff at vertical blank anyway), but perhaps that's even more difficult with a buffer..(?)

_________________
http://nes.goondocks.se/


Top
 Profile  
 
 Post subject: Re: Audio rendering?
PostPosted: Mon Jan 18, 2016 12:36 pm 
Offline

Joined: Sun Sep 19, 2004 11:12 pm
Posts: 19115
Location: NE Indiana, USA (NTSC)
Doing everything at vblank basis will make Big Bird's Hide & Speak not work, as it writes directly to a DAC at $4011 roughly 8000 times a second while Big Bird is speaking.


Top
 Profile  
 
 Post subject: Re: Audio rendering?
PostPosted: Mon Jan 18, 2016 3:24 pm 
Offline
User avatar

Joined: Wed Nov 10, 2004 6:47 pm
Posts: 1845
oRBIT2002 wrote:
But you still have to keep track of the waveform when it should "shift shape" (sorry If I use bad terminology here, but perhaps you get what I mean anyway).


That's what the period does. The logic here is actually pretty simple:

Every clock:
- if counter is nonzero, subtract 1 (and do nothing else)
- otherwise (counter is zero):
-- reset counter to period value
-- have channel take another step through its waveform generator (update duty for pulse, tri-step for tri, shift reg for noise, etc)


It's that easy. Channels produce output every CPU cycle (though, again, you want to downscale it) -- and that output is determined by the state of their waveform generator (duty, etc).

In the case of the pulse channels, the duty simply cycles through an 8 step sequence, with some steps being "high" (output current volume) or "low" (output 0)


oRBIT2002 wrote:
I had some hopes this could be done on vertical blank basis instead of after each clockcycle (since most ROMs afaik do their music-stuff at vertical blank anyway), but perhaps that's even more difficult with a buffer..(?)


You render audio samples the same way you render pixels. Just as 1 PPU cycle [roughly] generates 1 pixel.... 1 APU cycle generates 1 audio sample (though again -- scale it down)

You probably have a buffer that your PPU gradually fills up with graphic data as it progresses through the frame, right? So do the same with audio. Give your APU a buffer that it can progressively fill up with samples as it advances through the frame.


As tepples mentioned, you should not emulate the APU on a once-per-frame basis. For the same reasons you wouldn't want to emulate the PPU that way.


Top
 Profile  
 
 Post subject: Re: Audio rendering?
PostPosted: Tue Jan 19, 2016 12:20 am 
Offline
User avatar

Joined: Sun Mar 19, 2006 3:06 am
Posts: 583
Location: Gothenburg/Sweden
Disch wrote:
That's what the period does. The logic here is actually pretty simple:

Every clock:
- if counter is nonzero, subtract 1 (and do nothing else)
- otherwise (counter is zero):
-- reset counter to period value
-- have channel take another step through its waveform generator (update duty for pulse, tri-step for tri, shift reg for noise, etc)



With "period", are you refering to some special bits in the audioregisters?

_________________
http://nes.goondocks.se/


Top
 Profile  
 
 Post subject: Re: Audio rendering?
PostPosted: Tue Jan 19, 2016 12:41 am 
Offline
User avatar

Joined: Wed Nov 10, 2004 6:47 pm
Posts: 1845
I'm referring to the 11-bit F-value that is written to $4002,4003 to control the pitch.... and the similar values that exist on the other channels.

That value effectively sets the "size" of the clock divider -- which also means it controls the pitch of the generated tone. Ex, if $135 is written to $4002,4003... then Pulse 0's duty cycle generator takes one step every $135+1 APU cycles.


EDIT: Apparently wiki calls it the "timer". The 'T' bits outlined here:

http://wiki.nesdev.com/w/index.php/APU# ... 00-4007.29

I remember blargg calling it the period in his apu doc... I guess that name stuck with me. =P


Top
 Profile  
 
 Post subject: Re: Audio rendering?
PostPosted: Tue Jan 19, 2016 1:41 am 
Offline
User avatar

Joined: Sun Mar 19, 2006 3:06 am
Posts: 583
Location: Gothenburg/Sweden
So, when the frequency value (or "period" as you call it) reaches zero, has an entire waveform "passed" by then?

Thanks for taking the time to answering my newbie-questions btw. :)

_________________
http://nes.goondocks.se/


Top
 Profile  
 
 Post subject: Re: Audio rendering?
PostPosted: Tue Jan 19, 2016 2:25 am 
Offline
User avatar

Joined: Sun Jan 22, 2012 12:03 pm
Posts: 5732
Location: Canada
Frequency is how often something happens.

Period is how long something is.

If something is repeating, its frequency and period are very closely related, but the reason we call the values in the registers "period" is because as that value gets larger, the period of the waveform gets larger. They are a direct correlation. The frequency of the waveform, on the other hand, is its inverse (it gets lower as the register's value gets larger).


Top
 Profile  
 
 Post subject: Re: Audio rendering?
PostPosted: Tue Jan 19, 2016 3:25 am 
Offline
User avatar

Joined: Sun Mar 19, 2006 3:06 am
Posts: 583
Location: Gothenburg/Sweden
Perhaps I mixed up the terminology...
The 11-bit "length" describes the length of the squarewave (for example) then? When it reaches zero an entire squarewave has been "rendered"?

_________________
http://nes.goondocks.se/


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 34 posts ]  Go to page 1, 2, 3  Next

All times are UTC - 7 hours


Who is online

Users browsing this forum: Yodak and 2 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB® Forum Software © phpBB Group