Audio rendering?
Page 2 of 3

Author:  tepples [ Tue Jan 19, 2016 6:52 am ]
Post subject:  Re: Audio rendering?

Pulse waves in the 2A03 are made up of eight states, four high, and four low. It takes (period + 1) APU cycles, or (2 * period + 2) CPU cycles, to finish one of these states and advance to the next. This means it takes (8 * period + 8) APU cycles, or (16 * period + 16) CPU cycles, to finish one entire cycle of the square wave.

Author:  Disch [ Tue Jan 19, 2016 9:48 am ]
Post subject:  Re: Audio rendering?

I learn best by example... so let's look at an example.

Using this page for a quick reference:

If you look at the "Duty Cycle Sequences" section, you can see that when the 'D' bits of $4000 are set to %10, the channel is configured to have a 50% duty. This means it will walk through the below sequence:

0 1 1 1 1 0 0 0

With 1s signifying high output and 0s signifying low output.

Let's also say that the channel has an output volume of 7 (low 4 bits of $4000), and let's assume other tricks like envelope and sweep are disabled (don't worry about that stuff for now... the only things actually needed to generate sound are volume, duty, and period/length -- other things are just for effects and can be worked in later)

Lastly, let's say the channel's length/period/timer/whatever you want to call it (11-bit value set via $4002,4003) is set to $002 *

The channel will have two internal counters:
A) keeps track of the timer/period countdown
B) keeps track of its position in the duty cycle

Here's what will happen every APU cycle:
-  50% duty:  0 1 1 1 1 0 0 0
-  period = $002

  A    B    output
  2    0      0     <- A=period at start, duty starts at beginning.  Since duty[0] is 0, output is 0
  1    0      0     <- A decremented each cycle.  Since it was nonzero, nothing else changes, output still 0
  0    0      0
  2    1      7     <- since A was 0, it isn't decremented, and instead is reset to period and
  1    1      7         B is incremented.  Since duty[1] is 1, duty is high and the volume (7) is output
  0    1      7
  2    2      7     <- duty[2] is still high, so output is still 7
  1    2      7
  0    2      7
  2    3      7     <- duty[3] is still high
  0    4      7     <- duty[4] is still high
  2    5      0     <- duty[5], however, is low, so output is 0
  1    5      0
  1    7      0
  0    7      0     <- there are only 8 phases of duty, so it goes from
  2    0      0          7 to zero.  At this point the pulse wave has completed one full cycle
                         this process will just keep repeating over and over to generate the tone

Notice a few things:
- when period = 2, the duty is clocked every 2+1 cycles
- this will produce an alternating 0,7 output pattern, which is the generated pulse/square audio wave.
- volume controls the height of the wave. Taller = louder
- period controls the width of the wave. Higher period = Wider = lower pitch.

* a period of $002 would actually be crazy high pitched, and in fact the pulse channel's sweep unit would forcibly silence it, so this example is not really realistic and technically would result in all 0 output on the NES -- but nevermind the sweep unit for now. The above example is pretty much how it works and is conceptually sound. If I wanted to be hypertechnicaly I would have to have chosen a period >= 8, but I wanted to keep the example as short as possible, so whatever.

Author:  oRBIT2002 [ Tue Jan 19, 2016 11:59 am ]
Post subject:  Re: Audio rendering?

Learning by examples are great, I agree.

Thanks Disch for a great example! I think this stuff is really interesting..

EDIT: I made a simple C# prototype based on your example. Sounds really cool. :) Let's see how I should proceed with this stuff..

Author:  oRBIT2002 [ Wed Jan 20, 2016 1:08 pm ]
Post subject:  Re: Audio rendering?

I've started to implement this now, it logs to a pcm file at the moment and it seems to be working sort of.
Remains to understand the rest of the register-bits to get it to sound better. :)
The "envelope decay", is this some sort of volume-fading feature?

Author:  Disch [ Wed Jan 20, 2016 2:01 pm ]
Post subject:  Re: Audio rendering?

"Envelope decay" is an automatic fade-out (or automatic volume change).

It operates on the same 'period' concept... you provide a period P and after P+1 clocks, the output volume will change.

The key thing to note, though, is that the clocks do not come from the APU clock, but rather from the Frame Sequencer, which is almost like an entirely separate subunit within the APU.

APU clock -> Channel Timer -> Duty Cycle -> Channel Output

APU clock -> Frame Sequencer -> Envelope Period -> Volume change

The sweep unit operates similarly, but changes the pitch/timer, rather than the volume:

APU clock -> Frame Sequencer -> Sweep Period -> Channel Timer change

Author:  oRBIT2002 [ Thu Jan 21, 2016 1:30 pm ]
Post subject:  Re: Audio rendering?

If I understand correctly.. When bit 4 of $4000 is set, the volume is decreased at 240/(bit0-3 of $4000) Hz?

Author:  Zepper [ Thu Jan 21, 2016 1:38 pm ]
Post subject:  Re: Audio rendering?


You see. The NTSC refresh rate is 60FPS (or very near that). So, if a unit runs at 240hz, it's updated 4 times per frame. Well, you got it - 120hz means 2 times per frame, and so on. The frequency value is merely the number of CPU cycles to wait.

Author:  Disch [ Thu Jan 21, 2016 6:42 pm ]
Post subject:  Re: Audio rendering?

Hidden 4-bit counter: let's called this 'envvol'.
- envvol = $F after a write to $4003 (but not immediately after... see wiki for details)
- envvol is not directly accessible

$4000.0-3 (low 4 bits of $4000): let's call this 'V'

When $4000.4 is set: Envelope is disabled, V is used as output volume

When $4000.4 is clear: envvol is used as output volume... V is instead used as a period: envvol is decremented roughly 240/(V+1) times per second

Note that the 240 is not really 240 but is a clock signal that comes from the Frame Sequencer. See this page:

Author:  oRBIT2002 [ Fri Jan 22, 2016 12:41 am ]
Post subject:  Re: Audio rendering?

Someone should write a "NES Audio for dummies" document. :)

When reading about the framecounter on the wiki, it only specifies maximum 5 different steps but if the envelope is active there should be 15 (bit 0-3 of $4000)?

Author:  Disch [ Fri Jan 22, 2016 1:56 am ]
Post subject:  Re: Audio rendering?

The frame counter loops. So after it does those 4 (or 5) steps, it just keeps repeating them.

The thing is there's several APU cycles (~3700) between each step of the frame counter. So there are ~3700*(V+1) cycles between volume changes for the decay.

Author:  oRBIT2002 [ Sun Jan 24, 2016 12:47 pm ]
Post subject:  Re: Audio rendering?

How do you guys buffer your audio for playback? My initial thoughts was something like this:
Assume 2 buffers of XX ms.
1) Play buffer 1 & fill buffer 2
2) Play buffer 2 & fill buffer 1
Is there a smarter way of doing this? Probably? I'm using SlimDX/DirectSound if that matters...

Author:  tepples [ Sun Jan 24, 2016 1:08 pm ]
Post subject:  Re: Audio rendering?

Yes, double buffering in that manner is how it's usually done.

Author:  oRBIT2002 [ Mon Jan 25, 2016 2:58 am ]
Post subject:  Re: Audio rendering?

I'm struggling with playback of my buffers. I don't think it's a performance-issue but something else. I'm using two buffers that I'm switching between at X(X) frames interval, but the playback isn't smooth. The generated audio is OK so obviously something else is wrong.
I've tried playing the buffer at end of each NES-frame (or as I mentioned, at XX frames intervall) but nope. I've also tried setting up a Windows Timer (C#) that runs at 60fps but that doesn't work very good either.
Anyone with hints of what I might have missed?
Thanks in advance.

Author:  Disch [ Mon Jan 25, 2016 6:42 pm ]
Post subject:  Re: Audio rendering?

Audio streaming is tricky because unlike a graphic image, it is a function of time. What I mean by that is that you can generate a graphic image and put it on the screen for as long (or as short) as you want it... but with audio, your chunk of audio has an exact, fixed length of time, and it can't reach further or shorter than that. There are tricks you can do to stretch/compress audio so that it takes a shorter or longer length of time, but they're tricky.

Think of it like an hourglass. You have audio data (sand) that you pour into the top, and it will slowly drain out the bottom (play). If you keep filling the top at roughly the same rate it drains, you'll be fine. But problem come up in two instances:

1) You run out of audio/sand (buffer underrun). In which case audio playback abruptly stops and creates breaks or hiccups in your playback.

2) You have too much audio to fit in your buffer and end up having to discard some of it (buffer overrun). This will cause skips in the playback.

You almost certainly are having one of the above two problems.

Keep in mind the NES does not run at exactly 60 FPS (it's closer to 60.1). I've found that the easiest way to keep audio synced is to have the audio playback drive the framerate, and not the other way around. "Sync to sound" is what it's often referred to. Basically you keep an eye on how fast your audio is draining, and once it drains to a point where you have another frame's worth of free space, then you emulate another frame.

EDIT: note that even that gets tricky if your audio lib takes huge chunks of audio at a time instead of a gradual drain.

Author:  Rahsennor [ Mon Jan 25, 2016 6:45 pm ]
Post subject:  Re: Audio rendering?

I'm not sure about the specifics on Windows, but I think you're confused about something here. You shouldn't have anything time-related in your audio code. The sound hardware keeps its own time, and all you should care about is pumping data into it as fast as it can handle. Buffers should be completely filled and queued for playback as soon as that happens.

I need at least three buffers, preferably four, to get decent sound on Linux, but that's partially because it takes way too long to notice when a buffer is ready for more input. I also like to spin a second thread and pump the audio loop at a higher framerate than the video - it cycles at 100 FPS, but will output as much audio as it can, not the same amount each time. This has the added benefit of not stuttering when the framerate gets choppy.

Page 2 of 3 All times are UTC - 7 hours
Powered by phpBB® Forum Software © phpBB Group