IRQ nesting due to OAM DMA

Discuss technical or other issues relating to programming the Nintendo Entertainment System, Famicom, or compatible systems.

Moderator: Moderators

Post Reply
User avatar
za909
Posts: 209
Joined: Fri Jan 24, 2014 9:05 am
Location: Hungary

IRQ nesting due to OAM DMA

Post by za909 » Tue May 05, 2020 5:04 am

I have discovered an issue with my PCM player project in Mesen that is very serious and can easily crash a game once implemented in one.

Namely, there is a particular alignment of DMC IRQ, NMI and OAM DMA that can cause two IRQs to be nested in eachother, clobbering the variables for the IRQ at the higher level. This leads to a time wasting loop counter in the higher IRQ being decremented when already zero, causing 255 more loops to occur. This eats up a considerable amount of time (at least 20 scanlines).

This particular alignment can be broken down like so:
1. IRQ fires close to NMI and starts the DMC to set up the next IRQ and initializes the time wasting loop counter
2. NMI fires while the IRQ time wasting loop is being executed
3. NMI instantly clears the I flag to allow the PCM stream to continue
4. OAM DMA is executed, during which the DMC plays its sample and requests the next IRQ
5. When OAM DMA ends, the new pending IRQ is handled immediately , which sets up a third IRQ, and initializes the loop counter variable for itself, clobbering the value for the first IRQ
6. When the DMC requests the third IRQ or any subsequent IRQs, they are handled like normal, until the program exits NMI
7. The CPU returns to the IRQ that was intersected by NMI and the time wasting loop is executed 255 times.
8. Another IRQ (still set up from within an IRQ inside NMI) is pending by this time, so it is handled immediately.
9. The situation resolves itself until the next similar alignment occurs.

I see a few ways I could go about solving this, but I would welcome any other additional methods to the ones below:
- Make NMI aware that it interrupted an IRQ and somehow go back and finish handling that IRQ, by taking the return address off the stack and jumping to it (sounds time consuming).
- Delay NMI by never allowing it to interrupt an IRQ. Is it possible to turn off NMIs at the start of the IRQ handler and re-enable them at end and still "catch" the NMI if the PPU has gone into VBlank during the IRQ?
- Save all variables of IRQ before clearing the I flag in NMI, except for pointers to the PCM data, which have to be continuously updated. There might be some remaining jitter from handling the IRQ that gets interrupted by NMI but it is much more tolerable than the performance loss from earlier.

Thank you!

tepples
Posts: 22020
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Re: IRQ nesting due to OAM DMA

Post by tepples » Tue May 05, 2020 7:37 am

If vblank has begun and not ended, and $2002 has not been read, turning NMI back on will immediately fire NMI. Bases Loaded II depends on this. However, turning NMI on or off at exactly the right side of the picture can cause visible artifacts in some cases due to race conditions on $2000 writes vs, the resetting of the horizontal bits of the VRAM address.

Can you make the IRQ reentrant by detecting IRQ-in-IRQ state, incrementing a counter of IRQ-in-IRQs, and then having the IRQ handler run itself as many times as there were IRQ-in-IRQs?

User avatar
Bregalad
Posts: 7892
Joined: Fri Nov 12, 2004 2:49 pm
Location: Chexbres, VD, Switzerland

Re: IRQ nesting due to OAM DMA

Post by Bregalad » Tue May 05, 2020 7:59 am

- Delay NMI by never allowing it to interrupt an IRQ. Is it possible to turn off NMIs at the start of the IRQ handler and re-enable them at end and still "catch" the NMI if the PPU has gone into VBlank during the IRQ?
Yes, you can disable NMIs by clearing $2000.7 and re-enable by setting this bit, but be aware of the trick tepples mentionned above. The main issue is if you use all of VBlank time in your NMI routine, a late-NMI will screw up; similairly if you use NMI start for raster effect synchronisation (unlikely if you use IRQs at all...) then the raster effect will be screwed up.
- Save all variables of IRQ before clearing the I flag in NMI, except for pointers to the PCM data, which have to be continuously updated. There might be some remaining jitter from handling the IRQ that gets interrupted by NMI but it is much more tolerable than the performance loss from earlier.
Why do you even clear the I flag in the NMI in the 1st place ? The 6502 wasn't designed to be used that way, hence the name, non maskable.

User avatar
tokumaru
Posts: 11772
Joined: Sat Feb 12, 2005 9:43 pm
Location: Rio de Janeiro - Brazil

Re: IRQ nesting due to OAM DMA

Post by tokumaru » Tue May 05, 2020 8:28 am

Bregalad wrote:
Tue May 05, 2020 7:59 am
Why do you even clear the I flag in the NMI in the 1st place ?
Apparently so that APU IRQs will still fire while the NMI is running, to prevent any disruptions in the sound.

User avatar
Quietust
Posts: 1569
Joined: Sun Sep 19, 2004 10:59 pm
Contact:

Re: IRQ nesting due to OAM DMA

Post by Quietust » Tue May 05, 2020 10:31 am

Bregalad wrote:
Tue May 05, 2020 7:59 am
The 6502 wasn't designed to be used that way, hence the name, non maskable.
"Non-maskable" has nothing to do with interrupts overlapping - it has to do with the fact that NMI ignores the "Interrupt Mask" flag (while IRQs respect it, allowing you to ignore them when you don't want to deal with them).
Quietust, QMT Productions
P.S. If you don't get this note, let me know and I'll write you another.

turboxray
Posts: 83
Joined: Thu Oct 31, 2019 12:56 am

Re: IRQ nesting due to OAM DMA

Post by turboxray » Tue May 05, 2020 12:23 pm

Bregalad wrote:
Tue May 05, 2020 7:59 am
Why do you even clear the I flag in the NMI in the 1st place ? The 6502 wasn't designed to be used that way, hence the name, non maskable.
Doing preemptive ISRs on the 65x is really no different than anything else; re-enable the interrupts when inside an ISR that can be preempted by a higher level ISR.

User avatar
rainwarrior
Posts: 7824
Joined: Sun Jan 22, 2012 12:03 pm
Location: Canada
Contact:

Re: IRQ nesting due to OAM DMA

Post by rainwarrior » Tue May 05, 2020 1:06 pm

One option here is to just have your NMI increment a counter and RTI, and then handle the vblank update in the main thread after waiting on the counter. That way NMI won't be re-enabling the interrupt spuriously.

Unfortunately it leaves you with no way to e.g. keep music running smoothly during slowdown, but maybe that's acceptable here.


Another option is that instead of doing CLI indiscriminately in your NMI, you could check the flags on the stack, and only CLI if it was already clear on the stack.

User avatar
za909
Posts: 209
Joined: Fri Jan 24, 2014 9:05 am
Location: Hungary

Re: IRQ nesting due to OAM DMA

Post by za909 » Tue May 05, 2020 1:19 pm

tepples wrote:
Tue May 05, 2020 7:37 am
If vblank has begun and not ended, and $2002 has not been read, turning NMI back on will immediately fire NMI. Bases Loaded II depends on this. However, turning NMI on or off at exactly the right side of the picture can cause visible artifacts in some cases due to race conditions on $2000 writes vs, the resetting of the horizontal bits of the VRAM address.

Can you make the IRQ reentrant by detecting IRQ-in-IRQ state, incrementing a counter of IRQ-in-IRQs, and then having the IRQ handler run itself as many times as there were IRQ-in-IRQs?
I don't have a fixed idea of what scrolling needs there will be for a game using this PCM audio capability. Most likely having none, and taking the Battle Kid route of fixed single-screen rooms would be the least cumbersome. But it seems like horizontal mirroring avoids the high X bit issues altogether (technically, if I'm not mistaken it doesn't, it just causes wrong VRAM reads to point to a mirror anyway). So the easiest route seems like using horizontal mirroring, disabling NMI in IRQ before the DMC restart, and re-enabling NMI before exiting IRQ. Even if an IRQ happens right before VBlank and is now serviced before getting to the NMI, the time spent in IRQ is never more than 3 scanlines, which itself is rare and will only happen if the bank pointer needs to be incremented or a new "pattern" begins. So thankfully OAM DMA will never be delayed by a dangerous amount either. I need to test what the least amount of possible remaining bandwidth is for uploading to VRAM though. If there is no scrolling, or not more than what fits in two screens at a time, I'd like to squeeze in some tile animations (say 32 bytes + a full palette should fit in per frame, and I can draw new rooms with rendering off in the dark).

As for re-entrant IRQs... I'd avoid this if possible because it might introduce a few samples that will be played to quickly one after the other. The distortion coming from OAM DMA might be bad enough as it is, even though all this time I've been listening to a much worse distortion due to that runaway loop than I was supposed to and it still didn't sound too bad!

Fiskbit
Posts: 125
Joined: Sat Nov 18, 2017 9:15 pm

Re: IRQ nesting due to OAM DMA

Post by Fiskbit » Tue May 05, 2020 5:24 pm

You can fix the $2000 write glitch by writing to $2000 if your current nametable x is 0 and $2100 if it's 1. This causes the bad scanline to come from the appropriate nametable, hiding the issue. You're correct, though, that you don't have to worry about this if you use horizontal mirroring.

User avatar
za909
Posts: 209
Joined: Fri Jan 24, 2014 9:05 am
Location: Hungary

Re: IRQ nesting due to OAM DMA

Post by za909 » Wed May 06, 2020 12:50 am

The solution seems to work well now, there are still a few IRQs happening close to eachother sometimes soon after exiting NMI, but there is no more problematic clobbering for now, and OAM DMA delaying a few samples is barely audible on its own (a quiet 60Hz buzz is heard during very quiet passages in the music).

Code: Select all

IRQ:
sta irqStackA ; faster than pha
lda ppu_control
and #$7f
sta $2000

; do other IRQ stuff... and restart DMC

lda ppu_control
sta $2000
lda irqStackA
rti

Code: Select all

NMI:
bit $4015
bpl @irqnotpending
rti ; give back control to IRQ

@irqnotpending:
bit $2002 ; make generation of another NMI impossible
cli
pha
txa
pha
tya
pha

lda #$02
sta $4014

; do other NMI stuff... 

pla
tay
pla
tax
pla
rti

Post Reply