Can anyone help me figure out why my sprite update code works in 8bitworkshop but not MESEN?

Are you new to 6502, NES, or even programming in general? Post any of your questions here. Remember - the only dumb question is the question that remains unasked.

Moderator: Moderators

Post Reply
User avatar
SpencerKR
Posts: 19
Joined: Tue Aug 18, 2020 12:54 pm

Can anyone help me figure out why my sprite update code works in 8bitworkshop but not MESEN?

Post by SpencerKR » Thu Aug 20, 2020 8:59 am

Hi all. I'd like to start by apologizing for the amount of code I have to paste for this problem, but I've highlighted the parts which I believe are relevant.

The basic problem is that my game runs fine in the 8bitworkship IDE, but has serious issues in MESEN; none of sprite updates seem to work and I believe I have it narrowed down to problems in writing to the OAM RAM

The situation I'm in is that I've been developing a game in 8bitworkshop IDE for about a week, and I decided to make a ROM and see if it would run in MESEN. Much to my dismay, it does not.

I believe the problem comes when my code tries to write to OAM RAM. With other projects, I am able to debug and see nice healthy data changing in OAM RAM in just the way you'd expect, but the OAM RAM debug of MESEN doesn't match the OAM RAM debug I get from 8bitworkshop

Here's a screenshot of the OAM RAM debug, on the left is the healthy data from 8bitworkshop, on the right is the data from MESEN

https://imgur.com/fSbl3Ig

This is my code for moving actors (code for animating and flipping is very similar)

Code: Select all

MoveSprites: subroutine
	;rts ;test
	;ArgWord1 - Sprite pointer
        ;ByteArg1 - X delta
        ;ByteArg2 - Y delta
        
        ldy #ActorMetaSprite
        lda (ArgWord1),y ;get a pointer to the metasprite for the actor
        sta TempWord2
        iny
        lda (ArgWord1),y
        sta TempWord2+1
        
	lda #$00
        
        ldx #MetaSpriteHeaderSize
        lda #<OAM_RAM
        ldy #ActorOAMAddress 
	clc
        adc (ArgWord1),y ;look up the memory address the sprite wants to live in
        sta TempWord ;oam ram address to store
        lda #>OAM_RAM
        sta TempWord+1
        

        
        ldy #MetaSpriteDataLength
        lda (ArgWord1),y ;add the length of the sprite to the counter
        clc
        adc TempWord ;add the address of the sprite to the counter
        sta TempByte ;tracks length of metapsrite  
 

 	

        ldy #ActorXVBlankWaits
        lda (ArgWord1),y
        bne .applyXVBlankWaits
        
        ldy #ActorPositionX ;target position for whole metasprite gets added
        lda (ArgWord1),y
        clc 
        adc ByteArg1
        sta (ArgWord1),y
        jmp .doneCheckingXWait
        
.applyXVBlankWaits        
        lda NMICounterByte
        and #%00001111  ;knock off the highest nybble so the mod is faster
	sec  		;but this does mean a max vblank wait of 15 on animations
.Xmod        
        ldy #ActorXVBlankWaits
        sbc (ArgWord1),y
	bcs .Xmod
        
        
        adc (ArgWord1),y

        bne .doneCheckingXWait
        
        ldy #ActorPositionX ;target position for whole metasprite gets added
        lda (ArgWord1),y
        clc 
        adc ByteArg1
        sta (ArgWord1),y
        jmp .doneCheckingXWait
        
        
        
.doneCheckingXWait   


        ldy #ActorYVBlankWaits
        lda (ArgWord1),y
        bne .applyYVBlankWaits
        
        ldy #ActorPositionY ;target position for whole metasprite gets added
        lda (ArgWord1),y
        clc 
        adc ByteArg2
        sta (ArgWord1),y
        jmp .doneCheckingYWait

.applyYVBlankWaits

        lda NMICounterByte
        and #%00001111  ;knock off the highest nybble so the mod is faster
	sec  		;but this does mean a max vblank wait of 15 on animations
.Ymod        
        ldy #ActorYVBlankWaits
        sbc (ArgWord1),y
	bcs .Ymod
        
        
        adc (ArgWord1),y

        bne .doneCheckingYWait
        

        ldy #ActorPositionY
        lda (ArgWord1),y
        clc 
        adc ByteArg2
	sta (ArgWord1),y
.doneCheckingYWait   
        
.nextSprite:
	txa
        tay
        lda (TempWord2),y ;at this point, A has desired X position of sprite tile
        
        ldy #ActorPositionX ;target position for whole metasprite gets added
        clc
        adc (ArgWord1),y ;add the actor position to the sprite tile offset
        ldy #OAMSpriteX        
        sta (TempWord),y ;store in the OAM location to set tile's X position
       
        inx
        txa 
        tay
        lda (TempWord2),y 
        ldy #ActorPositionY
        clc
        adc (ArgWord1),y       
        ldy #OAMSpriteY        
        sta (TempWord),y
        
        inx 
	inx
        
        clc
        lda TempWord
        adc #SpriteLength
        sta TempWord
        inx
        cmp TempByte    
        bne .nextSprite  
        

.done
        rts
This is the full NMI code

Code: Select all


NMIHandler:

        jsr HandleInput
        lda WordReturn ;put return value from HandleInput into argument bytes
        sta ByteArg1
        lda WordReturn+1
        sta ByteArg2
        
        lda #<Cursor
        sta ArgWord1
        lda #>Cursor
        sta ArgWord1+1
        
        jsr MoveSprites
        
        lda #<Player
        sta ArgWord1
        lda #>Player
        sta ArgWord1+1
        
        jsr GetVectorToWalkTarget
        lda WordReturn
        sta ByteArg1
        lda WordReturn+1
        sta ByteArg2
                
       	jsr MoveSprites
        
        lda #<Player
        sta ArgWord1
        lda #>Player
        sta ArgWord1+1
        
        lda PlayerAnimation
        sta ArgWord2
        lda PlayerAnimation+1
        sta ArgWord2+1
        
        jsr AnimateSprites
        
        lda #<Player
        sta ArgWord1
        lda #>Player
        sta ArgWord1+1
        
        lda #<PlayerMetaSpritePointers
        sta ArgWord2
        lda #>PlayerMetaSpritePointers
        sta ArgWord2+1
        
        jsr SetMetaSpriteDirection
        
        lda #<Cursor
        sta ArgWord1
        lda #>Cursor
        sta ArgWord1+1
        
        lda #<CursorIdle
        sta ArgWord2
        lda #>CursorIdle
        sta ArgWord2+1
        
        jsr AnimateSprites       

      	lda ScrollState
        bne .skipScroll
        
      	inc ScrollPos	; increment low byte
        bne .noinc	; Z flag set if wrapped to 0
        inc ScrollPos+1	; increment high byte
.noinc
; store X and Y scroll position
        lda ScrollPos	; A -> low byte
        sta PPU_SCROLL	; set horiz scroll
        lda #0		; A -> zero
        sta PPU_SCROLL	; set vert scroll
; store 8th bit into name table selector
; name table A or B ($2000 or $2400)
	lda ScrollPos+1	; load high byte
        and #1		; select its low bit
	ora #CTRL_NMI	; set rest of bits
        sta PPU_CTRL
        
.skipScroll        
      	lda #$02
        sta OAM_DMA
        jsr ReadJoypad0
        inc NMICounterByte
	rti	  
This is how my actor data is structured

Code: Select all

Player: ;Label for the actor definition
	.byte #$18 ;no longer used
        .byte #$04 ;index of OAM where actor's sprites will live
        .byte #$00 ;current X position
        .byte #$30 ;current Y position        
        .byte #$70 ;current animation frame
        .word PlayerRightMetaSpriteData ;pointer to current metasprite
	.byte #DirectionRight
        .byte $00 ;X vblank waits (movement speed below 1)
        .byte $03 ;y vblank waits (movement speed below 1)
        
PlayerMetaSpritePointers:
	.word PlayerRightMetaSpriteData
        .word PlayerLeftMetaSpriteData

PlayerRightMetaSpriteData:  
	.byte #$18
	.byte -10,-24,$ba,2 ;data format:
	.byte - 2,-24,$bb,2 ;X, Y, Tile, Attribute
	.byte -10,-16,$bc,2
	.byte - 2,-16,$bd,2
	.byte -10, -8,$be,2
	.byte - 2, -8,$bf,2        
        
PlayerLeftMetaSpriteData: 
	.byte #$18
	.byte   2,-24,$ba,2|OAM_FLIP_H
	.byte - 6,-24,$bb,2|OAM_FLIP_H
	.byte   2,-16,$bc,2|OAM_FLIP_H
	.byte - 6,-16,$bd,2|OAM_FLIP_H
	.byte   2, -8,$be,2|OAM_FLIP_H
	.byte - 6, -8,$bf,2|OAM_FLIP_H

User avatar
Quietust
Posts: 1597
Joined: Sun Sep 19, 2004 10:59 pm
Contact:

Re: Can anyone help me figure out why my sprite update code works in 8bitworkshop but not MESEN?

Post by Quietust » Thu Aug 20, 2020 9:13 am

The most likely problem is that your NMI code is performing game logic first and Sprite DMA (and scrolling updates) last - Sprite DMA can only be done during the 20-scanline VBLANK window, so you need to do it first. The same also applies to VRAM/Palette and Scrolling updates. All of your game logic can be done during the other 240-ish scanlines.

It also wouldn't hurt to preserve registers during your interrupt handlers (PHA+TXA+PHA+TYA+PHA at the very beginning, and PLA+TAY+PLA+TAX+PLA just before the RTI at the end), especially if you decide to have game logic running outside of your NMI handler.
Last edited by Quietust on Thu Aug 20, 2020 9:15 am, edited 1 time in total.
Quietust, QMT Productions
P.S. If you don't get this note, let me know and I'll write you another.

User avatar
tokumaru
Posts: 11858
Joined: Sat Feb 12, 2005 9:43 pm
Location: Rio de Janeiro - Brazil

Re: Can anyone help me figure out why my sprite update code works in 8bitworkshop but not MESEN?

Post by tokumaru » Thu Aug 20, 2020 9:14 am

Without looking much into it, there's a HUGE flaw with your NMI handler: you seem to be doing a whole lot of game state processing (reading controllers, animating sprites, etc.) before finally doing the OAM DMA.

I'm aware that some tutorials have their first programs using this structure, which I strongly oppose to, precisely because it scales poorly and causes problems like the one you're probably facing right now.

You see, the vblank period is really short (around 20 scanlines out of the 262 of a complete frame), so you're not supposed to waste any of it doing things that could be done at other times. The NMI fires at the beginning of vblank, and you're wasting that precious time doing things that don't have to be done during vblank (updating state), so by the time all that's done, the vblank time is probably over, and the OAM DMA, which MUST happen during vblank, doesn't.

The first tasks in your NMI handler absolutely must be the PPU updates. Anything else you need to do (game logic, music, etc.) must come after that, or go in a separate thread (i.e. the main loop).

I can't say for sure that this is the cause of your problem right now, since I barely looked at your code, but even if it isn't, this is going to be a problem soon as your logic grows more complex, so it needs fixing anyway.

User avatar
SpencerKR
Posts: 19
Joined: Tue Aug 18, 2020 12:54 pm

Re: Can anyone help me figure out why my sprite update code works in 8bitworkshop but not MESEN?

Post by SpencerKR » Thu Aug 20, 2020 9:25 am

Thanks both. Moving the DMA load to the head of the NMI produces a slightly better result in Mesen. All of the sprites get initialized now, but they still don't update which is maddening

User avatar
tokumaru
Posts: 11858
Joined: Sat Feb 12, 2005 9:43 pm
Location: Rio de Janeiro - Brazil

Re: Can anyone help me figure out why my sprite update code works in 8bitworkshop but not MESEN?

Post by tokumaru » Thu Aug 20, 2020 9:38 am

It looks to me like you're trying to initialize RAM using .byte/.word statements, which doesn't work. If "Player" is in RAM, those .byte and .word statements only reserve the space, but don't initialize the memory, since RAM is volatile memory and doesn't come pre-initialized.

What you have to do is store those initial attributes somewhere in ROM, and manually copy them to the player's RAM during initialization.

What you did *does* work in home computers that run code from RAM though, because in that case programs are copied from permanent storage to RAM automatically, so the initial values are automatically transferred. On the NES though, where no automatic copy from ROM to RAM is made, and programs typically run entirely from ROM, you have to manually initialize all your RAM variables.

User avatar
SpencerKR
Posts: 19
Joined: Tue Aug 18, 2020 12:54 pm

Re: Can anyone help me figure out why my sprite update code works in 8bitworkshop but not MESEN?

Post by SpencerKR » Thu Aug 20, 2020 9:49 am

Ah, so the place where I pasted the actor definition from is in ROM. I'm pretty sure that part is working since all those initial values get set correctly during my InitSprites routine. I can see my two metasprites all loaded up with correct offsets, tiles, and attributes, the trouble is that with this code they won't change

This is what I see in MESEN, what appears to be a properly initialized screen but which never updates: https://imgur.com/hUWRBUF

Here is my initialization code, which has every indication of working:

Code: Select all

InitSprites: subroutine
	;lda ;ArgWord1 ;sprite pointer set before calling
        ;lda ;ArgWord1+1
        
	ldx #MetaSpriteHeaderSize
        lda #<OAM_RAM
        ldy #ActorOAMAddress 
	clc
        adc (ArgWord1),y ;look up the memory address the sprite wants to live in
        sta TempWord ;oam ram address to store
        lda #>OAM_RAM
        sta TempWord+1
        
        ldy #ActorMetaSprite
        lda (ArgWord1),y
        sta TempWord2
        iny
        lda (ArgWord1),y
        sta TempWord2+1
        
        ldy #MetaSpriteDataLength
        lda (TempWord2),y ;add the length of the sprite to the counter
        clc
        adc TempWord ;add the address of the sprite to the counter
        sta TempByte ;tracks length of metapsrite
        
.nextSprite:   
	txa
        tay
        lda (TempWord2),y ;at this point, A has desired X position of sprite tile
        clc
        ldy #ActorPositionX
        adc (ArgWord1),y
        ldy #OAMSpriteX        
        sta (TempWord),y
       
        inx
        txa 
        tay
        lda (TempWord2),y 
        clc
        ldy #ActorPositionY
        adc (ArgWord1),y
        ldy #OAMSpriteY        
        sta (TempWord),y
        
        inx 
        txa 
        tay
        lda (TempWord2),y
        ldy #OAMSpriteTile
        sta (TempWord),y
        
        inx
        txa
        tay
        lda (TempWord2),y
        ldy #OAMSpriteAttribute
        sta (TempWord),y
        clc
        lda TempWord
        adc #SpriteLength
        sta TempWord
        inx
        cmp TempByte; check if all sprites have been loaded into memory at the address
        bne .nextSprite  
        rts

User avatar
SpencerKR
Posts: 19
Joined: Tue Aug 18, 2020 12:54 pm

Re: Can anyone help me figure out why my sprite update code works in 8bitworkshop but not MESEN?

Post by SpencerKR » Thu Aug 20, 2020 9:53 am

Ohhhhhhhh crap. Maybe I'm trying to write to ROM addresses in my code? Oh heck. Lemme try some stuff.

User avatar
tokumaru
Posts: 11858
Joined: Sat Feb 12, 2005 9:43 pm
Location: Rio de Janeiro - Brazil

Re: Can anyone help me figure out why my sprite update code works in 8bitworkshop but not MESEN?

Post by tokumaru » Thu Aug 20, 2020 9:59 am

SpencerKR wrote:
Thu Aug 20, 2020 9:49 am
Ah, so the place where I pasted the actor definition from is in ROM. I'm pretty sure that part is working since all those initial values get set correctly during my InitSprites routine. I can see my two metasprites all loaded up with correct offsets, tiles, and attributes, the trouble is that with this code they won't change
So there's your problem: you're passing the address of "Player" to a subroutine in "ArgWord1", and this routine attempts to modify that memory (I see multiple instances of sta (ArgWord1),y), but since this is ROM, these values will never change.

During initialization, you need to copy the data from "Player" to a structure with the same format in RAM, and pass that to the subroutine.

User avatar
SpencerKR
Posts: 19
Joined: Tue Aug 18, 2020 12:54 pm

Re: Can anyone help me figure out why my sprite update code works in 8bitworkshop but not MESEN?

Post by SpencerKR » Thu Aug 20, 2020 11:02 am

I guess that's what makes it "read only" huh?

I feel a bit silly but I'm ELATED that it now works in Mesen!

For any future noobs who somehow run into the same problem, here's a summary of the changes I made

first, I laid out this structure in RAM (which I can probably make more efficient later)

Code: Select all

Cursor: 
	.byte ;#$04 ;number of bytes for data
        .byte ;#$00 ;beginning of address in OAM RAM
        .byte ;#$00 ;current animation frame
        .byte ;CursorXPosition ;current X position
        .byte ;CursorYPosition ;current Y position
        .word ;Cursor_MetaSpriteData
	.byte ;#DirectionRight
        .byte ;$00 ;X vblank waits (movement speed below 1)
        .byte ;$00 ;y vblank waits (movement speed below 1)
        
Player: ;Label for the actor definition
	.byte; #$18 ;no longer used
        .byte; #$04 ;index of OAM where actor's sprites will live
        .byte; PlayerXPosition ;current X position
        .byte; PlayerYPosition ;current Y position        
        .byte; #$70 ;current animation frame
        .word; PlayerRightMetaSpriteData ;pointer to current metasprite
	.byte; #DirectionRight
        .byte; $00 ;X vblank waits (movement speed below 1)
        .byte; $03 ;y vblank waits (movement speed below 1)

Then I ran this code in my initialization

Code: Select all

InitSprites: subroutine
	;lda ;ArgWord1 ;sprite pointer set before calling
        ;lda ;ArgWord1+1
        ;argword2 - pointer to actor's data in rom
        
        
        ldy #$00       
.loadActor
        
	lda (ArgWord2),y
        sta (ArgWord1),y
        iny
 	cpy #ActorLength
        bne .loadActor

Everything else is the same.

It's a little bit weird that it wasn't an issue in 8bitworkshop. I'll probably continue to use it since its so nice to have code changes instantly show in the emulator but I'll have to be on the lookout for that in the future

Thanks for the help everyone :)

User avatar
tokumaru
Posts: 11858
Joined: Sat Feb 12, 2005 9:43 pm
Location: Rio de Janeiro - Brazil

Re: Can anyone help me figure out why my sprite update code works in 8bitworkshop but not MESEN?

Post by tokumaru » Thu Aug 20, 2020 12:16 pm

SpencerKR wrote:
Thu Aug 20, 2020 11:02 am

Code: Select all

	.byte ;#$04 ;number of bytes for data
I don't know where this is being assembled, but most assemblers have separate directives for RESERVING memory and for DEFINING data (e.g. ASM6 has .dsb for reserving bytes and .db/.byte for defining bytes). Try to use the correct directives where appropriate in order to make your code clearer.
It's a little bit weird that it wasn't an issue in 8bitworkshop.
The fact that it allows VRAM operations outside of vblank and ROM to be written is indeed very worrying... it must be using a very inaccurate emulator.

User avatar
SpencerKR
Posts: 19
Joined: Tue Aug 18, 2020 12:54 pm

Re: Can anyone help me figure out why my sprite update code works in 8bitworkshop but not MESEN?

Post by SpencerKR » Thu Aug 20, 2020 2:03 pm

Just got word that 8bit workshop uses JSNES, which makes sense since the whole IDE exists as a webpage.

However, they did also let me know that 8bitworkshop can use the MAME NES emulator as well, so I'm not gonna give up on it just yet.

If you've never checked out the 8bitworkshop ide, here's a link: https://8bitworkshop.com

It can compile to like a jillion game consoles and early computers and even design hardware in verilog

Post Reply