Streemerz NES port

A place where you can keep others updated about your NES-related projects through screenshots, videos or information in general.

Moderator: Moderators

User avatar
Dwedit
Posts: 4236
Joined: Fri Nov 19, 2004 7:35 pm
Contact:

Post by Dwedit » Tue Mar 27, 2012 9:28 am

I tried HP Swfscan, and couldn't get any relevant scripts out of the game. Just useless crap like the preloader. I don't think there are any fully working AS3 decompilers yet.

Even SWFMILL can't get a correct disassembly.

I do know there are multiple DoAbcDefine tags, and that throws off some decomplers.

I think it's probably time to break out the video recorder (camstudio), record some gameplay, and play it back frame-by-frame to get a good idea of what's going on.
Here come the fortune cookies! Here come the fortune cookies! They're wearing paper hats!

User avatar
tokumaru
Posts: 11466
Joined: Sat Feb 12, 2005 9:43 pm
Location: Rio de Janeiro - Brazil

Post by tokumaru » Tue Mar 27, 2012 10:09 am

I've had some luck with this in the past: https://sites.google.com/site/as3extractor/

Not that a pirate portable version of Sothink SWF Decompiler is terribly hard to find. I don't really mean to encourage piracy, but since this is a one time thing that will not result in any financial gain for you, I wouldn't worry so much.

User avatar
thefox
Posts: 3141
Joined: Mon Jan 03, 2005 10:36 am
Location: Tampere, Finland
Contact:

Post by thefox » Tue Mar 27, 2012 10:37 am

tokumaru wrote:I've had some luck with this in the past: https://sites.google.com/site/as3extractor/
Thanks, this seems to work just fine! The extraction feature seems a little bit buggy, if I select stuff using space bar instead of clicking on it, it tells me I haven't selected anything. :) If I try to extract everything, it chokes on the JSONDecoder class. And after I deselect that class... it chokes on something else. Oh well, I don't need to extract them since I can view them using that program.

EDIT: Spoke too soon, the decompiled code isn't perfect. There's some stuff that's obviously wrong. But then again, so was in the code produced by Sothink SWF Decompiler. Just have to be careful, it should still be useful.

User avatar
tokumaru
Posts: 11466
Joined: Sat Feb 12, 2005 9:43 pm
Location: Rio de Janeiro - Brazil

Post by tokumaru » Tue Mar 27, 2012 1:23 pm

Here's another one you can try: http://code.google.com/p/asdec/

I have these programs hanging around my HD, I don't really remember how good they are. I guess that having options is always good, since one program might do what the other doesn't, so they complement each other.

User avatar
thefox
Posts: 3141
Joined: Mon Jan 03, 2005 10:36 am
Location: Tampere, Finland
Contact:

Post by thefox » Thu Mar 29, 2012 8:10 am

I added code to export the object data now. It's only exporting the object type and coordinates right now but it's not too hard to add the additional parameters which some of the object types need.

I also generated the level lists for different game modes and the level information itself (it contains references to each screen and object list that the level uses, as well as global properties like music used in the level).

Level data is almost complete now. Time to start working on the level loader.
tokumaru wrote:Here's another one you can try: http://code.google.com/p/asdec/

I have these programs hanging around my HD, I don't really remember how good they are. I guess that having options is always good, since one program might do what the other doesn't, so they complement each other.
Yep, this one gave pretty good results as well but some clear errors here as well. But yeah I'm going to cross verify the important stuff with the different programs if something seems shady.

User avatar
thefox
Posts: 3141
Joined: Mon Jan 03, 2005 10:36 am
Location: Tampere, Finland
Contact:

Post by thefox » Sun Apr 01, 2012 2:39 am

thefox wrote:Level data is almost complete now. Time to start working on the level loader.
Level loader done. Screen switcher also works (switches the screen when the player moves up/down). After I implement the doors the level lists will be fully traversable from beginning to end.

There was a minor bump in the road when I noticed that Nintendulator doesn't support CHR-RAM with FME-7 (which is the mapper I'm currently using for testing), but it was pretty easy to modify the Nintendulator mapper sources to fix it.

Next on the TODO list is object support. I think I'll start with The Door. Then I also need to start organizing the sprite tile data to see how much space it'll take, what tiles need to be available at any given time and what palettes they need.

User avatar
thefox
Posts: 3141
Joined: Mon Jan 03, 2005 10:36 am
Location: Tampere, Finland
Contact:

Post by thefox » Sun Apr 01, 2012 9:32 am

Doors work now, floated all the way through all of the levels with my Shatterhand sprite until I got stuck in a wall in the last level because my code doesn't handle the level start position properly yet.

User avatar
thefox
Posts: 3141
Joined: Mon Jan 03, 2005 10:36 am
Location: Tampere, Finland
Contact:

Post by thefox » Tue Apr 03, 2012 10:38 am

I worked on the sprites today and yesterday. First I copy pasted them all to a single image and started trying to figure out the palette sets. I managed to get a pretty good compromise once again. The "Superb Joe" and Leopardman sprites are problematic though, since some of the colors they use aren't used anywhere else. Technically they can be drawn with the current palette sets, but the sprites would be layered and end up using colors from three different palette sets, so it would probably flicker like hell, especially since Superb Joe and Leopardman need to be on screen simultaneously.

Anyway, next I assigned the palettes to each spritesheet, split them up with Imagemagick, and fed them to my sprite converter. Then I had a bunch of .SPR (the sprite definition) and .CHR files. I took the most common sprites (excluding the Master Y, Dr. Tary, Leopardman and Superb Joe sprites, since those won't be needed simultaneously with the normal player sprite) and combined them to a single .CHR file, optimized the tileset and guess what: it came out at EXACTLY 256 tiles (4KB). Pretty funny coincidence. Here's an image of the tileset in its current state (the tiles are in pseudo random order due to the way the optimizer works):

Image

You may notice that some of the tiles are very similar, but not quite. This is a small problem that occurs when using the sprite converter. It works on individual images, so it doesn't always produce optimal results if the size of the global tileset is a concern. Since it's a genetic algorithm it's also somewhat random in how the sprites are placed (several placements can give the "optimal" result, i.e. the same amount of sprites used).

There are some ways to fix this problem. I could do a first pass, looking for similarities in images, split them up from those areas, then convert, and then recombine them. This should produce a little bit smaller final tileset, maybe with the cost that the metasprites would use some more 8x8 sprites.

Another way to fix it would be to modify the genetic algorithm so that placements which re-use existing tiles (from the global tileset) get some extra "fitness" points. I'm probably not going to bother with that for now.

One way to get more wiggle room in the sprite bank would be of course to upload some of the sprite frames dynamically. Player sprites would be a good candidate for this. They take 8 tiles in the worst case I think, so that would be 8 * 16 bytes to be uploaded per frame, so 128 * 8 = 1024 CPU cycles with the fastest possible code. Considering the only other PPU uploads I need are the OAM and the palette, it should be very much doable.

I should modify my sprite converter (image -> sprite definition) tool to support 8x16 sprites, just to see what kind of results that would give.

...

On another topic, I've been thinking about how to implement the "ghost" in the original game. You see, there's a 5 minute time trial mode in the Flash game. If you complete it, the next time you'll race it you'll see a ghost run of your fastest run, with Leopardman (:)) playing the role of the ghost. Or so I've been told, I haven't actually been able to beat the time trial in 5 minutes yet.

So the problem is how to get 5 minutes worth of controller input / other state data squeezed so that it fits in the WRAM. (I think WRAM is absolutely necessary for this, especially given that there has to be two copies of the data -- one for the current run and one for the fastest run). The data would need to be compressed and decompressed in real time. Compressing just the controller input wouldn't be enough, because we don't have easy access to collision etc data of each screen all the time (and the player might be at a completely different screen than the ghost).

This is certainly not top priority, but it's fun to think about.

User avatar
tokumaru
Posts: 11466
Joined: Sat Feb 12, 2005 9:43 pm
Location: Rio de Janeiro - Brazil

Post by tokumaru » Tue Apr 03, 2012 10:59 am

thefox wrote:So the problem is how to get 5 minutes worth of controller input / other state data squeezed so that it fits in the WRAM. (I think WRAM is absolutely necessary for this, especially given that there has to be two copies of the data -- one for the current run and one for the fastest run). The data would need to be compressed and decompressed in real time.
I have routines in my game to record and playback RLE encoded controller data, just tell me if you want me to post them.
Compressing just the controller input wouldn't be enough, because we don't have easy access to collision etc data of each screen all the time (and the player might be at a completely different screen than the ghost).
Yeah, this is the real issue. You might have to run two game sessions side by side! I'm sure there will be enough RAM for this (since you're using WRAM), it's the processing time I'm worried about.

Actually I'm not even sure how this can work in the original game... I believe that enemies and objects do not "live" until you enter a room, so how can the ghost enter a screen before you do and interact with the objects in there?

User avatar
thefox
Posts: 3141
Joined: Mon Jan 03, 2005 10:36 am
Location: Tampere, Finland
Contact:

Post by thefox » Tue Apr 03, 2012 1:13 pm

tokumaru wrote:I have routines in my game to record and playback RLE encoded controller data, just tell me if you want me to post them.
Yeah sure, that would be nice to have. What kind of compression rate do you get with it generally speaking?
Yeah, this is the real issue. You might have to run two game sessions side by side! I'm sure there will be enough RAM for this (since you're using WRAM), it's the processing time I'm worried about.
About WRAM, I was thinking about making the game so that it can work without WRAM, but if WRAM it would have some extra goodies like the ghost thing we're talking about here.
Actually I'm not even sure how this can work in the original game... I believe that enemies and objects do not "live" until you enter a room, so how can the ghost enter a screen before you do and interact with the objects in there?
As far as I know, it simply stores the coordinates and the player sprite map index for each frame and probably some other state info for the streemer (this mode is capped at 5 minutes too, so it'll never generate a huge log). I don't think there's any interaction with the objects (and the only object I can think of that would be problematic regarding that is the breakable block, but that's not used in this level... and in any case it would simply just look funny).

I guess the log in the NES version would have to be stored on screen by screen basis (together with some state info for the beginning of the screen). If the player enters the screen while the ghost is in it, it would need to "rewind" so the ghost time matches the players time. If the ghost is not on the same screen as the player, it can simply step to the next frame. I don't know, we'll see if any of this is practical...

tepples
Posts: 21751
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Post by tepples » Tue Apr 03, 2012 1:21 pm

Player sprites would be a good candidate for this. They take 8 tiles in the worst case I think, so that would be 8 * 16 bytes to be uploaded per frame, so 128 * 8 = 1024 CPU cycles with the fastest possible code. Considering the only other PPU uploads I need are the OAM and the palette, it should be very much doable.
Yeah, five 32-byte copies from $0100-$019F to the PPU plus an OAM copy from $0200 will easily fit into vblank. If it's good enough for Battletoads... But then you have to make room for copying in the ghost's tile. And you don't need all rooms' sprites loaded at once because you can do like Battle Kid and load only those enemy sprites that a given room uses.
So the problem is how to get 5 minutes worth of controller input / other state data squeezed so that it fits in the WRAM. (I think WRAM is absolutely necessary for this, especially given that there has to be two copies of the data -- one for the current run and one for the fastest run).
How many keypresses are there in a typical run? If there are fewer than about 2,000 keypresses, then a single run can fit into 4 KiB. At 300 seconds, that'd be 6 presses per second. Each byte of the controller bytecode would look like this:

7654 3210: ccct tttt
t: number of frames to wait before processing next command
c: command number

Commands:
0: Do nothing
1: Release all buttons
2: Toggle button A
3: Toggle button B
4: Toggle button Up
5: Toggle button Down
6: Toggle button Left
7: Toggle button Right

Another way is to just sample X and Y every few frames, then interpolate at playback time. At 300 seconds and 4 samples per second, that's 2.4 KiB. This even solves tokumaru's problem of the other player interacting with enemies.

Would it be hard to make ghost support and other character support a compile time option?

User avatar
tokumaru
Posts: 11466
Joined: Sat Feb 12, 2005 9:43 pm
Location: Rio de Janeiro - Brazil

Post by tokumaru » Tue Apr 03, 2012 2:25 pm

thefox wrote:Yeah sure, that would be nice to have. What kind of compression rate do you get with it generally speaking?
I'm afraid that RLE will not be enough enough for 5 minutes... Sorry, but I can't give an idea of how well it compresses, but I really haven't tested it much!

Here's the encoder function:

Code: Select all

;-- SUBROUTINE --------------------------------------------------
;DESCRIPTION:
; Logs the buttom presses in a compressed format to an array that
; can later be used to demonstrate the levels in action.
;INPUT:
; DemoStatus: indicates the operation to execute;
; DemoData: address of the controller data;
; ControllerBuffer: current status of the controller;
;----------------------------------------------------------------
RecordDemo:

	;skip if this is not the first frame
	lda DemoStatus
	cmp #$ff
	bne +CompareButtons

	;indicate that a demo is being recorded
	dec DemoStatus

	;initialize the output pointer and start outputting data
	lda #$00
	sta DemoData+0
	lda #$60
	sta DemoData+1
	bne +WriteNewPair ;always

+CompareButtons:

	;branch if the button combination changed
	ldy #$01
	lda (DemoData), y
	cmp ControllerBuffer
	bne +AdvancePointer

	;increment the number of frames for which the combination is to be used
	dey
	lda (DemoData), y
	beq +AdvancePointer
	adc #$00
	sta (DemoData), y

	;return
	rts

+AdvancePointer:

	;advance 2 bytes
	clc
	lda DemoData+0
	adc #$02
	sta DemoData+0
	bcc +WriteNewPair
	inc DemoData+1

+WriteNewPair:

	;output the duration and the combination of buttons
	ldy #$00
	lda #$01
	sta (DemoData), y
	iny
	lda ControllerBuffer
	sta (DemoData), y

+Done:

	;return
	rts
And here's the decoder (it's not a function, but it's really simple and easy to figure out):

Code: Select all

;----------------------------------------------------------------
; Demo playback - START
;----------------------------------------------------------------

	;skip if the level is not in demo mode
	bit DemoStatus
	bpl +SkipDemo

	;overwrite the controller status, except for the start button
	dec DemoCount
	bne +
	ldy #$01
	lda (DemoData), y
	sta DemoCount
	clc
	lda DemoData+0
	adc #$02
	sta DemoData+0
	bcc +
	inc DemoData+1
+	ldy #$00
	lda ControllerBuffer
	and #BUTTON_STRT
	ora (DemoData), y
	sta ControllerBuffer

+SkipDemo:

;----------------------------------------------------------------
; Demo playback - END
;----------------------------------------------------------------
I was thinking about making the game so that it can work without WRAM, but if WRAM it would have some extra goodies like the ghost thing we're talking about here.
It would be really cool if the program could detect the presence of WRAM to enable the goodies, but still work without them on carts without WRAM.
I don't think there's any interaction with the objects (and the only object I can think of that would be problematic regarding that is the breakable block, but that's not used in this level... and in any case it would simply just look funny).
So balls and such do not affect the ghost?

User avatar
thefox
Posts: 3141
Joined: Mon Jan 03, 2005 10:36 am
Location: Tampere, Finland
Contact:

Post by thefox » Tue Apr 03, 2012 9:55 pm

tepples wrote:How many keypresses are there in a typical run? If there are fewer than about 2,000 keypresses, then a single run can fit into 4 KiB.
Not sure, we'll find out later.
Another way is to just sample X and Y every few frames, then interpolate at playback time. At 300 seconds and 4 samples per second, that's 2.4 KiB. This even solves tokumaru's problem of the other player interacting with enemies.
Yeah that might work.
Would it be hard to make ghost support and other character support a compile time option?
Shouldn't be too hard, I'll have to see later if there's any benefit to doing that (to have stripped down versions of the game etc).
tokumaru wrote:So balls and such do not affect the ghost?
Yeah. Well, the effects from those from the previous run are still stored in the log through the X and Y coords, so it doesn't matter. Of course it will look funny with any moving object (which might be out of sync) etc, but there's no way to get around that.

Anyway, I'm going to leave this whole ghost ordeal at rest for now, and start implementing the level objects.

Bananmos
Posts: 468
Joined: Wed Mar 09, 2005 9:08 am
Contact:

Post by Bananmos » Wed Apr 04, 2012 6:12 am

You may notice that some of the tiles are very similar, but not quite. This is a small problem that occurs when using the sprite converter. It works on individual images, so it doesn't always produce optimal results if the size of the global tileset is a concern. Since it's a genetic algorithm it's also somewhat random in how the sprites are placed (several placements can give the "optimal" result, i.e. the same amount of sprites used).

There are some ways to fix this problem. I could do a first pass, looking for similarities in images, split them up from those areas, then convert, and then recombine them. This should produce a little bit smaller final tileset, maybe with the cost that the metasprites would use some more 8x8 sprites.

Another way to fix it would be to modify the genetic algorithm so that placements which re-use existing tiles (from the global tileset) get some extra "fitness" points. I'm probably not going to bother with that for now.
I'm having the exact same issues in my sprite optimizing tool Tilificator and have had similar ideas (not implemented yet) on solving it. I'd be interested to know what solution you might implement :)

mic_
Posts: 917
Joined: Thu Oct 05, 2006 6:29 am

Post by mic_ » Wed Apr 04, 2012 11:06 am

I've used the NeuQuant algorithm frequently for color quantization of images. I wonder if it'd be difficult (and practical) to modify that algorithm so that each neuron represents an 8x8 tile instead of an RGB color - i.e. the output would be the N "best" tiles out of a larger set of tiles.

Post Reply