Source code collection

[SnowBro]

Hello. I'm releasing some NES source code I've written.
It basically contains most stuff necessary to make a full 2D scroller; a graphics engine, music+sfx engine, object engine (including collision detection, physics), animation, Huffman text decoding, palette fading, and a virtual machine for scripting game logic.
I've also integrated everything into a small demo which shows some of the capabilities. The total amount is 11,000+ lines of code split across 70 linkable units, which should be conveniently browsable by name.

The source code is currently at
http://www.stud.ntnu.no/~kenth/ntxdev.zip

You'll need the latest version of my assembler+linker, and (GNU) Make, to build it:
http://www.stud.ntnu.no/~kenth/xorcyst

I've compiled a ROM for those interested in that:

http://www.stud.ntnu.no/~kenth/ntxdev.nes.zip

I have made a number of tools to create levels, encode text, assemble game logic scripts etc., but it takes some effort to package it and document it properly, so I'll hold off releasing it until someone asks.

Most of the source code and internal data formats are well documented (I hope). I should really write some detailed articles though, but it takes a lot of time...

The code is still under heavy development, as there's much to be done before I'll be back at the functional level I was with the old X816 code base, but there's enough stuff in there already for starters.

Have fun!

(And thanks to Tootai for the monochrome bird used as test enemy (the image before conversion was much better, sorry about that...))

[Bregalad]

Wow, sick !!
Looks really great.

I should really write some detailed articles though, but it takes a lot of time...

Yeah, that would be really cool. Thanks a lot for your work !

By the way, I've got a few questions about your assembler :
-What's the use of the "proc" stuff ? What's the difference between a jsr to a procedure and a standard label ?
- Is it possible to pad the code into a file (for example I've main code at $e000-$exxx, DPCM samples at $f000-$fxxx and vectors at $fffa-$ffff, am I obligated to imput 3 different asm files ?
- Is there a way to definite an automatic-public data segment without definite 2 times a public variable ?
- What's the use of this below stuff ? Isn't is simpler to just remember wich bit has wich use ?

Code: Select all

; Records which describe PPU reg layouts.
.record ppu_ctrl0 nmi_on_vblank:1, pad0:1, sprite_size:1, bg_table:1, sprite_table:1, ppu_addr_inc:1, name_table:2
.record ppu_ctrl1 emph:3, sprite_on:1, bg_on:1, sprite_clip:1, bg_clip:1, mono:1
.record ppu_status vblank:1, sprite0:1, sprite_overflow:1, pad0:5

; Field masks.
PPU_CTRL0_NMI				.equ	mask ppu_ctrl0::nmi_on_vblank
PPU_CTRL0_SPRITE_SIZE		.equ	mask ppu_ctrl0::sprite_size
PPU_CTRL0_BG_TABLE			.equ    mask ppu_ctrl0::bg_table
PPU_CTRL0_SPRITE_TABLE		.equ	mask ppu_ctrl0::sprite_table
PPU_CTRL0_PPU_ADDR_INC		.equ	mask ppu_ctrl0::ppu_addr_inc
PPU_CTRL0_NAME_TABLE		.equ	mask ppu_ctrl0::name_table

Is it possible to assign more than one letter to one byte in a *.char file (for example : the=0x80; every=0x81; each=0x82; etc...) This could be an usefull and easy to code way to compress text data and many commercial NES games uses it (letters never takes a whole 8-bit table).
- What on God's green earth (lol) is an union ?
- Same as above for the enumerations
- When indirect indexed or indexed indirect adressing (i.e. lda [$00],Y or ldx [$00,X]) shall the argument be a byte or a word (I tried the word thig but it didn't work). So, is that right to imput .byte[2] ?
- Is there a way to mix up user-defined variables and auto-assigned variables, for example, to be sure to have save buffer to the SRAM at $6000-$7fff ?
Thanks in advance ![/code]

[Guest]

-What's the use of the "proc" stuff ? What's the difference between a jsr to a procedure and a standard label ?

No, at the moment there's really no difference between "proc" and a normal label. However, I have fun ideas for it, such as being able to inline a function defined in another unit, or the linker being able to extract a single procedure from one unit, as opposed to having to link in the whole unit. The basis of these expansions would be that the size of the procedure (in bytes) has to be known, which the assembler can find out if it's told that the label is a procedure and not a generic label (the procedure ends at the "endp" statement).

- Is it possible to pad the code into a file (for example I've main code at $e000-$exxx, DPCM samples at $f000-$fxxx and vectors at $fffa-$ffff, am I obligated to imput 3 different asm files ?

You can't pad to an absolute address, but you can use the current PC symbol, $, to achieve the proper result if you don't want to split up the files. For example,

Code: Select all

.pad $FFFA - $

simulates absolute padding to $FFFA.
Perhaps I should just change ".pad XXXX" to mean ".dsb XXXX - $", since people are probably used to the pad directive taking an absolute address (right now pad is an alias for dsb).

- Is there a way to definite an automatic-public data segment without definite 2 times a public variable ?

Hmm, I think you just found a bug. You're supposed to be able to start a data segment with

Code: Select all

.dataseg public

to make all the variables auto-public. But it doesn't work, even though it should, so I have to fix that.
You can also use

Code: Select all

.public my_var .byte

Which is equivalent to

Code: Select all

my_var .byte
.public my_var

- What's the use of this below stuff ? Isn't is simpler to just remember wich bit has wich use ?

I'm just showing off the .record directive. For the PPU regs, I agree, it's just as simple to type the masks directly, since they're not gonna change. But I think records are a life-saver for user-defined bitfields, and especially the ability to initialize the fields like a normal structure and have the assembler do the tiresome shifting, ANDing and ORing for you. Check out "warp2d.h"; back in the old days I had to generate the packed bytes of (area, section, location) manually, now the assembler does it for me, so I can easily change bitfield values when I build the level data.

Is it possible to assign more than one letter to one byte in a *.char file (for example : the=0x80; every=0x81; each=0x82; etc...) This could be an usefull and easy to code way to compress text data and many commercial NES games uses it (letters never takes a whole 8-bit table).

Nope, not yet. I'm using Huffman coding for my text, which gives very good compression. But I have to release the tools I've made for compressing text in order for it to be any good for others.

- What on God's green earth (lol) is an union ?

A union is like a structure, only all the fields share the same storage in memory. Check "effect.h" for example, which contains the data structures for the music player's effect generator. At any time, a sound channel can have only one effect, but there are several different effects to choose from. In order to use the most meaningful names according to the effect type, the effect states share the same data storage, I just use the effect.kind field to figure out which of the unions to use.

- Same as above for the enumerations

If you search for a C tutorial I'm sure you will find good explanations of both unions and enumerations. An enumeration is just a convenient way of defining a set of strictly increasing constants, without having to actually type the values in manually. It's a way of raising the abstraction level of your code and make it less error-prone.

- When indirect indexed or indexed indirect adressing (i.e. lda [$00],Y or ldx [$00,X]) shall the argument be a byte or a word (I tried the word thig but it didn't work). So, is that right to imput .byte[2] ?

The argument is an 8-bit address, but the actual data at that address is a 16-bit pointer. I use the "ptr" datatype defined in "ptr.h", so I can access it as my_ptr.lo and my_ptr.hi. Remember to specify the "zeropage" modifier when defining the pointer, otherwise it might not get mapped to zeropage and you will get an error message during linking.

- Is there a way to mix up user-defined variables and auto-assigned variables, for example, to be sure to have save buffer to the SRAM at $6000-$7fff ?

Well yeah, sort of... You can define a label to point to WRAM:

Code: Select all

.label wram = $6000

But you can't map an entire data segment to a specific location. However, you can use a structure, since a structure's field are guaranteed to be mapped sequentially in memory. So for example

Code: Select all

.struc wram_state
savegames .savegame[10] ; savegame struc defined elsewhere
temp_array .word[128]
...
.ends

.label wram = $6000 : wram_state

Then you can use wram.savegames etc. in your code. Just remember to not specify your reserved parts of WRAM in the linker script, so the linker won't try to allocate them.

Thanks in advance !

Thank you, I'm the only one who's used the assembler up until now I'm pretty sure, so I'm glad for all feedback on how others see it.

[tepples]

No, at the moment there's really no difference between "proc" and a normal label. However, I have fun ideas for it, such as being able to inline a function defined in another unit, or the linker being able to extract a single procedure from one unit, as opposed to having to link in the whole unit.

Sounds like .ifref in .CA65. In addition, CA65's .proc allows for procedure-local labels. Do you plan on adding that?

For example,

Code: Select all

.pad $FFFA - $

simulates absolute padding to $FFFA.
Perhaps I should just change ".pad XXXX" to mean ".dsb XXXX - $"

Such a change would at least help maintain consistency with x816. CA65, on the other hand, prefers that the linker script put a 6-byte "vectors" section into the ROM_C000 bank.

But I think records are a life-saver for user-defined bitfields, and especially the ability to initialize the fields like a normal structure and have the assembler do the tiresome shifting, ANDing and ORing for you. Check out "warp2d.h"; back in the old days I had to generate the packed bytes of (area, section, location) manually

If your assembler can handle the | operator, you don't need it. Define the masks in a header file, and OR them together in your constant data. Even better is to make a custom graphical editor for each data type, separate from your assembler.

I'm using Huffman coding for my text

I've read in Managing Gigabytes that a technique called "huffword", where each Huffman code maps to a word, provides good compression.

You can define a label to point to WRAM:

Code: Select all

.label wram = $6000

But you can't map an entire data segment to a specific location.

CA65 can.

However, you can use a structure, since a structure's field are guaranteed to be mapped sequentially in memory.

In fact, I use a technique like this in my GBA headers.

Thank you, I'm the only one who's used the assembler up until now I'm pretty sure, so I'm glad for all feedback on how others see it.

What do you plan on making your assembler do that CA65 doesn't already do?

[Guest]

What do you plan on making your assembler do that CA65 doesn't already do?

It's not really a matter of what an assembler can do, but how easy it allows the programmer to do it. I'm sure you can see that

Code: Select all

lda my_var + my_struc1::field1 + my_struc2::field2

is less intuitive than

Code: Select all

lda my_var.field1.field2

How can you define an array of structures with ca65's language? (xasm: like in C)
How can you initialize structures and unions statically (ROM data)? (xasm: like in C)
How can you index structures statically? (xasm: like in C)
I can't find directives equivalent to .DS(B|W|D) (.RES? Nah...).
And I still say records is more elegant than manual ORing in the way you describe. Besides, it doesn't HURT to have that feature since you can use |, & and << to your heart's content anyway if you want.
These are all features I found crucial for keeping my code "sane". A large part of the explanation for the omission of these features in ca65 is probably that it's meant to process code generated by a compiler, whereas my assembler targets code generated by mortal humans.

I don't see any point in listing more reasons why I wrote my own assembler instead of using ca65; I haven't used it anymore than you've used my assembler. If you're happy with ca65, there's no reason to switch. But I'm sure the code I've released (which is the real topic here) would look a lot uglier if I wrote it for ca65 since it's heavily based on (nested) unions and structs. Besides, writing the assembler and linker was a fun exercise, and I'm really comfortable with using it now. Hope I haven't offended you by releasing it to the public.
And if you have suggestions for features that can make it "stand out" more, I'm all ears. Constructive discussion on tools is fun.

And you're right about the graphical editor... I have an old, old one which just doesn't cut it anymore, so I eventually have to write a new one, but it's a timeconsuming project. I also have to write a new version of my old tool NSA (NES Screen Arranger) which is platform-independent and doesn't suck as much.

[SnowBro]

Cripes, forgot to enter my nick in the previous reply, I guess I should register here soon (I might be already though, it's been a while since I frequented the board).
Anyway, while we're talking about assemblers, I would like to give some heads up to Brian Provinciano. When I released version 1.0, I was perfectly content with releasing an "X816 clone", to put it blunt. But when I saw the screenshots of his NESHLA assembler on the Grand Theftendo page, I got inspired to add all the high-level junk I hadn't had the motivation to do before... And without that, there would be no NES source code to release now. So cheers for that!

[tepples]

I also have to write a new version of my old tool NSA (NES Screen Arranger) which is platform-independent and doesn't suck as much.

You could try starting with the source code for 8Name, an NSA clone I wrote when I couldn't get NSA to work reliably on Windows 2000. Find it along with my clone of (old) Tile Layer in Pin Eight NES Tools.


8Name screenshot

[RedComet]

[quote="SnowBro"I have made a number of tools to create levels, encode text, assemble game logic scripts etc., but it takes some effort to package it and document it properly, so I'll hold off releasing it until someone asks.
[/quote]

I'm rather interested in the tools for encoding text, is there any way you could upload those?

Anyway, great stuff there SnowBro.

[SnowBro]

You could try starting with the source code for 8Name, an NSA clone I wrote when I couldn't get NSA to work reliably on Windows 2000.

Have you seen my latest graphics editor, Tile Molester? I want to create a nametable editor with the same standard, modern software functionality: cut/copy/paste/move chunks of nametable data, undo/redo.
By the way, I tried "name" under MandrakeLinux 10.1, and when I right-click on the nametable, everything hangs for about 10-15 seconds, with the mouse flashing rapidly. I'm using Allegro 4.0.3. Must be something in the loop in the right_click() function that's causing it not to terminate. It's fine on XP.

[SnowBro]

I'm rather interested in the tools for encoding text, is there any way you could upload those?

I uploaded it here. You'll also need the file huffman.asm from the source archive (see top of this post) to actually decode the text; it's all very hastily described in the README.

[Bregalad]

No, at the moment there's really no difference between "proc" and a normal label. However, I have fun ideas for it, such as being able to inline a function defined in another unit, or the linker being able to extract a single procedure from one unit, as opposed to having to link in the whole unit. The basis of these expansions would be that the size of the procedure (in bytes) has to be known, which the assembler can find out if it's told that the label is a procedure and not a generic label (the procedure ends at the "endp" statement).

So is the ".proc / .endp" needed for every subroutine (so for 99% of the code) ?

You can't pad to an absolute address, but you can use the current PC symbol, $, to achieve the proper result if you don't want to split up the files.

Hem, you say I can't and you after show me how to do it, so I don't get it right.
So .pad $xxxx - $ would tell the linker to full all datas between current adress and ardess $xxxx, is that right ?

You can also use
Code:

.public my_var .byte


Which is equivalent to
Code:

my_var .byte
.public my_var

OK thanks. By the way, the .byte is there an alias for .db and could also be .db[64] for a buffer of 64 bytes, right ?

I'm just showing off the .record directive. For the PPU regs, I agree, it's just as simple to type the masks directly, since they're not gonna change. But I think records are a life-saver for user-defined bitfields, and especially the ability to initialize the fields like a normal structure and have the assembler do the tiresome shifting, ANDing and ORing for you. Check out "warp2d.h"; back in the old days I had to generate the packed bytes of (area, section, location) manually, now the assembler does it for me, so I can easily change bitfield values when I build the level data.

Oh, it's horrible, I'm unable to understand a single line in thoose .h files. Does this mean that the .record thing can automatically add logic instruction in your code ? So the carry flag or something could be affected in the programm, and this could be a source of bugs.

Nope, not yet. I'm using Huffman coding for my text, which gives very good compression. But I have to release the tools I've made for compressing text in order for it to be any good for others.

I guess that the MTE compression rate is as high than Huffman one, and it's much better to use and code (I didn't understand 100% how huffman works, trough. It's just a pointer pointing on other pointers, pointing themselves on other pointers, etc.... isn't it ?)

If you search for a C tutorial I'm sure you will find good explanations of both unions and enumerations. An enumeration is just a convenient way of defining a set of strictly increasing constants, without having to actually type the values in manually. It's a way of raising the abstraction level of your code and make it less error-prone.

Ah, okay, I got the enumeration thing. So

Code: Select all

.enum Color
Blue, Green
Yellow=4, Orange, Red

Is the equivalent to :

Code: Select all

Blue = 0
Green = 1
Yellow = 4
Orange = 5
Red = 6

Is that right ? (so What's the use of the enumeration's name ?)
I'm still not understanding anythiny to unions and structures. The C manual I looked at, were, let's say too "proffesionnal" for me. It's seeing examples of list of names, ages, and other thing like that (what's the use of names and ages for NES programming ??)

But you can't map an entire data segment to a specific location. However, you can use a structure, since a structure's field are guaranteed to be mapped sequentially in memory.

Ah, the day I'll got the structures may help me to understand that.
However, I found better to organize the RAM myself. OK I could make an error and assignate to variables to the same location, but I'm still sure of what I'm doing and I know witch part of the code I'm seeing while tracing/debuging my code.
For example, the tracers shows something like this

ldx $67
lda $39
sec
sbc $9f
asl A
sta $20
.label
lda $38
and $654,X
ora $20
sta $23f,X
dex
bne .label

You'll be totally lost if you don't know wich variables are $20, $38, $39, $68 and $9f (may $23f,X and $654,X could be resolved with a RAM wiewer). If you additionnaly adds automatic-logic instructions, thing will get worse.
Also, SRAM bankswitching like in a SOROM or ETROM game would be impossible, isn't it ?
Imagine a code like this :

Code: Select all

lda $200,X
cpx #$40
bcs .next
pha
lda #$00
sta #WRAMBankSwitchReg
pla
sta $7fc0,X   ;Oh, the RAM allocator overflowed the first bank
jmp .next2
.next
pha
lda #$01
sta #WRAMBankSwitchReg
pla
sta $5fc0,X    ;And cut the buffer into two parts (this ones writes to the SRAM because x is greather than $40)
.next2
.....

instead of :

Code: Select all

lda Buffer1,X
sta Buffer2,X
....

Wouldn't it be stupid, heh ? (the first code looks like a C generated code)

Thank you, I'm the only one who's used the assembler up until now I'm pretty sure, so I'm glad for all feedback on how others see it.

You just shall doccument it correctly. Otherwise your work could become useless and that would be a shame.
I'm interessing to switch from NESAsm to XRORCyst for my Ecological Evolution project, but I'm still in trouble (NESAsm really have a lack of options, but the XROCyst have too much, and not the one who I'm checking for, heh !)

[RedComet]

You'll also need the file huffman.asm from the source archive (see top of this post) to actually decode the text; it's all very hastily described in the README.

Thanks for uploading that. I've always thought huffman compression was a bit too difficult for the 6502 to accomplish, but I guess its not afterall!

What kind of a compression rate have you managed to squeeze out with huffman as opposed to say dictionary encoding or dual/multi tile encoding?

[Bregalad]

The dos version of the thing looks like to don't work.

[SnowBro]

The dos version of the thing looks like to don't work.

huffpuff < README > test.asm

works here (standing in the folder where you uncompressed it).

The program reads input from stdin and writes output to stdout, which is why you have to use < > to redirect the streams from/to file. You could just start the program without any redirection, in which case the input is read from your keyboard and the output appears on your console.

There's only one (optional) argument: The name of a .tbl (character map) file of the same format as used in my assembler (I forgot to include example.tbl in the binary releases of huffpuff, but it's in the source code release). It's only necessary to use a character map if you use a different character coding than ASCII. Then you can write the input file in ASCII, and huffpuff will convert the characters according to your custom character map BEFORE the string is Huffman-encoded.

[SnowBro]

What kind of a compression rate have you managed to squeeze out with huffman as opposed to say dictionary encoding or dual/multi tile encoding?

I've only tried it on Final Fantasy 1.
Original (DTE): 13,284 bytes
Huffman: 10,638 bytes

I haven't taken the size of the DTE table itself into account (anyone know the size?).
The Huffman decoding table is 266 bytes.