Including binary files that contain pointers to dynamic data

Discuss technical or other issues relating to programming the Nintendo Entertainment System, Famicom, or compatible systems. See the NESdev wiki for more information.

Moderator: Moderators

User avatar
rainwarrior
Posts: 8734
Joined: Sun Jan 22, 2012 12:03 pm
Location: Canada
Contact:

Re: Including binary files that contain pointers to dynamic

Post by rainwarrior »

tokumaru wrote:As my programs get more complex, I sometimes worry that the assembler is going to bump into some sort of limit and will not be able to handle the huge amount of labels I'm using...
My current project has ~2MB of generated assembly files worth of this kind of stuff and they assemble "instantly" and painlessly. If there's a practical limit, it's very high, I think.

The only time I've seen assemble times become significant with ca65, though, was with Movax12's "high level" SMB disassembly. In this case, all the complicated macros he used really do consume a lot of assembly time. (As I recall, it was on the order of "minutes".)

Though, even if you did run into assembly complexity issues, this can be helped a lot by breaking up your code into separate units and assembling them separately, letting you do incremental builds on only the stuff that's changed since the last build (makefiles make this easy to accomplish). Doesn't help link time, but unless you're also exporting/importing all these generated labels for some reason (why would you need to?), I don't imagine the generated data will give you link time problems anyway.

In my project, I am not even using makefiles, just because a need has never come up. I have a huge pile of assembly code, but the whole build process only lasts about a second, so... I never bothered. I just let it build from scratch every time.
User avatar
pubby
Posts: 583
Joined: Thu Mar 31, 2016 11:15 am

Re: Including binary files that contain pointers to dynamic

Post by pubby »

rainwarrior wrote:My current project has ~2MB of generated assembly files worth of this kind of stuff and they assemble "instantly" and painlessly. If there's a practical limit, it's very high, I think.
I hit ld65's limits pretty hard on that 8MB bebop.nes rom I made. There's a ~255 file limit when linking and each file can't have more than ~255 segments. It was not a fun experience getting it to work.

Smaller roms shouldn't have a problem though.
makefiles make this easy to accomplish
And they come with the sweet bonus of being able to compile your files in parallel.
Sik
Posts: 1589
Joined: Thu Aug 12, 2010 3:43 am

Re: Including binary files that contain pointers to dynamic

Post by Sik »

rainwarrior wrote:The only time I've seen assemble times become significant with ca65, though, was with Movax12's "high level" SMB disassembly. In this case, all the complicated macros he used really do consume a lot of assembly time. (As I recall, it was on the order of "minutes".)
(・□・ )

I hope nobody has ever have to touch ca65hl.h again.
User avatar
rainwarrior
Posts: 8734
Joined: Sun Jan 22, 2012 12:03 pm
Location: Canada
Contact:

Re: Including binary files that contain pointers to dynamic

Post by rainwarrior »

pubby wrote:I hit ld65's limits pretty hard on that 8MB bebop.nes rom I made. There's a ~255 file limit when linking and each file can't have more than ~255 segments. It was not a fun experience getting it to work.
Heh, I didn't know it even had an input file limit, or segment limit. :P I've never been anywhere near either of those. Your case seems rather unique. What about performance, though? How long did it take to build?

A tip that could help in your specific case, or in general:

You don't have to link all your banks at once. This is especially true of 32k banking. You can do each bank separately and just concatenate the results. Even with 16k (or other) banking, you can create a dummy segment (that doesn't output to a file) to generate references for other banks that you need to do cross-bank linking with.

I started doing it that way when I found I couldn't get bank info from the debug symbols, so all the symbols in the listing would overlap with no way to disambiguate. Splitting them up was a necessity for generating debug symbols for FCEUX, but once I started doing it I found it convenient in other ways too.
User avatar
pubby
Posts: 583
Joined: Thu Mar 31, 2016 11:15 am

Re: Including binary files that contain pointers to dynamic

Post by pubby »

rainwarrior wrote:What about performance, though? How long did it take to build?
Assembling and linking probably took about 20 seconds total. It was still pretty fast.

The slow part was the converter program I made. It would take 5+ minutes to convert the movie file into rom data.
You don't have to link all your banks at once. This is especially true of 32k banking. You can do each bank separately and just concatenate the results. Even with 16k (or other) banking, you can create a dummy segment (that doesn't output to a file) to generate references for other banks that you need to do cross-bank linking with.
I like the idea, but what does the dummy segment look like? Did you map out the cross-bank addresses by hand?
User avatar
rainwarrior
Posts: 8734
Joined: Sun Jan 22, 2012 12:03 pm
Location: Canada
Contact:

Re: Including binary files that contain pointers to dynamic

Post by rainwarrior »

pubby wrote:I like the idea, but what does the dummy segment look like? Did you map out the cross-bank addresses by hand?
A dummy segment looks exactly like the regular version of that segment except it is placed in a MEMORY block with file = "" instead. No, you don't map them out by hand, that's the whole point of the dummy segment.
zzo38
Posts: 1096
Joined: Mon Feb 07, 2011 12:46 pm

Re: Including binary files that contain pointers to dynamic

Post by zzo38 »

I have had similar kind of problems. In my case the issue was music, and I used the following postprocessor code:

Code: Select all

        ; Adjust music looping addresses
        lda #$C0
        sta <3
        lda #high(song1)-$40
        sta <1
        lda #low(song1)
        sta <0
        sta <2
        ldx #$00
        stx $2000
muadj   lda [0,x]
        cmp #mushs
        bne muadj1
        lda <0
        sta <2
        inc <0
        lda <1
        eor <3
        sta [0,x]
        jmp muadj3
muadj1  inc <0
        cmp #musl
        bne muadj3
        lda <2
        sta [0,x]
        lda <0
        sta <2
        inc <2
muadj3  inc <0
        bne muadj
        inc <1
        lda <1
        cmp #(high(endsong)&$1F)|$40
        beq ppend
        cmp #$60
        bne muadj
        lda #$E0
        sta <3
        lda #$40
        sta <1
        jmp muadj
Obviously you cannot use this code as is; it is for a specific use case which is different from your own. But this probably will not work for you anyways since most assemblers cannot execute 6502 code; this is a specialized assembler which includes a 6502 interpreter for use as a postprocessor (and I may be the only person who uses it).
(Free Hero Mesh - FOSS puzzle game engine)
Celius
Posts: 2158
Joined: Sun Jun 05, 2005 2:04 pm
Location: Minneapolis, Minnesota, United States
Contact:

Re: Including binary files that contain pointers to dynamic

Post by Celius »

I have taken the approach of generating .db statements instead of including binary files before. I believe I did this when working on my raycasting demo, where I had tables of information for every possible wall distance, which looked something like this:

Code: Select all

RaycastingWallHeightsDistance01:
  .db $05,$05,$05,$05,$05...
RaycastingWallHeightsDistance02:
  .db $05,$05,$04,$05,$05...
...
RaycastingWallHeightsDistanceFE:
  .db $00,$00,$00,$01,$01..
RaycastingWallHeightsDistanceFF:
  .db $00,$00,$00,$00,$01...
Then I needed pointers to these tables, so I just generated .db statements like this:

Code: Select all

RaycastingWallHeightsDistancesL:
   .db <RaycastingWallHeightsDistance01
   .db <RaycastingWallHeightsDistance02
   ...
   .db <RaycastingWallHeightsDistanceFE
   .db <RaycastingWallHeightsDistanceFF

RaycastingWallHeightsDistancesH:
   .db >RaycastingWallHeightsDistance01
   .db >RaycastingWallHeightsDistance02
   ...
   .db >RaycastingWallHeightsDistanceFE
   .db >RaycastingWallHeightsDistanceFF
This approach is actually pretty handy.
tomaitheous
Posts: 592
Joined: Thu Aug 28, 2008 1:17 am
Contact:

Re: Including binary files that contain pointers to dynamic

Post by tomaitheous »

If CA65 can assemble code or data on top of an already incbin file, that there's an easy(ish) way to do this. Store all the absolute addresses as whatever (doesn't matter), and have your program output a non binary file to include with the binary file. In the supplemental file, you'll need a series of equates and org/data defines so the assembler will re-assemble data on top of the already included binary, based on the entry point of where the assembler is assigning an address for this binary. I do this kinda stuff in PCEAS a quite often. Either for hacking roms, or writing self modifying code that needs to reside in ram (all absolute addresses have to be redefined), or just binary distribution without source.
__________________________
http://pcedev.wordpress.com
User avatar
rainwarrior
Posts: 8734
Joined: Sun Jan 22, 2012 12:03 pm
Location: Canada
Contact:

Re: Including binary files that contain pointers to dynamic

Post by rainwarrior »

tomaitheous wrote:writing self modifying code that needs to reside in ram (all absolute addresses have to be redefined)
ca65 has a built-in feature for this purpose, i.e. a segment has a "load" address for where it will be placed, and a "run" address for where it will be executed.
tomaitheous
Posts: 592
Joined: Thu Aug 28, 2008 1:17 am
Contact:

Re: Including binary files that contain pointers to dynamic

Post by tomaitheous »

rainwarrior wrote:
tomaitheous wrote:writing self modifying code that needs to reside in ram (all absolute addresses have to be redefined)
ca65 has a built-in feature for this purpose, i.e. a segment has a "load" address for where it will be placed, and a "run" address for where it will be executed.
Yeah.. I really need to switch over to ca65 and quit stalling on it.
__________________________
http://pcedev.wordpress.com
User avatar
Hamtaro126
Posts: 818
Joined: Thu Jan 19, 2006 5:08 pm

Re: Including binary files that contain pointers to dynamic

Post by Hamtaro126 »

tomaitheous wrote:Yeah.. I really need to switch over to ca65 and quit stalling on it.
Converting all your PCE source code to CA65 can be a good thing, But can be painful...

You may need to convert your code that uses the .func pseudo-op to raw data/code, or a macro, or a define
AKA SmilyMZX/AtariHacker.
Post Reply