utility: Fixed Bank Creator, for use with 32kB PRG banks

Discuss technical or other issues relating to programming the Nintendo Entertainment System, Famicom, or compatible systems.

Moderator: Moderators

User avatar
Memblers
Site Admin
Posts: 3770
Joined: Mon Sep 20, 2004 6:04 am
Location: Indianapolis
Contact:

utility: Fixed Bank Creator, for use with 32kB PRG banks

Post by Memblers » Wed Nov 14, 2018 12:19 am

Since I was writing some GTROM documentation, and wanting to include templates for a few different assemblers, I decided to include a simple utility for duplicating data among 32kB banks, rather than complicate the source code for the templates. I'll be using it myself with CA65. I don't know if anyone else wants it, but here it anyways. C source is included.

edit - bugfix v1.1 added 11-16-2018

Code: Select all

Fixed Bank Creator
    by Memblers
    
V1.0 - 11-14-2018 - initial release
v1.1 - 11-16-2018 - bugfix: blank check fail warning would corrupt source
			    bank selector

-- Introduction --

Fixed Bank Creator is a utility designed for NES developers who are using a 
mapper that bankswitches PRG-ROM in 32kB pages.  It will allow you to take 
code/data from one bank, and duplicate it to the same address in every bank.  
You could do this with your assembler/linker, but the methods for doing so 
vary by assembler and can become convoluted, with issues such as having 
conflicting label names within the same scope.  Because this program operates 
on a binary file, it doesn't matter which assembler you are using.

Whether you've already developed with 32kB banks and are tired of dicking 
around with an overly-complicated build process, or you are a UNROM veteran 
looking to shrink your fixed bank and free up more memory on boards such as 
GTROM, it is hoped that this program could be useful to you.

-- Usage --

Fixed Bank Creator is a command-line program, intended to be run as part of 
your normal ROM-building process.  You specify the input ROM file in .NES 
format, address range for the desired fixed bank, and (optionally) an output 
filename.  You do not need to specify a source bank, as the program will 
detect which bank already has data within that range.  If data exists within 
that range in multiple banks, a warning message will be output.

    fixbank.exe [.NES file] [begin address] [end address] [optional filename]
    
Usage examples:

    fixbank hotseat2.nes $C000 $FFFF
        Creates a UNROM-style 16kB fixed bank, saved as "output.nes"
    
    fixbank hatetank.nes FFFA FFFF romtest.nes
        Duplicates the vectors only, saved as "romtest.nes"
    
    fixbank romtest.nes 0xE000 0xE7FF
        Duplicates 2kB data, DPCM samples for example, saved as "output.nes"
		
		
When creating your ROM to use the program, what you are supposed to do is 
choose the region of memory (anywhere in $8000-$FFFF) that you would like to 
make into the fixed bank.  Organizing your program into 32kB banks, you then 
leave that region empty (padded with $00 or $FF bytes) in every bank, except 
for one of them.  The contents of that one bank will then be duplicated.
        
-- Notes and Tips --

Warning, Error, and Info messages are output to the screen.  The warning 
messages should not be ignored, as they are likely indicators of a 
non-working result.

'$F000', 'F000', and '0xF000' are all valid input syntax for this program.

After the source bank has been detected, the rest of the banks are scanned to 
check if they are blank.  If they are not, a warning is output and data will 
be overwritten.

Blank check requires memory to be padded with either $00 or $FF.  If you use 
$FF, FlashROM and EPROM chips may be slightly faster to program, as $FF is 
the typical blank state of non-volatile memory.

Searching for source data bank begins in the lowest bank.  It's recommended 
that you put your fixed bank source data in the lowest (first) bank.

It's best if you set up your assembler to reserve memory in your fixed bank 
area, as the error messages from your assembler will likely be more helpful 
than the warning messages from this program.

If you need to create multiple "mirrored" parts of memory, just run the 
program again on the output file.

This program is free and open source under the MIT license.  Additionally, 
any discordian pope may automatically grant themselves a license under the 
WTFPL, as desired.

-- Known Issues --

The source bank detection will fail if you were to create a bank that 
consists entirely of $00 and/or $FF bytes.  If, for some reason, you wanted 
to have nothing but patterns of $00 and $FF bytes in your fixed bank, this is 
bad news.
Attachments
fixbank.zip
fixbank v1.1 (bugfix)
(21.8 KiB) Downloaded 272 times

User avatar
Banshaku
Posts: 2328
Joined: Tue Jun 24, 2008 8:38 pm
Location: Fukuoka, Japan
Contact:

Re: utility: Fixed Bank Creator, for use with 32kB PRG banks

Post by Banshaku » Wed Nov 14, 2018 12:44 am

I never had to manage 32kB banks but there is one thing I would like clarification since maybe some of my assumption were wrong for ca65.

If I write code in assembler with ca65, create a scope then include a file in that scope, wouldn't that remove the label clash and allow the code to be exactly at the same place? I never tried but this is what I would do first if I had this issue.

I may try someday to see how the assembler react, out of curiosity and wanting to know more about it.

User avatar
tokumaru
Posts: 11466
Joined: Sat Feb 12, 2005 9:43 pm
Location: Rio de Janeiro - Brazil

Re: utility: Fixed Bank Creator, for use with 32kB PRG banks

Post by tokumaru » Wed Nov 14, 2018 1:38 am

Banshaku wrote:If I write code in assembler with ca65, create a scope then include a file in that scope, wouldn't that remove the label clash and allow the code to be exactly at the same place? I never tried but this is what I would do first if I had this issue.
Wrapping things in scopes does indeed solve the label clash problems, but there are a few catches. I tried creating a macro to automate this once, but it didn't work. Macros are kinda mystical in ca65, I guess. But it works if you do it in other ways. What I ended up doing was creating my own implementation of PROC, which checks whether the entry point label is already defined or not, to decide between creating a named scope or an unnamed one. With this I didn't have to explicitly wrap stuff in scopes, I simply included the subroutines multiple times and the custom PROC macro took care of creating global labels only the first time.

Anyway, Memblers' idea is pretty cool, but unfortunately it doesn't solve the problems I have with virtual fixed banks. First, having to know the address means I'd still have to change settings manually whenever I added/removed things to/from the fixed bank. I guess I could set a larger size and only shrink it at a later time, when the fixed stuff was probably already set in stone and the lower portions of the banks were getting full. But the other problem for me is that not all fixed banks are equal: while some things are present in all banks, other things are repeated only in specific banks. Banks with CHR data have code to copy/decode that to VRAM, banks with level maps have collision detection routines, stuff like that. It's kinda like another fixed bank below the general fixed bank. Multiple passes of Memblers' tool could maybe handle that, but again I'd have to know all the addresses beforehand and constantly change the settings during development.

User avatar
rainwarrior
Posts: 7677
Joined: Sun Jan 22, 2012 12:03 pm
Location: Canada
Contact:

Re: utility: Fixed Bank Creator, for use with 32kB PRG banks

Post by rainwarrior » Wed Nov 14, 2018 3:02 am

tokumaru wrote:But the other problem for me is that not all fixed banks are equal: while some things are present in all banks, other things are repeated only in specific banks. Banks with CHR data have code to copy/decode that to VRAM, banks with level maps have collision detection routines, stuff like that. It's kinda like another fixed bank below the general fixed bank. Multiple passes of Memblers' tool could maybe handle that, but again I'd have to know all the addresses beforehand and constantly change the settings during development.
I don't think these other things fit the category of "fixed bank". This is just duplicate code, not something that needs to be at a fixed location. The problem relaxes a bit if you don't include the additional constraint of putting it at the same location.

Stuff at a fixed location can be dealt with in a various ways (e.g. tepples' suggested this "workaround" recently, other possibilities have been discussed in the past, and now this tool is being offered, etc.). Relocatable code that's duplicated in some banks is an easier problem.

For instance, probably the simplest case: in ca65, if the duplicated code is in different modules (i.e. different assembles), as long as you don't export the symbols there is no duplicate name conflict when you link. You can reuse the code via .include without even having to scope it. The name conflict only comes up when you have to use the duplicated code across different assemblies. Just .include the duplicate code where needed, sort of like a C++ inline implementation.

User avatar
NOOPr
Posts: 64
Joined: Tue Feb 27, 2018 10:41 am
Location: Brazil
Contact:

Re: utility: Fixed Bank Creator, for use with 32kB PRG banks

Post by NOOPr » Wed Nov 14, 2018 4:24 am

Just to mention, this is an excellent option for those using cc65/ca65

User avatar
tokumaru
Posts: 11466
Joined: Sat Feb 12, 2005 9:43 pm
Location: Rio de Janeiro - Brazil

Re: utility: Fixed Bank Creator, for use with 32kB PRG banks

Post by tokumaru » Wed Nov 14, 2018 8:45 am

rainwarrior wrote:I don't think these other things fit the category of "fixed bank". This is just duplicate code, not something that needs to be at a fixed location.
I guess you could have the same code replicated in different addresses if a trampoline in each bank called the correct address for that bank, but calling conventions that don't use a dedicated trampoline (e.g. target address in RAM, or in ROM after the JSR to a generic trampoline) would require the repeated routines to be in the same place, otherwise you'd need a more complex mechanism to select an address based on the bank. For that and other consistency reasons, such as page alignment so that there are no variations in timing across different banks, it makes much more sense to have all instances of duplicated code run from the same address.

User avatar
gauauu
Posts: 659
Joined: Sat Jan 09, 2016 9:21 pm
Location: Central Illinois, USA
Contact:

Re: utility: Fixed Bank Creator, for use with 32kB PRG banks

Post by gauauu » Wed Nov 14, 2018 9:34 am

Thanks for making this!

Something that would make this cooler (but significantly more complicated) would be if, instead of specifying an address range, you could specify a ca65 segment. Then the program would read through the generated map file and automatically figure out what it needed to do.

Something like:

Code: Select all

fixbank.exe --by-segment FIXEDBANK [mapfile] [.NES file]
Of course, I guess I could just pretty easily write a wrapper scrap (in something like python that makes string parsing a little easier than C) that looks up the bank and then calls your utility, so I can't complain.

I could see this also being useful for the eternal question of how to handle multiple banks with C, where it needs to call C runtime functions. A post-processor like this would really simplify the question of how to trick the linker into putting the C runtime everywhere it needs to be.

User avatar
Memblers
Site Admin
Posts: 3770
Joined: Mon Sep 20, 2004 6:04 am
Location: Indianapolis
Contact:

Re: utility: Fixed Bank Creator, for use with 32kB PRG banks

Post by Memblers » Thu Nov 15, 2018 11:05 pm

Had a little copy+paste error in my code, added v1.1 to the first post. bug causes the "data will be overwritten!" warning to ensure you'll have a bad result.

Yeah when I was writing it I had started to consider cases where you'd have code in some banks and not others (like music replay engine in the music data banks), but specifying all that pretty much requires making some kind of per-project config file, and figured it's best to keep it simple for now, and simply copy into every bank.

I like the idea of having it read a CA65 map file to detect things. In another thread, I believe tokumaru showed a way to make a segment auto-aligned/justified to the end of memory. Parsing the map file would be most useful in that situation.

User avatar
koitsu
Posts: 4216
Joined: Sun Sep 19, 2004 9:28 pm
Location: A world gone mad

Re: utility: Fixed Bank Creator, for use with 32kB PRG banks

Post by koitsu » Thu Nov 15, 2018 11:21 pm

gauauu wrote:Of course, I guess I could just pretty easily write a wrapper scrap (in something like python that makes string parsing a little easier than C) that looks up the bank and then calls your utility, so I can't complain.

I could see this also being useful for the eternal question of how to handle multiple banks with C, where it needs to call C runtime functions. A post-processor like this would really simplify the question of how to trick the linker into putting the C runtime everywhere it needs to be.
I've said this before in other threads and I'll say it again: if you use Python, you've now ensured nobody except Linux and OS X users are going to use the tool. You can essentially throw away 95% of your user base. Suddenly there's a "wrapper script" that's included, dangling in front of the face of the non-*IX user (alongside a C program and a Windows .exe), while the tool that generates the said map file is a binary they're already using. Yeah, this is kinda insulting to the developer.

I'm dealing with this exact situation right now, where because NES tool authors seem to be all over the board with their PL use and what OSes they run, I'm forced to use VMs else bare metal systems (some FreeBSD, some Linux), and have to copy files between systems manually. It's a complete waste of a person's time.

In short: either provide statically-linked Win32 .exe files of the Python program (good luck! If it can be done, then awesome, problem solved!) alongside the code, use another PL that can do this (there are many), or just use C.

And no, string parsing in C for this particular purpose (parsing a ca65 map file) is not difficult. fscanf()/sscanf() and strtoul() are not advanced topics. This present-day fear of C is actually more scary than the language itself. If complex data structures were involved here, oh yes, I would definitely be scared, but they aren't.

User avatar
Banshaku
Posts: 2328
Joined: Tue Jun 24, 2008 8:38 pm
Location: Fukuoka, Japan
Contact:

Re: utility: Fixed Bank Creator, for use with 32kB PRG banks

Post by Banshaku » Fri Nov 16, 2018 12:16 am

The only time on window that it's less an issue is if you use win10 with the linux subsystem but not everyone do that so I can understand your pain.

tepples
Posts: 21754
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Re: utility: Fixed Bank Creator, for use with 32kB PRG banks

Post by tepples » Fri Nov 16, 2018 7:41 am

koitsu wrote:In short: either provide statically-linked Win32 .exe files of the Python program (good luck! If it can be done, then awesome, problem solved!) alongside the code, use another PL that can do this (there are many), or just use C.
Would it be acceptable to distribute the tools in source code form without charge but charge for the Windows and macOS executables as a way of recovering $119.99 for a Windows 10 license and $799 for a Mac mini computer (which is the least expensive way of obtaining a new macOS 10.14 license) with which to build and test the executables? True, one can cross-compile a Windows or macOS application using GNU/Linux, and even partially cross-test a Windows application with Wine, but recent experience with focus bugs in BGB (a Game Boy emulator) in Wine 3 that don't happen in Windows or in Wine 1 has reduced my faith in Wine as representative of the behavior of Windows, and there's no way to cross-test a macOS binary because GNUstep was never designed to be binary compatible.

User avatar
gauauu
Posts: 659
Joined: Sat Jan 09, 2016 9:21 pm
Location: Central Illinois, USA
Contact:

Re: utility: Fixed Bank Creator, for use with 32kB PRG banks

Post by gauauu » Fri Nov 16, 2018 8:13 am

koitsu wrote: I've said this before in other threads and I'll say it again: if you use Python, you've now ensured nobody except Linux and OS X users are going to use the tool. You can essentially throw away 95% of your user base. Suddenly there's a "wrapper script" that's included, dangling in front of the face of the non-*IX user (alongside a C program and a Windows .exe), while the tool that generates the said map file is a binary they're already using. Yeah, this is kinda insulting to the developer.
If I'm writing a weird wrapper script instead of modifying his source, I'm doing it for my own use-case, so I don't care about supporting windows. Linux is 100% of my target user base (me).
In short: either provide statically-linked Win32 .exe files of the Python program (good luck! If it can be done, then awesome, problem solved!) alongside the code, use another PL that can do this (there are many), or just use C.
I agree that if you're really targeting Windows users, providing an .exe makes the most sense, but Windows users CAN install python, just like Linux users CAN install Wine. I've never heard a developer complain about python being hard to install. In fact, most developers I know find it easier to install python than to compile an arbitrary C program into a windows command-line executable.

yaros
Posts: 145
Joined: Tue Aug 28, 2018 8:54 am
Location: Edmonton, Canada

Re: utility: Fixed Bank Creator, for use with 32kB PRG banks

Post by yaros » Fri Nov 16, 2018 10:27 am

gauauu wrote: I agree that if you're really targeting Windows users, providing an .exe makes the most sense, but Windows users CAN install python, just like Linux users CAN install Wine. I've never heard a developer complain about python being hard to install. In fact, most developers I know find it easier to install python than to compile an arbitrary C program into a windows command-line executable.
Yeah, I find it weird too. I don't develop in python, but I always have it installed on my machine for other people's scripts. Never had a problem. It might be an issue with people new to NES _and_ to programming in general. Because console is not what windows users usually use. But really, if you are digging into NES, might as well install Python and learn the console interface.

I personally write my scripts in C#, just because I know this language well, and use LINQPad as script runner, but those are for my personal use anyway. Although could be compiled to mono-compatible exe files.

User avatar
koitsu
Posts: 4216
Joined: Sun Sep 19, 2004 9:28 pm
Location: A world gone mad

Re: utility: Fixed Bank Creator, for use with 32kB PRG banks

Post by koitsu » Fri Nov 16, 2018 11:45 am

gauauu wrote:I agree that if you're really targeting Windows users, providing an .exe makes the most sense, but Windows users CAN install python, just like Linux users CAN install Wine. I've never heard a developer complain about python being hard to install. In fact, most developers I know find it easier to install python than to compile an arbitrary C program into a windows command-line executable.
You've never seen what happens if you install Python system-wide on Windows (specifically the involvement of Python-centric environment variables), and then use Python-using binaries of commercial apps at the same time (ex. Dropbox, EVE, etc.), have you? :-) "Just install the PL" is not really feasible; *IX is not Windows, and Windows is not *IX.

tepples
Posts: 21754
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Re: utility: Fixed Bank Creator, for use with 32kB PRG banks

Post by tepples » Fri Nov 16, 2018 11:53 am

In Windows PCs that I've used, I've had no problem using both Python.org Python and Dropbox, as Dropbox successfully sandboxes its embedded copy of Python so as not to interfere with a system-wide installation of Python.org Python. I haven't played EVE Online however.

Post Reply