It is currently Tue Jul 23, 2019 8:40 am

All times are UTC - 7 hours





Post new topic Reply to topic  [ 25 posts ]  Go to page 1, 2  Next
Author Message
PostPosted: Wed Nov 14, 2018 12:19 am 
Offline
Site Admin
User avatar

Joined: Mon Sep 20, 2004 6:04 am
Posts: 3697
Location: Indianapolis
Since I was writing some GTROM documentation, and wanting to include templates for a few different assemblers, I decided to include a simple utility for duplicating data among 32kB banks, rather than complicate the source code for the templates. I'll be using it myself with CA65. I don't know if anyone else wants it, but here it anyways. C source is included.

edit - bugfix v1.1 added 11-16-2018

Code:
Fixed Bank Creator
    by Memblers
   
V1.0 - 11-14-2018 - initial release
v1.1 - 11-16-2018 - bugfix: blank check fail warning would corrupt source
             bank selector

-- Introduction --

Fixed Bank Creator is a utility designed for NES developers who are using a
mapper that bankswitches PRG-ROM in 32kB pages.  It will allow you to take
code/data from one bank, and duplicate it to the same address in every bank. 
You could do this with your assembler/linker, but the methods for doing so
vary by assembler and can become convoluted, with issues such as having
conflicting label names within the same scope.  Because this program operates
on a binary file, it doesn't matter which assembler you are using.

Whether you've already developed with 32kB banks and are tired of dicking
around with an overly-complicated build process, or you are a UNROM veteran
looking to shrink your fixed bank and free up more memory on boards such as
GTROM, it is hoped that this program could be useful to you.

-- Usage --

Fixed Bank Creator is a command-line program, intended to be run as part of
your normal ROM-building process.  You specify the input ROM file in .NES
format, address range for the desired fixed bank, and (optionally) an output
filename.  You do not need to specify a source bank, as the program will
detect which bank already has data within that range.  If data exists within
that range in multiple banks, a warning message will be output.

    fixbank.exe [.NES file] [begin address] [end address] [optional filename]
   
Usage examples:

    fixbank hotseat2.nes $C000 $FFFF
        Creates a UNROM-style 16kB fixed bank, saved as "output.nes"
   
    fixbank hatetank.nes FFFA FFFF romtest.nes
        Duplicates the vectors only, saved as "romtest.nes"
   
    fixbank romtest.nes 0xE000 0xE7FF
        Duplicates 2kB data, DPCM samples for example, saved as "output.nes"
      
      
When creating your ROM to use the program, what you are supposed to do is
choose the region of memory (anywhere in $8000-$FFFF) that you would like to
make into the fixed bank.  Organizing your program into 32kB banks, you then
leave that region empty (padded with $00 or $FF bytes) in every bank, except
for one of them.  The contents of that one bank will then be duplicated.
       
-- Notes and Tips --

Warning, Error, and Info messages are output to the screen.  The warning
messages should not be ignored, as they are likely indicators of a
non-working result.

'$F000', 'F000', and '0xF000' are all valid input syntax for this program.

After the source bank has been detected, the rest of the banks are scanned to
check if they are blank.  If they are not, a warning is output and data will
be overwritten.

Blank check requires memory to be padded with either $00 or $FF.  If you use
$FF, FlashROM and EPROM chips may be slightly faster to program, as $FF is
the typical blank state of non-volatile memory.

Searching for source data bank begins in the lowest bank.  It's recommended
that you put your fixed bank source data in the lowest (first) bank.

It's best if you set up your assembler to reserve memory in your fixed bank
area, as the error messages from your assembler will likely be more helpful
than the warning messages from this program.

If you need to create multiple "mirrored" parts of memory, just run the
program again on the output file.

This program is free and open source under the MIT license.  Additionally,
any discordian pope may automatically grant themselves a license under the
WTFPL, as desired.

-- Known Issues --

The source bank detection will fail if you were to create a bank that
consists entirely of $00 and/or $FF bytes.  If, for some reason, you wanted
to have nothing but patterns of $00 and $FF bytes in your fixed bank, this is
bad news.


Attachments:
File comment: fixbank v1.1 (bugfix)
fixbank.zip [21.8 KiB]
Downloaded 176 times
Top
 Profile  
 
PostPosted: Wed Nov 14, 2018 12:44 am 
Offline
User avatar

Joined: Tue Jun 24, 2008 8:38 pm
Posts: 2304
Location: Fukuoka, Japan
I never had to manage 32kB banks but there is one thing I would like clarification since maybe some of my assumption were wrong for ca65.

If I write code in assembler with ca65, create a scope then include a file in that scope, wouldn't that remove the label clash and allow the code to be exactly at the same place? I never tried but this is what I would do first if I had this issue.

I may try someday to see how the assembler react, out of curiosity and wanting to know more about it.


Top
 Profile  
 
PostPosted: Wed Nov 14, 2018 1:38 am 
Offline
User avatar

Joined: Sat Feb 12, 2005 9:43 pm
Posts: 11376
Location: Rio de Janeiro - Brazil
Banshaku wrote:
If I write code in assembler with ca65, create a scope then include a file in that scope, wouldn't that remove the label clash and allow the code to be exactly at the same place? I never tried but this is what I would do first if I had this issue.

Wrapping things in scopes does indeed solve the label clash problems, but there are a few catches. I tried creating a macro to automate this once, but it didn't work. Macros are kinda mystical in ca65, I guess. But it works if you do it in other ways. What I ended up doing was creating my own implementation of PROC, which checks whether the entry point label is already defined or not, to decide between creating a named scope or an unnamed one. With this I didn't have to explicitly wrap stuff in scopes, I simply included the subroutines multiple times and the custom PROC macro took care of creating global labels only the first time.

Anyway, Memblers' idea is pretty cool, but unfortunately it doesn't solve the problems I have with virtual fixed banks. First, having to know the address means I'd still have to change settings manually whenever I added/removed things to/from the fixed bank. I guess I could set a larger size and only shrink it at a later time, when the fixed stuff was probably already set in stone and the lower portions of the banks were getting full. But the other problem for me is that not all fixed banks are equal: while some things are present in all banks, other things are repeated only in specific banks. Banks with CHR data have code to copy/decode that to VRAM, banks with level maps have collision detection routines, stuff like that. It's kinda like another fixed bank below the general fixed bank. Multiple passes of Memblers' tool could maybe handle that, but again I'd have to know all the addresses beforehand and constantly change the settings during development.


Top
 Profile  
 
PostPosted: Wed Nov 14, 2018 3:02 am 
Offline
User avatar

Joined: Sun Jan 22, 2012 12:03 pm
Posts: 7528
Location: Canada
tokumaru wrote:
But the other problem for me is that not all fixed banks are equal: while some things are present in all banks, other things are repeated only in specific banks. Banks with CHR data have code to copy/decode that to VRAM, banks with level maps have collision detection routines, stuff like that. It's kinda like another fixed bank below the general fixed bank. Multiple passes of Memblers' tool could maybe handle that, but again I'd have to know all the addresses beforehand and constantly change the settings during development.

I don't think these other things fit the category of "fixed bank". This is just duplicate code, not something that needs to be at a fixed location. The problem relaxes a bit if you don't include the additional constraint of putting it at the same location.

Stuff at a fixed location can be dealt with in a various ways (e.g. tepples' suggested this "workaround" recently, other possibilities have been discussed in the past, and now this tool is being offered, etc.). Relocatable code that's duplicated in some banks is an easier problem.

For instance, probably the simplest case: in ca65, if the duplicated code is in different modules (i.e. different assembles), as long as you don't export the symbols there is no duplicate name conflict when you link. You can reuse the code via .include without even having to scope it. The name conflict only comes up when you have to use the duplicated code across different assemblies. Just .include the duplicate code where needed, sort of like a C++ inline implementation.


Top
 Profile  
 
PostPosted: Wed Nov 14, 2018 4:24 am 
Offline
User avatar

Joined: Tue Feb 27, 2018 10:41 am
Posts: 49
Location: Brazil
Just to mention, this is an excellent option for those using cc65/ca65

_________________
itch.io | github


Top
 Profile  
 
PostPosted: Wed Nov 14, 2018 8:45 am 
Offline
User avatar

Joined: Sat Feb 12, 2005 9:43 pm
Posts: 11376
Location: Rio de Janeiro - Brazil
rainwarrior wrote:
I don't think these other things fit the category of "fixed bank". This is just duplicate code, not something that needs to be at a fixed location.

I guess you could have the same code replicated in different addresses if a trampoline in each bank called the correct address for that bank, but calling conventions that don't use a dedicated trampoline (e.g. target address in RAM, or in ROM after the JSR to a generic trampoline) would require the repeated routines to be in the same place, otherwise you'd need a more complex mechanism to select an address based on the bank. For that and other consistency reasons, such as page alignment so that there are no variations in timing across different banks, it makes much more sense to have all instances of duplicated code run from the same address.


Top
 Profile  
 
PostPosted: Wed Nov 14, 2018 9:34 am 
Offline
User avatar

Joined: Sat Jan 09, 2016 9:21 pm
Posts: 619
Location: Central Illinois, USA
Thanks for making this!

Something that would make this cooler (but significantly more complicated) would be if, instead of specifying an address range, you could specify a ca65 segment. Then the program would read through the generated map file and automatically figure out what it needed to do.

Something like:

Code:
fixbank.exe --by-segment FIXEDBANK [mapfile] [.NES file]


Of course, I guess I could just pretty easily write a wrapper scrap (in something like python that makes string parsing a little easier than C) that looks up the bank and then calls your utility, so I can't complain.

I could see this also being useful for the eternal question of how to handle multiple banks with C, where it needs to call C runtime functions. A post-processor like this would really simplify the question of how to trick the linker into putting the C runtime everywhere it needs to be.

_________________
My games: http://www.bitethechili.com


Top
 Profile  
 
PostPosted: Thu Nov 15, 2018 11:05 pm 
Offline
Site Admin
User avatar

Joined: Mon Sep 20, 2004 6:04 am
Posts: 3697
Location: Indianapolis
Had a little copy+paste error in my code, added v1.1 to the first post. bug causes the "data will be overwritten!" warning to ensure you'll have a bad result.

Yeah when I was writing it I had started to consider cases where you'd have code in some banks and not others (like music replay engine in the music data banks), but specifying all that pretty much requires making some kind of per-project config file, and figured it's best to keep it simple for now, and simply copy into every bank.

I like the idea of having it read a CA65 map file to detect things. In another thread, I believe tokumaru showed a way to make a segment auto-aligned/justified to the end of memory. Parsing the map file would be most useful in that situation.


Top
 Profile  
 
PostPosted: Thu Nov 15, 2018 11:21 pm 
Offline
User avatar

Joined: Sun Sep 19, 2004 9:28 pm
Posts: 4156
Location: A world gone mad
gauauu wrote:
Of course, I guess I could just pretty easily write a wrapper scrap (in something like python that makes string parsing a little easier than C) that looks up the bank and then calls your utility, so I can't complain.

I could see this also being useful for the eternal question of how to handle multiple banks with C, where it needs to call C runtime functions. A post-processor like this would really simplify the question of how to trick the linker into putting the C runtime everywhere it needs to be.

I've said this before in other threads and I'll say it again: if you use Python, you've now ensured nobody except Linux and OS X users are going to use the tool. You can essentially throw away 95% of your user base. Suddenly there's a "wrapper script" that's included, dangling in front of the face of the non-*IX user (alongside a C program and a Windows .exe), while the tool that generates the said map file is a binary they're already using. Yeah, this is kinda insulting to the developer.

I'm dealing with this exact situation right now, where because NES tool authors seem to be all over the board with their PL use and what OSes they run, I'm forced to use VMs else bare metal systems (some FreeBSD, some Linux), and have to copy files between systems manually. It's a complete waste of a person's time.

In short: either provide statically-linked Win32 .exe files of the Python program (good luck! If it can be done, then awesome, problem solved!) alongside the code, use another PL that can do this (there are many), or just use C.

And no, string parsing in C for this particular purpose (parsing a ca65 map file) is not difficult. fscanf()/sscanf() and strtoul() are not advanced topics. This present-day fear of C is actually more scary than the language itself. If complex data structures were involved here, oh yes, I would definitely be scared, but they aren't.


Top
 Profile  
 
PostPosted: Fri Nov 16, 2018 12:16 am 
Offline
User avatar

Joined: Tue Jun 24, 2008 8:38 pm
Posts: 2304
Location: Fukuoka, Japan
The only time on window that it's less an issue is if you use win10 with the linux subsystem but not everyone do that so I can understand your pain.


Top
 Profile  
 
PostPosted: Fri Nov 16, 2018 7:41 am 
Offline

Joined: Sun Sep 19, 2004 11:12 pm
Posts: 21511
Location: NE Indiana, USA (NTSC)
koitsu wrote:
In short: either provide statically-linked Win32 .exe files of the Python program (good luck! If it can be done, then awesome, problem solved!) alongside the code, use another PL that can do this (there are many), or just use C.

Would it be acceptable to distribute the tools in source code form without charge but charge for the Windows and macOS executables as a way of recovering $119.99 for a Windows 10 license and $799 for a Mac mini computer (which is the least expensive way of obtaining a new macOS 10.14 license) with which to build and test the executables? True, one can cross-compile a Windows or macOS application using GNU/Linux, and even partially cross-test a Windows application with Wine, but recent experience with focus bugs in BGB (a Game Boy emulator) in Wine 3 that don't happen in Windows or in Wine 1 has reduced my faith in Wine as representative of the behavior of Windows, and there's no way to cross-test a macOS binary because GNUstep was never designed to be binary compatible.

_________________
Pin Eight | Twitter | GitHub | Patreon


Top
 Profile  
 
PostPosted: Fri Nov 16, 2018 8:13 am 
Offline
User avatar

Joined: Sat Jan 09, 2016 9:21 pm
Posts: 619
Location: Central Illinois, USA
koitsu wrote:
I've said this before in other threads and I'll say it again: if you use Python, you've now ensured nobody except Linux and OS X users are going to use the tool. You can essentially throw away 95% of your user base. Suddenly there's a "wrapper script" that's included, dangling in front of the face of the non-*IX user (alongside a C program and a Windows .exe), while the tool that generates the said map file is a binary they're already using. Yeah, this is kinda insulting to the developer.


If I'm writing a weird wrapper script instead of modifying his source, I'm doing it for my own use-case, so I don't care about supporting windows. Linux is 100% of my target user base (me).

Quote:
In short: either provide statically-linked Win32 .exe files of the Python program (good luck! If it can be done, then awesome, problem solved!) alongside the code, use another PL that can do this (there are many), or just use C.


I agree that if you're really targeting Windows users, providing an .exe makes the most sense, but Windows users CAN install python, just like Linux users CAN install Wine. I've never heard a developer complain about python being hard to install. In fact, most developers I know find it easier to install python than to compile an arbitrary C program into a windows command-line executable.

_________________
My games: http://www.bitethechili.com


Top
 Profile  
 
PostPosted: Fri Nov 16, 2018 10:27 am 
Offline

Joined: Tue Aug 28, 2018 8:54 am
Posts: 143
Location: Edmonton, Canada
gauauu wrote:
I agree that if you're really targeting Windows users, providing an .exe makes the most sense, but Windows users CAN install python, just like Linux users CAN install Wine. I've never heard a developer complain about python being hard to install. In fact, most developers I know find it easier to install python than to compile an arbitrary C program into a windows command-line executable.


Yeah, I find it weird too. I don't develop in python, but I always have it installed on my machine for other people's scripts. Never had a problem. It might be an issue with people new to NES _and_ to programming in general. Because console is not what windows users usually use. But really, if you are digging into NES, might as well install Python and learn the console interface.

I personally write my scripts in C#, just because I know this language well, and use LINQPad as script runner, but those are for my personal use anyway. Although could be compiled to mono-compatible exe files.


Top
 Profile  
 
PostPosted: Fri Nov 16, 2018 11:45 am 
Offline
User avatar

Joined: Sun Sep 19, 2004 9:28 pm
Posts: 4156
Location: A world gone mad
gauauu wrote:
I agree that if you're really targeting Windows users, providing an .exe makes the most sense, but Windows users CAN install python, just like Linux users CAN install Wine. I've never heard a developer complain about python being hard to install. In fact, most developers I know find it easier to install python than to compile an arbitrary C program into a windows command-line executable.

You've never seen what happens if you install Python system-wide on Windows (specifically the involvement of Python-centric environment variables), and then use Python-using binaries of commercial apps at the same time (ex. Dropbox, EVE, etc.), have you? :-) "Just install the PL" is not really feasible; *IX is not Windows, and Windows is not *IX.


Top
 Profile  
 
PostPosted: Fri Nov 16, 2018 11:53 am 
Offline

Joined: Sun Sep 19, 2004 11:12 pm
Posts: 21511
Location: NE Indiana, USA (NTSC)
In Windows PCs that I've used, I've had no problem using both Python.org Python and Dropbox, as Dropbox successfully sandboxes its embedded copy of Python so as not to interfere with a system-wide installation of Python.org Python. I haven't played EVE Online however.

_________________
Pin Eight | Twitter | GitHub | Patreon


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 25 posts ]  Go to page 1, 2  Next

All times are UTC - 7 hours


Who is online

Users browsing this forum: No registered users and 3 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB® Forum Software © phpBB Group