Interest in new mapper document?

Discuss technical or other issues relating to programming the Nintendo Entertainment System, Famicom, or compatible systems.

Moderator: Moderators

Post Reply
eyebreakstuff
Posts: 5
Joined: Sat Mar 20, 2021 6:49 am

Interest in new mapper document?

Post by eyebreakstuff » Sat Mar 20, 2021 6:57 am

Not too sure if someone has done anything like this before or if there is any interest at all, but all the entries for the mappers for each NES carts I could find were the same text file. So, I took that and (I believe) accurately converted the raw text into csv, xlxs and json. If there's interest I would be happy to share (assuming that doesn't violate any terms).

lidnariq
Posts: 10459
Joined: Sun Apr 13, 2008 11:12 am
Location: Seattle

Re: Interest in new mapper document?

Post by lidnariq » Sat Mar 20, 2021 12:28 pm

You should really use our wiki instead of Firebug's and/or Disch's documents.

https://wiki.nesdev.com

User avatar
Controllerhead
Posts: 241
Joined: Tue Nov 13, 2018 4:58 am
Location: $4016
Contact:

Re: Interest in new mapper document?

Post by Controllerhead » Sat Mar 20, 2021 1:40 pm

I'm working on something similar with the goal of a universal ASM6F template... and eventually more goals...
http://forums.nesdev.com/viewtopic.php? ... 92#p266362

I'd be interested to see what you came up with.
Image

eyebreakstuff
Posts: 5
Joined: Sat Mar 20, 2021 6:49 am

Re: Interest in new mapper document?

Post by eyebreakstuff » Sat Mar 20, 2021 2:08 pm

lidnariq wrote:
Sat Mar 20, 2021 12:28 pm
You should really use our wiki instead of Firebug's and/or Disch's documents.

https://wiki.nesdev.com
So that was my original plan, only I wasn't able to get a dump that i could manage nearly as easily (oddly) as the TXT file, which I have successfully converted to JSON and CSV for use in my powershell script. My concern would be with with accuracy of the data

lidnariq
Posts: 10459
Joined: Sun Apr 13, 2008 11:12 am
Location: Seattle

Re: Interest in new mapper document?

Post by lidnariq » Sat Mar 20, 2021 2:13 pm

Yes, there's a bunch of errors we've fixed in the multiple-decades-old mappers docs.

You can download a raw ("wikitext") version using the mediawiki API - mediawikiwiki: API:Get the contents of a page

User avatar
Controllerhead
Posts: 241
Joined: Tue Nov 13, 2018 4:58 am
Location: $4016
Contact:

Re: Interest in new mapper document?

Post by Controllerhead » Sat Mar 20, 2021 2:43 pm

lidnariq wrote:
Sat Mar 20, 2021 2:13 pm
Yes, there's a bunch of errors we've fixed in the multiple-decades-old mappers docs.

You can download a raw ("wikitext") version using the mediawiki API - mediawikiwiki: API:Get the contents of a page
In that case, i've already "sanitized" a bunch of this data and have a work in progress JSON file. Here:
https://nesblast.com/NEStemplate/
https://nesblast.com/NEStemplate/mapper_data.json

Please note this work is incomplete. I don't like releasing incomplete things into the wild, but, it would be much easier than starting from scratch and (likely) repeating the same work that i have already done. Enjoy.
Last edited by Controllerhead on Sat Mar 20, 2021 3:25 pm, edited 1 time in total.
Image

tepples
Posts: 22333
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Re: Interest in new mapper document?

Post by tepples » Sat Mar 20, 2021 2:45 pm

Occasionally I upload a dump of the wiki to my NESdev Wiki mirror.

eyebreakstuff
Posts: 5
Joined: Sat Mar 20, 2021 6:49 am

Re: Interest in new mapper document?

Post by eyebreakstuff » Sat Mar 20, 2021 5:07 pm

Awesome, thanks all for the resources, these will work great

Pokun
Posts: 1822
Joined: Tue May 28, 2013 5:49 am
Location: Hokkaido, Japan

Re: Interest in new mapper document?

Post by Pokun » Sun Mar 21, 2021 7:11 am

Controllerhead wrote:
Sat Mar 20, 2021 2:43 pm
i've already "sanitized" a bunch of this data and have a work in progress JSON file. Here:
https://nesblast.com/NEStemplate/
https://nesblast.com/NEStemplate/mapper_data.json

Please note this work is incomplete. I don't like releasing incomplete things into the wild, but, it would be much easier than starting from scratch and (likely) repeating the same work that i have already done. Enjoy.
This is great stuff. I think anything that visually shows things is much better than only text.

eyebreakstuff
Posts: 5
Joined: Sat Mar 20, 2021 6:49 am

Re: Interest in new mapper document?

Post by eyebreakstuff » Sun Mar 21, 2021 12:37 pm

If there is any interest in my json or my Powershell script, see below. Note this can work with any json file assuming it has the correct headers. If there's any interest, I may expand to include other consoles. This was mainly a POC/test project for generating an intractable GUI interface in powershell. It's very much a work in progress.

https://github.com/jamjen831/dumpScriptAssist

User avatar
Controllerhead
Posts: 241
Joined: Tue Nov 13, 2018 4:58 am
Location: $4016
Contact:

Re: Interest in new mapper document?

Post by Controllerhead » Sun Mar 21, 2021 3:02 pm

Pokun wrote:
Sun Mar 21, 2021 7:11 am
This is great stuff. I think anything that visually shows things is much better than only text.
Thanks =) I'm glad you enjoyed it!

It makes fighting with HTML Tables and sanitizing much of this data by hand all worth it. I did it mostly to get a birds eye view of the mapper situation myself: it's fun using the up/down arrow keys to cycle through them. I guess i'm glad i didn't let it die a lonely death on my hard drive like thousands of other things =p

It's really all of the hard work and research of the NESdev community here: I just dolled it up a bit.

Unfortunately, much of this data is still a mess, which is why i was reluctant to release it, but, i'm excited to see what eyebreakstuff can come up with as far as automatically parsing what is on the wiki. Writing web crawlers or powershell scripts isn't my specialty per-se, i honestly just threw Lidnariq's HTML tables in a JSON formatter and did the rest by hand: I hope my incomplete work-in-progress here can help.

The wiki is an absolutely wonderful resource, but, from a software perspective, sanitized data is powerful. If we can generate graphics, well, we can generate other things: The eventual plan here is to generate ASM6F templates for mappers: gathering valid hardware configurations as well as register locations to write mapper dependent macros to handle them. Perhaps emulator authors could benefit from a standardized mapper handling format as well.

...It would be great for say, an NES game engine, to support multiple mappers, wouldn't it? =)
Image

eyebreakstuff
Posts: 5
Joined: Sat Mar 20, 2021 6:49 am

Re: Interest in new mapper document?

Post by eyebreakstuff » Thu Mar 25, 2021 8:36 pm

So I love the idea of scraping the wiki, but is I cannot for the life of me find any page(s) that contain this info

lidnariq
Posts: 10459
Joined: Sun Apr 13, 2008 11:12 am
Location: Seattle

Re: Interest in new mapper document?

Post by lidnariq » Thu Mar 25, 2021 8:46 pm

What is "this" info?

Pokun
Posts: 1822
Joined: Tue May 28, 2013 5:49 am
Location: Hokkaido, Japan

Re: Interest in new mapper document?

Post by Pokun » Sun Mar 28, 2021 8:12 am

And why would one want to scrap the best NES development wiki on the internet?

lidnariq
Posts: 10459
Joined: Sun Apr 13, 2008 11:12 am
Location: Seattle

Re: Interest in new mapper document?

Post by lidnariq » Sun Mar 28, 2021 11:13 am

Because english spelling is terrible. (Scraping is the participle of "scrape". "scrapping" would be the participle of "scrap")

Post Reply