How can we keep our NES development resources around?

Discuss technical or other issues relating to programming the Nintendo Entertainment System, Famicom, or compatible systems.

Moderator: Moderators

User avatar
dougeff
Posts: 2775
Joined: Fri May 08, 2015 7:17 pm
Location: DIGDUG
Contact:

Re: How can we keep our NES development resources around?

Post by dougeff » Sat Oct 19, 2019 4:24 pm

We know that NA as it exists today is set to disappear at the END OF THIS MONTH. OCTOBER 30TH.

So, direct and to the point, what information on NintendoAge, AtariAge, etc do we want to back up?

Oh crap.

Uh, the entire programming thread needs to be saved.
nesdoug.com -- blog/tutorial on programming for the NES

cppchriscpp
Posts: 93
Joined: Fri Dec 27, 2013 4:28 pm

Re: How can we keep our NES development resources around?

Post by cppchriscpp » Sat Oct 19, 2019 6:52 pm

Added to the nerdy nights backup, alongside a thread on atariage about the 6502. (And a hex editing page I had been scraping but forgot to expose before >_>) These are in a new "miscellaneous" section.

This is probably starting to push the limits for what I should store on a single html page. for now it works, though.
Last edited by cppchriscpp on Tue Oct 22, 2019 7:27 pm, edited 1 time in total.

User avatar
Memblers
Site Admin
Posts: 3890
Joined: Mon Sep 20, 2004 6:04 am
Location: Indianapolis
Contact:

Re: How can we keep our NES development resources around?

Post by Memblers » Sun Oct 20, 2019 2:07 pm

Yikes. I wasn't feeling too optimistic about NA's future after it was sold, but I didn't expect it to (possibly) die off that fast. Not sure if it makes it harder to manage and migrate, but from what I understand, the entire site/forum/database was custom made by the site founder.

And also, despite the name, AtariAge is not related to NintendoAge. What is related though is SegaAge, and I've heard there's a PlaystationAge, I've never visited either of those though.

8bitMicroGuy
Posts: 314
Joined: Sun Mar 08, 2015 12:23 pm
Location: Croatia

Re: How can we keep our NES development resources around?

Post by 8bitMicroGuy » Sun Oct 20, 2019 6:50 pm

If everything fails from those GitHub stuff, just use a web-scrapping algorithm! I think that Chrome has one of such plugins. I forgot how it's named. Should work on Opera considering that today's Opera has Chrome's guts.

I think you should start working on the web-scrapping approach now. I cannot help much because I suck so bad at HTML/CSS/JS that I might break something in the process.

tepples
Posts: 22161
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Re: How can we keep our NES development resources around?

Post by tepples » Mon Oct 21, 2019 10:36 pm

BB1 is preserved as static HTML. BB2 appears to be lost in the sense that the backup I made disappeared with Parodius. All I have left on my drive is a proof of concept of the SNES and GB sections, and then I ran the full scrape in a shell account on the server so as not to waste the web server's upstream bandwidth. And that server has been decommissioned for seven years. If you have a file called bb2slim.css anywhere in your archives, you may have BB2 and/or BB1. However, I don't count it as too much of a loss, as all of its posts were folded into BB3. I'm running a second scrape of BB3 that includes links to attachments so that I can preserve those.

There's more of a problem with the NESdev list on Yahoo! Groups. Membership in the list requires approval from the administrator, and I doubt that the list's administrator even remembers it. The content remotely worth saving is from 1999 through 2005, with only five messages (a notice that BB2 was down momentarily and some spam) after October 2005. ArchiveTeam can archive public lists but not lists that require membership approval without help from a member. Today I was able to restore access to my Yahoo! account, and sometime this week, I plan to get in touch with a member of #archiveteam on EFnet to figure out how to get it preserved.

That said, as for my own NES projects, I have six things left that I deem suitable for preservation:
  • Add Who's Cuter (requires new images)
  • Add Forbidden Four
  • Replace Sprite Cans music
  • Create repository for Cover Your Own (requires new book)
  • Create repository for President
  • Create repository for full-screen book reader
UPDATE (a few hours later): I have used IgnoredAmbience's archiver (as recommended by balrog in an ArchiveTeam-related channel on EFnet) to grab all 3,570 messages as JSON files and have saved them to a .7z file in my Dropbox. I might have something to show once I can turn them into HTML.

cppchriscpp
Posts: 93
Joined: Fri Dec 27, 2013 4:28 pm

Re: How can we keep our NES development resources around?

Post by cppchriscpp » Tue Oct 22, 2019 7:25 pm

Alright, time for another update...

I've decided to fork off a section of my site for things like this: http://archive.nes.science - I'll expand this as needed. Right now it has:
  • nerdy nights
  • the nesdev wiki
  • the nesdev forums.
(I have one more thing that may go up in the next couple days, but it's very large and my internet can't get much above 700KiB/s up)

I'd like to add the yahoo group too. tepples, It seems like you've already got a large list you're focused on. I don't mind whipping up a viewer and/or converter if you'd rather share the json. (If you want to convert it yourself, by all means! Just don't feel like you have to) I'm really glad that's done - thank you for getting it downloaded!

For the things I host that are still alive (forums and the nesdev wiki, mainly) I'll try to rig something up to update them on a regular schedule. (No ETA on that though)

User avatar
Electronoob
Posts: 11
Joined: Mon Sep 29, 2014 3:28 pm

Re: How can we keep our NES development resources around?

Post by Electronoob » Thu Nov 07, 2019 4:44 am

https://mega.nz/#F!EYxkkQQb!yA9eUqvDdV-EcLkznu0IVw

# nintendoage-mirror-wget

im hopeful but havent confirmed if this is a complete dump of the forum before it closed. wget was spidering the site even after the dns was moved over away from the old server etc. I'm assuming this is fine.

due to github file restrictions i had to gzip the archive.

how i compressed

# tar -czf - ./nintendoage.com | split -b 99M - archive.tar.gz

how to decompress

# cat archive.tar.* | tar -xzf -

To clarify that snapshot is done using wget. I probably used this commandline:
wget -mkEpnp http://nintendoage.com/forum/

although i packaged it for github i was unable to actually get it uploaded there so using mega.nz instead

User avatar
Electronoob
Posts: 11
Joined: Mon Sep 29, 2014 3:28 pm

Re: How can we keep our NES development resources around?

Post by Electronoob » Thu Nov 07, 2019 5:16 am

in addition to the above wget snapshot i have taken a copy from archive.org:

https://mega.nz/#F!wBonGKBS!HMRD4GB15wiQkd8nx3UEUw

# nintendoage-mirror-from-archive.org

this version of the nintendoage mirror is taken via archive.org's wayback machine.

how i compressed

# tar -czf - ./nintendoage.com | split -b 99M - archive.tar.gz

how to decompress

# cat archive.tar.* | tar -xzf -

User avatar
koitsu
Posts: 4218
Joined: Sun Sep 19, 2004 9:28 pm
Location: A world gone mad

Re: How can we keep our NES development resources around?

Post by koitsu » Sat Nov 09, 2019 7:07 pm

tepples wrote:BB1 is preserved as static HTML. BB2 appears to be lost in the sense that the backup I made disappeared with Parodius.
I have backups (yes, still, after 7 years). Give me details (as I do not understand what the above underlined sentence means/refers to) and I can very likely get you whatever you made/did (assuming it was in the UNIX account as of August 14, 2012). Please DM me on Discord rather than reply here, as I do not frequent the forum as often (almost never).

Post Reply