Local HTML Version

Discussion about the site's wikis, including bugs/issues encountered.

Moderator: Moderators

6T4
Posts: 48
Joined: Sat Feb 19, 2011 8:56 pm

Local HTML Version

Post by 6T4 »

In the navigation box of the wiki, there is a link to "Local html version", which is a version of the entire wiki that can be downloaded and accessed offline. However, the current file there is from February 2010, over a year ago! In that time, there has been a fair amount added to the wiki, including very useful information about emulator testing and information about more mappers. So, I believe we should update the local version. This wiki is one of the most valuable resources I have found (I'm writing an NES emulator), and I would imagine others feel the same, and would like to have access to a local version. Ideally, it should be updated every time a change is made in the future as well, but if there is a problem with that, I think it should updated at least a few times a year. If an administrator could take care of this, it would be much appreciated.
User avatar
koitsu
Posts: 4201
Joined: Sun Sep 19, 2004 9:28 pm
Location: A world gone mad

Re: Local HTML Version

Post by koitsu »

6T4 wrote:... Ideally, it should be updated every time a change is made in the future as well, but if there is a problem with that, I think it should updated at least a few times a year. If an administrator could take care of this, it would be much appreciated.
1) It should absolutely not be an automatic process done every time someone changes a page. That design model is severely flawed for a lot of reasons which I'd rather not go into here. As the server admin I can assure you I would be a very unhappy camper if I saw something like this implemented.

2) I fully agree that the local HTML version for download should be updated semi-often -- probably once a month.

If one of the other Wiki administrators could let me know how exactly a "local HTML version" is being created to begin with, I could possibly automate this process (run once a month with the archive automatically updated).

I do have one question for you, 6T4:

Is there a particular reason you rely on a local HTML copy of the Wiki? Is there something slow or unacceptable about using the version on the web? What issue, if any, justifies your need for a local copy on your hard disk? If there's something you're experiencing which justifies this, please let us know and I will try to address whatever the core issue is.
User avatar
tokumaru
Posts: 12427
Joined: Sat Feb 12, 2005 9:43 pm
Location: Rio de Janeiro - Brazil

Re: Local HTML Version

Post by tokumaru »

koitsu wrote:Is there a particular reason you rely on a local HTML copy of the Wiki?
I imagine that not everyone has access to the web at all times. I sometimes go to places where I don't have any form of internet connection, and if I wanted to work on something NES-related I imagine it would be helpful to have a local copy of the wiki. I actually have portions of it in a "reference" folder in my projects archive, just in case.
3gengames
Formerly 65024U
Posts: 2284
Joined: Sat Mar 27, 2010 12:57 pm

Re: Local HTML Version

Post by 3gengames »

tokumaru wrote: I imagine that not everyone has access to the web at all times. I sometimes go to places where I don't have any form of internet connection, and if I wanted to work on something NES-related I imagine it would be helpful to have a local copy of the wiki. I actually have portions of it in a "reference" folder in my projects archive, just in case.
This. I reference old code all the time to remember how stuff works and keep a text document of all basic registers I use often. I sometimes have my laptop at my grandmas and other place working on stuff and changing stuff, and it'd be great to have the whole wiki in one place. Where can I even download the current local HTML version?

And I think the whole NESDev site chain is amazingly fast. I don't see any problem with speed. These forums and the wiki are the fastest sites my computer loads.
User avatar
Banshaku
Posts: 2417
Joined: Tue Jun 24, 2008 8:38 pm
Location: Japan
Contact:

Post by Banshaku »

I would be the culprit in that case. I didn't have much time to take care of the wiki these days because of time constraints (kids, you know ^^;; ).

I'm the only one that know how to produce this local version since I'm the one that had the idea in the first and maintained it (somehow) ;)

If my memory is good, it is some kind of command line tool that will extract the complete wiki or a plug-in. But for some reason it doesn't work 100% so some links/css (my memory is quite vague there) and you have to fix a few things by hand.

Maybe I can try to see with Koitsu if it could be automated and someone could launch it every month and replace the main file. For now, since last year, it was done by hand only.

Sorry about that. I will see if I can find some time this week and talk with Koitsu for any possible solution on the subject.

@3gengames:

When you access the wiki the link is in the left pane for easy access. Give it a try, even though it's quite a old version. I found it useful when you didn't have any internet access.
User avatar
Zepper
Formerly Fx3
Posts: 3262
Joined: Fri Nov 12, 2004 4:59 pm
Location: Brazil
Contact:

Post by Zepper »

OP: Well, you could always save a page in your PC as HTML, like the most accessed ones by you.
tepples
Posts: 22705
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Re: Local HTML Version

Post by tepples »

koitsu wrote:Is there a particular reason you rely on a local HTML copy of the Wiki? Is there something slow or unacceptable about using the version on the web?
I do some coding on the bus to and from work, and Citilink in Fort Wayne doesn't provide Wi-Fi. I also do some coding while waiting for the bus, and Glenbrook Square in Fort Wayne doesn't provide Wi-Fi to guests, only to employees. Nor do I currently subscribe to cellular data service. I currently pay $60 per year for mobile phone service and don't want to upgrade to $60 per month.

I could just code up a scraper in Python based on MediaWiki APIs and upload it to ndwiki's FTP account.
User avatar
Banshaku
Posts: 2417
Joined: Tue Jun 24, 2008 8:38 pm
Location: Japan
Contact:

Re: Local HTML Version

Post by Banshaku »

tepples wrote:I could just code up a scraper in Python based on MediaWiki APIs and upload it to ndwiki's FTP account.
I think the best way to go would be to automate what I was using last year. First thing would be to look if there was any update for that thing that could fix the bug. If not, find a way to automate was is broken after the export to html. I will see what I can do once I can talk to Koitsu.
User avatar
koitsu
Posts: 4201
Joined: Sun Sep 19, 2004 9:28 pm
Location: A world gone mad

Post by koitsu »

Banshaku and I are working on this, for the record. It may take us some time to get something automated completed, but rest assured we're working on it despite silence in this thread.
User avatar
Banshaku
Posts: 2417
Joined: Tue Jun 24, 2008 8:38 pm
Location: Japan
Contact:

Post by Banshaku »

For now we couldn't move forward because issue with schedule (it's not easy with the 12h difference). I won't be in front of computer for the golden week vacation. I should be able to figure out something after the 7th of may.
User avatar
Banshaku
Posts: 2417
Joined: Tue Jun 24, 2008 8:38 pm
Location: Japan
Contact:

Post by Banshaku »

There is a new version of the dump available here. This dump is available from the side menu of the wiki like usual.

For now it was done by hand so I could find the current issues in the dumping process. Koitsu is working on automating that process. He will update this thread once it will be available.
User avatar
koitsu
Posts: 4201
Joined: Sun Sep 19, 2004 9:28 pm
Location: A world gone mad

Post by koitsu »

Automation part (script) is finished. Just a bunch of standard shell.

The content link Banshaku provided (and is now visible/live on the wiki itself) will be automatically updated every Sunday. There's also a checksum file which folks can use the validate the archive.

nesdev wiki admin folks (so far just Banshaku and I; I need tepples and memblers' Email addresses :-) ) will get an Email every Sunday with details of the auto-generation. In the case that the generation fails, the .zip file (for the following week) will probably be empty or very, very small. Hopefully the Email will provide some insight to what broke/failed in that situation, and we can rectify it if/when that happens.

I think that about does it. Yaaeey!
Last edited by koitsu on Sat May 14, 2011 11:47 pm, edited 1 time in total.
6T4
Posts: 48
Joined: Sat Feb 19, 2011 8:56 pm

Thank You

Post by 6T4 »

Thank you very much Banshaku and koitsu for working on this. I see that many other users wanted this feature as well, and I'm therefore glad I brought this to attention; hopefully we'll all benefit. I know it will help me in my quests to develop the best NES emulator and write the best NES documentation. Thanks again.
tepples
Posts: 22705
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Re: Local HTML Version

Post by tepples »

After a lengthy outage, the offline HTML zipfile is back. It's not updated automatically, but it is lightweight HTML and tablet-friendly in case you like to have your Nexus 7 next to your computer while you're coding.
tepples
Posts: 22705
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Re: Local HTML Version

Post by tepples »

And here's the tool that I wrote to make these snapshots. It uses Python 3 because another site that I use it on requires a newer version of SSL libraries than existed in the Python 2.6/2.7 era, and I didn't want to wait for PEP 466.
Post Reply