NES Cart DB down again
Moderator: Moderators
Re: NES Cart DB down again
Hi, I'm a long time lurker that just learned of this due to searching for answers as to why the database was down. I registered in order to post this actually. I have used your site for some time and would also love to help donate some funds ( Do you have a paypal registered email? Would that work for you?). I'm very sorry to hear about these unfortunate events, please do keep up your amazing work!
Re: NES Cart DB down again
I got all the data back, everything was recovered thankfully. It's so close to being back online but I'm running into one snag and it's really got me stumped. Something is going seriously wrong with PHP. Very shortly after scripts start to execute, it will crash, restarting apache as well. In the apache error log I get either "zend_mm_heap corrupted" or apache will just terminate with some random error code. It has nothing to do with the script running, it will just crash at some random point unless the script is very short.
The most puzzling part is i'm using the exact same versions of everything that I was before, with the exact same configs from before so I'm at a loss as to why this is happening. Any ideas anyone?
The final damage was $390 for the recovery + new HD. My paypal address is prlacey_6942 AT hotmail DOT com if anyone wants to help out, I would definitely appreciate it!
The most puzzling part is i'm using the exact same versions of everything that I was before, with the exact same configs from before so I'm at a loss as to why this is happening. Any ideas anyone?
The final damage was $390 for the recovery + new HD. My paypal address is prlacey_6942 AT hotmail DOT com if anyone wants to help out, I would definitely appreciate it!
Re: NES Cart DB down again
Hi BootGod,
The error apparently is because a bug in php [1], reading more here [2] talk about set "USE_ZEND_ALLOC=0" on php config. Maybe you can try that, or try upgrade the *AMP install, obviously in a separate server to keep the original untouched.
Cheers.
[1] http://drupal.stackexchange.com/questio ... mmatically
[2] https://developers.oxwall.com/forum/topic/41865
The error apparently is because a bug in php [1], reading more here [2] talk about set "USE_ZEND_ALLOC=0" on php config. Maybe you can try that, or try upgrade the *AMP install, obviously in a separate server to keep the original untouched.
Cheers.
[1] http://drupal.stackexchange.com/questio ... mmatically
[2] https://developers.oxwall.com/forum/topic/41865
Re: NES Cart DB down again
I second the suggestion to try newer PHP (especially PHP 7) on a different server.
Re: NES Cart DB down again
Done!BootGod wrote:The final damage was $390 for the recovery + new HD. My paypal address is prlacey_6942 AT hotmail DOT com if anyone wants to help out, I would definitely appreciate it!
Re: NES Cart DB down again
Long time lurker here. Great news Bootgod!!! Can't wait for the site to be back up.
I was also able to put together a mini version of NESCartDB based off one of your XML files. It's def not as comprehensive as your site but I streamlined it for anyone looking for a quick donor list. Also it's mobile friendly.
http://mimeones.com/nescartdb
Hope you guys find it useful and suggestions are welcome!
I was also able to put together a mini version of NESCartDB based off one of your XML files. It's def not as comprehensive as your site but I streamlined it for anyone looking for a quick donor list. Also it's mobile friendly.
http://mimeones.com/nescartdb
Hope you guys find it useful and suggestions are welcome!
Re: NES Cart DB down again
I did try that actually, but the server will still crash none-the-less, just without that error message :/dubcl wrote: The error apparently is because a bug in php [1], reading more here [2] talk about set "USE_ZEND_ALLOC=0" on php config. Maybe you can try that, or try upgrade the *AMP install, obviously in a separate server to keep the original untouched.
Easier said than done unfortunately. The code for the public version of the site is ancient and heavily makes use of the old magic quotes "feature" (I didn't know any better at the time). Currently site is using 5.2.9-2, magic quotes is still available I believe in 5.3.x but was removed after that. I did try popping in a version of 5.3 and with that, apache wouldn't even start up (and gave absolutely no indication as to why).tepples wrote:I second the suggestion to try newer PHP (especially PHP 7) on a different server.
I had been working on a new version of the site that's much cleaner code-wise and uses a newer PHP and no magic quotes nonsense. The database structure for the new site is completely different than the old one and the data is way out of sync. Not looking forward when the day comes to re-sync them either! In any case, the new version is not ready to be made live.
I'm sure I'll figure out something to get old one back up and running though.
Also thanks everyone for donations so far! I'll be sure to give credit on the site once its back up and running.
Re: NES Cart DB down again
Permalinks please. Could you make result pages have URLs, either through fragment manipulation or through history manipulation? I wanted to link "list of carts you could use to make Wheel of Fortune" in this post, but the URL in the location bar still points to a blank form.mimeoNES wrote:http://mimeones.com/nescartdb
Hope you guys find it useful and suggestions are welcome!
Re: NES Cart DB down again
I can organize a mirror at my home server.
Do you need it?
Do you need it?
Re: NES Cart DB down again
Can the NES Cart data be made available as a SQLite database?
(Free Hero Mesh - FOSS puzzle game engine)
Re: NES Cart DB down again
Or, if that's too much trouble, at least as a MySQL dump so that one of us can do the conversion as needed.
Re: NES Cart DB down again
Please god no. The internal DB format tends to change and be non-backwards-compatible quite often. We deal with this mayhem in FreeBSD regularly (once a year usually), because the package tools rely on SQLite 3.x as a database.zzo38 wrote:Can the NES Cart data be made available as a SQLite database?
Let me educate people: if you want to export a database into something that people can use themselves, what needs to be provided is a text-based format. I don't care if it's mysqldump with certain options disabled, XML with or without a DTD, or even a bloody CSV. UTF8 output would be great, but even ASCII (with \xXX for chars outside ASCII range) would be fine.
Also, people putting up "mirroring sites" of this: if you think you're just going to run some kind of recursive wget --mirror then I will remind you as the guy who ran Parodius that this is EXTREMELY RUDE AND UNRELIABLE. Don't do this. Instead, work with the site owner/admin (BootGod) and get something that doesn't scrape an entire site. There are TONS of ways to do this that don't destroy resources like a "website mirror" would.
I wish people would let BootGod get the thing back up and working first. He just spent US$390 out of his own pocket on data recovery. One step at a time, and please have some patience!
Re: NES Cart DB down again
Ironically, they say that using SQLite's file format as your own is a good idea (going as far as stating "Content stored in an SQLite database is more likely to be recoverable decades in the future"). Either something is wrong on your end, or something is wrong on theirs. (・_・)koitsu wrote:Please god no. The internal DB format tends to change and be non-backwards-compatible quite often. We deal with this mayhem in FreeBSD regularly (once a year usually), because the package tools rely on SQLite 3.x as a database.zzo38 wrote:Can the NES Cart data be made available as a SQLite database?
EDIT: also before I forget, text based formats (at least when they have newlines) have the advantage of being easy to compare with a diff, and even to merge two files that were updated separately. Doing that with a binary format is a pain in the ass.
Re: NES Cart DB down again
It's definitely not my end, not when I see commits coming across that mention such matters. The documentation you linked makes me chuckle, especially when you consider that to use said database, you need to use something that links in to their API -- which is regularly changing in incompatible ways. (Yes, that's not the file format, but it's equally as important)Sik wrote:Ironically, they say that using SQLite's file format as your own is a good idea (going as far as stating "Content stored in an SQLite database is more likely to be recoverable decades in the future"). Either something is wrong on your end, or something is wrong on theirs. (・_・)
The two "biggest" projects I know of which use SQLite exclusively are pkg and ircd-ratbox, and I have already seen several reports of "weird SQLite database corruption" with pkg (these PRs have gone completely unanswered/ignored, which never sits well with me in an open-source projkect) -- but I will say that bugs in pkg are certainly possible. It's also possible that the problems I read about were related to the filesystem used as the backing store; SQLite, unlike other databases, relies exclusively on filesystem-level locks (which on some filesystems, including NFS, are known to be somewhat problematic; don't get me started on lockf vs. flock vs. fcntl).
My point here is this: don't just copy around a binary database file, even with SQLite. Either use .dump or a CSV export.
And yeah, text is good for lots of reasons, and you covered one of several. :-)
Re: NES Cart DB down again
This seems to be true of all major libraries these days, sadly. One of the reasons I hate semantic versioning, people use it as an excuse to break old stuff that was otherwise working perfectly fine. When your library gets so cluttered that the only reasonable way to keep going forward is to break existing programs*, then maybe it just means the design is at its limit and you should consider starting a new library from scratch, with a new name, and leave the old one to be maintained by those who still rely on it.koitsu wrote:especially when you consider that to use said database, you need to use something that links in to their API -- which is regularly changing in incompatible ways.
I can see the idea behind semantic versioning but I'd like to get rid of its major version (effectively leaving just two numbers), which would leave no room for breaking backwards compatibility. People underestimate just how important it is to keep old things working.
*Ignoring those that were doing something that wasn't valid in the first place and hence were working just by pure luck =P