It is currently Sun Oct 22, 2017 4:11 pm

All times are UTC - 7 hours





Post new topic This topic is locked, you cannot edit posts or make further replies.  [ 84 posts ]  Go to page Previous  1, 2, 3, 4, 5, 6  Next
Author Message
PostPosted: Sat Jul 09, 2016 7:11 pm 
Offline

Joined: Sat Jul 09, 2016 7:06 pm
Posts: 1
Hi, I'm a long time lurker that just learned of this due to searching for answers as to why the database was down. I registered in order to post this actually. I have used your site for some time and would also love to help donate some funds ( Do you have a paypal registered email? Would that work for you?). I'm very sorry to hear about these unfortunate events, please do keep up your amazing work! :beer:


Top
 Profile  
 
PostPosted: Mon Jul 11, 2016 5:19 pm 
Offline

Joined: Wed Jul 13, 2005 3:14 pm
Posts: 355
I got all the data back, everything was recovered thankfully. It's so close to being back online but I'm running into one snag and it's really got me stumped. Something is going seriously wrong with PHP. Very shortly after scripts start to execute, it will crash, restarting apache as well. In the apache error log I get either "zend_mm_heap corrupted" or apache will just terminate with some random error code. It has nothing to do with the script running, it will just crash at some random point unless the script is very short.

The most puzzling part is i'm using the exact same versions of everything that I was before, with the exact same configs from before so I'm at a loss as to why this is happening. Any ideas anyone?

The final damage was $390 for the recovery + new HD. My paypal address is prlacey_6942 AT hotmail DOT com if anyone wants to help out, I would definitely appreciate it!


Top
 Profile  
 
PostPosted: Mon Jul 11, 2016 7:22 pm 
Offline

Joined: Thu Jun 30, 2016 2:11 pm
Posts: 2
Hi BootGod,

The error apparently is because a bug in php [1], reading more here [2] talk about set "USE_ZEND_ALLOC=0" on php config. Maybe you can try that, or try upgrade the *AMP install, obviously in a separate server to keep the original untouched.

Cheers.

[1] http://drupal.stackexchange.com/questio ... mmatically
[2] https://developers.oxwall.com/forum/topic/41865


Top
 Profile  
 
PostPosted: Mon Jul 11, 2016 7:25 pm 
Offline

Joined: Sun Sep 19, 2004 11:12 pm
Posts: 19115
Location: NE Indiana, USA (NTSC)
I second the suggestion to try newer PHP (especially PHP 7) on a different server.


Top
 Profile  
 
PostPosted: Tue Jul 12, 2016 4:32 am 
Offline

Joined: Tue Aug 07, 2012 11:01 am
Posts: 12
BootGod wrote:
The final damage was $390 for the recovery + new HD. My paypal address is prlacey_6942 AT hotmail DOT com if anyone wants to help out, I would definitely appreciate it!


Done!


Top
 Profile  
 
PostPosted: Tue Jul 12, 2016 5:04 pm 
Offline

Joined: Tue Jul 12, 2016 4:57 pm
Posts: 6
Long time lurker here. Great news Bootgod!!! Can't wait for the site to be back up.

I was also able to put together a mini version of NESCartDB based off one of your XML files. It's def not as comprehensive as your site but I streamlined it for anyone looking for a quick donor list. Also it's mobile friendly.

http://mimeones.com/nescartdb

Hope you guys find it useful and suggestions are welcome!


Top
 Profile  
 
PostPosted: Wed Jul 13, 2016 3:10 pm 
Offline

Joined: Wed Jul 13, 2005 3:14 pm
Posts: 355
dubcl wrote:
The error apparently is because a bug in php [1], reading more here [2] talk about set "USE_ZEND_ALLOC=0" on php config. Maybe you can try that, or try upgrade the *AMP install, obviously in a separate server to keep the original untouched.

I did try that actually, but the server will still crash none-the-less, just without that error message :/

tepples wrote:
I second the suggestion to try newer PHP (especially PHP 7) on a different server.

Easier said than done unfortunately. The code for the public version of the site is ancient and heavily makes use of the old magic quotes "feature" (I didn't know any better at the time). Currently site is using 5.2.9-2, magic quotes is still available I believe in 5.3.x but was removed after that. I did try popping in a version of 5.3 and with that, apache wouldn't even start up (and gave absolutely no indication as to why).

I had been working on a new version of the site that's much cleaner code-wise and uses a newer PHP and no magic quotes nonsense. The database structure for the new site is completely different than the old one and the data is way out of sync. Not looking forward when the day comes to re-sync them either! In any case, the new version is not ready to be made live.

I'm sure I'll figure out something to get old one back up and running though.

Also thanks everyone for donations so far! I'll be sure to give credit on the site once its back up and running.


Top
 Profile  
 
PostPosted: Thu Jul 14, 2016 4:39 am 
Offline

Joined: Sun Sep 19, 2004 11:12 pm
Posts: 19115
Location: NE Indiana, USA (NTSC)
mimeoNES wrote:
http://mimeones.com/nescartdb

Hope you guys find it useful and suggestions are welcome!

Permalinks please. Could you make result pages have URLs, either through fragment manipulation or through history manipulation? I wanted to link "list of carts you could use to make Wheel of Fortune" in this post, but the URL in the location bar still points to a blank form.


Top
 Profile  
 
PostPosted: Thu Jul 14, 2016 12:32 pm 
Offline

Joined: Mon Oct 21, 2013 11:32 pm
Posts: 18
I can organize a mirror at my home server.
Do you need it?


Top
 Profile  
 
PostPosted: Thu Jul 14, 2016 1:12 pm 
Offline
User avatar

Joined: Mon Feb 07, 2011 12:46 pm
Posts: 929
Can the NES Cart data be made available as a SQLite database?

_________________
.


Top
 Profile  
 
PostPosted: Thu Jul 14, 2016 1:27 pm 
Offline

Joined: Sun Sep 19, 2004 11:12 pm
Posts: 19115
Location: NE Indiana, USA (NTSC)
Or, if that's too much trouble, at least as a MySQL dump so that one of us can do the conversion as needed.


Top
 Profile  
 
PostPosted: Thu Jul 14, 2016 3:53 pm 
Offline
User avatar

Joined: Sun Sep 19, 2004 9:28 pm
Posts: 3192
Location: Mountain View, CA, USA
zzo38 wrote:
Can the NES Cart data be made available as a SQLite database?

Please god no. The internal DB format tends to change and be non-backwards-compatible quite often. We deal with this mayhem in FreeBSD regularly (once a year usually), because the package tools rely on SQLite 3.x as a database.

Let me educate people: if you want to export a database into something that people can use themselves, what needs to be provided is a text-based format. I don't care if it's mysqldump with certain options disabled, XML with or without a DTD, or even a bloody CSV. UTF8 output would be great, but even ASCII (with \xXX for chars outside ASCII range) would be fine.

Also, people putting up "mirroring sites" of this: if you think you're just going to run some kind of recursive wget --mirror then I will remind you as the guy who ran Parodius that this is EXTREMELY RUDE AND UNRELIABLE. Don't do this. Instead, work with the site owner/admin (BootGod) and get something that doesn't scrape an entire site. There are TONS of ways to do this that don't destroy resources like a "website mirror" would.

I wish people would let BootGod get the thing back up and working first. He just spent US$390 out of his own pocket on data recovery. One step at a time, and please have some patience!


Top
 Profile  
 
PostPosted: Thu Jul 14, 2016 4:47 pm 
Offline

Joined: Thu Aug 12, 2010 3:43 am
Posts: 1589
koitsu wrote:
zzo38 wrote:
Can the NES Cart data be made available as a SQLite database?

Please god no. The internal DB format tends to change and be non-backwards-compatible quite often. We deal with this mayhem in FreeBSD regularly (once a year usually), because the package tools rely on SQLite 3.x as a database.

Ironically, they say that using SQLite's file format as your own is a good idea (going as far as stating "Content stored in an SQLite database is more likely to be recoverable decades in the future"). Either something is wrong on your end, or something is wrong on theirs. (・_・)

EDIT: also before I forget, text based formats (at least when they have newlines) have the advantage of being easy to compare with a diff, and even to merge two files that were updated separately. Doing that with a binary format is a pain in the ass.


Top
 Profile  
 
PostPosted: Thu Jul 14, 2016 5:16 pm 
Offline
User avatar

Joined: Sun Sep 19, 2004 9:28 pm
Posts: 3192
Location: Mountain View, CA, USA
Sik wrote:
Ironically, they say that using SQLite's file format as your own is a good idea (going as far as stating "Content stored in an SQLite database is more likely to be recoverable decades in the future"). Either something is wrong on your end, or something is wrong on theirs. (・_・)

It's definitely not my end, not when I see commits coming across that mention such matters. The documentation you linked makes me chuckle, especially when you consider that to use said database, you need to use something that links in to their API -- which is regularly changing in incompatible ways. (Yes, that's not the file format, but it's equally as important)

The two "biggest" projects I know of which use SQLite exclusively are pkg and ircd-ratbox, and I have already seen several reports of "weird SQLite database corruption" with pkg (these PRs have gone completely unanswered/ignored, which never sits well with me in an open-source projkect) -- but I will say that bugs in pkg are certainly possible. It's also possible that the problems I read about were related to the filesystem used as the backing store; SQLite, unlike other databases, relies exclusively on filesystem-level locks (which on some filesystems, including NFS, are known to be somewhat problematic; don't get me started on lockf vs. flock vs. fcntl).

My point here is this: don't just copy around a binary database file, even with SQLite. Either use .dump or a CSV export.

And yeah, text is good for lots of reasons, and you covered one of several. :-)


Top
 Profile  
 
PostPosted: Thu Jul 14, 2016 6:14 pm 
Offline

Joined: Thu Aug 12, 2010 3:43 am
Posts: 1589
koitsu wrote:
especially when you consider that to use said database, you need to use something that links in to their API -- which is regularly changing in incompatible ways.

This seems to be true of all major libraries these days, sadly. One of the reasons I hate semantic versioning, people use it as an excuse to break old stuff that was otherwise working perfectly fine. When your library gets so cluttered that the only reasonable way to keep going forward is to break existing programs*, then maybe it just means the design is at its limit and you should consider starting a new library from scratch, with a new name, and leave the old one to be maintained by those who still rely on it.

I can see the idea behind semantic versioning but I'd like to get rid of its major version (effectively leaving just two numbers), which would leave no room for breaking backwards compatibility. People underestimate just how important it is to keep old things working.

*Ignoring those that were doing something that wasn't valid in the first place and hence were working just by pure luck =P


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic This topic is locked, you cannot edit posts or make further replies.  [ 84 posts ]  Go to page Previous  1, 2, 3, 4, 5, 6  Next

All times are UTC - 7 hours


Who is online

Users browsing this forum: Dwedit and 6 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB® Forum Software © phpBB Group