All's Well That Ends Well...

You can talk about almost anything that you want to on this board.

Moderator: Moderators

User avatar
koitsu
Posts: 4218
Joined: Sun Sep 19, 2004 9:28 pm
Location: A world gone mad

Post by koitsu » Sun Apr 22, 2012 3:57 pm

Tormenter wrote:You can get a years hosting for $100 that will allow enough traffic for this site. Im sure they can scrape enough in donations for that.
I think in your case it would help if I posted a full summary of the complexities involved in doing what I/we do. I'm still doing that write-up (it's up to about 10KBytes at this time), but I'll give you a little more monetarily-focused explanation.

I say all of this knowing that your above paragraph said, quote, "for this site". I'm not sure you know how much bandwidth/month this site uses, so I'll disclose that at the end.

We actually rent co-location (datacenter) 24U worth of rack space, and I personally own all the hardware in that rack. Switches, power management units, console redirection devices, all the servers themselves and all their components, etc.. The total cost of all the equipment comes to around US$15,000.

Our monthly cost (for rack space, bandwidth, etc.) is $775/month. My portion/cost of that (once excluding the dedicated server/business folks we have, who have their own boxes) comes to $650/month. This doesn't include things like general liability insurance (that's another $450/year), general upkeep (disk replacements), contracting fees (for people who I pay to assist when doing work in the datacenter), domain renewals, etc..

Finally, for the record: we get around $30/year, at most, in donations. Most of the time it's $20 one year, then a couple years will go by with nothing. I concluded long ago that Internet visitors in general are already having difficulty making ends meet (for themselves), thus they cannot afford such. Also, you have to remember that if you *rely* on donations to pay the bills, then they aren't really donations, are they? Those are more like "service fees", which isn't the kind of operation I run.

Could I have moved everything off to some hosting provider elsewhere for cheaper? Not easily. Each of our servers is physical/dedicated, rather than under a VM/hypervisor, so resources aren't shared (per se). The cheap hosting providers you see usually use VMs/HVs or offer things like a VPS. Dedicated servers (meaning you rent a physical box in a datacenter for your own use and nobody else shares it) are more like $150-200/month. Considering that we have 8 servers, well, you can do the math. :-)

As for nesdev's bandwidth -- this is what I sent Banshaku. Keep in mind when it comes to network traffic, kilo = 1000, not 1024.
nesdev.com (nesdev.com) -- not the Wiki part: The site takes up about 128kbit/sec worth of bandwidth (average). 128 * 60 * 60 * 24 * 30 = 331776000 kbits per month. That comes out to about 42GBytes/month average.

nesdev wiki (wiki.nesdev.com): The site takes up about 56kbit/sec worth of bandwidth (average). 56 * 60 * 60 * 24 * 30 = 145152000 kbits per month. That comes out to about 19GBytes/month average.

About disk space:

nesdev.com (nesdev.com) -- not the Wiki part -- takes up about 92MBytes of disk for just the website/data. The MySQL DB takes up 164MBytes of disk.

nesdev wiki (wiki.nesdev.com) -- takes up about 190MBytes of disk just for the website/data. The MySQL DB takes up 78MBytes of disk.
You'll find lots of hosting providers do a very clever thing -- they either offer you lots of traffic/month but very little disk space, OR, lots of disk space but very little traffic/month. They do this to keep people from getting hosting and using it as a large repository for large files (movies, music, etc.), since bandwidth is expensive. We don't operate this way. Disk is disk, network is network. Disk space is cheap, network traffic isn't.

Bisqwit
Posts: 248
Joined: Fri Oct 14, 2011 1:09 am

Post by Bisqwit » Sun Apr 22, 2012 4:18 pm

Ah. I think when I hosted TASVideos, the bandwidth peaked somewhere around 60 GB / month according to Webalizer logs, with an average around 40 GB / month. Plus added to that a BitTorrent seeder. Right now, with TASVideos removed, I get between 6 and 15 GB / month (which is surprisingly much, considering I only have a personal site and a Japanese dictionary right now).
I have no transmission quota; only an uplink bandwidth limit that is around 100 KiB/s, still about 10 times more than required for NESDev by average. (This ISP provides good downloads, but crappy uploads.)
I have no disk space problems for hosting either. So according to those specifications, I could well host NESDev wiki and forums for a while. But I would really need to virtual-machine it, because I don't want to give admins and/or code of possibly unknown nature access to my private files.

User avatar
infiniteneslives
Posts: 2100
Joined: Mon Apr 04, 2011 11:49 am
Location: WhereverIparkIt, USA
Contact:

Post by infiniteneslives » Sun Apr 22, 2012 5:09 pm

koitsu wrote:Our monthly cost (for rack space, bandwidth, etc.) is $775/month. My portion/cost of that (once excluding the dedicated server/business folks we have, who have their own boxes) comes to $650/month. This doesn't include things like general liability insurance (that's another $450/year), general upkeep (disk replacements), contracting fees (for people who I pay to assist when doing work in the datacenter), domain renewals, etc..
Wow I never realized the reality of this situation. We all owe you a great amount of gratitude for footing our bill over the years. I know may have said it, but THANK YOU koitsu! I don't know if we'll ever be able to return that large of a favor.

I don't think it's really fair for a single/few people to be solely financially responsible for our costs. There are lots of ways that we can improve upon the $20 annual income of nesdev responsibly. Should we start another discussion over ideas of how we might go about this that we can agree upon formally?

UncleSporky
Posts: 385
Joined: Sat Nov 17, 2007 8:44 pm

Post by UncleSporky » Sun Apr 22, 2012 5:12 pm

If I'd known it was that expensive (maybe I just need to pay more attention!) I'd have chipped in more than a few times already.

User avatar
koitsu
Posts: 4218
Joined: Sun Sep 19, 2004 9:28 pm
Location: A world gone mad

Post by koitsu » Sun Apr 22, 2012 7:24 pm

infiniteneslives wrote:I don't think it's really fair for a single/few people to be solely financially responsible for our costs.
I guess I see it slightly differently (I don't mean this in an argumentative or combative way, BTW). If I didn't want to be doing the free hosting thing, I wouldn't be doing it. That's one of the complexities that I had to figure out in my head over the past 6 months or so. "Hobby vs. business" so-to-speak.

I don't have any problem with spending money on a hobby -- and I considered the hosting thing a hobby for many years -- but it's gotten to the point where financially it's just not feasible. But finances aren't the only reason. I guess that's why I'm writing up a very large summary explaining all the reasons (there are 4 or 5 main ones). Parodius as a whole (not specific to nesdev) has gone from a hobby to a chore/task. And the dedicated customers (on their own boxes) are a completely separate debacle which I'm not even going to get into here...

What I'm trying to say is that my disclosure of how much our monthly costs are isn't me trying to ask for help -- I work a full-time job (well, until May... that's a separate and much more personal topic though) and thus can afford the co-lo costs. You gotta remember that that's $650/month that covers, well, everything -- and all the sites we host (not just nesdev!).

Could I take everything hosted here, shove it on to a $200/month dedicated server somewhere (vs. doing native co-lo), and accomplish the same? Yes. However, I'd probably lose OOB (out-of-band, meaning need for native VGA or serial console which I can access remotely), lose resiliency (nightly backups done in an efficient manner, stored on a dedicated + secure box), lose distribution of services (MySQL is on one box, web is on another, etc.), yadda yadda. That dedicated box would also have only 1 disk, while all of the Parodius hosting servers have 4 (1 OS disk + 3 in raidz (think RAID-5) for everyone's data). If I was just hosting one or two sites for fun and didn't care about Doing Things Right(tm), sure, that'd work. But I tend to apply what I've learned as a UNIX SA over the years to, well, a lot of things. :-) Thus it becomes more expensive.
infiniteneslives wrote:There are lots of ways that we can improve upon the $20 annual income of nesdev responsibly. Should we start another discussion over ideas of how we might go about this that we can agree upon formally?
If you guys want to discuss that with regards to whoever ends up taking over things (i.e. new hosting provider), that's cool. Maybe something can be reached where multiple people chipping in small amounts per month can result in the thing paying for itself. But my decision at this point, with regards to Parodius, is final.

User avatar
Jeroen
Posts: 1048
Joined: Tue Jul 03, 2007 1:49 pm

Post by Jeroen » Sun Apr 22, 2012 7:43 pm

The problem with the paying for itself thing is "what happens if we fall short" In the end to keep running that means someone has to foot the bill.
Now considering we only have nesdev + the wiki to host...our costs I assume can be considerably lower. Which is a big plus I suppose.

User avatar
infiniteneslives
Posts: 2100
Joined: Mon Apr 04, 2011 11:49 am
Location: WhereverIparkIt, USA
Contact:

Post by infiniteneslives » Sun Apr 22, 2012 8:28 pm

koitsu wrote:
infiniteneslives wrote:I don't think it's really fair for a single/few people to be solely financially responsible for our costs.
I guess I see it slightly differently (I don't mean this in an argumentative or combative way, BTW). If I didn't want to be doing the free hosting thing, I wouldn't be doing it. That's one of the complexities that I had to figure out in my head over the past 6 months or so. "Hobby vs. business" so-to-speak.
No I understand what your saying and can see why you've chosen to take this on in the past. I just took for granted what you were doing for us and it sounds like others were as well. I don't want to hold back someone who wants to contribute on a similar level as yourself. But I'm not going to assume that someone will. So I think there are a few people (including myself) who could find ways to contribute financially that we haven't previously considered.

But my decision at this point, with regards to Parodius, is final.
I COMPLETELY understand that and didn't mean to sound like we might be able to change your mind or anything. I was only speaking from a stand point for our future plans. But our future plans probably belong in another topic.

tepples
Posts: 22143
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Post by tepples » Sun Apr 22, 2012 8:31 pm

Go Daddy is offering an "Economy 4GH" package, which includes 10 GB of space, 10 MySQL databases (1 GB each), and unmetered bandwidth, for about $60 per year. There's also a plan that adds unmetered space (limited to half a million files) and SSL support (one domain-validated certificate and one IPv4 address) for twice that.

So we're looking at about 60 GB/mo of bandwidth, a couple hundred MB of static disk space, and well below 1 GB of MySQL. If it's just a file repository, a board, a couple static archived boards, and a wiki, then it'd probably fit on such a shared hosting account on DH/GD.

No referral URL because my only conflict of interest is that my employer uses Go Daddy hosting.

User avatar
Kit Sniper
Posts: 15
Joined: Sun Nov 02, 2008 11:16 pm
Location: Mexico
Contact:

Post by Kit Sniper » Sun Apr 22, 2012 9:26 pm

tepples wrote:Go Daddy
NO.

Get a REAL HOST.

Edit: Okay, considering the amount of programming content hosted here, the last thing this site needs is for whoever hosts it to wipe it at the first sign of trouble.

Xkeeper
Posts: 88
Joined: Fri Feb 29, 2008 10:35 am

Post by Xkeeper » Sun Apr 22, 2012 9:49 pm

I'm still offering to host it on my DreamHost account (for free*). I've been using it for about 4 years for various projects (including very high-bandwidth areas) and... no real issues.

(This offer sort of extends to a few other parodius things depending. Things like sl1me/blargg's domain, etc. I have no issues hosting them, as I do for BMF's old rustedmagick.com site.)



*I like to try to charge people $10 for real domain names (the cost to me to register/maintain them), but generally I can be coerced into giving it out for free. Otherwise, nobody really pays for their hosting.

Xkeeper
Posts: 88
Joined: Fri Feb 29, 2008 10:35 am

Post by Xkeeper » Sun Apr 22, 2012 9:54 pm

...to add something since there seems to be a lot of discussion elsewhere of how to move things:


- a tarball of the site files. (This is easy enough if it's stored sanely.)
- mysqldump of databases (compressed)

From there it's just extracting and changing the auth information and possibly the root paths if they're used. Trivial and a lot easier to do than scraping every page of the site (!).

User avatar
Banshaku
Posts: 2393
Joined: Tue Jun 24, 2008 8:38 pm
Location: Japan
Contact:

Post by Banshaku » Sun Apr 22, 2012 10:07 pm

@Xkeeper

thank you for the offer, I will keep it in mind regarding the wiki.

But first, we need to wait for Memblers to manifest himself regarding his intention about the BBS. Since he seems to have access to some private hosting, he may decide to use this if the requirements are good enough.

Let wait for him to give his opinion before moving forward too fast: we should be proactive, not reactive.

User avatar
Memblers
Site Admin
Posts: 3890
Joined: Mon Sep 20, 2004 6:04 am
Location: Indianapolis
Contact:

Post by Memblers » Mon Apr 23, 2012 2:19 am

Well nothing lasts forever, but going by archive.org, it's been maybe 12 years this month since NESdev moved here from Geocities. The Parodius server has been amazingly good, as far as fast response and reliability. So in those respects it's probably all downhill from here, but I'm sure we'll survive with the site being a little slower.

I didn't know it was so expensive for everything, I knew it wasn't cheap, but wow. Through bluehost, I have access to a cheap (shared) server that I pay for to use with my online store (still "coming soon"), wish I would have donated more instead of paying for that (and hardly using it so far), but seems like it would barely put a dent in the total cost. Thanks for keeping it running all this time, koitsu. And of course for setting up the forums, I still remember asking for help with that. And back when I felt I had to reply to every single post because there weren't any users, heheh.

So, I think it wouldn't be too difficult to move it to this onto cheap hosting for now, but how bad could it be? I'm already paid up for another 18 months or whatever, and there are no space/bandwidth limits (well, TOS says no more than 1000 tables in an SQL file, sounds like a lot right?) - I guess it just gets throttled if it uses too many resources. If it ends up not working for us, maybe we can move do a dedicated server later on. I wouldn't want to pay $100 a month by myself, but I'm sure something could be figured out if needed.

WhoaMan
Posts: 168
Joined: Sat Oct 02, 2004 12:07 pm

Post by WhoaMan » Mon Apr 23, 2012 10:48 am

I have a server currently co-located that could house nesdev.com. It would be shared with some hosted videos for a while... but bandwidth shouldn't be a problem and i could move it to its own 1u server in the near futur

Eclipsed Moon
Posts: 3
Joined: Mon Apr 23, 2012 10:41 am
Contact:

Post by Eclipsed Moon » Mon Apr 23, 2012 10:54 am

The Konfiskated Teknologies Network (kontek.net) would like to extend an offer of free hosting to any affected sites. We previously handled the relocation of many IGN hosted sites as they shut down, and would gladly take on some more.

E-mail me at eclipsedmoon@gmail.com for any inquiries, or use any of the IM screen names linked to my account here.

Post Reply