Recent Spammer Influx Warning. (Non-Spam Thread)

Found an issue with the phpBB system here at NESdev? Use this forum to report problems.

Moderator: Moderators

User avatar
Hamtaro126
Posts: 818
Joined: Thu Jan 19, 2006 5:08 pm

Recent Spammer Influx Warning. (Non-Spam Thread)

Post by Hamtaro126 »

Recently, A new bunch of Spammers had Spammed up the site again...

This needs to be taken care of again due to possible issues of external links that may possibly lead to a scam, malware, or even a virus/worm, but in most cases, the links are just there to be taken care of immediately.

Question: In what action, may we suggest, to stop most of it lately?

Answer: Most of the NESDEV Staff in my standpoint recommend the Sign-In Protection Scheme.

I do not know if this is right from a hearing point, but letting the people not only ''Sign in'', but to verify a person by putting in two or three randomly-generated ''Nintendo-related'' words via an Image, Followed by you typing it in...

Anyone want to test this out? I'd like to help as much as possible...

Any other Suggestions?
AKA SmilyMZX/AtariHacker.
3gengames
Formerly 65024U
Posts: 2284
Joined: Sat Mar 27, 2010 12:57 pm

Post by 3gengames »

My suggestion is to learn not to click bot links and wait for the mods to clean it up. :)
User avatar
Dwedit
Posts: 4924
Joined: Fri Nov 19, 2004 7:35 pm
Contact:

Post by Dwedit »

Let established members (20+ posts, and 2 weeks membership) report spam, and have it get auto-removed from the public once several people (maybe 3 to 5) report it. Then a moderator can confirm that the spam should be deleted, or reject the reports and make the post visible again. I doubt anyone here would abuse such a feature, but just in case, revoke the report feature to anyone who abuses it. Flamewars aren't common here.
Here come the fortune cookies! Here come the fortune cookies! They're wearing paper hats!
User avatar
Hamtaro126
Posts: 818
Joined: Thu Jan 19, 2006 5:08 pm

Post by Hamtaro126 »

I do not really click on these spam links, nor care about them.

And yesterday, the fact that I saw 3-4 spam links by a couple of spammers... not really the result of bad reports,

Next time, I will make sure you guys handle it, I am just not trying to make the forums a spam battlegrounds like some forums.
AKA SmilyMZX/AtariHacker.
tepples
Posts: 22708
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Post by tepples »

Dwedit: I'm not sure community moderation like that is possible in phpBB 2. What was the big reason against upgrading to 3 again?
User avatar
tokumaru
Posts: 12427
Joined: Sat Feb 12, 2005 9:43 pm
Location: Rio de Janeiro - Brazil

Post by tokumaru »

tepples wrote:What was the big reason against upgrading to 3 again?
The possibility of breaking everything? :shock:
User avatar
koitsu
Posts: 4201
Joined: Sun Sep 19, 2004 9:28 pm
Location: A world gone mad

Post by koitsu »

I thought we've covered this topic many times over already? Or at least *I* thought *I* have. Oh well, here I go again...

Anonymous posting isn't permitted on the board any more. You must have a login. Signing up requires a valid Email address (for verification). But this doesn't matter -- the accounts you've been seeing post spam ARE CREATED BY HUMAN BEINGS. Software is not being used to create and validate the accounts.

There are companies in Asia and India which exist solely to do exactly that -- create accounts on forums, then proceed to sell the account credentials to the highest bidder, so the bidder can begin using them for spam. There's an alternate method of business, which involves a customer paying one of these companies to actually sign up on forums and post links to products, put a URL in their profile which people will click, etc.. The companies have employees/contractors who read or speak English.

As such, any form of captcha is effectively pointless. So please don't propose anything like that.

Also, they have an additional advantage being in another timezone; while they're awake the forum mods here are asleep. It won't be until later in the day that the spams get removed.

And worse, some of these companies use compromised machines to siphon their requests, so blocking them by IP address doesn't always work either. Welcome to the Internet in 2011.

A "voting system" for posts would be absolutely pointless as well. Ten people mark up a post as spam; what should happen to it? What if a bunch of dickheads register accounts and start marking random posts as spam (yes this will happen, and the repercussions could be devastating).

If I were to disclose how I've been dealing with the spammers, all they'd do is read this post/thread and figure out new ways to get around it. The simple version, without giving away details, is that they're being blocked gradually. It takes time. Believe me, nobody is just sitting on their ass about this issue.

Really, the spam count around here is incredibly low compared to the rest of Internet forums, my Email box, etc... I'm not saying it's not annoying, I'm saying the *scale* of annoyance is quite low compared to the rest of what's online. Settle down, bud!
User avatar
Dwedit
Posts: 4924
Joined: Fri Nov 19, 2004 7:35 pm
Contact:

Post by Dwedit »

I recommended the voting system for Established Members Only, people who have been here a while and posted a lot.
Here come the fortune cookies! Here come the fortune cookies! They're wearing paper hats!
tepples
Posts: 22708
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Post by tepples »

koitsu wrote:Also, they have an additional advantage being in another timezone; while they're awake the forum mods here are asleep.
Do we have any regulars in Japan whom we could trust as global mods?
A "voting system" for posts would be absolutely pointless as well. Ten people mark up a post as spam; what should happen to it? What if a bunch of dickheads register accounts and start marking random posts as spam (yes this will happen, and the repercussions could be devastating).
To expand on what Dwedit said: Look at how Slashdot has handled this. Each comment has a score from -1 to 5 (most comments start at 1), and each user has a score from -10 to 50 (start at 0). Any account that is at least a year or two old, has a high enough score, and has a certain amount of recent activity will occasionally get a package of "mod points" that can be used for increasing or decreasing the score of a comment in any thread to which the user hasn't already posted along with the score of the user who posted the comment. (Site administrators get unlimited mod points.)
If I were to disclose how I've been dealing with the spammers, all they'd do is read this post/thread and figure out new ways to get around it.
Good. You don't need to spill the BEANS.
Really, the spam count around here is incredibly low
I agree. Perhaps the few that get through are so noticeable precisely because the count is so low.
User avatar
Gilbert
Posts: 564
Joined: Sun Dec 12, 2010 10:27 pm
Location: Hong Kong
Contact:

Post by Gilbert »

Maybe a first step is to provide users who have rights to edit the Wiki the power of marking posts as spam? They already had to ask for permission for the editing rights and had their requests reviewed so basically we can consider them all trusted users, right?
User avatar
Banshaku
Posts: 2417
Joined: Tue Jun 24, 2008 8:38 pm
Location: Japan
Contact:

Post by Banshaku »

tepples wrote:Do we have any regulars in Japan whom we could trust as global mods?
I still browse nesdev every day briefly but I'm quite busy these days. I could always see what I can do. I still check every day if any user get created on the wiki that could been a potential nesdev user and email them when it's the case. But I don't have access access to the internet as much as before thought.
User avatar
koitsu
Posts: 4201
Joined: Sun Sep 19, 2004 9:28 pm
Location: A world gone mad

Post by koitsu »

tepples wrote:
koitsu wrote:A "voting system" for posts would be absolutely pointless as well. Ten people mark up a post as spam; what should happen to it? What if a bunch of dickheads register accounts and start marking random posts as spam (yes this will happen, and the repercussions could be devastating).
To expand on what Dwedit said: Look at how Slashdot has handled this. Each comment has a score from -1 to 5 (most comments start at 1), and each user has a score from -10 to 50 (start at 0). Any account that is at least a year or two old, has a high enough score, and has a certain amount of recent activity will occasionally get a package of "mod points" that can be used for increasing or decreasing the score of a comment in any thread to which the user hasn't already posted along with the score of the user who posted the comment. (Site administrators get unlimited mod points.)
All this does is obfuscate and add unnecessary complexity to something which doesn't need it. Speaking strictly about the above "mod point" concept applied to this board:

1) People will quickly run out of mod points. Example: 20 spammers in a month. Uh oh, Hamtaro1234567890 is out of mod points within that month, and Ubernesdevdude who usually visits 5000 times a day (and has tons of mod points) happens to be on vacation in Jamaica. What now?

2) What happens to a post that gets modded down? Does it move to another forum?

3) Existing accounts cannot *safely* have mod points given to them. There are 3545 accounts on the forum:

Code: Select all

mysql> select count(*) from phpbb_users;
+----------+
| count(*) |
+----------+
|     3545 |
+----------+
1 row in set (0.00 sec)
Guess how many of those have logged in within the past YEAR?

Code: Select all

mysql> select count(*) from phpbb_users where user_lastvisit > unix_timestamp('2010-09-09 16:22');
+----------+
| count(*) |
+----------+
|      733 |
+----------+
1 row in set (0.02 sec)
3545 - 733 = 2812 accounts which haven't been logged into in the past year, but are probably still valid and could be used for whatever nefarious purposes.

Wait a minute, why would someone just log into them suddenly if they haven't been used in the past year? Who would do such a thing? The answer to that is implied in the next item:

4) I've been noticing spammers creating accounts on a daily basis -- usually multiples per day -- yet never posting anything. Our way of blocking spammers DOES block them from creating accounts AS WELL as posting, so the fact they're able to create accounts at all means they aren't being blocked and are therefore going to be able to post a post. They just haven't yet.

So, these spammers have a ton of usernames/passwords which they can sell to the highest bidder, or log in any time they want and use mod points (assuming we end up assigning mod points to existing accounts) to mod-up their own spams.

There's really no solution to this problem. I'm still waiting for Hamtaro389429835934 to provide his insights on how to solve this problem, since he's apparently fed up with the situation.

And as for upgrading to phpBB3 -- if that's going to happen, we need to "start over". The migration between major versions of phpBB never, EVER goes smoothly. We don't want to lose any of the existing posts (users/visitors have already stated they want those kept).
User avatar
koitsu
Posts: 4201
Joined: Sun Sep 19, 2004 9:28 pm
Location: A world gone mad

Post by koitsu »

By the way, some other statistics worth pointing out:

Number of accounts which have never posted a single thing:

Code: Select all

mysql> select count(*) from phpbb_users where user_posts = 0;
+----------+
| count(*) |
+----------+
|     2072 |
+----------+
1 row in set (0.02 sec)
Number of accounts which have posted only once and only twice, respectively:

Code: Select all

mysql> select count(*) from phpbb_users where user_posts = 1;
+----------+
| count(*) |
+----------+
|      289 |
+----------+
1 row in set (0.02 sec)

mysql> select count(*) from phpbb_users where user_posts = 2;
+----------+
| count(*) |
+----------+
|      202 |
+----------+
1 row in set (0.01 sec)
So uh, yeah... These numbers kinda speak for themselves.
tepples
Posts: 22708
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Post by tepples »

You might want to take a look at the "Comments and Moderation" section of the Slashdot FAQ as well as the history of how Slashdot got to the moderation system it uses.
koitsu wrote:1) People will quickly run out of mod points. Example: 20 spammers in a month. Uh oh, Hamtaro1234567890 is out of mod points within that month, and Ubernesdevdude who usually visits 5000 times a day (and has tons of mod points) happens to be on vacation in Jamaica. What now?
Slashdot mod points expire after a few days.
2) What happens to a post that gets modded down? Does it move to another forum?
The user can set two thresholds. Posts with a score >= the higher threshold get displayed in full. Posts with a score >= the lower threshold display only the author, subject, and first line (after any
elements have been stripped out). Posts with a score below the lower threshold display as "1 hidden comment". Users with JavaScript turned on can click a hidden or abbreviated comment to expand it to a full view.
3545 - 733 = 2812 accounts which haven't been logged into in the past year, but are probably still valid and could be used for whatever nefarious purposes.
But do they also have enough of a post count (let's say 50) to qualify for mod points? The 2,500-odd accounts with two or fewer posts certainly don't.
Wait a minute, why would someone just log into them suddenly if they haven't been used in the past year?
Slashdot gives mod points only to users who have been active recently.
So, these spammers have a ton of usernames/passwords which they can sell to the highest bidder
Only if a spammer has guessed the username/password, or the user has sold the account to a spammer.
And as for upgrading to phpBB3 -- if that's going to happen, we need to "start over".
Then we might have to handle it like the transition from wwwThreads to phpBB 2: start a new board, disable creation of new topics on the old board, and eventually disable replies on the old board.
User avatar
koitsu
Posts: 4201
Joined: Sun Sep 19, 2004 9:28 pm
Location: A world gone mad

Post by koitsu »

I finished reading the story of how Slashdot's moderation system came to be and its history. I didn't skim it, I read it. I did skim the FAQ however.

There is absolutely no mention of how they deal with actual spam (the kind we get here). The term "spam" throughout the document and the FAQ means "comments from a user that really don't amount to much value" -- the #1 cited example of spam is a "first post!!!" comment. This is not the kind of spam we get here.

Their karma system is intended to let moderators "rank" users based on whether or not what they say is worthwhile. Furthermore, I see absolutely zero indication that actual spam is *deleted* on Slashdot. Surely that shit gets deleted -- in fact, I KNOW it gets deleted. Someone is deleting it. It doesn't just get "modded down" and the user given bad karma -- I have seen spam show up on Slashdot and then the next day is completely gone (even with visibility sliders set to show everything possible). So someone there is having to do the same thing we do: delete the utter crap that comes in.

Finally, regarding the visibility slider situation -- which for those who aren't familiar with it, does exactly what tepples describes: it lets you define what sort of posts (based on their modded value) you see. This also doesn't solve the problem, it just masks it. It also introduces a horrible situation that really doesn't bode well (my opinion): Hamtaro83828342 starts using the new system, realises all the spam gets modded down, and adjusts his slider so he doesn't see modded down posts. Then suddenly he reads a thread and finds that certain posts are "missing" from the thread -- relevant posts to the subject matter. He then has to adjust the slider so he can view said posts. But now he's going to see the boatloads of spam he complained about in the first place. So he's going to spend more of his time fiddling with a slider in a web GUI than he is reading actual content. :-)

Next there's the issue of a new user who signs up and has highly useful things to say. Someone's slider settings may be filtering him out entirely. He won't get karma or mod-up points on his comments because nobody's seeing what he has to say, because of the spam. Sad panda situation.

Then finally there's the issue of efficiency: more accounts = only slightly slower board/forum. However, more posts (e.g. spams which aren't getting outright deleted) = *much* slower board/forum. "Wow, nesdev has over 50,000,000 posts! ... Oh, wait, 49,500,000 of them are spam..."

All this said, I am absolutely open to suggestions, and I appreciate the time people spend to propose suggestions. Honest. The only reason I'm highly critical of them is because -- and this is stated quite boldly in the Slashdot moderation story too! -- there's a balance that has to be achieved.

Furthermore, I think tepples and I would have to be the two who ended up coding the enhancements into the board (phpBB 2.x). This makes software upgrades a bitch cuz there's custom patches. Heck, we already have custom patches to make the board work properly with UTF-8, and our version of phpBB 2 is quite old/outdated (the Admin panel reminds me of this every time I use it. :-) ). We *should* upgrade, but oh god...

Anyway, sorry for the long-winded rant, these are just the thoughts that come into my head. I have no problem with additional forum moderators being made and so on, but then there's the issue of what happens if someone starts deleting posts / abusing their status -- we do MySQL backups daily, so reverting/restoring the effects of abusive behaviour is possible, but it's a tedious process.

I really hate the idea of having to manage people. I really do. I like everyone to just be reasonable; the Parodius FAQ is sort of an example of how I feel about things. "Just don't be a dick" sums it up (and that's coming from a guy who usually *IS* a dick), but not everyone sees eye to eye.

As far as trying other forum software, we can absolutely do that, and I can spend the time to set that up *in parallel* to this board still operating. We can try softwares, let people post stuff, mess about, etc. and see what sorts of havoc can occur. Think of it as a sandbox test to see what forum software might work well for us.

Oh, and I'll state this point blank: if there's **commercial** forum software people want or like (that runs on UNIX, under PHP or Perl -- please not ASP or Mono/.NET or anything nonsensical like that), let me know, I will be happy to purchase a license for the board.
Post Reply