BPL Forum Posting Policy Revision in Light of Recent SPAM Attacks

Dear All

This is ONLY A TEMPORARY FIX while BPL works out how to handle the problem!

Oh yes, we are very well aware of how many non-Members are valued contributors to the Forum channels. Please don't go away! We want to return to 'normal' ASAP.

I will repeat what has already been said elsewhere: this is a problem for a whole raft of website Forums across the Internet which were previously wide open: everyone is being hit. I suspect that the list of 'suitable spamming targets' has been massively increased just recently with the release of the 'upgraded' spamming SW, so that script kiddies have a whole new range of targets.

Let me also emphasise that the current spamming SW is extremely sophisticated. It knows how to create new member registrations and how to solve CAPTCHAS: the SW includes templates for this for hundreds (thousands?) of web sites. Most of the solutions which have been proposed on this Forum will not work against this SW.

But we WILL deal with it!

Cheers
Roger

Maybe it's possible to temporarily not allow any new people to register

Allow currently registered people to post

That would temporarily solve problem without denying registered non-members to post

Are you sure this isn't a conspiracy to deny rightwingers to post on the Romney/Ryan thread? : )

with the option to request, and receive, a refund after a vetting period for those who do not wish to become members? Say 1 month?

Edited: This is meant only as a stopgap measure until more sophisticated methods of blocking SPAM are developed.

This may turn out to be a positive shot in the arm for BPL. RJ is back at the helm, fixes may benefit all of us, and like a power outage we all need to appreciate what we hold in this community.Technology always comes with price but the simple pleasures of our outdoor adventures is the bond for all of us. Sure as of late a few cairns have been knocked over at BPL but we will still find OUR way.

Hoot

As a working web developer with lots of experience in maintaining forums both custom and off the shelf, there's a great many things you can do to minimize the spam bots. Many of them you've probably already thought of or are working to implement, so I won't ramble on, but here are my personal faves (feel free to message me if you want/need details):

– Captcha doesn't work, especially the commercial ones because they're widely used and as such spammers work to crack them constantly. Break a mainstream Captcha service so your bot can spam it and suddenly thousand of sites become accessible, as such, a one-off solution is best because you become a niche that requires special attention. Take-away here: don't pay big bucks to subscribe to a Captcha service because just like a $1 pay-wall, the hassle of trying to decode the really hard to understand graphics of numbers and stuff usually drives plenty of REAL people away from signing up.

– Randomized natural language question & answer. This one works well because bots can't read. Use some programming logic to generate questions such as "what's the third letter from the right in the word top-right of the screen" (if you're playing along, it would be "e" in "Help"). Sometimes you can get away with just having one question. For extra points, randomize the question between a dozen or so you setup in advance. For guru points generate the questions programmatically by having the logic "scrape" your own page during the sign-up process for a random word out of a list of generated words. The more one-off the logic but that continues to be pretty "natural language" in it's query to human users, the better.

– The Honey Pot. Most spam bots sign up accounts by scraping/crawling your sign-up page and looking for telltale field names in your sign up form. It does this by checking the "name" attribute typically and us programmers are a lazy bunch if given half a chance, and will name our fields appropriately. Spam bots look for typically used language and LOVE things named "email" and "username" and so on. One VERY successful method is to setup a simple field in the sign-up form (visible because bots ARE smart enough to ignore hidden things sometimes) and give the input field the "email" name, but on screen label it "If you're human, leave this empty". Bots will enter an email address, albeit fake, because they think it's a required field, and your own programming them throws out any submissions that includes a value when a human would know not to include anything. Extra Hater Points if you send the bot a "successful registration!" screen and email just to throw the persistent bots for a loop (bots will look for words like "success" in page after trying to sign up a spam account). It should be said you should rename your ACTUAL name, email, and so on, fields to something obfuscated. I'm somewhat partial to "fleamail" or sometimes "spammersshoulddieofcancer" for "email". Should probably check all your fields and just change them to something confusing behinds the scenes.

Anyhow, somehow I rambled. But yeah, you should be able to take care of making yourself a pretty low-yield target by creating a one-off system for sure. All told it's not a ton of programming (in fact it's a very minor amount) and should solve your problem.

Best of luck and let me know if you need any advice (I'm too busy to help code… sorry, day job of doing web stuff at a start-up means I get zero free time!) :(.

> This is intended to be a short term change in policy while we evaluate a number of options as we move forward. One of the options we will evaluate is whether or not to maintain this restriction indefinitely, or at least until we are able to upgrade to new forum software.

The solution to this is extremely simple. Simply add a question to the registration page that only a human with knowledge of the site can answer. Right now you have NO anti-spam measures that I could see other than checking for a valid email perhaps so it's a miracle we've not seen much more spam before this.

I've used this technique successfully for several years on my phpBB-based forum (thankfully it's built into the software now instead of having to mod it). I'm even 3 revisions behind in updating the forum software and haven't had a single spam registration since implementing this technique.

So just get your programmer to add a little code to add a question and check for a correct answer. Personally I would recommend something like "What is the 4-word slogan for BPL?" A: "pack less be more" (ignore case and punctuation). Problem solved with no cost and a few minutes of time.

"The solution to this is extremely simple. Simply add a question to the registration page that only a human with knowledge of the site can answer…"

Yeah, these are all good delaying tacticts. AI's have time. They can do anything you can think of. And this is another good solution.

AI has many definitions, but I like to think of them as Turing did, as a response generator whose responses cannot be distinguished, person and machine. After all, that IS what we are talking about. How to distinquish between man and machine?

Question:
Every potential new member could be a robot. How do you distinquish them apart?
Answer:
You can not. Someone will build a better AI and log in, eventually.

Only by checking a potential new member's intent to use his membership can we check. Like using a different algorithim to solve complex multiplication (used for encryption,) checking a persons "target of intent" becomes a matter of not validating his signing up (the obvious "become a member") but checking his posts for some valid content ("I have a question about sleeping bags…")

IFF the AI can solve this, then it doesn't matter, his posts are "valid" within the context of this site. I don't care if he is a person or machine. His posts are fine. He might start spamming after ten or twenty posts, but even people's computers have been known to be hacked and start spamming.

IFF his first post is spam, well, he needs to be gotten rid of(I favor hanging by his protruding member rendering him impotent…metaphorically speaking, of corse.)

So, I would suggest a "logic" check by simply moderating his first couple posts rather than an "item" check that can be solved by brute force.

But, there is never a guarentee that a bot will not figure out something general in response, like "This is a great site!" Sorry, it needs to show a specific intent. Something that is NOT gleaned from anything on the web page, and by extension, the web site. And not so general that it can be responded to by simple word substitution for phrases on the page. "I like backpacking" from the web site would not be good enough. "I like" is general, "backpacking" was gleaned from the web page.

I think Roger is Australia's AI. How do I know? Maybe I am an AI. How do you know? Does it matter as long as Roger stays on subject, there, and I stay on the subject, here?

Let me preface this by pointing out that I am absolutely sure what the headache the Admins have feels like and that I know multiple postings with "just ______, it' so obvious" in the theme can be frustrating.

However, some of the postings are full of good input from pros that know what they're doing. I like Mr. Eriksson's posting (even though I have a higher opinion of Captcha) and he provides some very good ideas that really can work. Where you have admins of phpBBS boards telling you what works for them, this is also significant, because that platform's not the most robust out there in terms of security — so, if they've got solutions, they've probably been tested…

One thing I am sure of: an effective repair for this highly-disrupted site will involve replacement of the forums software. Replacing the front end is a big deal, but if the BBS can be severed and there's a functional bridge out there for the CMS/front to the forums, THIS IS THE FIX.

Of course, a new forum package would necessarily include a modernization of look and function. Note, however: modern features take more bandwidth and will allow more traffic, potentially driving overhead at BPL Corporate Towers UP. Who's gonna pay for the upgrades, programming, time and higher operating costs?

> Yeah, these are all good delaying tacticts. AI’s have time. They can do anything you can think of.

Quite true, but unless you don’t use an appropriate question, the Q&A method has been very effective for at least 3 years now. And as Erik pointed out above, phpBB is one the most popular boards to hack and spam since it has one of the largest user bases and this is still the most effective single solution.

I disagree that allowing a potential spammer to register in the first place is an acceptable thing since it will just take up the mods time to check the intent of the first few posts of each new user. I sure wouldn’t want that job! It’s far better to prevent them from registering in the first place. I do agree with HOW you filter spambots is crucial, which is why I suggested the slogan since it’s contained within a picture rather than plain text. Yes, AI can “read” pics of course but it’s going to take a long time to randomly pick the right combination of words you may be looking for since the slogan is not obvious.

> One thing I am sure of: an effective repair for this highly-disrupted site will involve replacement of the forums software.

While I know many would like to see that (I’m one of the few that don’t mind the spartan software), that’s not true in my solution. Simply add a question to the registration page and the code to check it to allow the registration process to continue. Literally a 5 min job for whoever designed it. At least it was just a couple of minutes for me to edit the php files before they finally added it into the base code.

Edit: I see Alexander’s post above now (hadn’t read it the first time I saw this thread) and I like the idea of “completing” the registration process with a fake success message if they fail the Q&A. So maybe that would take 10 minutes. :)

"Quite true, but unless you don't use an appropriate question, the Q&A method has been very effective for at least 3 years now. And as Erik pointed out above, phpBB is one the most popular boards to hack and spam since it has one of the largest user bases and this is still the most effective single solution. …"
Yes, of course. I didn't mean not using automated defenses, too. We definitly don't want to waste a moderators time.

I've been having really good results (ie. perfect) with the Q&A method for a number of years now.

A few BPL related examples that could be incorporated into the registration process:

Q) When it's winter, precipitation most commonly falls as…
A) Snow

Q) What you use to hold your gear (hint: goes on your back).
A) Backpack

glad you provided the answers Dan- I was at a loss there :)

I should point out that captcha isn't all snake-oil, they definitely work. I just tend to discount them because most/all sites I've ever maintained would rather not erect a potential barrier to entry (and captchas, or for a lot of users anything that takes them longer to fill out than 20 seconds) which could dissuade people from signing up and participating in the first place. Usually we had a "well if someone wants to spam us, at least that means we've made it, then we'll address the problem" attitude towards things but obviously that doesn't work everywhere. We dubbed this "a problem we'd like to have". So in the end yeah there's some strong captcha services but I think between the fact that they can be expensive, onerous for users, and if a spammer breaks one service they have access to spam everyone using that service, facts, that I tend to pass.

Plus like others have pointed out, the Q&A method does seem to be oddly effective. Between Q&A and the honey-pot I've never had to deal with spammers. Ironically, when we used phpBB on a site and DID get spammed, it was because we were a single release behind and the captcha we were using was breached.

Oh and like someone else mentioned, Q&A and honey-pots can be very cheap/fast/easy to implement. I wouldn't suggest going the "get all new software" route until you've exhausted all the custom approaches since migrating content and user accounts is a BIG deal. Ugh.

You have my sympathies BPL gang!

Bad plan! Think longterm. And please…no CAPTCHA or similar. Impossible.

I agree with Dan – registration questions would likely do the trick. Another similar option (but not just text based) – have a picture and ask the registrant: "How many people are in the picture above?"

I prefer the Q&A idea. CAPTCHA always feels more like GOTCHA to me because half the time I can't read the darn thing. It seems to do a better job blocking people than bots.

From Wikipedia –

"CAPTCHA is vulnerable to a relay attack that uses humans to solve the puzzles. One approach involves relaying the puzzles to a group of human operators who can solve CAPTCHAs. In this scheme, a computer fills out a form and when it reaches a CAPTCHA, it gives the CAPTCHA to the human operator to solve.

Spammers pay about $0.80 to $1.20 for each 1,000 solved CAPTCHAs to companies employing human solvers in Bangladesh, China, India, and many other developing nations.[23] Other sources cite a cost as low as $0.50 for each 1,000 solved."

Someone is out there waiting to make $.001 on your challenge….

Nothing is as simple as you might hope.

Dan writes:
>Q) When it's winter, precipitation most commonly falls as…

A) Rain

At least within 100 miles of here. Lots of rain where I live. Snow is headline news.

I'd probably fail this test.

Maybe tests like this should allow for a small number of reasonable answers, e.g. snow, rain, tree drip :-)

And my friends in Houston would also answer "rain".

"From the redwood forest, to the gulf stream waters
This land was made for you and me."

Yeah, a simple methode for spammers is to simply register themselves, as humans. Then bot as many posts as the can before getting knocked off. Obviosly, this has been tried and results in large amounts of spam.

If a spammer can make money on a spam messages, then spamming will be done. Someone, somewhere will benefit. Besides using automagic tecniquies, which can all be broken, there needs to be a cost associated with it that allows the registering party to have some confidence that it is MORE expensive to register and spam than simply ignore the site. Using "slave" labour means no cost. If he has to pay, he won't.

Even delaying him by moderating his posts really is not a solution. It is only another delay. In any group of ten people, chances are they can type ten reasonble messages to get by this moderation…then spammmmmmmmmmm.

I would suggest a refundable cost, or garantee, when registering. If you start spamming, it is forefit. Or a membership fee, as is now in place. Anything that makes the spm more expensive than not.

Just another thought to add to the mix…

It seems to me if spammers can hire cheap labor to solve "captchas", they could hired to solve Q&A's as well. Perhaps a trial membership, returnable (prorated) if no spamming took place, would be the best option? Maybe a separate category for industry reps?

Having just spent the last four years of my life in Houston, I think an appropriate answer would actually be "what's winter?". ;-D

> It seems to me if spammers can hire cheap labor to solve "captchas", they could hired to solve Q&A's as well. Perhaps a trial membership, returnable (prorated) if no spamming took place, would be the best option?

You would kill your forum if you instituted such a policy. It's bad enough our's is still blocked after 5 days. It would be interesting to know if anyone has joined.

I've never heard of a forum do that, and human spammers have never been a problem for any forum I'm aware of that uses a reasonable anti-spam measure such as Q&A.

Just use the QA solution as a temporary stopgap instead of the payed membership. Waaay better in the short term, and you have to work on a long term solution either way.

Maybe the spamming is an inside job! HaHa!

I am always perplexed by the people that complain about having to cough up $25.00 for something they love. What a joke. Some people say there are other forums. I say go to your other forums. There is only one BPL. Viva BPL! At least this will end the spam on the Carbon Flame War thread or somebody will have to cough up at least $25.00!

From another point of view; Anyone that studies forums much will notice that it's always the same damn people that post – to me, that gets old fast – that to me means a forum is dead. It even looks as if the people that participate in them get paid to be there. It seems different here at BPL. I think it may be more effective if people pay to talk rather than get paid to talk. There seems to be a wider participation here than the usual forums – people even pay to play. There is more quality here and a quality that is worth investing in. I think much of the quality here comes about because people are attracted that have the ability to assess the value of things – it makes for a more colorful playing field. It sure is a good thing that if someone does not like that, they can go elsewhere to play – would be terrible if they couldn't!

Just spammed again, by a non member.

Better get with it. Should have been fixed already. It's embarrassing how long you have been talking about updating the site. And now it has come to this. Seriously, It has been year upon year of talk. I can dig up links, I have the free time.

I'd tell you to pat yourself on the back, but I'm actually thinking a bit lower. But I'll bet your kicking yourself now anyway.

This sucks. Bad.

Perfect timing with your trip too. What cosmic powers are at work there? Having a spammer on the trip could be fun.

+1 with Dan. Viva BPL

Once again I ask for an exact $ figure on what it will cost to fix this problem. You might just get it. The forums are certainly valuable to a lot of us. Perhaps many of us find it more valuable than you do. Stating that only 20% of MLifers participate, access the forums, I can see how you can look past it/us.

Don't overlook the offers of help with this problem from forum members who work in this field. The answers to your problems may lie closer than you think.

My dear old mother has a saying, Sh*t or get off the pot.

The time for action has long since passed.

It's a big dang mess that we all saw coming. Told you so.

Edit: reversing my previous mo I added text to this post not deleting.