Lemur Catta
 
 
 
Lemur CATTA is a system for Comment Annoyance Trapping by Testing Aptitude.
 
It automatically protects blogs, forums, and other online conversations from annoying comments.
 
Depending on its implementation, Lemur CATTA can block spammers, idiots, and assholes, without interfering with normal usage by qualified participants.
 
Thursday, September 27, 2007 Our Boat Trip in Greece
 
The Failure of CAPTCHA
 
The idea behind a CAPTCHA is to have a task that is trivially accomplished by humans, but is nearly impossible for machines. Typically, you're shown an image of a random assortment of letters and numbers and asked to type what you see. This is several problems.
 
First, because of optical character recognition technology, most CAPTCHAs are nearly unreadable for humans, making them a constant source of frustration.
 
Second, even newer CAPTCHA schemes that fix the first problem are all vulnerable to a mechanical turk attack. That is to say, even if they are literally impossible for a machine to do, a human willing to work for pennies a click can always be recruited to defeat them.
 
Finally, while a CAPTCHA may block spam, spam is typically not the biggest problem faced by online conversations. As a conversation grows in length and popularity, it will eventually collapse under its own intellectual weight as people become too lazy to read and understand what's already been said before add their two cents.
 
For the sake of brevity we'll call these people idiots. This is distinct from people whose intended contribution is meant only to be disruptive, who we'll call assholes.
 
Spammers, Idiots, and Assholes
 
Lemur CATTA attempts to address these shortcomings by presenting a short, contextual aptitude test as a barrier for participation. Unlike a CAPTCHA, a Lemur CATTA makes use of what we know about the user. For example, we know the last thing they read: the content we are trying to protect.
 
In an attempt to balance the effectiveness of the test, the level of aptitude required to pass it, and the amount of additional work needed from the site's author, Lemur CATTA is divided into three levels.
 
 
Level 1 - Completion
 
A Level 1 Lemur CATTA is a completion test. It simply ensures you've completed some task, such as reading a blog. This test has to be simple enough to be generated automatically, requiring no additional work.
 
An example of a Level 1 Lemur CATTA would be a form that presents you with a list of sentences, some of which are from the content, some of which are from elsewhere. To gain entry, you simply choose the correct sentences.
 
  1. • For the sake of brevity we'll call these people idiots.
  2. • I think that's the one issue both sides can agree on.
  3. • Any time you learn something, it changes your brain.
 
A human who just read something should have no trouble recognizing what they just saw, while someone who hasn't read the content will have to cheat by scouring the content for the sentences to derive the answer.
 
Depending on how hard we make that, this should raise the barrier of entry enough to keep out many idiots and assholes, as well as a good number of spammers. The effectiveness is very much implementation dependent. Simply presenting the text verbatim is easily defeated with a simple search, taking maybe a minute. This is better than nothing, but less than ideal.
 
A better implementation would be to present the sentences as rendered images, which would require the use of OCR technology or manually typing the phrases in, raising the amount of trouble considerably.
 
Another idea would be to make minor changes to the text that would go unnoticed by humans, but make a text search harder. This is not unlike zero-cell DVD copy protection, which relies of the over-preciseness of the ripping process to stumble on minute details the relatively sloppy playback process won't notice.
 
For example, given the sentence in text:
  1. I don't think that's something we will do.
 
We could present the user with the sentence:
  1. I do not think that is something we shall do.
 
The meaning is the same, but the text is not. Another idea would be to take advantage of implementation details, replacing characters that look the same to humans, but not to computers.
 
  1. I don't think that's something we will do.
  2. I don't think that's something we will do.
 
These two sentences look the same to us, but are not mathematically equivalent, because one uses non-breaking spaces. This is not a detail humans care about, but it would trip up a casual idiot.
 
Level 2 - Comprehension
 
A Level 2 Lemur CATTA is a comprehension test. It requires the user to not simply parrot what was said, but to actually understand what was said. For example, given some ideas, you'd have to select the ones that jive with the point of the content.
 
  1. • Lemur CATTA, unlike CAPTCHA, can block idiots.
  2. • Spam is the biggest problem with online conversations.
  3. • Lemur CATTA requires you to pick the picture of a lemur.
 
Such a test would still be trivial to anyone who's read and understood the content, but take a decent amount of thought and research for an idiot. It would be very nearly impossible for a machine.
 
At this level, Lemur CATTA is actually more effective than any other CAPTCHA yet devised, because a legitimate participant is already armed with the knowledge necessary to pass the test, but a mechanical turk would have to take the time to study the content.
 
A mechanic turk works by assuming it will take a you no time at all to accomplish this task, so I can pay you a very small amount of money to complete. If, however, you have to actually read a passage and understand it, it won't be worth your time, making such an attack economically unfeasible.
 
The major weakness of Level 2 is it requires a content's author (or other site administrator) to compose the questions, creating extra work. However, with proper implementation, this problem can be ameliorated.
 
Consider the life of a blog entry. It is created, and receives some comments. The stupidness of the comments is directly related to how many comments there are. If there are only three comments, there's not going to be much of a problem.
 
If an issue is easy to understand, or not particularly controversial, it's less prone to being commented on by idiots. Similarly, a blog's small, regular readership are less trouble than people who come from a site like Digg, who are likely to be commenting only on the brief excerpt presented on their site.
 
As such, an author can produce an entry and do nothing, leaving its protection to the automated CATTA. Should the article prove sufficiently popular as to require intervention, the author can create the questions necessary for the next level.
 
That's less troublesome than it may at first sound. Consider the author is likely to already be responding to the comments, possibly out of frustration at having to answer the same question over and over again.
 
  1. For the last time, a Lemur CATTA is an aptitude test. It is not a comment engine, and it is not a substitute for moderation.
 
The author could, instead, enter a password that would cause the comment to be parsed and added to the CATTA, so creating it takes no additional work.
 
  1. • Lemur CATTA is an aptitude test.
  2. • Lemur CATTA is a comment engine.
  3. • Lemur CATTA is a substitute for moderation.
 
It could be pretty well assumed that having to choose the correct answer from a list is going to prevent that question from coming up again.
 
Level 3 - Affinity
 
A Level 3 Lemur CATTA is an affinity test. It is potentially very similar to a Level 2 CATTA, except that its answers cannot be found by simply reading the content. It is not only a more effective block against spammers and idiots, but it also blocks assholes.
 
  1. • Mike Lee is the World's Toughest Programmer.
  2. • The iPhone refund was an insult to early adopters.
  3. • The most important thing is for code to be fast.
 
If you're a fan of this blog, choosing the "correct" sentence should be just as trivial as any of the other tests, but if you're an outsider, and therefore not to be trusted, you should find it difficult, requiring, at the very list, a nontrivial amount of research to defeat.
 
As it relies on a greater context, Level 3 can actually be easier to use, if harder to implement, than Level 2. That's because, once a sufficient body of trivia has been assembled, there's no need to manually create questions for every entry. New questions can be added as necessary.
 
There is a certain amount of flexibility that can come by moving between the levels. For example, a Level 3 CATTA could be deployed to limit the discussion to fans, with entry-specific questions being filled in as they become necessary to keep the conversation focused. These new questions would shift the CATTA toward Level 2, and add to the master database for future instances.
 
A CATTA that asks questions that are broader than the views expressed in the blog blurs the line between Levels 2 and 3, requiring a comprehension of the relevant issues, as well as a kind of affinity, at least for the issue. Informed, but dissenting, opinions cannot (and should not) be blocked.
 
Especially at Level 3, the questions can actually be as fun as they are effective. Imagine a forum for fans of a certain band. This is an area prone to attack by assholes. Now imagine that, at some point, be it when entering the site, before leaving a comment, or when signing up for an account, you were played a 30 second snippet of a song and asked to name it.
 
For a fan of that band, this would be both an easy test and a fun game. For an asshole who just wants to write something disruptive, this would be an extremely high barrier to entry. Are you going to listen to and learn all of a band's songs just to insult their fans? Probably not.
 
There are several techniques that can be added to the implementation of a Lemur CATTA to increase its effectiveness. For example, choosing 3 things out of 9 is a lot more difficult to do by chance than choosing 1 in 3. Even with randomization, it would only take 3 tries to defeat a 1 in 3 test, but it would take (and feel free to correct my math) 504 tries to defeat a 3-in-9 test.
 
You might argue it doesn't take a computer long to do something 504 times, but limiting the number of tries to some reasonable amount, say 3 failed attempts in a row, would protect against brute force attacks.
 
Timeouts can also be added to increase effectiveness, since passing a Lemur CATTA assumes you are using knowledge you've already arrived with, it shouldn't take you very long to answer. Thus, even if one could use Google to defeat an affinity test, it would take them significantly longer than a legitimate fan.
 
Finally, moderation is always important in an online conversation. There's always going to be some level of determination that will defeat any test. There may well be a person who is such an asshole they are willing to listen to a bands music to insult its fans. The key to defeating that kind of person is to swiftly delete their comment.
 
Without Lemur CATTA, assholes can attack a forum as fast as it can be moderated, but with Lemur CATTA they will have to spend so much time crafting their attack that having their work immediately deleted will be immensely frustrated, discouraging future attempts.
 
Using Lemur CATTA
 
The ideas behind Lemur CATTA, as well as my specific implementation and source code, can be licensed for any use under the terms of the Lemur Charity License. In a nutshell, rather than pay me money to use it, you donate that money directly to charity. I have named the Madagascar Fauna Group as the license's beneficiary, due to their efficiency as a charity organization and their holistic approach to fighting the extinction of lemurs.
 
To use Lemur CATTA to protect a web site, whether personal or commercial, simply requires a reasonable one-time donation. To actually use Lemur CATTA in a commercial venture requires a donation of 10% of the profit. In the likely case Lemur CATTA is used as part of a suite of services, a reasonable valuation of its contribution to profits should be made.
 
To protect the brand and license of Lemur CATTA, I also require your implementation retain them both. If you make improvements to the source, or come up with better ideas for tests, I ask, but do not require, you contribute those back to the main project.
 
The purpose of the charity license is two-fold. First, it will raise awareness and funds for my cause. Second, it will prevent a large company from stealing it. I believe Lemur CATTA to be a patentable idea, but patents will not protect it from a company with enough money to out-litigate me. With the charity license, stealing Lemur CATTA would be a PR nightmare.
 
Lemur CATTA in Action
 
I have a simple comment engine, called Alison, running on my site, which provides Lemur CATTA protected comments for the latest entry on my blog. If you would like to see a real-life implementation of Lemur CATTA, that would be a good place to go. In the future, as Lemur CATTA is adopted by other sites, or if other implementations appear, I will add a link to those here.