Figure 1: Blue Frog agent sending an "opt out" request on behalf of a user
[This article was updated in November, 2005. Apparently my original article found its way to the attention of Eran Aloni, at Blue Security. We swapped a couple of Emails and he corrected me regarding a few technical changes that have been made to Blue Security's system since the original article was written. If you want to get a good idea what the folks at Blue Security are up to, their site has an active forum in which Blue Security staff describe the most recent changes to the software, as well as actions taken based on complaints.
Main changes in Blue Security's service since the first draft of this article:
These changes show that Blue Security's software is evolving along with their approach. Is it working? Some 25% of their users have reported a reduction of up to 50% of their incoming spam. That indicates that complaining is working.
]
Blue Security has embarked on a uniquely creative approach to reducing the amount of unsolicited bulk email ("spam") received by its members. By maintaining a "do not email" registry and then enabling the complaints of tens of thousands of members, Blue Security hopes to invert the value proposition of unsolicited bulk email so that commercial email senders have an incentive to respect the "do not email" registry.
In the US CAN-SPAM Act of 2003, Congress required The Federal Trade Commission to report on the feasibility of a "national do not email registry." In its report [1] The FTC essentially concludes that a do not email registry would do more harm than good, since it would potentially serve instead as a "do email" registry. The FTC's primary concern with the do not email registry, however, is that it would be difficult to make it anything other than advisory - junk email senders could simply ignore it, along with the rest of CAN-SPAM's provisions, and target recipients on the list regardless. This is a valid concern if you would be in the position, as The FTC would be, of taking action against those law-breakers. After all, it is easier to throw your hands up in the air and exclaim, "it cannot be done," than to have to deal with hundreds of thousands of what would then be provable instances where CAN-SPAM was being violated.
Blue Security's approach to reducing unsolicited email is to combine a do not email registry with a mechanism that automates and simplifies the user's process of complaining about violations. If messages are sent to Blue Security members, in violation of Blue Security's do not email registry, Blue Security identifies the merchant advertised in the messages and issues an initial complaint. The initial complaint is sent to the merchant, the merchant's domain registry technical contact, and the merchant's Internet service provider. If the initial complaints are not resolved satisfactorily within a ten day grace period, Blue Security writes a script that guides the member's desktop computer in submitting a complaint via the merchant's web site. Each member who receives subsequent email in violation of the do not email registry may send an automated complaint. The total number of complaints sent will always be less than or equal to the number of messages received that violate the do not email registry. The fundamental economics of sending unsolicited emails change when this happens, because the sender now has to ensure that their site has the capacity to potentially handle hundreds of thousands of simultaneous complaints.
Many in the industry have complained that Blue Security's approach may be unethical, for various reasons such as:
Concerns that the wrong web site might receive complaints
Concerns that the flood of complaints amounts to a "denial of service attack"
Belief that there are more effective ways of dealing with unsolicited Email
Concerns on the part of service providers that it will drive up their costs
It is the author's belief that these concerns, while worth taking into account, are adequately addressed by Blue Security's process. In this paper, we will describe Blue Security's process and comment on the ethics of and potential effectiveness of their approach.
The first piece of junk email that I received was in 1995, and, if I recall correctly, I responded with a message consisting of a two-megabyte uuencoded file of white noise entitled "I_AM_VERY_INTERESTED.DOC." I received no response for several days, but later got a bounce message from the mailer software at the sender's site explaining that my message could not be delivered because the sender's inbox was full. This was my first introduction to the junk email wars. Now, more than ten years later, my mail server has processed over one million junk emails destined for mjr@ranum.com. Since I'm the domain registrar for a handful of early internet sites, have been participating on public mailing lists since the early 1980's, and am a published author who writes columns for magazines, it's virtually certain that any time someone harvests email addresses, they will come across mine. As blocking and filtering technologies appear, I try them, and enjoy the peace and quiet until the junk emailers figure out a new way of encoding their messages to get past my guard. This has been going on for a decade, and I don't think I've ever gotten as much satisfaction out of a junk email blocking solution as I did the first time I got the "inbox full" error reply.
Since the passage of the US CAN-SPAM act, I have not noticed any appreciable reduction in my junk email volume. As part of preparing for this paper, I did a brief review of my junk-box and would estimate that about twenty percent of the messages are pornographic/sexual unsolicited emails that violate the terms of CAN-SPAM. Consistently, those emails attempt to direct the reader to a web site, presumably offering some service or membership. In the past when I naively attempted to reply to the messages that offered an "opt out" I simply got added to more mailing lists. I think it's safe to say that CAN-SPAM hasn't worked, The FTC either has no teeth or is not showing them, and we computer users are left on our own, to try to defend ourselves.
Most of us would agree that a customer, or potential customer, has a right to complain and request redress if a vendor is sending annoying messages. Indeed, the legal system has acknowledged this right with respect to telephony and bulk (surface) mail by requiring telemarketers to respect a "do not call" list - fines and penalties are assessed against companies that ignore the list. If you receive harassing telephone calls from telemarketers you can complain to your phone service provider or request the telemarketer, "please add me to your permanent do not call list," whereupon they are required to not call you again. As a consumer, you can call catalog companies and ask them to, "please stop sending me your catalog." You can call Equifax and ask them to put you on a list to stop receiving pre-approved credit card offers. In short, virtually every industry that performs mass marketing implicitly recognizes your right to complain and request that they stop sending you unsolicited messages.
Except for the email marketers.
Figure 1: Blue Frog agent sending an "opt out" request on behalf of a user
Telemarketers and surface mail marketers incur a cost whenever they contact a customer, regardless of whether or not the customer is interested. A catalog company is willing, in fact, eager, to save the cost of mailing printed materials to someone who will simply throw them away. The reason that telemarketers and surface mail marketers show a degree of social responsibility is not because they want to, but because the economics of their medium force them to. If a catalog company's call center was swamped with fifty thousand customers calling to complain, it would cost them a fortune. If a credit card company's card application processing center was swamped with one hundred thousand customers returning their applications after writing on them "do not bother me any more" it would cost them a huge amount of postage and a great deal of time to sort the real credit applications from the complaint responses. In fact, such protests have happened, and they are part of the reason why effective regulations governing telemarketing and surface mail marketing have been put in place.
Sending bulk email does not have the same cost dynamics as telemarketing or surface mail marketing. Contracting a bulk email service to send fifteen million emails costs between one hundred and two hundred dollars. Illicit bulk emailers can send tens of millions of emails for free by using networks of hacked "zombie" computers - often home users' systems who are blissfully unaware that their broadband internet connection is being used to annoy millions of people. The home users are often doubly victimized when their service provider terminates their service for violating usage policies, or their system is placed in one of the Internet "mail black holes" so that any legitimate messages they send are lost. To change the cost dynamics of unsolicited commercial email, it is necessary to place the cost burden for the bulk mailing in the correct place. Early thoughts on how to do this are led by Paul Graham[2] and others, who describe prototype bulk email blockers that would "strip mine" the websites provided as links in incoming messages. Few people are currently pursuing such approaches because of the difficulty of automatically distinguishing legitimate sites from sites sending inappropriate or unwelcome traffic.
Blue Security's approach to protecting their members from unsolicited email is based on the notion of enabling and empowering a customer complaint campaign. In a sense, it is very similar to how online political movements orchestrate letter-writing campaigns in order to influence candidates: portions of the letter-writing campaign are automated, while other portions are carefully controlled by human operators to ensure that the campaign does not go awry. By making it easy for the citizens to get involved by exercising their right to write their elected representatives, the campaign's organizers increase the likelihood that they will take the time to do so. The result is an impressive deluge of letters in the candidate's post-office box. Blue Security's system results in a deluge of legitimate complaints, sent by individual members that received unauthorized commercial email, one complaint per unauthorized message received.
When a member joins the Blue Security community, they install the Blue Frog agent on their desktop system. The member's email addresses are added to Blue Frog's do not email registry, after being hashed using the NBS SHA-1 cryptographic hashing algorithm. The member is now fully signed up.
Blue Security offers the do not email registry hash codes and a reference implementation (source code included) for a routine to check whether a given email address' hash code is in the registry. This reference implementation of the checking routine could be built into a bulk email delivery program as a "do not send" filter or could be used offline to prune out addresses that are flagged as not wishing to receive unsolicited email.
In the event that a Blue Security member receives an unsolicited email, they can forward it to Blue Security's mail analysis center, where it will be reviewed in a combination manual/automated procedure to determine whether or not it represents a bulk mailing incident. For example, if thousands of instances of the same message are simultaneously forwarded as violations to the Blue Security mail analysis center, it will be readily apparent that this is not an isolated incident of a legitimate commercial contact from a business partner or customer. This type of message analysis process has been successfully used for years by commercial junk email-blocking companies such as Postini and Spamhaus and is widely considered to be a legitimate way of identifying objectionable messages.
Another path by which unsolicited email reaches the Blue Security mail analysis center is via the honeypot mail accounts. The messages coming in on the honeypots will almost always be unsolicited bulk emails.
Once Blue Security has a clear violation of the do not email registry, the Blue Security staff will browse to the web site listed in the bulk email, and will ascertain whether or not there is a site contact or opt-out method provided as required by CAN-SPAM. If there is an opt-out method or contact provided, the Blue Security staff will send a warning explaining that multiple messages were received in violation of the do not email registry and to cease and desist. The service provider that is hosting the offending web site will also be notified of the violation. Once notification has been sent, Blue Security will allow the sender a ten day grace period.
Suppose another email is sent on behalf of the same site, once again violating the do not email registry - at this point, Blue Security's analysts will search the web site for an appropriate place to lodge a complaint. In some cases, the site may not have a CAN-SPAM compliant opt-out form, however, in order to engage in commerce on the Internet there must be some type of order form or order entry system available (see Figure 1). Blue Security's analysts determine how one could best lodge a complaint through the site, and develop a script for the Blue Frog agent, that will control it through the process of automatically accessing the site and submitting a complaint on the user's behalf.
Once the complaint-lodging script is available to the members' Blue Frog agents, the agents will contact the web site of the violator and lodge one complaint for each message that the member received in violation of the do not email registry. It is important to emphasize this: the Blue Frog agent does not generate any more traffic than would normally be generated by the desktop users' surfing to the site (as they were invited to) and manually exercising their right to lodge a complaint. Certainly, a massive response of complaints will have a potentially serious impact on a commercial site. On the other hand, a commercial business that has been warned that it is annoying its potential customers and then proceeds to do so, has no right to expect anything other than a flood of angry complaints - whether they are delivered manually or automatically.
Figure 2: Lodging a Complaint Using an Order Form
7: Use of Honeypots
Like many anti junk email providers, Blue Security sets up a number of honeypot accounts to act as collecting targets for junk messages. A honeypot mail account is an account that never receives legitimate email - for example, it might be a duplicate screen-name or a simple mail forwarder that is not monitored by a human (e.g.: fishlips@ranum.com <- NEW! fishlips@ranum.com is no longer a live Email address! Follow the link to see why not!) Since the honeypot email address never originates email, it should never get any traffic whatsoever - messages coming in on the honeypot account are usually a sign that someone has been "harvesting" sites to try to discover target addresses for bulk email. "Harvesting" is explicitly prohibited by the CAN-SPAM act, and honeypots are a very effective way of increasing the accuracy of the junk email identification process because the recipient doesn't need to invest any effort in trying to separate the "good" email from the bad - it's all bad if it lands in the honeypot mailbox.
Blue Software's complaint system has human beings controlling the important stages of the process, so that there is no way an automated complaint will be sent to a site that has not been first warned which then engaged in another violation. One of the primary reasons that automated "strike back" systems have never been widely fielded in computer security is simply because they are automated and therefore can be fooled. With Blue Software's control loop, a situation where an innocent site is "framed" simply won't happen because the Blue Software analysis team will be contacting the real site owners with the initial complaint and there is adequate time to resolve the situation. Automated complaints would not be made unless there was a subsequent violation. Having intelligent human beings in the critical decision-path eliminates the concern that an innocent site would be subjected to unjustified complaints.
Figure 3: Activity Reports
Secondly, there is the question of who benefits. Email marketers are sending their messages for commercial gain, not for fun. In order for their messages to have value, they must somehow reference a site whereby a potential customer could engage in some form of transaction. It would be pointless for one site to launch a marketing campaign in violation of CAN-SPAM on behalf of another site - even a competitor - because the deception would be uncovered and they would become the legitimate recipient of future complaints if they repeated the bulk emailing. Since the Blue Security analysts are looking at the beneficiary of the bulk email campaign, there is no chance of an innocent home broadband user being mis-identified as the origin of the emails - even if the broadband user's machine was used as a transmission station for the bulk email. The contents of the offending email would still point to the beneficiary of the marketing campaign - and the eventual recipient of any justified complaints.
If you don't like the idea of Blue Software's automating your complaints to bulk emailers, you are still free to manually complain - it is your right as a consumer. Indeed, if you are a US Citizen, and are receiving unsolicited commercial email that is in violation of the CAN-SPAM act, you can and should file a complaint with The Federal Trade Commission. Directions for how to contact The FTC are on[3] or you can call toll-free at 1-877-FTC-HELP. As a taxpayer, it is your right to take advantage of the services of The FTC and as a citizen of the Internet it is your right to complain if someone emails you objectionable materials of any sort.
Is Blue Security's approach ethical and moral? Many people have asked, "How is this different from launching a distributed denial of service?" Unlike a denial of service attack, in which a single attacker wields a remote-controlled network of computers to unilaterally attack a target, Blue Security's Blue Frog agent sends a single complaint from each affected user for each email that they received. Unlike a denial of service attack, sites that have done nothing wrong will receive no complaints. Furthermore, the complaints are simply messages asking the site's administrators to comply with CAN-SPAM and a do not email registry, not maliciously-crafted packets designed to interfere with the normal operation of the computer. It's the difference between orchestrating a letter-writing campaign with thousands of like-minded people peacefully protesting, and a lone bomber sending a package-bomb. Both have an impact on the recipient, but they exist on two totally different moral dimensions.
In The FTC's report on the feasibility of a national do not email registry,[1] they conclude that a registry would be a greater detriment to the Internet community than it would be a benefit. The report even considers the possibility of using a hashing algorithm to make it impossible for the "harvester" to directly use the registry as a recipient list, and concludes that hashing would not help because the harvester could use the same hashing approach to validate addresses from their existing email recipient list, thereby defeating the purpose of the hash. In fact, this is poor logic because there is no evidence that bulk emailers care about the accuracy of their lists - since it costs them nothing to send the messages in the first place, there is no reason for them to concern themselves with ensuring that their lists are accurate. Furthermore, if the Blue Security registry were used by an offender to improve their recipient list, they would be including in that list a significant number of the honeypot addresses, which would prove the fact that they were intent on ignoring the do not email registry.
The way that Blue Security is using their do not email registry includes an important mechanism that The FTC's national do not email registry lacks: a mechanism for enforcement that ensures negative consequences for intransigent offenders. If a bulk emailer knows that they will have to sort thousands of complaints out of their transaction records if they abuse the Blue Security member's registry, then there is an incentive for them to prune the Blue Security members' addresses out of their lists. Blue Security's approach is to make it not worth it to have their members on an email list; if they accomplish that goal, they will succeed.
Recently, one of my readers E-mailed me to observe that there is an increasing problem with spammers and phishers using 'botnets of compromised computers to proxy their websites through the compromised system. The problem, my reader points out, is that if Blue Security's complaint system gets launched against a proxied connection through some innocent home user's computer, it won't harm the junk emailer and will doubly victimize the owner of the 'botted system. If Blue Security's response were automated and instantaneous, this would probably be the case. By having a human being in the loop, Blue Security prevents this from happening - recall that the research team will investigate the site and issue a warning to the site's administrators, first. If the site were being virtualized behind a 'botnet, this would be fairly apparent to the researchers. Furthermore, when Blue Security's researchers are investigating a complaint, they are also looking at the domain registration for the site in question and the abuse policy of the site's hosting service - a virtualized site would be rapidly identifiable. At this time, however, commercial junk emailers are not using 'botnets for hosting. A commercial junk emailer needs a website that can remain operational for a day or two (at least) so that they can take a few orders and transact some business; a 'botnet would not be reliable enough and would represent a clear indicator of criminal activity. It would be awkward for a junk emailer to explain how they had come to virtualize their site behind a cluster of compromised computers. In short, for the typical commercial junk emailer, 'botnets represent more trouble than they are worth.
Phishing scammers have a different technical problem and are capable of using 'botnets both to transmit their messages as well as to virtualize their sites. After all, they are already engaging in an illegal activity (felony fraud) and are not even attempting to appear legitimate. Unlike commercial junk emailers, the additional complexity of a 'botnet is to the benefit of the phishers, since it further complicates investigation for law enforcement. The typical phishing scammer's site, however, does not remain active very long. By the time Blue Security's research team has identified the scammer's site, it will likely have been taken down. Scammers, having no legitimate commercial concerns that can be placed in check, are not very susceptible to deterrence.
I believe that Blue Security has come up with a creative combination of community activism, centralized analysis, and distributed processing that offers a decent chance at re-writing the cost equation for bulk email.
As I studied Blue Security's approach, I can see immediately why there have been some strong negative reactions to it. Obviously, the people who really don't like Blue Security's system are running the sites that send unsolicited commercial email. They are staring at the possibility of a radical shift in their cost equation, since many of them rely on hosting services that charge overage for high bandwidth usage. Currently, they can predictably spend one hundred dollars to host a site, and email out fifteen million ads for free - netting a few thousand customers will make them very profitable. With Blue Security's system in place, their revenues will be dramatically cut because they will spending more time coping with complaints than processing orders. Personally, I am not particularly concerned if they are inconvenienced; they don't appear to be concerned whether I am inconvenienced, either.
The regulators probably aren't very happy to see someone attempting to establish a do not email registry after they had written a report to the US Congress saying "It can't be done." Bulk email is a problem for ISPs that have to carry it across their networks, but although most ISPs have strict abuse policies regarding bulk emailers abusing their resources, there still are a number that make a lot of their revenue from hosting shady web sites run by junk emailers. These junk email-friendly ISPs are probably not very happy, either, since a storm of consumer complaints may represent a load spike for virtually any hosting service and may consume their network bandwidth. The cost equation for hosting those web sites might change unpredictably for those ISPs, and they're understandably reluctant to discover what might happen. After all, unsolicited bulk email isn't a problem for them if it's being sent from unsuspecting broadband customers on another carrier's network.
Where could this all end up going? I'm looking forward to seeing Blue Security's approach evolve, as it surely will. With recent legal decisions in favor of AOL and Microsoft against senders of unsolicited commercial email, it appears the courts are finally establishing precedents in favor of the rights of computer users to enjoy the Internet in peace. Perhaps a future version of Blue Frog will automate the process of forwarding offending emails to The FTC, or to a lawyer who is pursing a class-action lawsuit against the sender.
I think that what Blue Software is doing is a worthy experiment and I appreciate the elegance of how they've structured their system and process so that innocent parties don't get hurt or inconvenienced. I look forward to watching their system evolve and - maybe someday - I can look forward to a nearly junk-free email inbox.
[1] http://www.ftc.gov/reports/dneregistry/report.pdf - The Federal Trade Commissions report to Congress regarding the feasibility of a national "Do Not Email" registry.
[2] http://www.paulgraham.com/ffb.html - "Filters That Fight Back" - Paper by Paul Graham, in which he describes the denial of service effect that might be achieved by bulk email filters that follow links contained in the messages to simultaneously improve the statistical analysis and provide negative feedback against bulk emailers' web sites.
[3] http://www.ftc.gov/bcp/conline/pubs/alerts/dnealrt.htm - "Keep your Email Address Unlisted - there is no 'National Do Not Email Registry'" The Federal Trade Commission describes scammers and spammers using "sign up here for no email" as a harvesting technique. The FTC's recommendation: don't publish or use your Email address (i.e.: FTC encourages you to accept victimization).
http://www.ftc.gov/bcp/conline/pubs/buspubs/canspam.htm - The Federal Trade Commission's Requirements for Commercial Emailers. A succinct description of what a commercial emailer should do in order to be in compliance with The CAN-SPAM Act.
http://www.ftc.gov/bcp/conline/edcams/spam/rules.htm - The Federal Trade Commission's summary page of Rules, Regulations and Acts regarding unsolicited commercial Email, pornographic and offensive Email, and Email fraud.
http://www.paulgraham.com/spam.html - "A Plan For Spam" - Paper by Paul Graham, which widely popularized the use of statistical categorizers for bulk email. Graham's paper offers a fairly good treatment of the economics of bulk emailing.
