Spam Survey Results, Dec. 2, 2002

Spam Survey Results, Dec. 2, 2002

Summary
Original Posting
Responses:
Spam Bait

Summary

I posted a question in October to the SANS University Security mailing list, unisog-at-sans.org, and several other University mailing lists, asking what people are doing about spam. Below is the original posting followed by the responses with minor editing.

Many sites are using or evaluating SpamAssassin, http://spamassassin.org, or a commercial product such as PureMessage (formerly PerlMX), http://www.activestate.com/products/PureMessage/, to tag spam. These applications create a score based on over 150 regular expressions, plus they optionally interface to black hole lists and to spam databases.

Vipul's Razor,http://razor.sourceforge.net/, is one of the spam databases. Razor is linked to SpamNet, http://www.cloudmark.com/products/spamnet/. The Spamnet web site currently lists these statistics:

SpamFighters 220,745
Emails processed today 5,117,295
Spam caught today 1,729,869

Another spam database is DCC, the Distributed Checksum Clearinghouse, http://www.rhyolite.com/anti-spam/dcc/. Currently, the DCC's graphs show an average of 60% of messages submitted are "likely spam". SpamAssassin and PureMessage can be configured to use fuzzy matches against these databases in creating a spam score for a message.

Many sites are using black hole lists to either block mail from certain hosts or in scoring messages in SpamAssassin, etc. However, many replied that the black hole lists are not always trustworthy.

Generally blocking and tagging is done site-wide with no user option. There are some features which sites have made "opt-out". No one has an "opt-in" solution.

Finally I have "munged" all the addresses in this page, to make it harder for spambots to harvest them to add to spam lists. Replace the "-at-" by "@" to use the addresses. See the last entry for more on munging.

There is a lot of information here. Thanks to all who replied.

Jerry Berkman, UC Berkeley, jerry-at-uclink.berkeley.edu

Original Posting ...

From jerry-at-uclink.berkeley.edu Wed Nov 27 15:00:58 2002
Date: Mon, 14 Oct 2002 16:27:37 -0700 (PDT)
From: Jerome M Berkman <jerry-at-uclink.berkeley.edu>

We have noticed a huge increase in spam this year.  We are trying to
figure out what to do about it, and we are wondering what others are
doing, and especially what has proven successful.

You can send your response to me and I will summarize; if you want, I
will summarize without saying what each site is using.

The server I help administer, uclink.berkeley.edu, has 40,000+ accounts
(students, staff, and faculty).  Users access their mail via POP and IMAP
clients such as Eudora, Netscape, and Outlook Express.

Please let me know if you have tried any of the following (or anything
else) and whether it was useful, especially on a large scale system:

- blackhole lists, RBLs, DULs, etc.  Which do you use or did you 
create your own?

- if you block, do you block on content, on IP address, or on "from "
address?

- tar pits to slow up the spam arrival rate 

- open source applications such as spamassassin or spamcop

- commercial solutions, such as those offered by BrightMail,
TrendMicro, and Sendmail, Inc.

Is your system opt-in or opt-out?

Do you use spamtraps?  What types?

Do you block spam or put it in a "grey" folder for the user to decide
what to do with?  If you block, do you block during the SMTP protocol
or bounce it later or just delete it without notice?

How do you prevent harvesting of addresses in your University's web
pages?  In departmental directories on the web?  From ldap directories?

	- Jerry Berkman,
	  UC Berkeley, jerry-at-uclink.berkeley.edu, 1-510-642-4804

Responses ...

From iglesias-at-draco.acs.uci.edu Wed Nov 27 15:00:58 2002
Date: Mon, 14 Oct 2002 21:18:34 -0700
From: Mike Iglesias <iglesias-at-draco.acs.uci.edu>

We're using Mailscanner and SpamAssassin for all email that goes to
user-at-uci.edu.  That mail is then forwarded on to the user's delivery
point, either a departmental system or one of our central mail servers.

If the header 

  X-UCIRVINE-SpamCheck:

exists in the email, SpamAssassin thought it was spam.  So our users can
use the filtering options in their mail programs to filter spam into
another folder (we don't recommend they just delete it).  They can look
at the "spam" folder when they feel like it to make sure no legitimate
stuff got in there.

Read more about it at http://www.nacs.uci.edu/email/spam-assassin.html

Mike Iglesias                          Internet: iglesias-at-draco.acs.uci.edu
University of California, Irvine       phone:    949-824-6926
Network & Academic Computing Services  FAX:      949-824-2069

From pckizer-at-tamu.edu Wed Nov 27 15:00:58 2002
Date: Tue, 15 Oct 2002 01:10:33 -0500
From: Philip Kizer <pckizer-at-tamu.edu>

Jerome M Berkman <jerry-at-uclink.berkeley.edu> wrote:
>The server I help administer, uclink.berkeley.edu, has 40,000+ accounts
>(students, staff, and faculty).  Users access their mail via POP and IMAP
>clients such as Eudora, Netscape, and Outlook Express.
>
>Please let me know if you have tried any of the following (or anything
>else) and whether it was useful, especially on a large scale system:

We have ~60,000 local users, and roughly 90,000 deliverable addresses in
the top-level @tamu.edu directory.  We have a central email system, but
there are a significant number of people that still use departmental mail
servers.  Almost all off- to on-campus SMTP traffic is handled by central
servers running sendmail.

We are finishing the process of installing PerlMx for virus filtering
and making use of their Spam scanning to "tag" messages that are
suspected of being spam for analysis and possibly future opt-in to
blocking.  PerlMx operates via sendmail's MILTER interface and includes
RBL checking, though I plan to put some RBL checks into mandatory
blocking mode (like rfc-ignorant that's based on
strict technical criteria).  Virus blocking is mandatory and a hit
results in a 500-level SMTP response so our servers are never
responsible for attempted delivery of either the virus messages or the
bounces.

>How do you prevent harvesting of addresses in your University's web
>pages?  In departmental directories on the web?  From ldap
>directories?

We're still pondering that one.

-philip

-- 
Philip Kizer, Senior Lead Systems Engineer, Texas A&M University
USENIX Liaison to Texas A&M University         <usenix-at-tamu.edu>
Texas A&M CIS Operating Systems Group, Unix   <pckizer-at-tamu.edu>

From rcgraves-at-brandeis.edu Wed Nov 27 15:00:58 2002
Date: Tue, 15 Oct 2002 02:08:40 -0400 (EDT)
From: Rich Graves <rcgraves-at-brandeis.edu>

On Mon, 14 Oct 2002, Jerome M Berkman wrote:

> - blackhole lists, RBLs, DULs, etc.  Which do you use or did you 
> create your own?

The various free open relay databases are horribly inaccurate and can't
distinguish between spam houses and mom-and-pop ISPs that host non-profits
associated with the university.  I tried a few supposedly conservative
lists for a while, had to stop. Lots of false positives and collateral
damage, very little actual spam stopped that we wouldn't have caught some
other way.

SPEWS is *way* too aggressive and sloppy. A typo blocked 1/4 of AOL some
months back, and it took them 16 hours to fix.

I am beginning to consider DUL-type lists now that SMTP AUTH and webmail
have become viable options for roving users.

DUL blocks suck for legitimate Linux users for whom direct-to-mx makes
a lot more sense than smarthost, though.

> - if you block, do you block on content, on IP address, or on "from "
> address?

All, and more.

About 1% of all email is blocked with "relaying denied." Almost all of this
is Klez and similar viruses that guess an SMTP server based on collected
email addresses.

About 1% of all email is blocked by Subject: and other header patterns in
sendmail.cf. Klez patterns, toner supplies, ADV:, long strings of
whitespace followed by hexadecimal tags intended to confuse checksumming
filters.

About 3-5% of all email is blocked by a simple body string-matching Milter.
Patterns are entered by hand for persistent sources of spam. Slightly more
viruses are blocked than spams. The same milter also renames executable
attachments that get through from virus.exe to virus_exe. It's a very
simple hack in C, no attempts made to get inside Base64 encoding or to
understand MIME structures. Sampling one MX for the last day, first
character starred out so that I'm not prevented from sending this to you:

    279 virus: *iframe src=3Dcid:
    125 spam body: *betahost.net/
     52 spam body: *tnsoft.com/~mailtouch/
     33 spam body: *ebt Consolidation Quote
     31 spam body: *ww.emailremovals.com
     22 spam body: *DBGMGR.EXE
     22 spam body: *com?subject=remove
     15 spam body: *ustomerService, 427-3 Amherst Street, Suite 319, Nashua, NH
     11 virus: *he attachment is the original mail
     10 spam body: *ll our mailings are sent complying to the
					proposed H.R. 3113 Unsolicited
     10 spam body: *free3host.net
      9 virus:
	*v+LRCQID7dIDFEECggDSLm9df8C/zSNKDBBAAoGA0AEUQ+FEN23f7doqAT/dCQk/xWcEQmDxCTD
      8 spam body: *NLARGE YOUR PENIS
      7 virus: *iframe src=cid:
      7 virus: *he file is the original mail
      6 spam body: *nudecelebsluts.com/
("spam body" and "virus" lines split across two lines for readability)

I've got an email/domain-based accessdb with a few hundred entries.

4 or 5 netblocks (including flowgo.com) are completely blocked at the
campus border. This is an extreme step taken only for persistent sources of
spam actually received at Brandeis. I verify that they are also blocked by
multiple DNSBLs and also refer to
http://www.stanford.edu/group/itss-ccs/security/filters.html

We don't block everything listed there, and I'm not sure Stanford does, 
either. Several of the IP ranges they list are no longer associated with 
the companies they list. This is a risk with any IP-based block list; if 
you succeed in getting the spammers booted, you punish the next colo 
customer.

This regex helps a lot with "random" emails:

#  Regular expression to reject:
#    * Exactly 16-character usernames @yahoo.co.uk
#    * numeric-only localparts from aol.com and msn.com and brandeis.edu
#    * localparts starting with a digit from juno.com and hotmail.com
#    * localparts longer than 16 characters from aol.com
#    * localparts w/ _ and longer than 16 characters and at least 1 digit
#      @(hotbot|juno|rocketmail|excite|hotmail|mail).com
#    * test*@test*.com
#    * COVAD/Cox spammer pattern oqvg_quqe_s_c_f@brandeis.edu
#
Kcheckaddress regex -a@MATCH
   ^(mailer\-daemon[0-9]+.*<@.*|
	orgasm[0-9]+<@.*|
	subscriber_services[0-9]+<@.*|
	test.*<@test.*\.com|
	.{16}<@yahoo.co.uk|
	[0-9]+<@(aol\.com|msn\.com|brandeis\.edu)|
	[0-9][^<]*<@(hotmail|juno)\.com|
	.{16}[^<]+<@aol\.com|
	.{10}.*_.{2}.*[0-9].{2}.*<@
		(hotmail|juno|rocketmail|hotbot|excite|yahoo|msn|mail)\.com|
	.*_...._._._.<@.*brandeis\.edu)\.?>
(regular expression above split across lines to improve readability; join
together with no white space for actual use in sendmail)

> - tar pits to slow up the spam arrival rate 

I have define(`confBAD_RCPT_THROTTLE',`10')dnl but don't see it helping 
much. Spammers don't often get addresses wrong.

> - open source applications such as spamassassin or spamcop

I tested spamassassin but it seemed a pig, and unreliable too. The code
seems very messy and the support community insufficiently technical IMO.

I just use eu.org's vbsfilter.c with a static array of strings to reject.
Stupid, but effective and with only very minor performance impact.

> Is your system opt-in or opt-out?

Opt-out.

No one but the help desk has opted out. I think I've done a reasonable job
warning people about what I'm doing. Their faith in me is a bit disturbing.

You wouldn't be able to get away with it at Berkeley. I do audit results
and take blame -- all rejects make it clear that postmaster-at-brandeis.edu
is doing it, and postmaster-at-brandeis is immune from filtering.

> Do you use spamtraps?  What types?

There are a couple hidden mailto: links on www.brandeis.edu intended as
spamtraps. What I've found is that they attract 4x as much Microsoft
Outlook viruses as spam. Since we don't want to block everyone who could
visit our web pages and get Klez this makes the spamtraps pretty much
useless. We certainly can't automate anything based on spamtrap input, we'd
block too much legitimate mail from victims of viral forgeries.

Another form of spamtrap is procmailing everything in a Korean character
set sent to hostmaster-at-brandeis.edu and a few other administrative
addresses in a Korean character set to a spam folder. Obviously we're not
going to censor Korea as a rule, but if it's a persistent source hitting
both hostmaster and historydeparment then it goes into accessdb or if a
body match is required the milter.

> Do you block spam or put it in a "grey" folder for the user to decide
> what to do with?  If you block, do you block during the SMTP protocol
> or bounce it later or just delete it without notice?

The milter or sendmail rules block during SMTP conversation.

I would prefer to both reject and deliver to a spam folder so that I can
search for false positives, but there just doesn't seem to be a reasonable
way to do that with the milter API. Return to sender seems a better 
approach from the privacy perspective anyway.

The greymail idea is attractive, but we don't have the resources. I also 
think it's a cop-out. Legit email delivered to greymail without notice to 
the sender is MUCH WORSE than legit email bouced.

> How do you prevent harvesting of addresses in your University's web
> pages?  In departmental directories on the web?  From ldap
> directories?

ldap ports 389 & 636 are blocked at the border.

finger @brandeis.edu is rate-limited by xinetd.

Our web directory returns the formulation rcgraves<em>at</em>brandeis.edu
to non-Brandeis IPs. Evangelizing similar formulations elsewhere on the
Brandeis web.

I not-so-recently spammed this to all students.

Everybody: Dealing with email viruses and junk email

 On the average day, Brandeis refuses about 5% of all incoming email as
 identifiable viruses or spam. Some people are delighted with this;
 some don't believe we block anything because they get so much; on
 rare occasions we block legitimate email.

 It's difficult to make everyone happy. Understand that because we
 prefer to work in an uncensored academic environment, avoiding the
 accidental loss of legitimate email takes priority over the elimination
 of unwanted email.

Some guidelines for dealing with the problem:

 * The second easiest way for junk mailers to get your email address
 is to find it on a web page. If you have your email address listed
 on a public web page (simply search http://www.brandeis.edu/search/
 to find out), consider mangling it in ways that are obvious to 
 humans but not to spam robots. For example, the Brandeis online
 directory now returns rcgraves<em>at</em>brandeis.edu for queries
 from off campus. It's too bad that we have to do this, but we do.
 The first way for junk mailers to get your email address is for
 you to give it to them directly. Approach "free contests" and "free
 registration" with suspicion.

 * Junk email may be bounced (please include full headers) to
 spam-at-brandeis.edu. Viruses that seem to originate at Brandeis
 (first Received: header includes 129.64) may be bounced to
 security-at-brandeis.edu. Junk mail by its nature generates a high
 volume of duplicate reports, so do not expect a personal response.

 * Don't worry too much about Windows email-borne viruses if you
 read your email through brandeis.edu. You will receive them, but
 they will be renamed so as to be harmless. Most recent 
 infestations have come from AOL or Hotmail accounts. For technical
 details see http://my.brandeis.edu/bboard/seb/0000Jo.html

 * Don't put too much stock in From: lines. Modern viruses scrounge
 your hard drive for any addresses and forge messages from each
 of them. This means you can't trust email messages simply
 because they appear to be from a friend, and it also means that
 there's usually little point in replying to the supposed sender
 to warn them that they have a virus. Usually, they don't; it was
 a forgery by a virus on someone else's computer.

 * If your email is bounced inappropriately, redirect the complete 
 bounce message to postmaster-at-brandeis.edu. The addresses 
 postmaster, spam, abuse, and security are exempt from almost all 
 forms of junk email and virus filtering.

 * If you have a question, comment, or complaint that others might
 share, please consider posting it to the tech support bboard at
 http://my.brandeis.edu/bboard/q-and-a?topic_id=21 Responding
 to questions there is a high priority because answers posted in
 public can help more people.

 * Norton Antivirus is on the UNet CD and distributed via
 http://virus.brandeis.edu/.  If you are running Microsoft Windows
 and you do not have current antivirus software installed, you
 need to. Consult http://www.brandeis.edu/its/ or the Help Desk.

From lsheldon-at-creighton.edu Wed Nov 27 15:00:58 2002
Date: Tue, 15 Oct 2002 7:48:49 CDT
From: Larry Sheldon <lsheldon-at-creighton.edu>

> - blackhole lists, RBLs, DULs, etc.  Which do you use or did you 
> create your own?

We do some of our own--I'd like to use SPEWS but some of my customers 
are not quite ready for that.

> - if you block, do you block on content, on IP address, or on "from "
> address?

IP address or address range, for ours--dunno what Postini's method is.

> - tar pits to slow up the spam arrival rate 

Fun, but in-effective.

> - open source applications such as spamassassin or spamcop
> 
> - commercial solutions, such as those offered by BrightMail,
> TrendMicro, and Sendmail, Inc.

Postini, http://www.postini.com.

> Is your system opt-in or opt-out?

I don't understand the question in this context.

We are trying to get all mail going through Postini (AV is the justification,
and AV service is not optional within Postini, spam is the attraction and
individuals have great latitude in spam filtering.

> Do you use spamtraps?  What types?

No.

> Do you block spam or put it in a "grey" folder for the user to decide
> what to do with?  If you block, do you block during the SMTP protocol
> or bounce it later or just delete it without notice?

Postini "quarantines" in their system with auto discard in 14 days.
The stuff blocked by list refuses the mail.  (Egregious cases are dropped
silently by the boundary router.

> How do you prevent harvesting of addresses in your University's web
> pages?  In departmental directories on the web?  From ldap
> directories?

Dunno.

-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
. L. F. (Larry) Sheldon, Jr.                                            .
- Unix Systems and Network Administration                               -
. Creighton University Computer Center-Old Gym                          .
- 2500 California Plaza                                                 -
. Omaha, Nebraska, U.S.A.  68178       Two identifying characteristics  .
- lsheldon-at-creighton.edu               of System Administrators:     -
. 402 280-2254 (work)                Infallibility, and the ability to  .
- 402 681-4726 (cellular)               learn from their mistakes.      -
. 402 332-4622 (residence)                (Adapted from Stephen Pinker) .
- http://www.creighton.edu/~lsheldon                                    -
.              Si hoc legere scis nimium eruditionis habes              .
-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-

From jtillots-at-pharmacy.purdue.edu Wed Nov 27 15:00:58 2002
Date: Tue, 15 Oct 2002 08:31:04 -0500 (EST)
From: Jenett Tillotson <jtillots-at-pharmacy.purdue.edu>

I've installed spamassassin on my Solaris sendmail box.  I add
****SPAM**** to the header which bothers me cause I don't like changing
the original email message.  But the users would never notice any extra
header I add to the mail message.  I also add a new header which has one
"*" for every "spam point" the message got.  I had to whitelist our domain
or email the dean sent would get marked as spam, and that's bad.

I've heard only good things about this system.  Some email does get tagged 
as spam, but most users are using it to quickly determine what is spam and 
what isn't.  Some users delete everything which gets tagged as spam (which 
can be bad) and love that.  Some users are smart enough to use the "*" 
header line to delete stuff that's has a large enough spam score to really 
be spam, but doesn't delete stuff like the CNN report they have mailed to 
their mailbox every morning.

I love spam assassin cause I can easily modify it to meet our needs.  And 
it does a lot more than just spam filtering, like I use it to capture 
attachments with bad extensions - that is the real reason I installed spam 
assassin.  The spam tagging is just icing on the cake.

Jenett Tillotson
School of Pharmacy
Purdue University

From dwhitesi-at-ist.uwaterloo.ca Wed Nov 27 15:00:58 2002
Date: Tue, 15 Oct 2002 10:02:37 -0400 (EDT)
From: Dawn Keenan Whiteside <dwhitesi-at-ist.uwaterloo.ca>

> The server I help administer, uclink.berkeley.edu, has 40,000+ accounts
> (students, staff, and faculty).  Users access their mail via POP and IMAP
> clients such as Eudora, Netscape, and Outlook Express.

At the University of Waterloo, our user community is about 30,000
accounts (students, staff and faculty).  Users access mail via POP,
IMAP, web mail (mostly IMP, by Endymion MailMan in a few places still)
or on the server itself (Pine, elm, mutt, and even mail).

> - blackhole lists, RBLs, DULs, etc.  Which do you use or did you 
> create your own?

We have far more SMTP servers on campus than an institution of our size
has any business supporting, for historical and political reasons.
Most of these systems use a centrally managed, locally tweakable,
sendmail configuration.  The default config currently uses the
relays.osirusoft.com list and allows the sysadmin to not use that list
or add other suggested lists.  I review external RBLs about 3-4 times a
year and make adjustments to the default/suggested blacklists in the
sendmail configuration.  I was planning on switching to njabl because of
its reputation, but now that it looks like that RBL is going to move to
a payment scheme it will remain just a 'suggested' blacklist for a while.

Because we occasionally see legitimate sites being blacklisted or have
mail recipients complaining they're not getting mail from blocked
sites, we have a web interface to RBL exemptions (see
http://ego.uwaterloo.ca/RejectedMail.html, which is included in the
sendmail rejection message for blacklisted IPs).  Anyone can get a
temporary exemption for an IP, but they need to communicate with an
authorized person to get a permanent exemption.  Authenticated users
can opt out of RBL filtering altogether.

> - if you block, do you block on content, on IP address, or on "from "
> address?

We block on IP only.

> - tar pits to slow up the spam arrival rate 

Nothing above and beyond the sendmail defaults.

> - open source applications such as spamassassin or spamcop

We're deploying SpamAssassin and are quite happy with it.  I recently
gave a talk for local system administrators on setting up SpamAssassin: 
	http://ist.uwaterloo.ca/~dkeenan/talks/spamassassin/

> - commercial solutions, such as those offered by BrightMail,
> TrendMicro, and Sendmail, Inc.

We've been approached by vendors of commercial solutions and not
found anything in their systems that justifies spending the $$.

> Is your system opt-in or opt-out?

By default, all inbound mail is checked for RBL senders and users can
opt out.  When SpamAssassin is more fully deployed, it will also be
opt-out (and I'd better get that web interface for user configuration
changes done soon).

> Do you use spamtraps?  What types?

Not institution wide.

> Do you block spam or put it in a "grey" folder for the user to decide
> what to do with?  If you block, do you block during the SMTP protocol
> or bounce it later or just delete it without notice?

We tag spam and don't put it anywhere.  We provide information on how
to filter mail using a variety of clients and encourage users to not
automatically delete any mail.

> How do you prevent harvesting of addresses in your University's web
> pages?  In departmental directories on the web?  From ldap
> directories?

Our default sendmail configuration doesn't allow EXPN,VRFY or any other
information-disclosing verbs.  Our CSO/PH-compatible directory interface
has a limit of about 20 hits returned fo a search.  We don't allow guest
access to our Active Directory.

--
Dawn Keenan (Whiteside), IST, University of Waterloo, etc. etc.

From wcolburn-at-nmt.edu Wed Nov 27 15:00:58 2002
Date: Tue, 15 Oct 2002 08:14:14 -0600
From: "William D. Colburn (aka Schlake)" <wcolburn-at-nmt.edu>

I use two major methods.  First, I take URLs from SPAM and add them to a
content block list.  The idea being that they are positively SPAM, and I
can blow them away.  Then, I watch the logs for sites that send nothing
but SPAM, and block those sites directly.  Another useful trick is to
check which "User unknown" recepients get the most mail, and then trap
that mail as known SPAM.

Mail containing SPAM is refused unless it is addressed to postmaster.
Mail from sites that send nothing but SPAM is refused an a connection
milter, so they never even get a chance to say HELO.

Yesterday, my milter matched 671 emails with spam in the content and
refused them.  It matched 217238 unrepentent spammers by hostname, and
refused to even talk to them, but 169126 of those connections were a
single (badly written) spam engine belonging to Cross Media Marketing
Corporation.

My spam that I have received is at http://www.nmt.edu/~wcolburn/spam/
if you want to look at it.  Overall, I have had only had three actual false
positives that have been reported (each blocked message gets my phone
number in the error message) in the past year and a half.  I had some
fake false positives when I accidentally added the empty-string to the
list of known spam.  I rewrote my milter to detect the empty string
after it happened a couple of times.

I consider ~0 false positives to be a very good thing.  I am blocking
spam on a daily basis, but I do not think I'm getting anywhere near to
blocking even half of it.  This is just a guess though, since I have no
way to know if messages I suspect are SPAM are actually SPAM unless I
read someone elses email or they report to it me.  Few people seem to
care to report SPAM, they just consider it "normal" to receive that
much.

I have found, as other people have reported, that most SPAM comes from a
small group of spammers.  People who get 10+ messages a day, and write
them off as normal, are quite suprised when they finally report it to me
and it completely stops the next day.

--
William Colburn, "Sysprog" <wcolburn-at-nmt.edu>
Computer Center, New Mexico Institute of Mining and Technology
http://www.nmt.edu/tcc/     http://www.nmt.edu/~wcolburn

From Mike.Carter-at-Colorado.EDU Wed Nov 27 15:00:58 2002
Date: Tue, 15 Oct 2002 08:42:04 -0600 (MDT)
From: Mike Carter <Mike.Carter-at-Colorado.EDU>

Jerome;

We're in the pre-pilot stages of a new anti-spam initiative, so take this
for what it is worth, ie not in production, yet.

> The server I help administer, uclink.berkeley.edu, has 40,000+ accounts
> (students, staff, and faculty).  Users access their mail via POP and IMAP
> clients such as Eudora, Netscape, and Outlook Express.

My group admins a collection of mail servers that have about the same number
of active accounts.  The largest majority of these are IMAP only but we
do serve POP as well and shell access via Pine and Elm.

> Please let me know if you have tried any of the following (or anything
> else) and whether it was useful, especially on a large scale system:
> 
> - blackhole lists, RBLs, DULs, etc.  Which do you use or did you 
> create your own?

I have been experimenting with RBL+ from mail-abuse.com for about six months.
This is the pay-for version where we zone transfer the DNS information to
our servers.  While this does appear to be a pretty well maintained service,
the problem that we're struggling with is how to informatively let our
users know what to do about problems.  About the only solution we have 
come up with for that is to have them tell a confused sender to resend a
message to a non-blocked address and have that address auto-answer by doing
an RBL+ check on message headers.  We could likely accomplish this using
something like spamassassin.

We have also maintained a local blacklist for three or four years using
sendmail's access database.  While the group that manages this has come
up with some creative automated mechanisms for putting domains/IPs on this
list, it is very resource intensive to maintain.

> - if you block, do you block on content, on IP address, or on "from "
> address?

The local blacklist blocks on domain/IP.

> - tar pits to slow up the spam arrival rate 

No but we do have sendmail log file alerts that notify the black list
maintainers when it looks like we are being spammed.

> - open source applications such as spamassassin or spamcop

Our current direction is to use spamassassin on a global level, ie from
the system procmailrc.  We've been a procmail site for many years so this
piece will be easy for us to implement.  We also use procmail to defang
attachments for viruses at this level.  Our pilot will implement sa simply
to tag messages (subject and standard sa headers) but will do nothing with
the results besides passing it along to the user's inbox.  Users will be
responsible for filtering their own messages.  We are opting for the 
global procmailrc initially to be able to get something out sooner rather
than later, ie non-maintenance of N-thousand individual procmailrcs.  We
have successfully deployed a highly available spamd (runs on a Sun Cluster
with scripts/crons to transfer the service in the event of host crash, etc.)

> - commercial solutions, such as those offered by BrightMail,
> TrendMicro, and Sendmail, Inc.

We have done some basic evaluation of PerlMX and it looks like a good product
but we're going to start "free" with sa and see how it feels.  Our Health
Sciences campus has adopted PerlMX and seems to be happy with the product.

> Is your system opt-in or opt-out?

Our global procmailrc will allow for opt-out but we will initially not
advertise this as to do this for the POP/IMAP folks requires the development
of a secure/authenticated web mechanism for doing this.  While we do have
various mechanisms to provide this, our goal at the present is to give the
users something as fast as possible.  Opt-out will require a contact with
our IT Service Center.

> Do you use spamtraps?  What types?

Not sure what you're referring to here?

> Do you block spam or put it in a "grey" folder for the user to decide
> what to do with?  If you block, do you block during the SMTP protocol
> or bounce it later or just delete it without notice?

We're going to leave that up to the user.  We would obviously prefer that
their filters delete it but we'll leave it up to them.

                                - Mike

From ptemples-at-temples.com Wed Nov 27 15:00:58 2002
Date: Tue, 15 Oct 2002 10:52:32 -0400
From: Phil Temples <ptemples-at-temples.com>

Jerome,

*Definitely* go with spamassassin, combined with an HTML interface that 
allows users to access their quarantined (spam) email.  I haven't 
bothered with the latter, as my system does not serve users. But 
spamassassin has worked wonderfully for me.

Phil

From sasprague-at-ucdavis.edu Wed Nov 27 15:00:58 2002
Date: Tue, 15 Oct 2002 08:14:23 -0700
From: Sharie Sprague <sasprague-at-ucdavis.edu>

    [The following text is in the "iso-8859-1" character set]
    [Your display is set for the "US-ASCII" character set]
    [Some characters may be displayed incorrectly]

Jerome,

My responses are in UPPERCASE.  Please note that we operate a departmental
Exchange server.

~Sharie 

Sharie Sprague/Information Systems Planning & Support 
Offices of the Chancellor & Provost/Mrak Hall, 5th Flr/One Shields Ave. 
University of California/Davis, CA   95616 
Voice: 530-752-9928/FAX 530-752-3832/E-Mail: sasprague-at-ucdavis.edu 

-----Original Message-----
From: Jerome M Berkman

...

Please let me know if you have tried any of the following (or anything
else) and whether it was useful, especially on a large scale system:

- blackhole lists, RBLs, DULs, etc.  Which do you use or did you 
create your own?  WE TRIED USING THESE AND FOUND THEY FLAGGED ONE OF OUR
CAMPUS MAIL SERVERS (MILLARD.UCDAVIS.EDU) AND FLAGGED MANY MESSAGES THAT
WERE LEGITIMATE.

- if you block, do you block on content, on IP address, or on "from "
address?  WE BLOCK SOME ON FROM:, SOME ON SUBJECT CONTENT, SOME ON BODY
CONTENT, NONE ON IP ADDRESS.

- tar pits to slow up the spam arrival rate HAVEN'T TRIED THIS.

- open source applications such as spamassassin or spamcop HAVEN'T TRIED
THIS.

- commercial solutions, such as those offered by BrightMail,
TrendMicro, and Sendmail, Inc.  WE USE NEMX POWERTOOLS FOR EXCHANGE
(http://www.nemx.com/products/powertools/index.asp).  INITIALLY WE
PURCHASED THE PRODUCT TO LIMIT RECEIPT OF EXECUTABLE FILES (ALSO .BAT,
.PIF, ETC.), BUT NOW WE'RE USING THE FILTERING ASPECTS FOR FROM:,
SUBJECT, AND BODY FILTERING.

Is your system opt-in or opt-out?  OPT-OUT

Do you use spamtraps?  What types?  NO.

Do you block spam or put it in a "grey" folder for the user to decide
what to do with?  If you block, do you block during the SMTP protocol
or bounce it later or just delete it without notice?  WE QUARANTINE THE
MESSAGES INTO A FOLDER ONLY OUR DEPT. EXCHANGE ADMINS CAN ACCESS---THIS IS
IN CASE WE CONFIGURE A RULE WITH UNINTENDED CONSEQUENCES.

How do you prevent harvesting of addresses in your University's web
pages?  In departmental directories on the web?  From ldap
directories? OUR E-MAIL ADDRESSES ARE POSTED ON OUR WEB SITES (WE FEEL IT'S
GOOD CUSTOMER SERVICE).

From anonymous (by request)

We have also noticed a large increase in spam, although it
hadn't reached a point where is was causing a problem with
the mail server in terms of storage or bandwidth, it
certainly was enough to cause a few users to begin to
complain about it.  We made a few changes that seem to be
making a difference so far, although I have found it
difficult to quantify just how much of a difference.

I would appreciate if our site identity not be included in
the summary.

We are using Sendmail 8.11.6 utilizing the RBL feature as
well as the access.db feature to block spammers. we use
three blacklists, relays.ordb.org, spews.org,
monkeys.org(for open proxies).  spews seems to block the
most out of the three, though our configuration checks the
open relay database first.  the other feature of Sendmail
that seems to block a fair number of connections is denying
unresolved domain names of the sender address.  all these
block at the smtp level, and we reject with a custom message
depending on which list caused the rejection.

I have been testing spamassissin and spamcop on a personal
level for several months and like both. spamcop's RBL will
probably be added to our configuration sometime in the
future.  We haven't decided on implementing spamassissin
system wide yet, though it seems to work very well, it would
probably be and opt-in feature.

since we are a only a department on a branch campus our
system has ~3000 users. however, someone in our department
could use one of three addresses (xxx-at-sub.dept.domain.edu,
xxx-at-dept.domain.edu, xxx-at-domain.edu). the problem now is that spam gets
forwarded through other servers on this campus and the main
campus which are not, to my knowledge, implementing blocks.
this is unfortunately where most of my spam comes from now.

I would be interested in seeing how others are quantifying
the improvements from blocks they've implemented, since it
is such a subjective thing.  one tool I've utilized is BMS
(Bad Mail Stats http://www.ginini.com/software/bms/ )
for analyzing sendmail logs for rejected mail.

From hillman-at-sfu.ca Wed Nov 27 15:00:58 2002
Date: Tue, 15 Oct 2002 09:20:26 -0700
From: Steve Hillman <hillman-at-sfu.ca>

At 08:00 AM 10/15/2002 -0700, you wrote:

> > ...

We have about 35,000 accounts. POP/IMAP access, but also Elm/Pine via NFS 
(going away at Xmas). MTA is Sendmail 8.12.x, POP is a modified Qpopper.

> >
> > Please let me know if you have tried any of the following (or anything
> > else) and whether it was useful, especially on a large scale system:
> >
> > - blackhole lists, RBLs, DULs, etc.  Which do you use or did you
> > create your own?

Last October, we started using Orbz. There were a few complaints from 
Faculty about being unable to receive mail from sites they needed to 
receive from, so we added in Sendmail's Access.db support and stuck their 
e-mail addresses in as "SPAM FRIENDS".  A few people complained that we 
were blocking legitimate sites, but when we explained that those legitimate 
sites were "spam friendly", they accepted that it was the right thing to do.

The spam level dropped significantly at first. Last spring Orbz went away 
and we switched to ORDB. Unfortunately, even though we're still getting 
lots of "hits" for "your site refused because it's on the ORDB", the spam 
level has still risen sharply recently (I get about 100 per day, but I get 
it from sys, lpr, adm, root and all the rest as well..

> >
> > - if you block, do you block on content, on IP address, or on "from "
> > address?

If we notice a massive pattern from one site, we'll block it in hosts.deny, 
or add a rule to the sendmail.cf to reject them with a 500-series code. We 
did that last week when we got inundated with optprofessionals.com spam.

> >
> > - tar pits to slow up the spam arrival rate
> >
> > - open source applications such as spamassassin or spamcop

I'm just in the process of building a SpamAssassin box now. I hope to put 
it into production today or tomorrow.

Here, we're making some compromises. SA is intended to be invoked by the 
user at delivery time so that it can cull that user's settings and apply 
them to the potential spam. Unfortunately, we accept about 75,000 msgs per 
day, but deliver about 300,000 (after mailing list and alias expansion). If 
we did spam scanning at delivery time, we'd be scanning 4 times the volume. 
Our SA box is going to be a dedicated 4 processor Sun E450, but I'm fairly 
sure it wouldn't be able to keep up. At 75,000 msgs, I'm *Hoping* that it 
will (that's still one message per second). SA has in excess of 500 rules 
that it applies, and it's written in Perl, so it's pretty CPU intensive.

So we're using Sendmail's Milter interface to pipe the message to SA at 
arrival time. It'll mark it up and then pass it back to Sendmail for 
delivery. That's Phase 1 of the project. Phase 2 will be to write some 
standard rules for Procmail to process the SA headers at delivery time. 
Users will be able to use a webpage to choose one of a few simple options 
("Reject above level x", "Accept from this whitelist", etc). It would be 
nice to reject it at the Milter stage with a 500-series code (we do that 
for a Virus Scanner I added last Spring), because it would reduce the load 
on Sendmail, but at the moment, I can't think of how we could do it on a 
global scale.

> > Is your system opt-in or opt-out?

Well, ALL messages will get marked up. Users can then opt-in to some 
procmail rules to throw away spam at delivery time.

> >
> > Do you use spamtraps?  What types?

Thinking about it.. SA supports several "spam databases", at least one of 
which can be run and managed locally (Razor?). We could then use spamtrap 
e-mail addresses to pipe spam to razor for processing.

> >
> > Do you block spam or put it in a "grey" folder for the user to decide
> > what to do with?  If you block, do you block during the SMTP protocol
> > or bounce it later or just delete it without notice?

I took a look at BSM Development's code (free!), but he did too much MIME 
processing in with the Spam processing. We need users who want to receive 
attachments to be able to receive them unscathed. But BSM has the option of 
doing "corralling". Unfortunately, a quick calculation tells you that if 
you have 40,000 users and you send out notification e-mails twice a day 
telling them of all the spam they've received, you've just added 80,000 
msgs per day to your load, PLUS the load to generate those messages 
(running through your spam corral to summarize everything.

> >
> > How do you prevent harvesting of addresses in your University's web
> > pages?  In departmental directories on the web?  From ldap
> > directories?

We don't, unfortunately. We give the users the ability to opt-out of the 
campus directory though.

Steve Hillman                           hillman-at-sfu.ca
Senior Systems Administrator            (604) 291-3960
Simon Fraser University

From lovan-at-lifesci.ucsb.edu Wed Nov 27 15:00:59 2002
Date: Tue, 15 Oct 2002 09:53:38 -0700
From: Shea Lovan <lovan-at-lifesci.ucsb.edu>

Jerry-

I manage the email systems for the two biology departments and the 
psychology department at UCSB.  Like you, over the last year or so, we've 
seen a huge increase in the quantity and offensiveness of spam received on 
our systems.  I've looked at a variety of options, but with minimal success.

-  Blackhole lists:  We have been filtering inbound connections via 
blackhole lists for several years.  We are currently using the RSS and RBL 
lists from MAPS (I refuse to use the DULS list).  We had better luck with 
ORBS, but had to fall back to MAPS after ORBS shut down.

-  Open source filters:  I have done some extensive testing of Spambouncer 
(www.spambouncer.org).  It was good at identifying spam, but had an awful 
false-positive rate (about 30% of my mail).  I could hack the procmail 
rules a bit, but that would negate the advantage of picking up periodic 
filter updates.  I am looking to test SpamAssassin next, but must confess 
that I have pretty low expectations.

I've also thought about using Vipul's Razor.  However, I've seen that a 
high percentage of what people call spam is actually content from lists 
they subscribed or from vendors they purchased from.  Rather than 
unsubscribing, they get frustrated and call it "spam."  Based on this 
observation and the automated features of Vipul's Razor, it seems very 
likely to block valid messages (such as content from Bugtraq) on a regular 
basis.

-  Commercial solutions:  They are just too expensive to fund out of the 
departments' budgets.  I about died when BrightMail quoted a minimum price 
of $45k for a two year contract.  That's about five times what our 
department could reasonably spend.

-  Implementation details:  I've spent a lot of time considering how to 
actually implement a spam solution.  I think the ideal would be to give 
each user the option of setting which rule-sets to use (commercial junk, 
pornography, profanity, etc.) and how to handle each rule (tag the message, 
file it, or send it to /dev/null).  I think the default would be to 
activate all the rules and set them to tag the messages, though until I 
find something that works I can't be sure.

This is a rough problem.  In even broaching the concept of content 
filtering, I have encountered some pretty heavy opposition until I point 
out the "acceptable use" portion of the ECP.  However, I don't see any 
solution that won't involve some type of content filtering.

I'm interested in hearing what other people are doing around the 
system.  If you get anything that sounds promising or even just novel, I'd 
love to see it.

Thanks!
-Shea

>------------------------
>Shea A. Lovan
>Manager, IT Operations
>UCSB Biological Sciences
>Santa Barbara, CA 93106
>805.893.5294
>fax: 805.893.4724

From jkwong-at-borderware.com Wed Nov 27 15:00:59 2002
Date: Tue, 15 Oct 2002 13:15:31 -0400
From: Jason Kwong <jkwong-at-borderware.com>

Jerry,

Seems like there are lots of companies implementing different ways
of getting at SPAM. Either at the desktop, internal mail server,
mail firewall, managed solution i.e. Brightmail, content filtering,
they can be open source, commercial lots of different ways.

I think the way to go is to have a solution on site. Outsourced
solutions leave the possibilities of false positives being overlooked.

Using the same idea as the firewall and blocking malformed packets
at the perimeter/front door of your network, SPAM needs to be blocked
at the front door of the network. It shouldn't even touch your internal
mail systems. Malformed messages and viruses should also be blocked at
the front door, not handled at the mail server.

What do you think of an "e-mail firewall", something specifically designed
to look at and filter e-mail and only e-mail? Perhaps a Unix based solution
is the way to go, giving you granularity control and speed. A certified,
hardened OS, would be part of the e-mail firewall.

Tell me what you think, I'd like to hear what you have to say. I actually
work for a company that makes network security. Go figure... I do technical
support in Toronto.

Jason

From stevev-at-darkwing.uoregon.edu Wed Nov 27 15:00:59 2002
Date: Tue, 15 Oct 2002 11:35:43 -0700
From: Steve VanDevender <stevev-at-darkwing.uoregon.edu>

Jerome M Berkman writes:
 > 
 > Please let me know if you have tried any of the following (or anything
 > else) and whether it was useful, especially on a large scale system:
 > 
 > - blackhole lists, RBLs, DULs, etc.  Which do you use or did you 
 > create your own?

We currently use the MAPS RBL+ (http://mail-abuse.org/, educational zone
transfer subscriptions are pretty cheap; we pay $250/year for 25,000
users), Spamhaus SBL (http://spamhaus.org/sbl/), Blitzed OPM (Open Proxy
Monitor for unsecured HTTP/SOCKS proxies, http://blitzed.org/opm/), and
currently the open proxies part of Osirusoft
(http://relays.osirusoft.com/).

 > - if you block, do you block on content, on IP address, or on "from "
 > address?

In general we block on originating IP address or on sender domain.  We
do some limited header checks (for valid syntax of the Message-ID
header, and a few specific, common To: and From: header addresses used
in spams).

 > - tar pits to slow up the spam arrival rate 

Nope.

 > - open source applications such as spamassassin or spamcop

We have spamassassin available for optional local use.  I have not heard
really good things about trying to use spamassassin on a site-wide
basis.

 > - commercial solutions, such as those offered by BrightMail,
 > TrendMicro, and Sendmail, Inc.

Nope.

 > Is your system opt-in or opt-out?

I guess it's more "love it or leave it".  More seriously, most of our
current spam-blocking cannot be optionally enabled or disabled on a
per-user basis.  This has tended not to be a problem; most of our local
blocks are created in response to user reports of spam.

 > Do you use spamtraps?  What types?

We have a guy who seems to be a walking spamtrap, as he generates most
of our user reports (some forwarded from other users who report their
spam to him, since he's in charge of our user support stuff).  But we
don't have any artificially-created spam traps, which is probably what
you're asking.

 > Do you block spam or put it in a "grey" folder for the user to decide
 > what to do with?  If you block, do you block during the SMTP protocol
 > or bounce it later or just delete it without notice?

We recommend that people who decide to hook spamassassin into their
.procmailrcs have it throw suspect mail into a folder.  I am a very
strong believer in the idea that spam blocking should _never_ cause mail
to just vanish; mail should be either delivered or bounced.  In the
occasional case of false positives or "collateral damage" due to an
excessively wide IP block, this at least allows the sender to know the
mail was not accepted and to report the problem via other channels.
Should we ever decide to implement a site-wide content-filtering scheme
I would do my best to ensure that it bounced mail instead of just eating
it.

Since most of our blocking occurs during the SMTP protocol exchange,
appropriate use of 5xx replies causes mail to bounce.

 > How do you prevent harvesting of addresses in your University's web
 > pages?  In departmental directories on the web?  From ldap
 > directories?

I don't think we do, at this point.

From bfriday-at-lasierra.edu Wed Nov 27 15:00:59 2002
Date: Tue, 15 Oct 2002 12:17:38 -0700 (PDT)
From: Brian Friday <bfriday-at-lasierra.edu>

Hi Jerome,

Took the day "off" to work on this myself due to the problems we are
having here at La Sierra University. I'll try to answer your questions
inline. If you have any further questions just give me a hollar.

On Mon, 14 Oct 2002, Jerome M Berkman wrote:

> We have noticed a huge increase in spam this year.  We are trying to
> figure out what to do about it, and we are wondering what others are
> doing, and especially what has proved successful.

Don't know about you, but our increase in spam was quantifiable starting
about September. I went from receiving 1 "urgent business proposal" scam
mail to no less that 14 per week.

> The server I help administer, uclink.berkeley.edu, has 40,000+ accounts
> (students, staff, and faculty).  Users access their mail via POP and IMAP
> clients such as Eudora, Netscape, and Outlook Express.

Currently we are in the midst of a mail system upgrade but we service
about 2000 accounts which allow: Shell access (for Mutt, Mailx, PINE this
is going away relatively soon), IMAP, and POP3. Our MTA of choice here is
Postfix (didn't notice this in your questions but might be one to add).

> - blackhole lists, RBLs, DULs, etc.  Which do you use or did you
> create your own?

We currently use ordb.org as our only off-site RBL
We also keep a actively update list of spammers (or as active as I can
manually keep it) which show up on the radar enough to block via IP or
domain.

> - if you block, do you block on content, on IP address, or on "from "
> address?

All of the above, we use basic Postfix UCE rules, plus have access maps
which allow us to block via domain name, IP address, any header, and
specific (limited) body checks.

I expect I'll be using more body checks in the future as a recent snapshot
of postfix which is going into testing soon will be supporting checking
only the first 50K of a message body. Which should considerablly speed
things up in that somewhat inefficient area.

> - tar pits to slow up the spam arrival rate

I wish, unfortunately as the only system admin here I spend only about 20
hours a month on spam which IMHO is totally inadequate but all I can spare
usually. Therefore were just doing the basic stuff and haven't gotten
fancy yet.

> - open source applications such as spamassassin or spamcop

Since this massive increase in spam recently I've been researching how to
intergrate spamassassin into our postfix MTA systems. If I can get it to
work properly and it doesn't bog down the system its on then it will
become a regular feature of our system. So far I've been unsuccessful in
getting them to intergrate but its been more lack of time and effort then
deficiences in the software.

> - commercial solutions, such as those offered by BrightMail,
> TrendMicro, and Sendmail, Inc.

Would love to implement something I could get support for but budget being
what it is and knowing that its only getting smaller for us right now I'm
going to be lucky to pickup Verixa Mailarmor for our virus scanning on the
server level... these software packages are way out of our league at the
moment.

> Is your system opt-in or opt-out?

Not sure what you mean, regarding who gets their mail scanned for spam?
Opt-out or rather no opt I suppose, SpamAssassin doesn't actually
(although it can) delete e-mail, it only tags it as "probable spam" thus
allowing the users to filter out the spam better on their level.

> Do you use spamtraps?  What types?

nope see above, very basic system here

> Do you block spam or put it in a "grey" folder for the user to decide
> what to do with?  If you block, do you block during the SMTP protocol
> or bounce it later or just delete it without notice?

Once I get spam assassing running we should be able to do both, right now
we block any hits we get on the SMTP level (via header checks which have
the bulk of our spam fighting ability) or via RBL's. We block all forms of
double attachment types via body checks but that occurs after the message
has been successfully accepted by our server (ie costs more to reject then
in cpu terms).

> How do you prevent harvesting of addresses in your University's web
> pages?  In departmental directories on the web?  From ldap
> directories?

I rely on our webmaster to deal with this, and we are actively looking
into better ways to do this. We are just in our infancy of implementing
LDAP directories (or directories of any kind, we are implementing a
directory more for admin duties and less for general use).

Hope this helps.  If you have any questions, thoughts etc feel free
to give me an e-mail or a call!

Sincerely,

Brian Friday
Systems Administrator
La Sierra University

From jason.frand-at-anderson.ucla.edu Wed Nov 27 15:00:59 2002
Date: Tue, 15 Oct 2002 14:11:20 -0700
From: Jason Frand <jason.frand-at-anderson.ucla.edu>

Thanks Jerome for putting this together. 

We don't do anything substantive to prevent or 
block SPAM. We do block certain subject lines, 
but this is for virus prevention only, not SPAM.

I look forward to your survey recults,
jason

_____________________________
Jason L. Frand, Ph.D.
Assistant Dean and Director
Anderson Computing and Information Services
The Anderson Graduate School of Management at UCLA
web:    http://www.anderson.ucla.edu/faculty/jason.frand
phone:  (310) 825-2725       fax (310) 825-4835

From r.fulton-at-auckland.ac.nz Wed Nov 27 15:00:59 2002
Date: 16 Oct 2002 12:48:01 +1300
From: Russell Fulton <r.fulton-at-auckland.ac.nz>

On Tue, 2002-10-15 at 12:27, Jerome M Berkman wrote:
> 
> We have noticed a huge increase in spam this year.  

you  me and the rest of the bloody net, sigh...

> 
> The server I help administer, uclink.berkeley.edu, has 40,000+ accounts
> (students, staff, and faculty). 

All our mail goes through a two central MTAs and then to a multitude of
delivery servers including one large one (running cyrus) which servers
30,000 students and a central server for staff (~1000 accounts?) many
small departmental servers.  We are trying to stamp out the small
servers but they are like the proverbial mushrooms ,

 Users access their mail via POP and IMAP
> clients such as Eudora, Netscape, and Outlook Express.
you name it we have it!

> 
> Please let me know if you have tried any of the following (or anything
> else) and whether it was useful, especially on a large scale system:
> 
> - blackhole lists, RBLs, DULs, etc.  Which do you use or did you 
> create your own?

we use MAP's RBL+  it reduce it a bit but 80% of a flood is still a
flood :(

> 
> - if you block, do you block on content, on IP address, or on "from "
> address?
> 
> - tar pits to slow up the spam arrival rate 

no. but I'd be interested in how well these work.

> 
> - open source applications such as spamassassin or spamcop

WE have been playing with spamassassin.  Our primary mail server is a
mirapoint box and they are promising support for spamassassin or
something with same functionality RSN.

> 
> - commercial solutions, such as those offered by BrightMail,
> TrendMicro, and Sendmail, Inc.

No.  I'm fighting off the surfcontrol salesman at the moment.

> 
> Is your system opt-in or opt-out?

Every one is subject to RBL. We plan to use spamassassin (or whatever)
to tag messages so users can deal with them.  We have also setup support
for scripting on several of our major mail servers (the new ietf one --
can't remember its name) and we have web interface that allows users to
setup filters for the mailboxes.  When we have the tagging going we plan
to have an even simpler interface for spam with 3 options: delete,
deliver to folder, ignore.  If the the system includes content tags
(PORN, INVESTMENT etc) then we will have pull down lists wich allow
people to select the categories.

> 
> Do you use spamtraps?  What types?

NO, but it is something I'm interested in.  We are just about to
decommission our original public unix system (ccu1.auckland.ac.nz) - not
the original hardware I might add, it is the service that is being
discontinued.  These addresses have not been advertised (except as the
source of usenet postings) for 10 years and we see lots of spam coming
in and it would make a good spam trap.  Anyone sending mail direct to
this machine can go into a local RBL.

If you come up with any good links or documents on building spam traps I
would be interested.

> 
> Do you block spam or put it in a "grey" folder for the user to decide
> what to do with?  If you block, do you block during the SMTP protocol
> or bounce it later or just delete it without notice?

As above, RBL+ blocked at SMTP, plan to give users the option on content
filtering.  

> 
> How do you prevent harvesting of addresses in your University's web
> pages?  In departmental directories on the web?  From ldap
> directories?

We don't make our directories publically available :(  (a ploicy I
disagree with). It does not seem to make much difference.

Cheers,  

-- 
Russell Fulton, Computer and Network Security Officer
The University of Auckland,  New Zealand

"It aint necessarily so"  - Gershwin

From teus-at-NLnet.nl Wed Nov 27 15:00:59 2002
Date: Wed, 16 Oct 2002 12:18:19 +0200
From: Teus Hagen <teus-at-NLnet.nl>

Jerome,
I will try to answer your email. But would like some comments of an
initiative w're thinking of to fund. See below.

Jerome M Berkman wrote:

>Please let me know if you have tried any of the following (or anything
>else) and whether it was useful, especially on a large scale system:
>
>- blackhole lists, RBLs, DULs, etc.  Which do you use or did you 
>create your own?
>
Use RBL (MAPS) but do not see much SPAM bounced with these help lately.

>
>
>- if you block, do you block on content, on IP address, or on "from "
>address?
>
On "from" address. This is the most successfull we encountered yet.
But only OK for small scale usage (e.g. the address yahoo,com could be
needed by some VIP).
At the MTA (sendmail) side.

>
>
>- tar pits to slow up the spam arrival rate 
>
>- open source applications such as spamassassin or spamcop
>
spamassassin at the user side (this variety of checks finishes off
what came through the MTA) by about 20% of SPAM. Reliability seems
to be very good. The enduser has the power, adjustments and mistakes.

>
>- commercial solutions, such as those offered by BrightMail,
>TrendMicro, and Sendmail, Inc.
>
>Is your system opt-in or opt-out?
>
>Do you use spamtraps?  What types?
>
>Do you block spam or put it in a "grey" folder for the user to decide
>what to do with?  If you block, do you block during the SMTP protocol
>or bounce it later or just delete it without notice?
>
We flag the header so the user has the possibility to use different "grey"s
(spamassassin gives enough handles).

>
>How do you prevent harvesting of addresses in your University's web
>pages?  In departmental directories on the web?  From ldap
>directories?

My question:
We (NLnet foundation) have received a proposal for SPAM handling.
It is an end user thing, which classifies email (not on keywords as e.g. 
spamassassin)
with a learning set. Legal requirements forces you either to flag the 
header of emails
from a central side or (proposed approach) to allow the enduser to do
everything (learning set).

The implementation work is estimated to cost one manyear of costs. The
implementation will be GPL'd. If there is enough interest NLnet will sponsor
this fully.
However we have questions:
Are current anti-spam measurements already sufficient?
Is this end user approach valuable?
If valuable to proceed I would love to see some cooperation from other
universities (the proposal came from a university) with other and more
practical backgrounds.

Any comments?

Thanks

teus hagen
Stichting NLnet http://www.nlnet.nl

From jar-at-ucsc.edu Wed Nov 27 15:00:59 2002
Date: Wed, 16 Oct 2002 17:30:08 -0700
From: "Janine A. Roeth" <jar-at-ucsc.edu>

Jerry,

I'm going to answer your questions rather generally, as I'm the manager of 
the group that would implement spam filtering.  We're at the point of 
discussing our approaches and the characteristics of our 
implementation.  I'm happy to follow up with more specifics as they develop.

At 04:27 PM 10/14/2002 -0700, Jerome M Berkman wrote:
>Please let me know if you have tried any of the following (or anything
>else) and whether it was useful, especially on a large scale system:
>
>- blackhole lists, RBLs, DULs, etc.  Which do you use or did you
>create your own?

We are considering the use of blackhole lists, perhaps as a contributor to 
the scoring of a message as SPAM.  Messages that exceed a certain score 
would have {SPAM?} added to their subject line.

>- if you block, do you block on content, on IP address, or on "from "
>address?

Currently, our sendmail access database can block upon IP address, from 
address,
or to address.

>- tar pits to slow up the spam arrival rate

We aren't doing any kind of tar pit.

>- open source applications such as spamassassin or spamcop

We are considering the use of SPAMAssassin for central email spam 
filtering.  Our abuse-at-ucsc.edu team regularly reports spam to SpamCop.

>- commercial solutions, such as those offered by BrightMail,
>TrendMicro, and Sendmail, Inc.

Not yet...

>Is your system opt-in or opt-out?

Neither.  We will "deliver to the desktop" which means that all mail will 
be delivered.  We're in discussion about the thresholds that will determine 
what gets {SPAM?} added to the subject line.

>Do you use spamtraps?  What types?

We do not currently use any spamtraps.  We've been looking into them, 
thinking of recommending the "user+spam" address as a sort-of 
spamtrap.   However, I understand from my guys that spam engines are 
getting smarter now, and are filtering out the +extension. It may turn out 
to not be very useful anymore.

>Do you block spam or put it in a "grey" folder for the user to decide
>what to do with?  If you block, do you block during the SMTP protocol
>or bounce it later or just delete it without notice?

As noted, we aren't blocking spam per se (we do block dialup servers 
though).  We see this as a possibility for the future, and we don't yet 
have the resources to redirect spam for users to review

>How do you prevent harvesting of addresses in your University's web
>pages?  In departmental directories on the web?  From ldap
>directories?

Right now this is a topic of discussion - we have little in place.

From cab-at-tc.umn.edu Wed Nov 27 15:00:59 2002
Date: Tue, 29 Oct 2002 15:15:26 -0600
From: Christopher A Bongaarts <cab-at-tc.umn.edu>

[Warning: long, but hopefully worth it.]

As Jerome M Berkman once put it so eloquently:

> We have noticed a huge increase in spam this year.  We are trying to
> figure out what to do about it, and we are wondering what others are
> doing, and especially what has proved successful.
[...]
> The server I help administer, uclink.berkeley.edu, has 40,000+ accounts
> (students, staff, and faculty).  Users access their mail via POP and IMAP
> clients such as Eudora, Netscape, and Outlook Express.

At the University of Minnesota, we have a set of servers with about
120,000 accounts for students, staff, faculty, deparments, student
organizations, and alumni.  We offer POP and IMAP service (with and
without SSL) and support pretty much the same client set that you
folks do.

At the beginning of October, we put a new spam-blocking system into
operation.  It is completely homebrew, and is not suitable for general 
distribution, as it uses hooks into our commercial mail routing
software (Syntegra's Mail*Hub).  But the ideas behind it may prove
useful to others.

Two aspects of our system are not commonly found among most solutions
I've heard of:  (1) We make decisions based on several criteria;
i.e. we use MAPS RSS, but don't necessarily block you unless other
criteria based on IP address or other blackhole lists also match, and
(2) it is configurable on a per-user basis.

> Please let me know if you have tried any of the following (or anything
> else) and whether it was useful, especially on a large scale system:
> 
> - blackhole lists, RBLs, DULs, etc.  Which do you use or did you 
> create your own?

We use several lists as part of our criteria, but as noted above, we
do not use any as a sole reason for blocking.  So if a local system
gets listed on one of these, our policy for our IP space will prevent
any messages from being blocked.  Currently we use MAPS RSS, MAPS RBL, 
SBL, ORDB, RSL, and DSBL as inputs to our decisionmaking process, as
well as a dialup-line list (I forget who runs it - MAPS DUL?).

> - if you block, do you block on content, on IP address, or on "from "
> address?

We do not block on content.  Our blocking operates at the SMTP RCPT TO 
level (to implement per-user configurability).   We are planning to
add (user-selectable) SpamAssassin filtering to our blocking policy,
so users can add their own spam filtering criteria to their email
client.

The basis of our criteria for blocking starts with the IP address of
the connecting MTA.  We have a DNS-based database that tells what
policies apply to various IP ranges (so we can say "block all mail
from 211.* that does not have a reverse DNS name, except for 211.x.y.z 
which we know is good", or "block all mail in 24.* that appears in
MAPS RSS, ORDB, RSL, or DSBL").

> - tar pits to slow up the spam arrival rate 

Heck no.  Wastes our time more than theirs.

> - open source applications such as spamassassin or spamcop

SpamAssassin is in testing right now.  Our plan is to have a centrally
administered set of rules/whitelist/blacklist and to simply set a
header or rewrite the subject line for mail identified as spam.
We also add a header with a URL so that if mail is classified as spam
when it is not, the user can report that fact to us and we can adjust
our rules accordingly.

> - commercial solutions, such as those offered by BrightMail,
> TrendMicro, and Sendmail, Inc.

No, except to the extent that we continue to use a commercial sendmail 
replacement (as we have for the last 10 years) and our system
interfaces directly with it.

> Is your system opt-in or opt-out?

We spammed our users (no, the irony is not lost on me) a couple weeks
before we implemented the system, giving them the option to adjust
their mail control settings before it went live.  We have three
options: (1) allow mail from "well-behaved" MTA's only, the default;
(2) allow all mail from everywhere, for the diehard freedom types, and 
(3) allow only mail from umn.edu MTA's, i.e. local only.

Since the default setting is "block from non-well-behaved", it's
techincally opt-out.

Further, every user can set up to 20 exceptions (based on the SMTP
MAIL FROM address) for email that would otherwise be blocked.  This
operates on a per-user basis as well.

How does a user know 

> Do you use spamtraps?  What types?

We plan to use dormant accounts as a means of detecting compliance
with opt-in/opt-out policies.  If an account that has been inactive
for 6 months suddenly gets 

> Do you block spam or put it in a "grey" folder for the user to decide
> what to do with?  If you block, do you block during the SMTP protocol
> or bounce it later or just delete it without notice?

We block at the SMTP RCPT TO level (returning 5xx errors for users
that have blocking enabled).  Then we take the information we have (IP
address/DNS name of MTA, SMTP MAIL FROM address, date/time, and
reason(s) for blocking) and put it in a database that keeps the
previous two weeks of block activity.

Users can query the database to see what messages have been blocked,
and can select messages to add an exception for the sender (and thus
allow future emails from them).  This is very useful from a marketing
standpoint, too: one of the complaints from users early on (while we
were slowing turning up the blocking criteria) was that it wasn't
doing anything, or "made it worse" (since the optprofessionals spam
started shortly after we turned it on, and we didn't block it directly
right away, although reverse DNS checks stopped some of it).  This
page gives us a tangible way to say to users "*you* *personally* had N
messages blocked in the last two weeks" and show them that it's
working.

Exceptions are especially effective for users who have selected "allow
umn.edu MTA's only", as they can allow the few outside users who send
them mail and block everything else.  One user doing this had over
2,000 messages blocked the first two weeks of implementation; all of
them were spam.  He's down to about 800 in the last two weeks right
now.  My other favorite benchmark user has about 500.  I personally
have 229.

Currently, there are 2,944,629 blocked emails in the database (for the
last two weeks), for 54,258 different accounts.  That's a lot of saved
mail spool disk space and email server cycles. 812 users have granted
1187 exceptions.

The 5xx error that is returned to the blocked sender contains a URL
that if viewed will show the reason for the block and allow the sender
to request an exception of the recipient (this involves a step where
we send mail to the origination address to verify its validity).  The
intended recipient can then either allow the exception, actively
reject the request for exception, or ignore it (and thus silently
reject the request).  The URLs involved are encrypted to prevent
forgery.  We have had about 150 hits on the initial URL, and about
half of them actually requested an exception.

There is also a header in every message with a URL that allows users
to report a particular message as spam.  The result of visiting the
URL is a page that varies based on our classification of the sending
MTA.  Unclassified MTA's will tell the user "We Will Investigate";
MTA's that we know have a working abuse@ address will tell the user to
report it there; known or suspected "good" bulk mail sites will tell
the user to try to unsubscribe from the list (the old advice about
"don't unsub, the spammers will know your address is good" is bogus -
the spammers know your address is good as soon as they get the 200
repsonse on the RCPT TO, and most of them don't care, they just keep
blasting away; in fact, failure to accept a delivery status
notification is one of the reasons we use for blocking mail).

Regardless of what we tell the user, we log the fact that they
reported it as spam, and we can use that information to make decisions
on our blocking and classification.  For example, if we have
classified a site as "try to unsub", and we get repeated spam reports
from the same user for messages, that may indicate that the site is
not actually unsubbing the user, and maybe we should block them.

In the past 11 days, we have had about 38,750 spam reports.

The future SpamAssassin option will allow for users to implement their 
own "grey" folder.

> How do you prevent harvesting of addresses in your University's web
> pages?  In departmental directories on the web?  From ldap
> directories?

Currently we have nothing preventing this other than manual detection
and nullrouting.  In the future, I plan to add an authentication
option to our directory lookup web page, where unauthenticated users
get a very small number of searches per time period, and authenticated 
users get unlimited searches.

----

As you might have guessed, we spent a lot of time thinking about this, 
and continue to assimilate user feedback and observe patterns in
blocked mail to refine things.  The goal is not so much to eliminate
100% of spam, so much as it is to reduce the amount of spam we have to 
process and store, and prevent wasting end users' time.  We try to err 
on the side of permissiveness, but we can feel somewhat safe knowing
that there is a way for users to allow mail that gets blocked as
"collateral damage".

Hopefully other universities can take some of these ideas and
implement them in ways that make sense at their institutions.

I may try to convince my cohorts to get together and write something
up for a future USENIX or something.  But we will definitely wait
until the system has been running for a while so we have more
experience with it.

%%  Christopher A. Bongaarts  %%  cab-at-tc.umn.edu       %%
%%  Internet Services         %%  http://umn.edu/~cab  %%
%%  University of Minnesota   %%  +1 (612) 625-1809    %%

(Note by Jerry: yahoo.com and qmail return a 200 response
to the RCPT TO even for bogus addresses)

Finally, I am including some bogus addresses to see whether the form of the address makes any difference to spambots harvesting addresses from the web.

Mail to URL (mails to trapper01)	Webmaster
Raw address	trapper02@uclink.berkeley.edu
Munged, "@" replaced by "-at-"	trapper03-at-uclink.berkeley.edu
Munged, "@" replaced by "<B>@</B>":	trapper04@uclink.berkeley.edu

I'll see how much spam each of these addresses get, and post the results in the future. Of course, the best thing to do is connect to a CGI to send mail to an address on a web page, or send email addresses to the requestor, or ..., but that is another discussion.

Jerry Berkman, Dec. 5, 2002