Become a fan of Slashdot on Facebook


Forgot your password?
Check out the new SourceForge HTML5 internet speed test! No Flash necessary and runs on all devices. ×
Communications Spam The Internet IT

The Hostile Email Landscape ( 217

An anonymous reader writes: As we consolidate on just a few major email services, it becomes more and more difficult to launch your own mail server. From the article: "Email perfectly embodies the spirit of the internet: independent mail hosts exchanging messages, no host more or less important than any other. Joining the network is as easy as installing Sendmail and slapping on an MX record. At least, that used to be the case. If you were to launch a new mail server right now, many networks would simply refuse to speak to you. The problem: reputation. ... Earlier this year I moved my personal email from Google Apps to a self-hosted server, with hopes of launching a paid mail service à la Fastmail on the same infrastructure. ... I had no issues sending to other servers running Postfix or Exim; SpamAssassin happily gave me a 0.0 score, but most big services and corporate mail servers were rejecting my mail, or flagging it as spam: accepted my email, but discarded it. GMail flagged me as spam. MimeCast put my mail into a perpetual greylist. Corporate networks using Microsoft's Online Exchange Protection bounced my mail."
This discussion has been archived. No new comments can be posted.

The Hostile Email Landscape

Comments Filter:
  • by 0xG ( 712423 ) on Monday October 19, 2015 @11:05AM (#50758311)
    I run a small email system ~2500 users and don't have your problems...
    • Re: (Score:3, Insightful)

      by billyswong ( 1858858 )
      Maybe your little email server is old enough to escape the now-current hostility?
      • by beelsebob ( 529313 ) on Monday October 19, 2015 @11:21AM (#50758479)

        More likely, the original poster simply has his DNS misconfigured in some weird way, and doesn't know it.

        • by Anonymous Coward on Monday October 19, 2015 @11:38AM (#50758649)
          I second that emotion. Current *big* players are trying to limit spam and phishing, and require a few ducks in a row before you stop getting caught in their filters. I suspect proper analysis of the configurations and logs would pinpoint the issue. DNS would be a quick start but the problem could be in a few places depending on what mail implementation he's using. On another note, is it possible OPs domain has been used for spam/phishing in the past? The UNI I work has dealt with blacklists in the past and it was merely a case of spoofing and those adding us to blacklists didn't do their diligence in tacking it down properly. *Posted anon as to not get fired*
        • by Z00L00K ( 682162 ) on Monday October 19, 2015 @11:47AM (#50758725) Homepage

          It's usually the case when the reverse lookup don't point back to the same domain/name as the server identifies itself with.

          And it's the ISP that need to change the pointer from some generic name to a specific.

          • by Anonymous Coward on Monday October 19, 2015 @12:08PM (#50758923)

            Rejections in my experience have nearly always always been related to the PTR record needs to be pointing to the domain actually sending the email, not the domain name in the email address. My limited understanding is this:

            So if my email address uses on port 25 to send email then the PTR needs for the ip address isp,com sends from needs to say not as you might expect.

            when talks to another smtp server it will be asked to id itself. The server should reply with its FQDN and it is this that the PTR record for the servers id needs to point to . Even if that server hosts hundreds of websites and email accounts.

            I believe most VPS hosts allow this to be changed to whatever you want if you are given a fixed ip address. If they don't allow this to be changed then problems will occur and if you are handling emails you need to check before signing up. The PTR record is not applicable to a domain but to an IP address. You can only have one PTR record for an IP address.

            That is if my memory serves correctly. When I set up email servers, I always seem to forget this until I do sending tests to yahoo and other big boys. Then I set it properly and things behave.

            Other problems happen if using microsoft exchange and the srv fields in txt records for the dns are not set exactly right. Though I don't have to fiddle with this for obvious reasons.


          • by Anonymous Coward on Monday October 19, 2015 @12:59PM (#50759391)

            There are several factors that I've seen with my mail server.

            1) Do not try to work over a standard ISP service - one that assigns your IP dynamically - because most blacklists and major corporations blacklist dynamic IP pools
            2) Don't host in any of those cheap virtual hosting services - many of them are also blacklisted
            2) Setup DKIM signing (sendmail config and DNS record)
            3) Setup SPF DNS record

            Basically, one has to avoid running one's mail server someplace that is cheap because that is where the SPAMers put their mail servers as well (because they are cheap and easier to do anonymously).

        • What Z00L00K said. Also: Many corporate email systems, especially the larger ones, are configured to ignore anything from a dynamic IP address. The email must have a fixed IP address or they'll just plain ignore you. This is ostensibly for "security" but I suspect there's some barrier-to-entry aspect of it too. Also, by law, you have to be allowed to get yourself removed from grey- and black-lists. It's a pain in the butt, but it can be done.
        • by Anonymous Coward on Monday October 19, 2015 @12:48PM (#50759255)

          He's doing it wrong. Most probably he's not using SPF nor signing with domainkeys. That's expected today by most providers.

          If he's especially naive he's operating an open relay, which will warrant him to be blacklisted FAST.

          Another cause is, he could be operating his mail server from a "dialup" IP range, one declared as being assaigned to residential connectivity, which are usually blacklisted. I disagree with this practice, but that's how things go.

          Also most providers now require TLS support. So you need to generate certificates(self signed is not enough, but your own unofficial CA is enough usually, but make sure you're not using SHA1).

          Also, I happened to configure a mail server on a newly acquired IP from an hosting company a year ago or so and the IP they gave me was already tainted as being on a few blacklists. This can be solved too. I took the pain to discover which blacklists and followed their procedures to be taken out. Sometimes It was some automated procedure which just requested the server to be scanned again to make sure it follows best practices(as stated above). OOther times I had to politely ask and in one case even have the provider confirm the IP was actually reassigned.

          After this I have not seen a single email being rejected as spam.

          Operating mailservers could have been easy in the '80s and first half of the '90s when most mail server really were open relays and nobody cared, just because nobody was taking advantage of that. Nowadays it's become complicated because even the slightest misconfiguration will be attacked and exploited. It's in the general interest to request mail servers to be configured to a minimum standard that is getting relatively high, or we could really loose control of the email system.

      • Or maybe his top level domain is old enough?

        As many others have posted, this cheap new TLDs have had their reputations tarnished. My system's count of TLDs that are blocked by default is over 20, and includes such "winners" as .ninja, .space, .science, .audio, .xyz, .link, .rocks, .click, .work, .party, .review, .date, .eoc, .website, .eu, .win, .racing, .pro, .asia, .download, .faith, .wang, and .top, with more added as the spam load rotates through them.

        As for hosting on a virtual server out "in the cloud

    • Re: (Score:2, Interesting)

      by Anonymous Coward

      I've been running mail system for myself for the past 5 years or so. Gmail has begun tagging my messages as spam, starting a few months ago.

    • by LDAPMAN ( 930041 )

      I used to run multiple email domains. Some of them had few issues, others were constantly being blacklisted. It really depends on who you interact with. I found that often users never realized there was an issue as the messages were just silently dropped. In the end I got tired of fighting with it and moved them all to gmail. If your not having issues you are likely just very lucky or the services you interact with are the less zealous ones.

      • by Rei ( 128717 )

        I had the same issue. I fought with it for years, trying to fight one obscure reason after another for my mail getting bounced or worse (as you mention) silently dropped from different recipients, and eventually gave up. I still host my own incoming mail server but I switched to routing outgoing mail through existing services.

      • I got tired of fighting with it and moved them all to gmail.

        I suppose this is the 'plan'... *All your email are now belong to Google(TLA)*

        • I suppose this is the 'plan'... *All your email are now belong to Google(TLA)*

          I doubt it. The biggest source of spam is from botnets of hijacked machines. Most (>99%) of those machines don't have their ducks lined up when it comes to DNS. It's not a surprise that it's harder to start an email server these days. The sheer volume of spam is maddening.

    • by acoustix ( 123925 ) on Monday October 19, 2015 @11:43AM (#50758689) Homepage

      I run a small email system ~2500 users and don't have your problems...

      You probably have a dedicated/static IP and it isn't tainted from others who have used it before you.

      For people trying to run their own email server at home it can be a real pain. ISP's blocking 25 and 587. DHCP means that your IP pool has a bad reputation. Etc...

      • by Frosty Piss ( 770223 ) * on Monday October 19, 2015 @12:05PM (#50758901)

        Who in their right mind runs an email server without a static IP?

        • Re: (Score:2, Informative)

          by houghi ( 78078 )

          Script kiddies. Even if the provider doesn't block ports, all email should be rightfully be blocked as there is no Reversed DNS. And by all things practical, a reversed DNS is only possible with a fixed IP.

          And that is only for outgoing email. Letting incoming email depend on a non-fixed IP could lead to serious problems. I could mean somebody else receives your email on your (previous) IP address.

          Also: if the provider leaver port 25 open for non-fixed IP addresses (we are unable to run an email server, beca

    • Re: (Score:3, Informative)

      by bsdasym ( 829112 )
      I'm with you here. OP sounds like just being paranoid and probably is not quite properly setup. I setup a new domain last month with it's own self-hosted email and had no problems at all getting email through to any of the major providers. To avoid trouble, you need at a minimum:
      - An IP address in a block that doesn't already have a terrible reputation.
      - Working, correct reverse DNS that matches the SMTP banner.
      - Working, correct forward DNS for the MX records that also matches the SMTP banner.
      - Corr
  • There's little more (Score:2, Interesting)

    by Anonymous Coward

    There's little more to the article than the summary.

    How does the person in question solve their mail issue? They don't, they went back to Google Apps.

    Now you don't have to read it.

  • Welcome... (Score:4, Insightful)

    by Lisias ( 447563 ) on Monday October 19, 2015 @11:10AM (#50758363) Homepage Journal

    ... to this new Brave New Internet.

    Fighting SPAM was easy since the beginning. In the early 2k years, most of the SPAM fighting techniques was already somewhat prototyped on the mailing lists I was following,

    Now, 15 years later, I think I know why nobody did anything for a decade and a half - control. Now it's God Damn easy to drop someone from the mail system - you can render a company inoperative if it dare to run his own mail system.

    And so, for "safety", you need to pay for some bug corporation to run it for you - while harvesting you mail on the process.

  • I've been running my own mail server for a year or two now, the only places I've had reject my mail have been small businesses/organizations that have more restrictive policies. I haven't been flagged as spam on, gmail, or yahoo mail and even my workplace's server has accepted them. Perhaps this person got flagged early on as a spam source and didn't realize it?

    • by Jhon ( 241832 )

      "I've been running my own mail server for a year or two now,"

      Unless I'm reading this wrong the article indicates that the problem is NEW email servers. From TFA:

      IPs not previously used to send email typically don’t have any reputation built up in our systems. As a result, emails from new IPs are more likely to experience deliverability issues. Once the IP has built a reputation for not sending spam, will typically allow for a better email delivery experience.

      Now, I've no idea if that is

      • by Z00L00K ( 682162 )

        Just realize that the junk mail filter that Microsoft has is really bad. And there's no way to configure it or teach it good manners. You are entirely in the hands of Microsoft.

        Thunderbird is better on handling junk mail, but that's not really the point in this case.

        To me it looks like the big ones on the market (Gmail, Microsoft etc.) do what they can to snare new clients by making any newcomers seem unreliable. So next step is that we might see more cases where smaller companies soon will have problems wi

    • I still run mutt + courier imapd + postfix on my home box (though I admit I don't use it much anymore since juggling a few gmail accounts worka very well now compared to the old days of yahoo / netscape / hotmail / etc.)

      Simple way to boost your reputation is to simply configure a smarthost to send outgoing mail securely. There are plenty of tutorials on using gmail or several ISP smarthosts (like Verizon Business FIOS.

      Yeah, it's not an ideal solution, compared to, say, making everyone use GnuPG signatures

  • by neilo_1701D ( 2765337 ) on Monday October 19, 2015 @11:16AM (#50758421)

    I run my own mailserver, mostly "just because".

    The reputation problem I encountered early on was because of a lack of a reverse DNS entry. Easily fixed; I simply asked my VPS provider to create one.

    The next problem that started about 18 months ago was reputation: my little server simply wasn't a trusted service.

    Because of the (unbelievable) amount of spam hitting my server, I had taken out a Comodo AntiSpam Gateway subscription about two years earlier. It was initially free, but after a year or so they wanted money. Since the service rocks, I happily pay my ~$30 annually.

    What CASG also offers is outbound scanning: if I tell my server (an Exchange 2010 server) that the outbound smarthost is CASG, my email all of a sudden piggybacks Comodo's reputation. Voila, email flows without incident.

    Problem solved.

    • Reverse DNS is the big one but adding a real SSL certificate to the mail server lends credibility as well. It's definitely not trivial to setup a credible mail server these days but, it's certainly not an insurmountable task. I've run mine for about two years and, once I jumped through all the hoops to get it setup right, I've never had any issues with sending or receiving mail. The OP may want to look at some tools that will query the MX record and then test the mail server for various common pitfalls.

      • by tepples ( 727027 )

        The OP may want to look at some tools that will query the MX record and then test the mail server for various common pitfalls.

        Do " and Port25", mentioned in the featured article, have known problems?

      • The OP may want to look at some tools that will query the MX record and then test the mail server for various common pitfalls. This site has a number of useful tools for dealing with personal e-mail servers: []

        Who, me? If so, I've an SSL certificate and MX Toolbox reports no problems. Comcast was (silently) dropping emails so I ended up going the smarthost route.

      • SSL cert? ha, never needed it. neither do my domains and SPF or TXT records. I can send to all major providers without issue. Reverse DNS and A / MX records matching is all I have.

  • SPF+DKIM (Score:4, Interesting)

    by Meneth ( 872868 ) on Monday October 19, 2015 @11:23AM (#50758513)
    I run my own mail server on a dyndns connection. At first, Google would filter out my mails, but once I set up SPF and DKIM records, they became much more friendly. Haven't tried, but (also owned by M$) works fine.
    • by tepples ( 727027 )

      At first, Google would filter out my mails, but once I set up SPF and DKIM records, they became much more friendly.

      The featured article mentions having set up not only SPF and DKIM but also DMARC, reverse DNS, and checking against blacklists. Which step was missed? Or how many months did it take for Gmail to become more friendly?

      Haven't tried, but (also owned by M$) works fine. and are two domains owned by the same service.

      • How about he fucked up? Seriously you seem to be jumping every post going "But he DID THAT!!!!"

        Simple logic, if 10+ plus people did xyz and it worked flawlessly(just in this thread), and the OP "claims" to have done xyz, but it didn't work. The most basic simple answer is that the OP messed up xyz.

        Seriously, what is more likely:
        1. That everyone in this thread is lying about setting up E-mails servers just cause
        2. Their is a mass conspiracy targeting just the OP?
        3. The OP didn't configure something right..

    • I run my own mail server on a dyndns connection.

      I'm surprised you don't have issues with that. There are RBL's that specifically list IP address blocks that are thought to be dynamic address pools, and some servers will reject you for nothing more. Also, how do you handle reverse DNS with a dynamic IP?

  • PTR, SPF, DKIM, a clean IP and a properly configured SMTP server will work just fine. You're doing something wrong. Slashdot please improve your quality.
  • Settings to check: (Score:4, Insightful)

    by fraxinus-tree ( 717851 ) on Monday October 19, 2015 @11:30AM (#50758573)
    0. Previous RBL history for the IP address and the block
    1. Not being an open relay for any amount of time while setting up
    2. Reverse DNS
    3. SPF
    4. SMTP server host name 5. Retry delay not less than 1 hour. And e-mail starts running.
    • by tepples ( 727027 )

      The featured article mentions having already checked 0, 2, and 3 using and Port25. So of the remainder ("Not being an open relay for any amount of time while setting up", "SMTP server host name", and "Retry delay not less than 1 hour."), which is most likely?

  • Echoes my experience (Score:5, Interesting)

    by isj ( 453011 ) on Monday October 19, 2015 @11:31AM (#50758583) Homepage

    I've been running my own mailserver since 2003, and I have seen my share of problems.
    1: mailservers blocking mail based on spamhaus DUL. You can delist your IP. But still, blocking exclusively on that?
    2: accepting emails and then discarding them silently. No trace of them. No bounce. Recipient did not have it in their spam folder or anything. This was several years ago, so perhaps it's better now. But discarding emails after promising to deliver them without any possibility for the recipient to control it: bad idea.
    3: Various greylisting email servers. Not really a problem as my MTA will retry and the email is only delayed for a few minutes.
    4: rejecting emails sent over IPv6 but happily accepting them over IPv4. It turned out to be a problem with their parsing of SPF records, and apparently fixed now. But I did find out that there is no reasonable way to contact the gmail team.
    5: rejects emails due to FBLW15, whatever that means. It seems you can get whitelisted, but it appears that a lot of hosts are being hit by it for no reason.
    6: office365 bouncing emails due to "protection" with no explanation given, and direction to contact the recipient by other means to get whitelisted. This was for a the official email address listen on a company website. I decided that my email wasn't important enough. Their loss.

    Bottom line: If you run your own email server then expect to occasionally do some manual whitelisting etc. And expect some email servers to be uncooperative and/or RFC-clueless.

  • I generally do not have a problem. Obviously an outbound spam filtering service will deal with the issue.

    Did you do a slow start? Most common cause of this in the hosting industry is some guy gets a domain setups up email on a VPS then spams his entire contact list with a hey this is my new email to watch it get blocked, bounced etc. Oddly all the big guys seeing a mass mailing as the first thing they get from an IP they flag it.

    Fastmail frankly it sounds like you're a spammer er opt in marketing company. Your looking to startup a paid email service, what sets you apart from the market?

  • Different subject but now every time I try to send my dad an email with a link through it comes back saying it's SPAM. Each time I just copy and paste to Yahoo mail and it goes through fine. PissedAtCharterDotNet
  • Loose the .xyz TLD (Score:5, Informative)

    by JimMcc ( 31079 ) on Monday October 19, 2015 @11:34AM (#50758607) Homepage

    My guess is that the problem lies in the fact that the OP is using a garbage TLD. I've configured our mail server to silently drop all traffic from many of the new garbage TLDs, including .xyz. It does wonders for cutting down the spam levels. Sadly it's just a new version of Whack-a-Mole. Neither I, nor any of my users, appear to have gotten a legitimate email from any other these domains. I'll bet if the OP were to use a more traditional TLD, like .com, .uk, etc. there wouldn't be problems.

    • new garbage TLDs, including .xyz [...] Neither I, nor any of my users, appear to have gotten a legitimate email from any other these domains.

      Let me guess: neither you nor your users owns any shares of Google's parent company Alphabet Inc., whose web site is [] .

    • I've configured our mail server to silently drop all traffic from many of the new garbage TLDs, including .xyz. It does wonders for cutting down the spam levels.

      Neither I, nor any of my users, appear to have gotten a legitimate email from any other these domains.

      Gee, I wonder if that's at all related.

    • I put a rule in fail2ban to kill IPs relaying mail from .xyz domains for 86,400 seconds, on the first offense.

      Greylisting kills botnet spam. Using unique email addresses and routing them to /dev/null as they get compromised kills semi-legit spam. Fail2ban kills dictionary spammers, and, like the current .xyz spammers, acute problems that occasionally slip through the other layers.

  • You can thank years and years of spammers. The sad part to me isn't really that independent hosts are consider spam by default, its the fact, even that being the case, my independent hosted email accounts are *STILL* getting hundreds of spam mails each day. Very annoying.

  • I have continuously run my own email server since around 1990 in one form or another. Established a vanity domain in the mid 1990's and started hosting email on my own domain. I must say that has been a more difficult task as time has gone on and has required I be more savvy about IP reputation and how to maintain it. Sometime last year I moved my email server from a VPS to a dedicated host and my wife began complaining over this past summer that she could no longer send email to and friends
  • by taubz ( 322102 ) on Monday October 19, 2015 @11:44AM (#50758697) Homepage

    Try out [], a project I began a few years ago to make hosting your own mail much easier.

    It includes comprehensive diagnostics to ensure everything is configured correctly, including reverse DNS, which is the most common issue that leads to mail not being deliverable / going into spam. This doesn't solve every problem, but lots of people have had good results with this project.

  • Really, the whole closing of the open internet is about control and power. Just as FB is not about "connecting people to their friends!", it's about control and power then leveraging that power into more power, more money more control.

    The exact same thing is going on with Cloudflare which is about inducing site owners to select options which preclude anonymity.

  • I had a very similar issue with Gmail when I started sending legitimate mail. Thankfully, it was pretty easy to resolve. Maybe look at their support page for ways to fix your sender-side issues. Make sure to have domain keys, SPF, opt out trailer links, etc.. []

    Also make sure your host / server IP aren't black listed out of the gate. Generally speaking all ISP dynamic IP address blocks are marked potential spam since no customer-end's should be hosting the

  • So the big dominant e-mail providers are abusing their dominance to shut out independent competition, eh? Sounds like we should all set up private e-mail servers and then sue.

  • Problems? (Score:4, Insightful)

    by DaMattster ( 977781 ) on Monday October 19, 2015 @12:03PM (#50758875)
    I run my own email server as well. But it's not as simple as an MX record. I use domainkeys and spf as well. None of the major services flag me as spam.
  • by ledow ( 319597 ) on Monday October 19, 2015 @12:44PM (#50759215) Homepage

    A domain without information is untrusted.

    SPF tells them that you're trying to combat spam from pretending to come from you.

    Similarlt for DKIM, that also tells them that you are checking and explicitly marking every message you send out from your domain and absence of such signing should be treated as suspicious.

    Put both of those on, in a decent static IP range (nobody sensible accepts email from dynamic IP's!), and you're good to go. How do I know? My own domains are ALL run by me, on Postfix. They even forward some mail addresses to providers like GMail as a matter of course.

    The only problem I ever have for delivery is when *I* have accepted a spam message and try to forward it on to someone like GMail (harder to stop than it sounds, even with greylisting, etc.). They spot spam that my system can't, even on a re-forwarder.

    Hell, I IPv6'd my domain too. So long as you have valid PTR records for your reverse, places like GMail are perfectly happy with that. Never had a problem. (But if you can't set your reverse for your IPv6, there's a way to turn off using IPv6 and fallback to IPv4 just for GMail, etc. when using postfix - google it).

    My entire email for the last 5 years at least has been self-hosted. I've been using tiny startup services for about 10-15 years before that without issue. If anything, I have significantly more issues with the big-brand provider we use as smarthost for the Exchange servers in work, which are routinely blacklisted for spam and I have to fallback to manual sending from our leased lines, than anything to do with my self-hosting personal email domains.

    Just don't expect your no-name mailer on a dynamic range without even the simplest of anti-spam measures to be accepted by places like Google, and you're golden.

  • Maybe he got flagged as spam/spammer because it was spam? Wheres his images of the said emails? why hasn't he produced any data on how he configured his mail-server. Just stating the obvious.
  • A big part of the problem is that ISPs, not end users, now do most of the spam filtering. Under the old scheme, each user trained her own email client by using Spam/Not Spam buttons until the program learned that user's specific patten of expected mail. The complaint I get from users now is "My spam buttons stopped working!" By which they mean that they are seeing spam for which the Spam button in their client stays grayed out because the ISP decided the message was spam. Worse, they are seeing an increasin

  • Most of the larger email providers are very, very fickle about receiving email from major domains that aren't already recognized (gmail, yahoo, outlook, hotmail, etc).

    They either silently drop the mail I send to them or reject it outright, arbitrarily labeling it as "spam" or "suspicious".

    Several of the sites I run (some which sell products and some which are just forums for groups of like-minded people) almost never manage to successfully get an email to the people who are signing up and/or who have just b

  • I feel like this article was written by someone who hasn't been paying attention to the email landscape for the past twenty years. The checking services that the author lists don't make sure your DNS PTR records are correctly set up. They don't make sure that your server isn't an open relay. And they don't insure that your server is RFC compliant. They run your content through SpamAssassin and invert the score to rate your chances of successfully delivering a marketing message. I also run my own mail server
  • by Tony Isaac ( 1301187 ) on Monday October 19, 2015 @03:56PM (#50760811) Homepage

    Remember the wild west, when you could just pull off the Oregon Trail, build yourself a shack, and call it home? Nobody told you how to build your house, or how big your yard could be. But when you had a visit from a thief, there was no police to call, and if you had a fire, you lost everything. It was up to you to defend your own life at all times.

    Sure, life might have been simpler back then. But who would want to go back there?

    The Internet is the same story. In the good old days, everything was free for the taking, but it was the wild west. Now the city slickers want to put up fences, and the cowboys want to tear them down. Whether we like it or not, the Internet is changing, becoming more regulated, and some people aren't going to be happy about it.

"I have five dollars for each of you." -- Bernhard Goetz