Slashdot is powered by your submissions, so send in your scoop

 


Forgot your password?
Close
typodupeerror
×
Communications AT&T Yahoo!

Yahoo DMARC Implementation Breaks Most Mailing Lists 83

Posted by Soulskill from the we-can-think-this-through-after-it's-pushed-live dept.
pdclarry writes: "On April 8, Yahoo implemented a new DMARC policy that essentially bars any Yahoo user from accessing mailing lists hosted anywhere except on Yahoo and Google. While Yahoo is the initiator, it also affects Comcast, AT&T, Rogers, SBCGlobal, and several other ISPs. Internet Engineering Council expert John R. Levine, a specialist in email infrastructure and spam filtering, said, 'Yahoo breaks every mailing list in the world including the IETF's' on the Internet Engineering Task Force (IETF) list.

DMARC (Domain-based Message Authentication, Reporting & Conformance) is a two-year-old proposed standard previously discussed on Slashdot that is intended to curb email abuse, including spoofing and phishing. Unfortunately, as implemented by Yahoo, it claims most mailing list users as collateral damage. Messages posted to mailing lists (including listserv, mailman, majordomo, etc) by Yahoo subscribers are blocked when the list forwards them to other Yahoo (and other participating ISPs) subscribers. List members not using Yahoo or its partners are not affected and will receive posts from Yahoo users. Posts from non-Yahoo users are delivered to Yahoo members. So essentially those suffering the most are Yahoo's (and Comcast's, and AT&T's, etc) own customers. The Hacker News has details about why DMARC has this effect on mailing lists. Their best proposed solution is to ban Yahoo email users from mailing lists and encourage them to switch to other ISPs. Unfortunately, it isn't just Yahoo, although they are getting the most attention."
This discussion has been archived. No new comments can be posted.

Yahoo DMARC Implementation Breaks Most Mailing Lists More

Yahoo DMARC Implementation Breaks Most Mailing Lists

Comments Filter:

  • Darmok and Jalad (Score:5, Funny)

    by tylersoze ( 789256 ) on Wednesday April 09, 2014 @04:00PM (#46708331)

    DMARC and SMTP at Yahoo, mail broken.

    • "Dmarc, his eyes close. His sails furl."

      • Worst. Episode. Ever.
  • With the 'new' (sucky) web client -- I've started to move away from Yahoo. Bad news: Not gone yet. Biggest problem: Getting my old email messages out. (Need them for several reasons -- including legal)

    Time to move out of Yahoo... (adding another buzz kill!)

    • Re: (Score:1)

      by Khyber ( 864651 )

      If you need your emails regarding legal matters, throw a subpoena duces tecum at them.

      They'll unlock those accounts so goddamned quick, lest they be held in contempt of court.

    • Re: (Score:2)

      by 1u3hr ( 530656 )
      You can get all your Yahoo email by POP. You can pay about $20 a year; or do it free if you change your location to a non-US location, like e.g. Singapore. Then you can turn on POP. You can't send via their SMTP though, which wasn't a problem till this week. Now I put my ISP address as "From" and Yahoo in my "Reply-To". But it's pushing me to give up Yahoo entirely. I've been uneasy about them since MS bought them anyway.

  • SPF.. (Score:4, Interesting)

    by Bert64 ( 520050 ) <.moc.eeznerif.todhsals. .ta. .treb.> on Wednesday April 09, 2014 @04:14PM (#46708471) Homepage

    Implementing SPF can also do the same thing, the issue is that mailing lists don't rewrite the from headers so despite having been forwarded through the mailing list server the original sender is still shown in the headers, only the mailing list server isnt really supposed to be sending mail *from* other people's addresses...

    So either you allow mail to come from anywhere with any sender address, which lets mailing lists and email forwarding work fine but also makes spoofed spam very easy...
    Or you don't, and break the above...

    Really legit mailing lists should be rewriting the sender headers to reflect that the mail has been redelivered by the mailing list, the only difficulty this would cause is when users try to reply directly to messages rather than forwarding their replies to the list itself.

    • Re: (Score:3)

      by Minwee ( 522556 )

      Really legit mailing lists should be rewriting the sender headers to reflect that the mail has been redelivered by the mailing list, the only difficulty this would cause is when users try to reply directly to messages rather than forwarding their replies to the list itself.

      There really ought to be a better way to handle this [ietf.org].

      And it really should be implemented properly everywhere. Oh, and I want a pony too.

      • Re:SPF.. (Score:5, Informative)

        by Obfuscant ( 592200 ) on Wednesday April 09, 2014 @04:49PM (#46708721)

        There really ought to be a better way to handle this.

        RFC822 has been obsoleted at least twice now. The current standard (RFC5322) says this [ietf.org] about the origination headers:

        The originator fields indicate the mailbox(es) of the source of the message. The "From:" field specifies the author(s) of the message, that is, the mailbox(es) of the person(s) or system(s) responsible for the writing of the message. The "Sender:" field specifies the mailbox of the agent responsible for the actual transmission of the message.

        In other words, any mailing list that rewrites the From header field is wrong. It is also wrong for it to rewrite the Sender field, since the mailing list is not the "agent" responsible for the actual transmission of the message. It is only a transport agent, not an initiator. In the contextual history of RFC*22, the Sender is the person (secretary, e.g.) who sent the message when that person is not the author.

        And, additionally: "In all cases, the 'From:' field SHOULD NOT contain any mailbox that does not belong to the author(s) of the message." While that's only a SHOULD not, it is still relevant and shows the intent of that header.

        I've found the room full of horse droppings. I'm sure there's a pony around here somewhere. I'll let you ride him when I find him.

        • RFC5322 also says this:

          Note: Reintroducing a message into the transport system and using
          resent fields is a different operation from "forwarding".
          "Forwarding" has two meanings: One sense of forwarding is that a
          mail reading program can be told by a user to forward a copy of a
          message to another person, making the forwarded message the body
          of the new message. A forwarded message in this sense does not
          appear to have come from the original sender, but is an entirely
          new message from the forwarder of the message. Forwarding may
          also mean that a mail transport program gets a message and
          forwards it on to a different destination for final delivery.

          So, one could make the case that a list server is a robot reading and forwarding messages, therefor it is technically not wrong for the list server to put its own address in the From field and a contact address for the list owner in the Sender field. Note that list servers that batch posts in to messages containing several posts already do this.

          (Replies to the author and/or list could be directed by the Reply-To and Cc fields. Suggest author in Reply-To and list in Cc.)

          Of course, best

          • So, one could make the case that a list server is a robot reading and forwarding messages, therefor it is technically not wrong for the list server to put its own address in the From field and a contact address for the list owner in the Sender field.

            Other than such action would be a direct violation of the section of the RFC I quoted. The "robot" is not the author; its mailbox does not appear in the header field intended for the mailbox of the author. The "robot" is also not the agent that introduced the message for transmission, it is retransmitting a message already in the system.

            Note that list servers that batch posts in to messages containing several posts already do this.

            Not all mailing lists do this, many mailing lists allow recipients to determine whether this happens or not, and ones that do create digests are not the topic of discussio

            • such action would be a direct violation of the section of the RFC I quoted. The "robot" is not the author; its mailbox does not appear in the header field intended for the mailbox of the author. The "robot" is also not the agent that introduced the message for transmission, it is retransmitting a message already in the system.

              If I had a secretary and I instructed him to forward messages related to certain topics to designated recipients, he would be the author of the new messages that contain the original messages. The section I quoted allows this. How is this different from having a list server perform the same task?

              A multi-post digest is reasonably consided a new message. One that is "authored" by the list server. With the list owner as the responsible agent. As best I can decern, the people at IETF do not think this is a viol

              • If I had a secretary and I instructed him to forward messages related to certain topics to designated recipients, he would be the author of the new messages that contain the original messages.

                Nonsense. Whoever wrote the original message is the author. Your secretary would be the sender and his mailbox would appear in the Sender header field.

                Here's just a trivial reason why that must be. The boss sends an email describing a new policy regarding the use of slashdot during working hours. Your secretary forwards that to others and in doing so obeys your instruction to become the author. Let's say I'm on that forwarding list. Now I would ask "why is a secretary writing a policy regarding use of sla

        • Currently, all mailing lists implementations break DMARC specs. At first glance it would appear that the Mailing List specs and the DMARC specs are incompatible with each other...

          HOWEVER, There IS a way to be compliant with both specs.

          The mailing list is just a transport agent of list messages right? Well it can also be the transport agent of how users' actual email addresses are handled, between their real email address and usernames that obfusicates their actual email address.

          For example:
          * User "Bob Smith

    • Re:SPF.. (Score:4, Interesting)

      by Zocalo ( 252965 ) on Wednesday April 09, 2014 @04:27PM (#46708567) Homepage
      A better solution might be to move the original sender's "From" to another header ("Return-Path", "Reply-To", - whatever works best for the list software/admin) and set a new "From" to an address that would feed any replies to the list's submission/moderation queue. If the address of the person replying is on the mailing list or the list accepts any submission address, it goes into the normal queue for remailing, if not it either gets discarded as a bogus reply that is probably spam or goes into a moderation queue, depending on the list.

      This is still an implementation flaw in the way DMARC and SPF work with mailing lists rather than a problem with mailing lists though, so the onus really belongs with DMARC and SPF to better provide a way to support mailing lists. Including a way to specify in the DMARC/SPF configuration for the that the sender is a mailing list and that they need to validate the original sender against a different header instead - "X-Originally-From", rather than the mailing list's domain in the current "From", perhaps?

      • I would say it is a problem with mailing lists. They are taking mail, rewriting it to say something different, then delivering it in such a way that they claim they didn't change it (with broken digital signatures). This isn't Yahoo breaking mailing lists. This is just mailing lists doing something stupid. The fix is for them to stop doing MITM attacks on people's mail or to do it, but to resign the mail themselves so they take responsibility for it.

        It's not like DKIM is new by the way, mailing list develop

    • Re: (Score:2)

      by EvanED ( 569694 )

      Really legit mailing lists should be rewriting the sender headers to reflect that the mail has been redelivered by the mailing list, the only difficulty this would cause is when users try to reply directly to messages rather than forwarding their replies to the list itself.

      No no no no no no no. "When users try to reply directly" is a significant problem -- "reply" going to the sender only is fail-safe. "reply" going to the list results in things like "take me off this list" bombs, people accidentally disclo

    • Really legit mailing lists should be rewriting the sender headers to reflect that the mail has been redelivered by the mailing list, the only difficulty this would cause is when users try to reply directly to messages rather than forwarding their replies to the list itself.

      Or the fucking email providers could not be dipshits and white-list the stuff you actually subscribed to when you validate your damn email address. This can be done with existing email solutions by offering an option:

      "This is the first message from this sender, allow further unsolicited messages [_] Yes [o] No?"

      The whole sender-provider and DMARC BS is fucking irrelevant since we've had white lists and PGP for essentially ever.

  • It looks to be blocking relayed email, from a domain that it shouldn't originate from. I would think that is what we would want... mail can't come from one domain and claim to be from another. If this is the case, shouldn't the mailing list actually rewrite that it comes from the domain of originating mailing list? Because it is essentially coming from the mailing list

    • Re:Am I understanding this correctly? (Score:4, Insightful)

      by pdclarry ( 175918 ) * on Wednesday April 09, 2014 @04:30PM (#46708585)

      It's not blocking relayed mail in the usual sense. Most mailing lists use the original poster's email address as the FROM field so everyone on the list knows who posted the message. The SENDER field contains the actual list address. And that should match the sending server's IP address. So reverse DNS and SPF (and DKIM if enabled) will validate the SENDER as the list server software. The REPLY TO will be either the list or the original poster, depending on list policy. DMARC requires that the FROM field also match the sending server, and ignores SPF and DKIM.

      • DKIM validates off of the 'd=' in the DKIM signature. If the mailing list software alters the message (by adding an unsubscribe notice or other list decoration, e.g.), then the original DKIM signature is invalid regardless of any header address.

        SPF validates the sending IP to the SMTP mFROM claim. Most list software changes the mFROM to a list bounce address, and therefore SPF at least passes.

        DMARC does a couple of things to validate messages... First, it compares DKIM and SPF domains to the header From dom

      • Re: (Score:2)

        by bmo ( 77928 )

        DMARC requires that the FROM field also match the sending server

        REALLY?

        This is the stupidest thing I've seen in a long time.

        In spam fighting circles, the FROM header is universally ignored, because it can be anything. We don't fucking care what the FROM header says. Indeed, treating the FROM header as "accurate" leads to insanity like joe-jobs.

        Whoever came up with that idea should schedule a meeting with Bill Mattocks' wooden mallet.

        "My sense of personal integrity is none of your concern."
        -thus spake Walt

    • I would think that is what we would want... mail can't come from one domain and claim to be from another.

      Of course it can. It is perfectly reasonable that I use one email address for all my correspondence but use different outgoing SMTP servers depending on where my network connection is at the moment. Or I may want to use the address of a mail forwarder I use (one of many professional organizations that provide this service, e.g. IEEE) while sending an email.

      Because it is essentially coming from the mailing list

      Mailing list software is not the author of any email it distributes on behalf of a list user. The standards define what the From header contains.

      • Re: (Score:2)

        by Dog-Cow ( 21281 )

        Mailing list software is not the author of any email it distributes on behalf of a list user.

        That is only true in the sense that the list-serv isn't creative in the Human sense, and did not author the words contained in the message. From a technical standpoint, however, it is perfectly valid to state that list-serv software is authoring the message. It just happens to be quoting verbatim from some random message it received.

        • From a technical standpoint, however, it is perfectly valid to state that list-serv software is authoring the message.

          That's utter nonsense.

          It just happens to be quoting verbatim from some random message it received.

          If it is quoting verbatim then the outgoing headers will be the same as the ones it got. And quoting verbatim means that it is NOT the author, the person or entity it is quoting from is still the author.

          You might want to note that any MTA is simply "quoting verbatim" the "random message" it receives, and yet nobody in their right mind would try claiming that the MTA has assumed authorship of the message and should fiddle with the origination headers. You might also note that many de

    • Comment removed based on user account deletion

      • Forwarded email breaks all these kinds of "sender authentication" systems, and that's unlikely to change in the near future. Mailing lists are one type of forwarded email, but not the only type.

        Properly used, the Resent-From and Resent-Sender fields could help with this. Of course, this would require the Sender Authentication systems to properly handle these fields.

        Another option occurred to me since I made my previous post. The original message could be made an attachment to the message sent by the list server. This way, both the list message and the original message would be available for DMARC/SPF/whatever sender authentication.

        • The original message could be made an attachment to the message sent by the list server. This way, both the list message and the original message would be available for DMARC/SPF/whatever sender authentication.

          So not only do you suggest that the "robot" who is handing the mailing list forge emails to look like they were written by it instead of the real author, you want it to now use arbitrary contents of the body of the message (an attachment) for SPF and DMARC analysis? The body, which is under complete and full control of any spammer who happens to figure out there is a new way of bypassing spam filters by just putting something that looks like valid email transport information as an attachment to his spam?

          • No.

            Are digest messages considered forgery?

            Nor am I suggesting a back door for spammers. I do think it is likely that list servers will not be trusted to do proper Sender Authentication. Both the list message and the original message would have to pass sender authentication.

            If the list server acted exactly as a proper MTA would, then the message would only be subject to a single level of sender authentication. My idea would subject the forwarded message to double authentication: Once for the original sender

            • Are digest messages considered forgery?

              Digests are not relevant to this discussion because digests won't be blocked by DMARC. They are compilations of messages, not the original messages themselves. They already contain, or should contain, origination header fields that will pass SPF or whatever valid spam detection techniques are available.

              If the list server acted exactly as a proper MTA would, then the message would only be subject to a single level of sender authentication.

              If the list server acts as a proper MTA would, then there would still be a problem with DMARC because the list server won't fiddle with the origination headers.

              My idea would subject the forwarded message to double authentication: Once for the original sender and the second for the list server.

              Your idea is a waste of time, simply because

  • Anything that helps isolate Yahoo from currently uninfected sectors is good by me. If I never see that virulent purple abomination again it'll be too soon.

  • More ammunition for the members of various online communities I participate in for switching to some stupid forum...

  • In your quest to 'revitalize' your user-base by throwing out the loyal veterans, you pissed off people who have been members since eGroups and OneList by throwing that purple-abomination Neo web-interface at them... but still they refused to go away, they just relied more heavily on their 90's-style mail clients for access.

    This strikes at the heart of that persistence. I do believe you've found a way to get rid of your remaining loyalists. Well done.

  • Back when the Internet Mail Consortium was a thing (Score:3)

    by tlambert ( 566799 ) on Wednesday April 09, 2014 @06:52PM (#46709571)

    Back when the Internet Mail Consortium was a thing, we established best common practices for mailing lists, and most of them were vehemently against mailing list servers rewriting mail headers. Some popular MLM software rewrites standard headers, which breaks DMARC SPF implementations.

    The thing to do here is to fix the MLM software to use the correct additional headers, rather than rewriting the headers the DMARC policy feels are important; in addition, this would allow the DMARC policy to "whitelist" based on the attached headers, assuming everything else wasn't a black mark, and avoid the "greylisting" that would happen ordinarily with most SPAM filtering systems in "medium posture" rather than "low posture" (i.e. the ones that have the concept of "suspect email" as a middle ground).

    The idea that this "breaks all the IETF mailing lists" is basically alarmist BS - the IETF mailing lists are run on an individual basis, they aren't all hosted on a single machine out there, which is why they have varying degrees of SPAM and signal/noise ratios. So to claim that e.g. Namedroppers (the IETF DNS Working Group) mailing list server is impacted the same way the one Levin is all upset about is, is disingenuous.

    • Re:Back when the Internet Mail Consortium was a th (Score:5, Informative)

      by pdclarry ( 175918 ) * on Wednesday April 09, 2014 @07:58PM (#46710035)

      The thing to do here is to fix the MLM software to use the correct additional headers, rather than rewriting the headers the DMARC policy feels are important; in addition, this would allow the DMARC policy to "whitelist" based on the attached headers, assuming everything else wasn't a black mark, and avoid the "greylisting" that would happen ordinarily with most SPAM filtering systems in "medium posture" rather than "low posture" (i.e. the ones that have the concept of "suspect email" as a middle ground).

      I think you will find that most MLM software uses correct additional headers. At least listserv and mailman (for the lists that I manage) do. We've been playing nicely with ISPs for years on our lists, we create no spam (once we fixed the bounceback spam problem 3 years ago) and generally are among the more well-behaved email users around. The problem is that Yahoo's implementation of DMARC is not using the additional headers. All it looks at is From.

      • I think you will find that most MLM software uses correct additional headers. At least listserv and mailman (for the lists that I manage) do. We've been playing nicely with ISPs for years on our lists, we create no spam (once we fixed the bounceback spam problem 3 years ago) and generally are among the more well-behaved email users around. The problem is that Yahoo's implementation of DMARC is not using the additional headers. All it looks at is From.

        Not a problem, if you leave the "From:" line the hell alone, and only add new headers, per RFC 5322, and RFC 2919, etc.. It can look at the From line all it wants, and as far as it's concerned, as long as the rest of the headers are unadulterated, your list server is an intermediate relay server in the SMTP routing path.

  • Yet another Yahoo SNAFU. The seem very intent on killing their own company.
    • This is what I was thinking. Although, in this case, it doesn't appear to be Yahoo!'s fault. I miss (the real, pre-Yahoo!) Flickr. :-(
  • ..leads me to have sympathy for Yahoo. Over a decade ago, I was partly in charge of maintaining the mailing lists at the IETF Secretariat - so I remember what volume of email they were working with in 2001, and I would never want to manage a mailing list that big again (certainly not in 2014). In hind-sight it wasn't so bad then, I recall about 47,800 messages in 4 days @ roughly 85% spam for the whole IETF mailing list, but that was in 2001. We had to implement anti-spam filters for lists of people with

  • I just received a private communication from the moderator of a Google Group. He says that mail from Yahoo members is being blocked by Comcast and Yahoo. Now that it's Google's ox being gored perhaps something will be done about it.

  • seriously drop all mail incoming and outgoing to/from yahoo groups. Put them out of business instantly.

    • Sounds simple, but for some of us dropping all mail to/from Yahoo groups isn't feasible. Of six mailing lists that I receive messages from semi-regularly for work related purposes, three of them are on Yahoo groups. All are specific to niche software packages and for two of the three at least represent THE primary source of "support" for those packages. Taking your suggestion wouldn't necessarily be career suicide, but could hurt me.

      Having said that, I receive those mailing lists at my work address and not

Slashdot Top Deals

If all else fails, lower your standards.

Close