Bug 414666 - opensuse.org emailadresses will fail for default SPF configurations
Summary: opensuse.org emailadresses will fail for default SPF configurations
Status: VERIFIED INVALID
Alias: None
Product: openSUSE.org
Classification: openSUSE
Component: 3rd party software (show other bugs)
Version: unspecified
Hardware: Other Other
: P4 - Low : Critical with 18 votes (vote)
Target Milestone: ---
Assignee: Matthias Boettger
QA Contact: E-mail List
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2008-08-05 10:17 UTC by Felix-Nicolai Müller
Modified: 2009-10-20 20:41 UTC (History)
9 users (show)

See Also:
Found By: Beta-Customer
Services Priority:
Business Priority: 99
Blocker: ---
Marketing QA Status: ---
IT Deployment: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Felix-Nicolai Müller 2008-08-05 10:17:22 UTC
If the sending and the the receiving party of an Email (to @opensuse.org) use providers that use SPF the Email will be bounced as suse.de has not implemented SRS. Instead, they only redirect the Email, which looks like they are trying to fake the sender.
This bug is in my eyes very critical as the @opensuse.org receivers do not get notified for faild attempts. I only got aware of it after six month and many many people have failed to send me email to my opensuse.org adress (and finally some managed to complain about it). If the Email is forwarded to gmx or googlemail for instance, it will be turned down by the respective server. I suspect many many opensuse members are hit by this.

I already contacted postmaster@suse.de and got this reply:

"Hi,
disable your rejects on SPF records or whitelist our server on _YOUR_
server.

--> host mx1.fnmueller.de[217.13.200.26] said: <--

If you want to use you opensuse.org address, you have to disable SPF
checks. The SUSE email server only redirect messages to you and do not
rewrite the sender addresses. Your server thinks we are sending
unauthorized emails (spam) with @gmx sender addresses.

Cheers Matthias"

(Sadly, I never got a reply on why SRS is not implemented.)

This is imho the worst way to go. All opensuse members would have to be notified as well as they would have to be able to make their providers make the respective changes, which is quite hard for providers like google or gmx. Just not using SPF is not an option either. 

So SRS should be implemented on suse.de as this will transparently solve the problem for all now existing and future members.
Comment 1 Benjamin Weber 2008-08-05 11:45:15 UTC
Hmm, this could explain why I didn't get any reply to some important emails last week :(.
Comment 2 Boyd Gerber 2008-08-05 13:52:05 UTC
I have had the problems and the same results.  That is why I do not use my @opensuse.org email.  I have to use SPF.  My domain is constrantly trying to be abused.  I just this week had major forgery attemps of my Domain.  Trying to get everyone to white list is not an option.
Comment 4 Per Jessen 2008-08-06 14:07:06 UTC
I could be completely wrong here, but I don't see how this has to do with SPF - "opensuse.org" does not publish an SPF record/entry, so there's no SPF check to be done on the receiving side. 

Even if "opensuse.org" did publish an SPF record, the problem would be quickly avoided by having a '?all' instead of '-all'.  Having '-all' on a forwarding domain would be pretty silly anyway.
Comment 5 Boyd Gerber 2008-08-06 14:13:44 UTC
You do not understand.  zenez.com publishes an SPF record.  I use the -all
When I use an gerberb@opensuse.org, the email looks like it came from
gerberb@zenez.com instead of gerberb@opensuse.org.  It then hits my 
system claiming to be from gerberb@zenez.com and because of my record
it is rejected.  With SRS I do not have this problem.
Comment 6 Boyd Gerber 2008-08-06 14:20:34 UTC
I have everything you need in my home directory on the BS.  I use pysrs.

http://download.opensuse.org/repositories/home:/gerberb/

I use pysrs.  You can get information on the spf discuss list or help
lists.  Also for specific srs you it's list.

srs-discuss@v2.listbox.com

The author of pysrs is there.

I am using it.  I use the complete set in my home project on BS.
I use them to also gain reputation.

So yes it does all work with postfix.  Even the very latest version
of postfix.

I have not been able to get these from my home project to python or
server:mail
Comment 7 Per Jessen 2008-08-06 14:46:37 UTC
(In reply to comment #5 from Boyd Gerber)
> You do not understand.  zenez.com publishes an SPF record.  I use the -all
> When I use an gerberb@opensuse.org, the email looks like it came from
> gerberb@zenez.com instead of gerberb@opensuse.org.  It then hits my 
> system claiming to be from gerberb@zenez.com and because of my record
> it is rejected.  With SRS I do not have this problem.
> 

I think that's purely a configuration issue on your side and nothing to do with
opensuse at all. And you don't need SRS to solve it either, Postfix will do it
for you. 

I'm using "per@computer.org" from my own mailserver (enidan.com) as well, but
when I send mails from here, they don't look like they came from "enidan.com",
they look like they came from "per@computer.org", but obviously from an
"enidan.com" mailserver.  "enidan.com" does publish an SPF record, although
with "?all", but there's no way that'll somehow influence who will receive my
mails from "per@computer.org".  (computer.org does also not publish an SPF
record).
Comment 8 Boyd Gerber 2008-08-06 14:52:55 UTC
The problem is more on reception @opensuse.org:
user@domain1.com sends a mail to user2@opensuse.org

user2@opensuse.org is configured to forward a the mail to
user2@domain2.com

domain1.com is exporting SPF records, domain2.com is verifying on SPF
records.

thus:
MX of domain2.com gets a mail from user@domain1.com, transmitted from
the smtp server opensuse.org

if domain1.com is strict (so no -All in the SPF), domain2.com will deny
reception of that mail.
Comment 9 Boyd Gerber 2008-08-07 03:54:10 UTC
I was just told that pysrs is not recommended for postfix.  There are issues.  I was informed that it is best to use header check.
Comment 10 Boyd Gerber 2008-08-07 23:17:46 UTC
pysrs and the following for our purposes, we could configure postfix to use a proxy as a mail relay
(say at address 127.0.0.2).  It reads lines, and echoes lines to the real
mail relay (and reads lines from the real mail relay and echoes to
postfix).  A useful python smtp proxy has hooks to call a function when
postfix sends HELO, MAIL FROM, RCPT TO, and maybe even data (except an
SMTP proxy has to keep processing to a minimum to avoid triggering
incorrect timeouts).  For our purposes, just keeping track of when SMTP is
in DATA state and triggering on MAIL FROM is sufficient.


There is even a proxy class already provided in the smtpd module.
Just derive from smtpd.PureProxy and override process_message().

Since SMTPServer collects the entire HELO+MFROM+RCPTs+DATA before
calling process_message, I'm not sure about timing for large messages.
But it may do the trick for you.

Untested code:

import asyncore
import smtpd
import SRS
from ConfigParser import ConfigParser

cp = ConfigParser({
  'secret': 'shhhh!',
  'maxage': '8',
  'hashlength': '8',
  'separator': '=',
  'socket': '/var/run/milter/pysrs'
})

class SRSProxy(smtpd.PureProxy):
  def __init__(self,listen,relay):
    self.srs = SRS.new(
      secret=cp.get('srs','secret'),
      maxage=cp.getint('srs','maxage'),
      hashlength=cp.getint('srs','hashlength'),
      separator=cp.get('srs','separator'),
      alwaysrewrite=True  # skip calling us for local domains
    )
    self.fwdomain = cp.get('srs','fwdomain',None)
    smtpd.PureProxy.__init__(self,listen,relay)

  def process_message(self,helo,mfrom,rcpts,data):
    # ... lots of special cases for no-srs, signonly, etc excluded
    new_address = self.srs.forward(mfrom,self.fwdomain)
    smtpd.PureProxy.process_message(self,helo,new_address,rcpts,data)
    # prolly want to log stuff

cp.read(["/etc/mail/pysrs.cfg"])
proxy = SRSProxy('127.0.0.2','relay.fwdomain.com')
asyncore.loop()
Comment 11 Boyd Gerber 2008-08-07 23:25:35 UTC
But it may do the trick for you.

should have been

But it may do the trick for us.
Comment 12 Felix-Nicolai Müller 2008-08-10 09:42:05 UTC
are there any updates on the bug? A usable opensuse.org emailaddress would be really helpful and great. :-)
What is the action plan?
Comment 13 Per Jessen 2008-08-10 13:04:31 UTC
That's probably a little much to expect in less than a week. 

Second, the proposed solutions are untried and untested - I don't think it's a good idea to rush into a deployment on a high volume mail server.

Third, AFAICS, this problem does not affect everyone with an @opensuse.org alias - in fact, it is only a problem when your forwarding-to address is on a mail-server with strict SPF enforcement.

There used to be a scheme called "trusted forwarders" for problems exactly like this, in fact the website is still up http://www.trusted-forwarder.org.  I don't know if it works anymore nor if anyone is using it, but you never know. (the latest zone file is from february 2008).
Comment 14 Felix-Nicolai Müller 2008-08-10 13:27:57 UTC
"That's probably a little much to expect in less than a week. "
All I want is something like a status update (see below).

"Second, the proposed solutions are untried and untested - I don't think it's a
good idea to rush into a deployment on a high volume mail server."
How do you know that? I personally was in contact with the suse.de postmaster and his answer indicated that nothing will be done if I leave it up to just him. I don't know how opensuse folks can talk internally to suse.de folks, I don't know which way they want to go or how much priority is given to this issue. And I need some safety to deploy my opensuse.org address, which I have already used in congresses (not a good idea as it now turned out!). 
This _is_ a highly critical issue if you give someone at a congress an openuse.org emailaddress to show the flag and then this only causes problems.

"Third, AFAICS, this problem does not affect everyone with an @opensuse.org
alias - in fact, it is only a problem when your forwarding-to address is on a
mail-server with strict SPF enforcement."
Just plain wrong. The forwarding-from address / server is "the problem" (well, not it SPF was properly supported by suse.de). Therefore, if you don't have your opensuse.org address forwarded to a server you have complete control of  - you have a problem.

"There used to be a scheme called "trusted forwarders" for problems exactly like
this, in fact the website is still up http://www.trusted-forwarder.org.  I
don't know if it works anymore nor if anyone is using it, but you never know.
(the latest zone file is from february 2008)"
This only worked if all the mailservers using SPF would use it and trust this list....

Baseline is: 
We need a working opensuse.org address as soon as _possible_. And it seems suse.de postmaster is the only one able to solve this for every opensuse member with one change in his configuration.
In effect, every opensuse member can be hit by this. In effect, a problemfree account might not work tommorrow if the provider changes his mind to use SPF (which more and more are doing).

I know every change needs to be tested. I know it is sunday and I don't expect an answer today. But slowing down and taking pressure away is no solution, nor is it a solution to not use the opensuse.org address or for each member to run his own mailserver. If presenting opensuse to the outside can be done in a good way by having a sometimes working emailaddress..... 
Fixing this asap and for everyone in a way that no action has to be taken by each individual member is in the interest of opensuse, novell and suse and each member. 

Open communication and a status update from time to time would certainly keep me calmer.
Comment 15 Per Jessen 2008-08-10 14:12:37 UTC
(In reply to comment #14 from Felix-Nicolai Müller)

> "Second, the proposed solutions are untried and untested - I don't think it's 
> a good idea to rush into a deployment on a high volume mail server."
> How do you know that? 

Allright, I'm guessing - like Boyd Gerber said a while ago "pysrs is not recommended for postfix", and libsrs2 is still only out for postfix 2.1.4 (whilst postfix has moved on to 2.5.3).  Does not sound like tried and tested to me.

> "Third, AFAICS, this problem does not affect everyone with an @opensuse.org
> alias - in fact, it is only a problem when your forwarding-to address is on a
> mail-server with strict SPF enforcement."
> Just plain wrong. The forwarding-from address / server is "the problem" (well,
> not it SPF was properly supported by suse.de). Therefore, if you don't have
> your opensuse.org address forwarded to a server you have complete control of-
> you have a problem.

Well, I can only say that my company does a LOT of email forwarding, and the only time SPF causes a problem is when the recipient mail server has strict SPF enforcement. So far we have only hit this problem with one single mail server.

> Open communication and a status update from time to time would certainly keep
> me calmer.

The report has not yet been assigned to anyone. I'm sure something will happen tomorrow. 
Comment 16 Boyd Gerber 2008-08-10 14:30:34 UTC
While I do control my mail server, not having a strick SPF policy is not an option for us.
I currently have some spammer forging my email address to send bulk email.
I get 2000 replies per day telling me my address was forged and that my email
has not reached it's user thinking I need to know that my domain uses a
strick SPF policy and the email was not sent.  I get between 1000 and 5000 email from
other MAIL DEAMON's telling me what ever results.  These are just from forged 
gerberb... or gerber...  There is a spammer in the UK that loves to use my domain
as the from user.  They send out 10,000,000 to 40,000,000 emails per week
Without SPF and my strick policy they would allow these email to not be rejected at the boarder
and then more intensive measures would be requirred to note they are forged.  I rejected about 30000 email just
for SPF FAIL. per day.  There are some days that this number reaches
100,000 SPF rejects.  I had logging of SPF FAIL to get a feel for this 
on my small domain.  So I really see the benefits of a strick SPF policy.

I use the white list method for forwarders.  

Saddly trusted forwarders has been depricated.  As SPF is now considered a standard.

Saddly it is only a standard for testing.  RFC 4408.  This was caused by MS and their 
Sender ID.  I prefer DKIM over Sender ID.  The more people that have a
strick SPF record the better because it allows reject at the boarder and no further 
processing is needed.  Sadly I have been trying to get SRS on the SUSE systems for
over 2 years.  I really do not see them doing anything soon sadly.  I had hope
that this bug might make it happen, but I am beginning to doubt that it will
make much difference.
Comment 17 Andreas Jaeger 2008-08-10 19:04:07 UTC
Ad #12: Please give us some more time, this needs proper investigation.  I'm gratefull for the help provided in this but we cannot start right away:

It's vacation time - Henne is on FTO right now and Adrian who owns the bug was at LWE last week and will return the next days...
Comment 18 Adrian Schröter 2008-08-11 08:48:43 UTC
The non-ml adresses are handled by IS&T and not lists4 server.

Low prio, since this affects only servers with non-common features.
Comment 19 Matthias Boettger 2008-08-11 10:26:12 UTC
I don't know what I or IS&T should do here, from our side everything is working! Opensuse.org addresses are normal aliases which will redirected to the final destination. This is normal way like email aliases work!

Rejecting emails based on SPF records is absolute idiocy and makes only trouble. You can use SPF records for filtering (e.g. with spamassassin) but never for blocking! Only RBLs are common ways for blocking emails! If you have trouble with spam on your domain, try to use greylisting, which will lock out a lot of spammers. If you have trouble with your ISP, rejecting based on SPF records, contact your ISP. Neither SUSE IS&T nor OpenSUSE is able to change anything here. If you need to block using SPF records, whitelist our servers! If you don't want to do that, don't use your opensuse.org address or tell us a different destination address.

We will not change anything on our email servers here. If the OpenSUSE team want to have a fix here with dirt hacks, please do it on an own server. I will not play with our high traffic email relays! 
If you want to have SRS, implement it on SLES with postfix and we will use it. 

We use what we sell!
Comment 20 Jim Stutz 2009-10-20 17:18:41 UTC
Closing bugs to meet ACS SLA requirements.
Comment 21 Jim Stutz 2009-10-20 20:41:35 UTC
This bug is skewing the SLA results, marking it so that it work be used in the SLA calculation.