Bug 164751 - [AI] Mailinglist migration
Summary: [AI] Mailinglist migration
Status: VERIFIED FIXED
: 140500 (view as bug list)
Alias: None
Product: openSUSE.org
Classification: openSUSE
Component: Action Items (show other bugs)
Version: unspecified
Hardware: Other Other
: P5 - None : Normal (vote)
Target Milestone: ---
Assignee: Hendrik Vogelsang
QA Contact: Adrian Schröter
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2006-04-09 22:11 UTC by Christian Boltz
Modified: 2007-03-31 23:37 UTC (History)
2 users (show)

See Also:
Found By: Other
Services Priority:
Business Priority:
Blocker: ---
Marketing QA Status: ---
IT Deployment: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Christian Boltz 2006-04-09 22:11:42 UTC
from http://en.opensuse.org/2006-01-24-status-meeting

Action Item skh, henne:
- Prepare and announcement for list migration to the internal SUSE lists.

Action Item skh, henne:
- risk assesment for the internal packagers list.

Action Item adrian:
- check again with IS&T about searchable archives.
Comment 1 Christian Boltz 2006-04-09 22:46:59 UTC
http://en.opensuse.org/Meetings/Status_Meeting_2006-02-21

Action Item henne/Klaus
- find out if we can help heiko with the postfix rewrites for the new list
  server
Comment 2 Sonja Krause-Harder 2006-04-10 13:01:13 UTC
Do we have a status for that?
Comment 3 Hendrik Vogelsang 2006-04-10 16:00:06 UTC
*** Bug 140500 has been marked as a duplicate of this bug. ***
Comment 4 Christian Boltz 2006-04-10 18:25:55 UTC
some notes from the duplicate (bug #140500) shouldn't get lost:

* Candidates are: packagers, suse-network, devel-ftl, devel-gcc?, mobile?
  Non-candidates are: kernel and other mailing lists where security issues
  or servicepacks are discussed.

* from bug #141155 (dupe of the dupe ;-)): mailinglist archives contain lots of 
  spam
Comment 5 Christian Boltz 2006-05-09 21:24:00 UTC
Henne, just FYI: Highlighting the current mail in the MHonArc archives
seems to be possible - see
    http://www.mhonarc.org/archive/html/procmail/2006-05/msg00020.html
for a live demo.
Comment 6 Christian Boltz 2006-06-30 22:16:23 UTC
Henne, I've just seen the new design on lists4.opensuse.org. Looks very
good! :-)

Beta-testing the archives, I found some things that could be made better ;-)
- please don't spam-protect the message-IDs - they can be very useful when 
  searching for a specific mail
- the mbox files (except the current month) should be compressed
- the "search result" page (search.cgi) is not correctly themed yet and has 
  a broken link to /images/opensuse.gif (which doesn't exist)
- the search results also match on the navigation - this can be annoying when 
  searching for author names. [1]
  Is there a way to exclude the navigation from the search index?
  (a similar solution is mentioned in bug 173441: <!--htdig_noindex-->)
- according to the headers in the mbox, opensuse-test seems to run on the new 
  server already. I'm missing the X-Message-Number-For-Archive: header, can you
  please re-add it?
- There seems to be an issue with decoding special characters in Subject: and
  From: headers - see http://lists4.opensuse.org/opensuse/2006-06/msg00010.html 
  and http://lists4.opensuse.org/opensuse/2006-06/author.html for examples
- a "scroll to top" link would be useful on nearly all pages
- what about removing the message numbers in the "Follow Ups" and "References" 
  column?
- very minor: on the overview pages, the "," in "by thread, by date, by author,
  in mbox format" should not be part of the link ;-)


[1] Example: search for "Jaeger" in opensuse-announce - the result will contain 
    things like "This thread: Andreas Jaeger ... Andreas Jaeger ... Follow ups 
    to this message: 00003 by Andreas Jaeger"
Comment 7 Christian Boltz 2006-07-01 18:08:18 UTC
Two additional issues:
- the 2006-07 archives show the Subject: twice instead of the Message-ID: 
  header, see http://lists4.opensuse.org/opensuse/2006-07/msg00001.html
  (verified in different lists, bug is new in 2006-07)
- the overview pages (like http://lists4.opensuse.org/opensuse/) don't show the 
  favicon (broken <link> "favicon.ico", should be "/skin/favicon.ico")
Comment 8 Hendrik Vogelsang 2006-07-03 14:25:14 UTC
- I can spamprotect either everything or nothing
- they will be. they are not at the moment because im testing...
- that are old broken search indexes. will be fixed once im done testing
- search pages are not css'ified yet
- X-Message-Number-For-Archive isnt possible with the new mailinglist manager
  i might be able to hack something together. Why do you need it?
- encoding is already fixed
- i will rework the navigation a bit anyway
- message numbers removed
- done
Comment 9 Christian Boltz 2006-07-03 21:20:48 UTC
Let's try with some quoting ;-)

(In reply to comment #8)
> - I can spamprotect either everything or nothing

That's bad :-(
No spam protection isn't a good idea - on the other hand the spam protection makes it impossible to contact someone. Simply masking/removing the @ and the dots (replace them with a space for example) would be better IMHO.
Maybe you should ask on the mailinglist if someone can implement this and also un-protect the Message-ID ;-)

> - X-Message-Number-For-Archive isnt possible with the new mailinglist manager
>   i might be able to hack something together. Why do you need it?

It's a good way to check if you received all mails from the list - which might be essential for security announcements (yes, they are numbered in the subject, but the "summary reports" are not) and is at least useful for all other lists, even if it's "only" for statistical purposes or guaranteed complete local archives.

BTW: I have a script (from David Haller ;-) to check the completeness of a mbox - I can attach it if you are interested.

BTW2: I guess the "CSS'ify mailinglists" on opensuse.org/Tasks is obsolete now - please remove it if you agree...
Comment 10 Hendrik Vogelsang 2006-07-04 10:50:46 UTC
Try the spam protection now. Most spam harvesters should be able to parse it anyway tho. Im not so sure this is a good idea...

Each archive prints now how many messages it holds.
Comment 11 Christian Boltz 2006-07-04 13:50:00 UTC
(In reply to comment #10)
> Try the spam protection now. Most spam harvesters should be able to parse it
> anyway tho. Im not so sure this is a good idea...

Well, IMHO you have to find a middle way between spam protection and usability of the archives ;-)

You should also have in mind that spammers can subscribe to any mailinglist (with an "anonymous" gmx etc. adress) and collect adresses there...
(I strongly assume this already happens on suse-security :-/ - my mail adress used there clearly causes the most spam.)


About the "encryption" of the mail adress:

name<at>domain<dot>tld  is an acceptable way. Maybe it is too common for spam harvesters, don't know - but my experience shows that _any_ encryption will work ;-))

Some other options:

I use   n a m e (at) d o m a i n . d e   on my homepage and never got spam on this address. Combined with a reduced letter-spacing (CSS) it doesn't look too bad ;-)  I guess you can even use a plain @ instead of "(at)" - a  @  with a space before and after won't attract spammers too much ;-)

The solution from KDE bugzilla also didn't result in any spam message yet - they simply remove the @ and all dots. Example:  name domain de

On pages where people requested working mailto: links, I use some ways of JavaScript encryption. This is also a well-working solution.


Summary: I guess _every_ encryption will work. I can live with the one you implemented now, however it's not the one I prefer because of readability ;-)
Comment 12 Hendrik Vogelsang 2006-07-06 14:04:28 UTC
I cant change letter spacing for body parts and translating it in general looks pretty ugly. See

http://lists4.opensuse.org/opensuse-test/2006-07/msg00046.html

especially long addresses get _very_ long if i space them...
Comment 13 Christian Boltz 2006-07-06 17:57:16 UTC
(In reply to comment #12)
> I cant change letter spacing for body parts and translating it in general 
> looks pretty ugly.

Indeed, it isn't a good solution here.

What about the "KDE Bugzilla" way "name domain de" (replace "@" and all dots with spaces)?
Comment 14 Hendrik Vogelsang 2006-07-07 09:57:42 UTC
Hm how about that?

suse-linux-unsubscribe©suse·com

instead of "at" i use "copyright" and instead of "dot" i use "middot"

thats at least a pretty unusual obfuscation
Comment 15 Hendrik Vogelsang 2006-07-07 10:23:35 UTC
Okay after thinking about it again i will reenable the stripping onf the domainpart. The unique part of the message id isnt touched. This makes it possible to search for it (in the web-archive and in your mailer) and i dont have to deal with obfuscation which is only a race with the harvester scripts. Every regex i might think of is probably already thought of by the spammers anyway.
Comment 16 Hendrik Vogelsang 2006-08-17 18:49:20 UTC
i consider this half done now.
Comment 17 Christian Boltz 2006-09-10 12:26:33 UTC
OK, then let me summarize the open parts of this AI ;-)

- migration of SUSE-internal lists
  Candidates are: packagers, suse-network, devel-ftl, devel-gcc?, mobile?
  Non-candidates are: kernel and other mailing lists where security issues
  or servicepacks are discussed.
- migration of @suse.com lists (suse-linux etc.)
- lists.suse.com archives contains lots of spam - will hopefully be solved when 
  migrating to the new list server
- mbox files should be compressed (gz or bz2) because
  a) less disk space usage
  b) less traffic
  c) harder for spammers (the uncompressed files are available in Google, 
     making spam-protection in the web archives quite pointless)

BTW: opensuse-announce doesn't have a list footer (with unsubscribe address etc.) - please add it ;-)
Comment 18 Christian Boltz 2006-09-12 08:09:36 UTC
One more item for the TODO list: don't deliver spam...
http://lists.opensuse.org/opensuse/2006-09/msg00162.html has 17.5 spamassasin points on relay1.suse.de (and 20.3 SA points on my server) - such mails should never be delivered to the mailinglist!

Oh, and the list archive does not contain the content of HTML-only mails (like this spam mail) ;-)
Comment 19 Hendrik Vogelsang 2007-02-01 11:50:12 UTC
this is done
Comment 20 Christian Boltz 2007-03-31 23:37:14 UTC
VERIFIED and CLOSED - thanks for your work on setting up the new list server!