ThunderBayes 1.0

SpamBayes integration for Thunderbird.

UPDATE: ThunderBayes 1.0.1 is now available.

Download ThunderBayes 1.0

Overview

ThunderBayes provides a toolbar button similar to Thunderbird’s Junk button with which email can be classified as Spam or Ham. Clicking the button causes two things to happen: (1) it sends the source of the selected messages to SpamBayes to be classified and (2) it optionally moves the messages to a folder of your choice (this can be configured in the extension options). It includes a custom version of SpamBayes, and provides a simple preference page in the Thunderbird Account Settings where the SpamBayes POP3 proxy and message filters can be configured.

Changes in this version

  • Added integrated SpamBayes proxy server (can be disabled/bypassed).
  • Added ThunderBayes configuration in Thunderbird Account Settings (POP3 accounts only).
  • Implemented REST API for SpamBayes, which allows more efficient training and automatically updates the SpamBayes review page when training in Thunderbird.

Requirements/prerequisites

Everything needed to run ThunderBayes on Windows is included in the XPI file.

Non-Windows users must have Python 2.2 or later (2.3 recommended) installed. It works great with the Python 2.3.5 that ships with Mac OS X 10.4 (it should work fine with earlier versions of Mac OS X as well).

Recommended configuration:

  • On the main menu bar select Tools->Account Settings…
    • Select the “ThunderBayes” pref page on any POP3 account with which you would like to use ThunderBayes spam classification and filtering.
    • Check “Enable ThunderBayes proxy for this account”. This will copy the real hostname and port for this account from the Server pref page to the ThunderBayes page. The Server page will be populated with the proxy hostname and port for this account. Note: this may not work if you are using an externally configured version of SpamBayes (i.e. not the internal version that ships with ThunderBayes).
    • Enable and configure filters if desired (you may want to do this later after you have trained SpamBayes and are confident that it is filtering mail correctly). WARNING: renaming the filters created by the ThunderBayes pref pane will prevent them from being edited from the pref pane–you will need to use the Message Filters list to edit them if you rename them.
  • Enable the “Spam Status” column in your Inbox and other folder views by clicking the right-most column header (you should see a menu). Select the “Spam Status” item so there is a check mark beside it. The Spam Status column uses four icons to denote message status:
    • Spam – red circle with slash
    • Unsure – yellow question mark
    • Ham – gray dot
    • Unknown – gray question mark (this means the message did not have a “X-Spambayes-Classification” header, which probably means it did not pass through the SpamBayes proxy)
  • Add the ThunderBayes “Spam” button to the toolbar. To add the button to the toolbar, right-click the toolbar and select “Customize…” Then drag the “Spam” icon into your toolbar and click “OK” on the “Customize Toolbar” dialog.

Migrating to ThunderBayes 1.0
(from any previous version)

As of version 1.0, ThunderBayes ships with a Python interpreter (Windows only)
and SpamBayes 1.0.4. This makes it very easy for anyone with Thunderbird to get
ThunderBayes installed and operational. Previously it was necessary to install
and configure SpamBayes before ThunderBayes actually did anything useful.

If you are one of the early adopters of ThunderBayes and went through all the
nuisance of setting up and configuring SpamBayes in addition to installing the
ThunderBayes extension then this section applies to you. Everyone else may skip
ahead to the next section.

These are the steps to migrate the data from your externally configured
SpamBayes to ThunderBayes’ internal version of SpamBayes. Read all of the
instructions before you start the migration process.

  1. Shutdown Thunderbird.
  2. Shutdown your SpamBayes proxy (disable automatic start as well).
  3. Backup your Thunderbird profile. It’s easy. Instructions here: http://www.mozilla.org/support/thunderbird/profile#backup
  4. If you skipped the previous step, go back and do it…really, you might need it if you do something wrong in the following steps. I usually like to make a zip archive of my profile so I don’t accidentally delete or change any of the files contained in it. That, and it’s easy to move around when it’s a single compressed file.
  5. Make a new folder in your Thunderbird profile named ‘spambayes_data’. The path should be something like this (the ‘default.xyz’ part will be different): <…>/Thunderbird/Profiles/default.xyz/spambayes_data
  6. Find out where your existing SpamBayes data is stored. On Windows it might be
    C:\Documents and Settings\\Application Data\SpamBayes\Proxy
    On Mac OS X it might be in /Library/SpamBayes. If you can’t find it just search for some of the files in the next step–that should lead you to the right place.
  7. Copy the following files and folders from your existing SpamBayes directory:
    • _pop3proxyham.mbox
    • _pop3proxyspam.mbox
    • bayescustomize.ini
    • hammie.* (i.e. any files that start with ‘hammie.’)
    • pop3proxy-ham-cache/* (folder and all files inside it)
    • pop3proxy-spam-cache/* (folder and all files inside it)
    • pop3proxy-unknown-cache/* (folder and all files inside it)
    • spambayes.messageinfo.db
  8. Mac OS X and other Linux/Unix users: edit the bayescustomize.ini file and change each port listed on the line beginning with listen_ports to something higher than 1023. Then change each account in Thunderbird to use the new port assigned to that account (after the next step). Explanation: on Unix ports lower than 1024 are privileged and cannot be used by non-root users.
  9. Start Thunderbird
  10. If you haven’t already, install ThunderBayes 1.0 and restart Thunderbird

If everything went as planned you should have a fully functional ThunderBayes setup with the same SpamBayes configuration that you had before, except its now running the internal version of SpamBayes distributed with ThunderBayes.

After all that, I will tell you that migrating the SpamBayes database to use the internally bundled version of SpamBayes is not strictly necessary–the internal version can simply be disabled, and you may continue to use your externally configured version of SpamBayes. However, there are a few (minor) features in ThunderBayes that will not work correctly:

  1. The ThunderBayes server name/port configuration in Thunderbird’s Account Settings will not work. Filter configurations will work, although you’ve probably already configured your own which will continue to work as well.
  2. ThunderBayes ships with a patched version of SpamBayes that automatically updates the “Review” page in the SpamBayes proxy web interface upon training with a given message. If you don’t know what that is you don’t have to worry about it. However, if you opt to use an external version of SpamBayes this feature will not work in ThunderBayes 1.0. It did work in ThunderBayes 0.3, which was never formally released. The feature was reimplemented in ThunderBayes 1.0 as a custom extension to SpamBayes, which is much more efficient and makes the code much simpler.
  3. ThunderBayes will not automatically start/stop SpamBayes when Thunderbird starts/stops (obviously).

To disable the internal SpamBayes shipped with ThunderBayes go to Tools->Add-
ons->Extensions->ThunderBayes->Preferences. Uncheck the “Use internal SpamBayes
proxy” checkbox.

Finally, a simpler option is just to disable your old SpamBayes proxy and install the new version of ThunderBayes. Configure your accounts in Thunderbird, and train from scratch. This will of course be a pain at first because you have to re-establish all of your training data, but it is very easy to get going
quickly. NOTE: when you configure your accounts you will need to set the server name and port in Server Settings back to the real server name and port BEFORE you configure the ThunderBayes page in Account Settings.

11 Responses to “ThunderBayes 1.0”

- Omar

Hi Daniel,

Thanks for your efforts in developing this useful extension for ThunderBird!

This is the first time I’m using it, certainly hope it helps sort out the junk since I’m getting tired of doing it manually. 😉

I just wanted to let you know that I’ve installed ThunderBayes 1.0 on ThunderBird 2.0.0.6 (20070728) but I’m not sure if it’s working…

I went into Tools->Accounts, disabled the built-in adaptive junk filters, and enabled ThunderBayes for each account.

I also added the Spam/Ham button to the toolbar (the spam status & probability columns were added automatically).

I have noticed the following (possible) issues:

– If I go into Tools->Accounts and check the ThunderBayes Settings page for each account, the ‘Enable ThunderBayes for this account” and the two filter Settings checkboxes are cleared

– I can select all the checkboxes, but as soon as I do anything, even as simple as flipping to the ‘Junk Settings’ page (for example) and then back to the ‘ThunderBayes’ page, all three ThunderBayes checkboxes are cleared

– Clicking on the ‘Configure’ button for either of the two ThunderBayes filters results in a ‘Cannot find ThunderBayes-spam/unsure filter.’ alert

So I’m not even sure if it’s working… I think it is though – I’ve manually marked a few hundred messages as spam and ham, and this at least, is accurately reflected in the Spambayes web Interface, which says:

POP3 proxy running on 53100, proxying to pop.xxxxx.com:110.
Active POP3 conversations: 0.
POP3 conversations this session: 0.
Emails classified this session: 0 spam, 0 ham, 0 unsure.
Total emails trained: Spam: 209 Ham: 326

Thanks again for all your hard work. 🙂

- Julian

A very good integration of Spambayes with minimal setup. It just works whereas I had problems with 0.2.2 and the external Spambayes.

One for the wish list – please would it be possible to have separate buttons for Ham and Spam and not the drop down selection? There is also no Spam /Ham button when the email is open.

I gather there is a newer version of Spambayes on the way (1.1) will this be integrated into Thunderbayes?

- Julian

Then Mozilla Thunderbird Add-ons site still shows version 0.2.2 of Thunderbayes.

- Rob

Hi

I’m a little bit confused about what’s going on.

I have Thinderbird 2.0.0.6 on Suse 10.0 and I’ve downloaded and installed the add-on v1.

In the add-on prefs I selected “use internal spambayes proxy”. In the acct prefs though it lets me select the enable thunderbayes on this account check box but won’t save it.

It doesn;t let me configure the filters. It says Cannot find Thunderbayes filter.

Another question. Do we leave the Thunderbird junk filtering enabled?

- Martin

I’ll probably go for the “simpler option” when upgrading from 0.2.2 later this day ;-). Now all I need is a whitelist so I can put certain e-mail addresses on there.

Just one more question: Will Mozbackup now backup everything related to Thundebayes so I can easily move all my data from my desktop to my notebook when I’m traveling?

Keep up the good work!

Cheers,
Martin

- Giuliano

I Daniel,
first of all I have to thank you for your Thunderbayes. It is probably the only thing I really missed since I left Outlook for Thunderbird.
Is there any forum about Thunderbayes? If not I wish you’ll have 5 minutes for me:
I installed Thunderbayes 1.0 following your instruction about migrating from a previous version (0.2.2).
It works fine but each time I close Thunderbayes and restart it I loose the configurations and I have to go inside the config page to write again the Remote servers and the spambayes ports.
I disinstalled the spambayes 1.0.4 application, but nothing changes, and I have the same problem on both the desktop and the notebook.
I use Thunderbird 2.0.0.4 and Thunderbayes 1.0
Let me know if you may need any other info and thanks a lot for your help.

- daniel

For everyone who had problems with release 1.0, I’m very sorry. A new version has been released. See my post on ThunderBayes 1.0.1 for more details and a download link.

Julian – Separate Spam/Ham buttons may be added to a future version (I’ve gotten many requests for that feature). I know about the lack of a button when the email is open; that should be fixed soon. Finally, SpamBayes 1.1 will be bundled with ThunderBayes as soon as it is beyond alpha status (or when I deem it stable enough).

Martin – I’m not sure about MozBackup. Let me know what happens if you try it. One way to make sure you’ve got everything is to copy the entire profile directory (all spambayes settings are stored there unless you’re using an external version of spambayes).

- Rob

Hi

I finally got 1.0 working after installing, uninstalling, reinstalling, reuninstalling, creating a new profile etc etc.

In the end I restored my original profile, installed Thunderbayes, swore a couple of times and then deleted the spambayes_data directory … et voila. It works like a charm.

Finally some real spam filtering. Good work! I appreciate it.

PS I’ll vote for two buttons as well.

- Jerome

What is the status with Spambayes and SSL connection ?
I recall spending quite a lot of time to get Spambayes 1.0.4 (march 2005) to work for thunderbird having to use Stunnel to be able to connect to my corporate mail server that requires SSL

- Don

I have installed ThunderBayes on a computer running Windows XP Pro with Thunderbird and can’t find a button to put on the Task Bar. Using the latest versions of both. What happened??

- Terry

Daniel, thank you for Thunderbayes – I installed it recently, and it works really well for me. Keep up the great work!