Hyperkitty

Hyperkitty is a Django based archiver and archive interface for Mailman.

Installation

To use Hyperkitty, a working web server setup is required (e.g. using Apache HTTP Server to forward to the WSGI directly, or using Nginx forwarding requests to an application server such as UWSGI).

Install the hyperkitty package.

Warning: Hyperkitty should only be accessed over TLS (unless only accessed directly from the machine running it for testing purposes), as it otherwise exposes passwords and user data to the network.

Configuration

The web application is configured in /etc/webapps/hyperkitty/settings_local.py (which is included by the default configuration in /usr/share/webapps/hyperkitty/settings.py).

Note: Hyperkitty should store user sensitive data (e.g. sqlite database) in /var/lib/hyperkitty/data/, as that directory is only accessible by root and the application itself.

Change the default secret for the application:

/etc/webapps/hyperkitty/settings_local.py

SECRET_KEY = 'something-very-secret'

Make sure to disable debugging when running in production:

/etc/webapps/hyperkitty/settings_local.py

DEBUG = False

Add a valid email configuration (so that the Django application can verify subscribers):

/etc/webapps/hyperkitty/settings_local.py

EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend'
EMAIL_HOST = 'localhost'
EMAIL_PORT = 25
EMAIL_HOST_USER = username
EMAIL_HOST_PASSWORD = password

Note: The DEFAULT_FROM_MAIL and SERVER_MAIL configuration options can be used to define the From: header of mails sent for internal authentication and error reporting, respectively.

To connect with a running mailman instance's REST API, configuration options have to be added to hyperkitty's configuration.

/etc/webapps/hyperkitty/settings_local.py

MAILMAN_REST_API_URL = 'http://localhost:8001'
MAILMAN_REST_API_USER = 'rest_admin'
MAILMAN_REST_API_PASS = 'rest_admin_password'

To configure the archive integration with a mailman instance first setup the integration with hyperkitty on mailman's side and then configure hyperkitty to accept those connections:

/etc/webapps/hyperkitty/settings_local.py

MAILMAN_ARCHIVER_KEY = 'SecretArchiverAPIKey'
MAILMAN_ARCHIVER_FROM = ('127.0.0.1', '::1')

The valid hosts or domain names for the application need to be defined:

/etc/webapps/hyperkitty/settings_local.py

ALLOWED_HOSTS = [
    'localhost',
    'lists.example.com'
]

Hosting

This article or section needs expansion.

Reason: Example for a setup using Apache HTTP Server. (Discuss in Talk:Hyperkitty)

Note:

Hyperkitty needs to be run as its own user and group (i.e. hyperkitty). It is using /etc/webapps/hyperkitty/, /var/lib/hyperkitty/ and /run/hyperkitty/ for configurations, static caches and (potentially) sockets, respectively.
As the static files are by default served from a top-level static/ directory, which will conflict with postorius on the same (sub)domain, it is advisable to set the STATIC_URL variable in /etc/webapps/hyperkitty/settings_local.py to something unique (e.g. '/hyperkitty_static/').

Nginx and uWSGI

Hyperkitty comes with a working uWSGI configuration file in /etc/uwsgi/hyperkitty.ini.

Install nginx and uwsgi-plugin-python, create a per-application socket for uWSGI (see UWSGI#Accessibility of uWSGI socket for reference) and activate the [email protected] unit.

For a local test setup, serving Hyperkitty at http://localhost/hyperkitty/ add the following Nginx configuration to your setup:

/etc/nginx/hyperkitty.conf

server {
  listen 80;
  server_name localhost;
  charset utf-8;
  client_max_body_size 75M;
  root /usr/share/webapps/hyperkitty;
  access_log /var/log/nginx/access.hyperkitty.log;
  error_log /var/log/nginx/error.hyperkitty.log;

  location /hyperkitty_static {
    alias /var/lib/hyperkitty/static;
  }

  location ~^/(hyperkitty|user-profile)/(.*)$ {
    include /etc/nginx/uwsgi_params;
    uwsgi_pass unix:/run/hyperkitty/hyperkitty.sock;
  }
}

Setup

Note: Run the following commands as the hyperkitty user (e.g. using sudo or su).

After first installation make sure to generate a database:

[hyperkitty]$ django-admin migrate --pythonpath /usr/share/webapps/hyperkitty/ --settings settings

Afterwards, the static data for the application needs to be collected:

[hyperkitty]$ django-admin collectstatic --pythonpath /usr/share/webapps/hyperkitty/ --settings settings

To compress the data, run the following:

[hyperkitty]$ django-admin compress --pythonpath /usr/share/webapps/hyperkitty/ --settings settings

Enable and start the hyperkitty-qcluster.service systemd service for required asynchronous operations on the web application.

Populate the database with default data (when setting up for the first time):

[hyperkitty]$ django-admin loaddata --pythonpath /usr/share/webapps/hyperkitty/ --settings settings first_start

Create a superuser account for the Django application:

[hyperkitty]$ django-admin createsuperuser --pythonpath /usr/share/webapps/hyperkitty --settings settings

Log in to the admin interface of the Django application at http://localhost/hyperkitty/admin to be able to add more Sites besides the default example.com or to add additional Mail domains.

Note: After adding a new site make sure to set the SITE_ID variable in /etc/webapps/hyperkitty/settings_local.py to the respective ID!

Tips and tricks

Importing mailman2 archives

Hyperkitty can import archives from mailman < 3.0.

Note: Run the following commands as the hyperkitty user (e.g. using sudo or su).

[hyperkitty]$ django-admin hyperkitty_import --pythonpath /usr/share/webapps/hyperkitty --settings settings -l ADDRESS mbox_file [mbox_file ...]

Here ADDRESS is the fully-qualified list name (e.g. [email protected]) and the mbox_file argument represents existing archives (in mbox format) to import (usually found in /var/lib/mailman/archives/private/LIST_NAME.mbox/LIST_NAME.mbox).

Afterwards the full-text search index can be updated manually:

[hyperkitty]$ django-admin update_index_one_list --pythonpath /usr/share/webapps/hyperkitty --settings settings ADDRESS

Note: The full-text search index should be created by the minutely running cron-job automatically.

Content-Security-Policy Header

It is possible to define a Content-Security-Policy HTTP response header for HyperKitty. This allows content and resources only from specified locations.

When using nginx add the following to the server directive in which HyperKitty is setup in:

/etc/nginx/hyperkitty.conf

[..]
add_header Content-Security-Policy "default-src 'self'; connect-src 'self'; img-src 'self' https://secure.gravatar.com; script-src 'self' 'unsafe-eval' 'unsafe-inline'; style-src 'self' 'unsafe-inline'";
[..]

Disabling Gravatar support

The builtin Gravatar support can be disabled in the configuration:

/etc/webapps/hyperkitty/settings_local.py

HYPERKITTY_ENABLE_GRAVATAR = False

This option was introduced in Hyperkitty 1.3.4. In earlier versions, use this instead:

/etc/webapps/hyperkitty/settings_local.py

GRAVATAR_SECURE_URL = ''

Saving mail attachments to disk

By default Hyperkitty stores mail attachments in its database. However, it can be configured to save the attachments to disk instead:

/etc/webapps/hyperkitty/settings_local.py

HYPERKITTY_ATTACHMENT_FOLDER = /var/lib/hyperkitty/data/attachments

Note: The location needs to be accessible and writable by the hyperkitty user.

Template customization

Using Django's TEMPLATES-DIRS capabilities, it is possible to override the following templates to change the looks of the application:

hyperkitty/headers.html: the content will appear before the </head> tag
hyperkitty/top.html: the content will appear before the <body> tag
hyperkitty/bottom.html: the content will appear before the </body> tag

Xapian search backend

Hyperkitty can make use of a Xapian based search backend. Install the python-xapian-haystack package and configure the backend:

/etc/webapps/hyperkitty/settings_local.py

HAYSTACK_CONNECTIONS = {
    'default': {
        'ENGINE': 'xapian_backend.XapianEngine',
        'PATH': "/var/lib/hyperkitty/data/xapian_index",
    },
}

Make sure to create the search index for all lists afterwards. Run the following command as the hyperkitty user (e.g. using sudo or su):

[hyperkitty]$ django-admin update_index --pythonpath /usr/share/webapps/hyperkitty --settings settings

Troubleshooting

SMTP AUTH extension not supported by server

If upon first login with the admin user a SMTNotSupportedError is thrown (logged to /var/log/hyperkitty/hyperkitty.log), make sure to verify the EMAIL_* settings in /etc/webapps/hyperkitty/settings_local.py.

By default e.g. EMAIL_USE_TLS is set to False which might trigger a failed login via SMTP.