How to build your own cloud for you and your friends on Debian Wheezy

Get back your privacy and control over your data in just a few hours: build your own cloud for you and your friends

Some of the most personal pieces of your identity are stored on servers around the world beyond your control

Let's say you've been a Gmail user between 2006 и 2013 like me, meaning you received 30'000+ emails and wrote about 5000 emails over that 7 year period. Some of the emails you sent or received are very personal, maybe so personal that you probably wouldn't like even some family members or close friends to go through them systematically. Maybe you also drafted a few emails that you never sent because you changed your mind at the last minute. But even if you never sent them, these emails are still stored somewhere on a server. Като резултат, it's fair to say that Google servers know more about your personal life than your closest friends or your family.

Statistically, it's a safe bet to consider that you've got a смартфон. You can barely use the phone without using the contacts app which stores your contacts in Google Contact on Google servers by default. So not only does Google know about your emails, but also about your offline contacts: who you like to call, who calls you, whom you text, and what you text them about. You don't have to take my word for it, you can verify for yourself by taking a look at the permissions you gave apps such as the Google Play Service to read the list of people that called you and the SMS you got. Do you also use the calendar app that comes with your phone? Unless you explicitly opted out while setting up your calendar, this means that Google knows precisely what you're up to, at every time of the day, day after day, година след година. The same applies if you chose an iPhone over an Android phone, except Apple gets to know about your correspondance, contacts and schedule instead of Google.

Do you also take great care to keep the contacts in your directory up-to-date, updating your friend's, colleagues's and and family's email addresses and phone numbers when they move to a new job or change carrier? That gives Google an extraordinarily accurate, up-to-date picture of your social network. And you love the GPS of your смартфон which you use a lot together with Google Maps. This means Google not only knows what you do from your calendar but also where you are, where you live, where you work. And by correlating GPS location data across users, Google can also tell with whom you may socializing with right now.

Your daily habit of handing out your most personal information will impact your life in a way that no one can even forsee

To summarize, if you are an average internet user, Google has up-to-date, in-depth information about your interests, worries, passions, questions, over almost 10 години. It has a collection of some of your most personal messages (emails, СМС), an hour-by-hour detail of your daily activities and location, and a high-quality picture of your social network. Such an intimate knowledge of you likely goes beyond what your closest friends, семейство, or your sweetheart know of you.

It wouldn't come to mind to give this mass of deeply personal information to complete strangers, for instance by putting it all on a USB key and leaving it on a table in a random cafe with a note saying 'Personal data of Olivier Martin, use as you please'. Who knows who might find it and what they would do with it? Yet, we have no problem handing in core pieces of your identity to strangers at IT companies with a strong interest in our data (that's how they make their bread) and world-class experts in data analysis, perhaps just because it happens by default without us thinking about it when we hit that green 'Accept' button.

With so much high-quality information, over the years, Google may well get to know you better than you can ever hope to know yourself: heck, crawling through my digital past right now, I can't remember having written half of the emails I sent five years ago. I am surprised and pleased to rediscover my interest in marxism back in 2005 and my joining ATTAC (an organization which strives to limit speculation and improve social justice by taxing financial transactions) the next year. And god knows why I was so much into dancing shoes back in 2007. These is pretty harmless information (you wouldn't have expected me to reveal something embarassing here, would you? ;-). But by connecting the dots between high-quality data over different aspects of your life (what, when, with whom, where, ...) over such time spans, one may extrapolate predictive statements about you. Например, from the shopping habits of a 17-year-old girl, supermarkets can tell that she is pregnant before her dad even hears about it (true story). Who knows what will become possible with high-quality data the like Google has, which goes well beyond shopping habits? By connecting the dots, maybe one can predict how your tastes or political views will change in the coming years. днес, companies you have never heard of claim to have 500 data points about you, including religion, sexual orientation and political views. Speaking of politics, what if you decide to go into politics in 10 years from now? Your life may change, your views too, and you may even forget, but Google won't. Will you have to worry that your opponent is in touch with someone who has access to your data at Google and can dig up something embarassing on you from those bottomless wells of personal data you gave away over the years? How long until Google or Facebook get hacked just like Sony was recently hacked and all your personal data end up in the public sphere forever?

One of the reason most of us have entrusted our personal data to these companies is that they provide their services for free. But how free is it really? The value of the average Google account varies depending on the method used to estimate it: 1000 USD/year accounts for the amount of time you invest in writing emails, while the value of your account for the advertisement industry is somewhere between 220 USD/year и 500 USD/year. So the service is not exactly free: you pay for it through advertisement and the yet unknown uses that our data may find in the future.

I've been writing about Google mostly because that's the company I've entrusted most of my digital identify to so far and hence the one I know best. But I may well have written Apple or Facebook. These companies truly changed the world with their fantastic advances in design, engineering and services we love(d) to use, every day. But it doesn't mean we should stack up all our most personal data in their servers and entrust them with our digital lives: the potential for harm is just too large.

Claim back your privacy and that of people you care for in just 5h

It does not have to be this way. You can live in the 21st century, have a smartphone, use email and GPS on daily basis, and still retain your privacy. All you need to do is get back control over your personal data: emails, calendar, Контакти, файлове, и т.н.. Най- уебсайт lists software that help controlling the fate of your personal data. Beyond these options, the safest and most powerful way to get back control over your personal data is to host your cloud yourself, by building your own server. But you may just not have the time and energy to research how exactly to do that and make it work smoothly.

That's where the present article fits in. In just 5 часа, we will set up a server to host your emails, Контакти, calendars and files for you, your friends and your family. The server is designed to act as a hub or cloud for your personal data, so that you always retain full control over it. The data will automatically be synchronized between your PC/laptop, your phone and your tablet. Essentially, we will set up a system that replaces Gmail, Google Диск / Dropbox, Google Contacts, Google Calendar and Picasa.

Just doing this for yourself will already be a big step. But then, a significant fraction of your personal information will still leak out and end up on some servers in the silicon valley, just because so many of the people you interact with every day use Gmail and have smartphones. So it's a good idea to have some of the people you are closest to join the adventure.

We will build a system that

  1. supports an arbitrary number of domains and users. This makes it easy to share your server with family and friends, so that they get control over their personal data too and can share the cost of the server with you. The people sharing your server can use their own име на домейн or share yours.
  2. lets you send and receive your emails from any network upon successfully logging in onto the server. Насам, you can send your emails from any of your email addresses, from any device (PC, телефон, tablet), and any network (at home, at work, from a public network, ...)
  3. encrypts network traffic when sending and receiving emails so people you don't trust won't fish out your password and won't be able to read your private emails.
  4. offers state-of-the-art antispam, combining black lists of known spammers, automatic greylisting, and adaptative spam filtering. Re-training the adaptative spam filter if an email is misclassified is simply done by moving spam in or out of the Junk/Spam folder. Също, the server will contribute to community-based spam fighting efforts.
  5. requires just a few minutes of maintenance once in a while, basically to install security updates and briefly check the server logs. Adding a new email address boils down to adding one record to a база данни. Освен това, you can just forget about it and live your life. I set up the system described in this article 14 months ago and the thing has just been running smoothly since then. So I completely forgot about it, until I recently smiled at the thought that casually pressing the 'Check email' button of my phone caused electrons to travel all the way to Iceland (where my server sits) and back.

To go through this article, you'll need a minimum of technical capabilities. If you know what is the difference between SMTP and IMAP, what is a DNS, and have a basic understanding of TCP/IP, you know enough to follow through. You will also need a basic working knowledge of Unix (working with files from the command line, basic system administration). And you'll need a total of 5 hours of time to set it up.

This article was inspired by and builds upon previous work

This article draws heavily from two other articles, namely Xavier Claude's and Drew Crawford's introduction to email self-hosting.

The article includes all the features of Xavier's and Draw's articles, except from three features that Drew had and which I didn't need, namely push support for email (I like to check email only when I decide to, otherwise I get distracted all the time), fulltext search in email (which I don't have a use for), and storing emails in an encrypted form (my emails and data are not critical to the point that I have to encrypt them locally on the server). If you need any of these features, feel free to just add them by following to the respective section of Drew's article, which is compatible with the present one.

Compared to Xavier's and Drew's work, the present article improves on several aspects:

  • it fixes bugs and typos based on my experience with Drew's article and the numerous comments on his original article. I also went through the present article, setting up the server from scratch several times to replicate it and make sure it would work right out of the box.
  • low maintenance: compared to Xavier's work, the present article adds support for multiple email domains on the server. It does so by requiring the minimum amount of server maintenance possible: basically, to add a domain or a user, just add one row to a mysql table and that's it (no need to add sieve scripts, ...).
  • I added webmail.
  • I added a section on setting up a cloud, to host not just your emails but also your files, your addressbook / contacts (emails, phone numbers, birthdays, ...), calendars and pictures for use across your devices.

Get a Virtual Private Server, a domain name, and set them up

Let's start by setting the basic infrastructure: our virtual private server and our име на домейн.

I've had an excellent experience with the Virtual Private Servers (VPS) на 1984.е и Linode. В тази статия, we will use Debian Wheezy, for which both 1984 and Linode provide ready-made images to deploy on your VPS. I like 1984 because the servers are hosted in Iceland which run exclusively on renewable energy (geothermical and hydropower) and hence does not contribute to the climate change, unlike the coal power plants on which most US-based datacenters currently run on. Също, they put emphasis on civil liberties, transparency, freedom и Free Software.

It could be a good idea to start a file to store the various passwords we will need to set on the server (user accounts, mail accounts, cloud accounts, база данни accounts). It's definitely a good idea to encrypt this file (maybe with GnuPG), so that it won't be too easy to attack your server even if the computer you use to set up your server gets stolen or compromised.

For registering a domain name, I've been using the services of gandi for over 10 years now, also with satisfaction. For this article, we will set up a zone with the name We then add a host named to it, set the MX record to that host. While you're at it, set short Time To Lives (TTL) to your records like 300 seconds so that you'll be able to make changes to your zone and test the result rapidly while you're setting up the server.

накрая, set the PTR record (reverse DNS) so that the IP адрес of the host maps back to its name. If you don't understand the previous sentence, read this article to get the background. If you use Linode, you can set the PTR record in the control panel in the Remote Access section. с 1984, contact the tech support who will help you with it.

On the server, we will start by adding a non-privledged user, so that we don't end up working as root all the time. Също, to log in as root will require an extra layer of security.

 adduser roudy

Тогава, в /etc/ssh/sshd_config, we set

 PermitRootLogin no

and reload the ssh server

 service ssh reload

Тогава, we'll need to change the hostname of the server. Edit /etc/hostname so that it has just a single line with your hostname, in our case


Тогава, edit the ssh server's public key files /etc/ssh/, /etc/ssh/, /etc/ssh/ so that the end of the file reflects your hostname, or instance [email protected]. Then restart the system to make sure the hostname is fixed wherever is should be


We will update the system and remove services we don't need to reduce the risk of remote attacks.

 apt-get update apt-get dist-upgrade service exim4 stop apt-get remove exim4 rpcbind apt-get autoremove apt-get install vim

I like to use vim for editing config files remotely. For this, it helps to automatically turn on syntax highlighting. We do so by adding

 syn on

да се ~/.vimrc.

Set up postfix and dovecot to send and receive email

 apt-get install postfix postfix-mysql dovecot-core dovecot-imapd dovecot-mysql mysql-server dovecot-lmtpd postgrey

In the Postfix configuration menu, we select Internet Site, and set the system mail name to

We will now set up a database to store the list of domains hosted on our server, the list of users for each of these domains (together with their password), and a list of mail aliases (to forward email from a given address to another one).

 mysqladmin -p create mailserver mysql -p mailserver mysql> GRANT SELECT ON mailserver.* TO 'mailuser'@'localhost' IDENTIFIED BY 'mailuserpass'; mysql> FLUSH PRIVILEGES; mysql> CREATE TABLE `virtual_domains` (   `id` int(11) NOT NULL auto_increment,   `name` varchar(50) NOT NULL,   PRIMARY KEY (`id`)   ) ENGINE=InnoDB DEFAULT CHARSET=utf8; mysql> CREATE TABLE `virtual_users` (   `id` int(11) NOT NULL auto_increment,   `domain_id` int(11) NOT NULL,   `password` varchar(106) NOT NULL,   `email` varchar(100) NOT NULL,   PRIMARY KEY (`id`),   UNIQUE KEY `email` (`email`),   FOREIGN KEY (domain_id) REFERENCES virtual_domains(id) ON DELETE CASCADE   ) ENGINE=InnoDB DEFAULT CHARSET=utf8; mysql> CREATE TABLE `virtual_aliases` (   `id` int(11) NOT NULL auto_increment,   `domain_id` int(11) NOT NULL,   `source` varchar(100) NOT NULL,   `destination` varchar(100) NOT NULL,   PRIMARY KEY (`id`),   FOREIGN KEY (domain_id) REFERENCES virtual_domains(id) ON DELETE CASCADE   ) ENGINE=InnoDB DEFAULT CHARSET=utf8;

We will host the домейн. If there are other domains you'd like to host, you can also add them. We also set up a postmaster address for each domain, which forwards to [email protected].

 mysql> INSERT INTO virtual_domains (`name`) VALUES (''); mysql> INSERT INTO virtual_domains (`name`) VALUES (''); mysql> INSERT INTO virtual_aliases (`domain_id`, `source`, `destination`) VALUES ('1', 'postmaster', '[email protected]'); mysql> INSERT INTO virtual_aliases (`domain_id`, `source`, `destination`) VALUES ('2', 'postmaster', '[email protected]');

We now add a locally hosted email account [email protected]. първи, we generate a password hash for it:

 doveadm pw -s SHA512-CRYPT

and then add the hash to the database

 mysql> INSERT INTO `mailserver`.`virtual_users` (`domain_id`, `password`, `email`) VALUES ('1', '$  6$  YOURPASSWORDHASH', '[email protected]');

Now that our list of domains, aliases and users are in place, we will set up postfix (SMTP server, for outgoing mail). Replace the contents of /etc/postfix/ with the following:

 myhostname = myorigin = /etc/mailname mydestination = localhost.localdomain, localhost mynetworks_style = host  # We disable relaying in the general case smtpd_recipient_restrictions = permit_mynetworks, reject_unauth_destination # Requirements on servers that contact us: we verify the client is not a # known spammer (reject_rbl_client) and use a graylist mechanism # (postgrey) to help reducing spam (check_policy_service) smtpd_client_restrictions = permit_mynetworks, reject_rbl_client, check_policy_service inet: disable_vrfy_command = yes inet_interfaces = all smtpd_banner = $  myhostname ESMTP $  mail_name (Debian/GNU) biff = no append_dot_mydomain = no readme_directory = no  # TLS parameters smtpd_tls_cert_file=/etc/ssl/certs/cloud.crt smtpd_tls_key_file=/etc/ssl/private/cloud.key smtpd_use_tls=yes smtpd_tls_auth_only = yes smtp_tls_security_level=may smtp_tls_loglevel = 1 smtpd_tls_loglevel = 1 smtpd_tls_received_header = yes smtpd_tls_session_cache_database = btree:$  {data_directory}/smtpd_scache smtp_tls_session_cache_database = btree:$  {data_directory}/smtp_scache  # Delivery alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases message_size_limit = 50000000 recipient_delimiter = +  # The next lines are useful to set up a backup MX for # relay_domains = # relay_recipient_maps =  # Virtual domains virtual_transport = lmtp:unix:private/dovecot-lmtp virtual_mailbox_domains = mysql:/etc/postfix/ virtual_mailbox_maps = mysql:/etc/postfix/ virtual_alias_maps = mysql:/etc/postfix/ local_recipient_maps = $  virtual_mailbox_maps

Now we need to teach postfix to figure out which domains we would like him to accept emails for using the database we just set up. Create a new file /etc/postfix/ and add the following:

 user = mailuser password = mailuserpass hosts = dbname = mailserver query = SELECT 1 FROM virtual_domains WHERE name='%s'

We teach postfix to find out whether a given email account exists by creating /etc/postfix/ with the following content

 user = mailuser password = mailuserpass hosts = dbname = mailserver query = SELECT 1 FROM virtual_users WHERE email='%s'

накрая, postfix will use /etc/postfix/ to look up mail aliases

 user = mailuser password = mailuserpass hosts = dbname = mailserver query = SELECT virtual_aliases.destination as destination FROM virtual_aliases, virtual_domains WHERE virtual_aliases.source='%u' AND virtual_aliases.domain_id = AND'%d'

With all this in place, it is now time to test if postfix can query our database properly. We can do this using postmap:

 postmap -q mysql:/etc/postfix/ postmap -q [email protected] mysql:/etc/postfix/ postmap -q [email protected] mysql:/etc/postfix/ postmap -q [email protected] mysql:/etc/postfix/

If you set up everything properly, the first two queries should return 1, the third query should return [email protected] and the last one should return nothing at all.

Сега, let's set up dovecot (the IMAP server, to fetch incoming mail on the server from our devices). Edit /etc/dovecot/dovecot.conf to set the following parameters:

 # Enable installed protocol # !include_try /usr/share/dovecot/protocols.d/*.protocol  protocols = imap lmtp

which will only enable imap (to let us fetch emails) and lmtp (which postfix will use to pass incoming emails to dovecot). Edit /etc/dovecot/conf.d/10-mail.conf to set the following parameters:

 mail_location = maildir:/var/mail/%d/%n [...] mail_privileged_group = mail [...] first_valid_uid = 0

which will store emails in /var/mail/domainname/username. Note that these settings are spread at different locations in the file, and are sometimes already there for us to set: we just need to comment them out. The other settings which are already in the file, you can leave as is. We will have to do the same to update settings in many more files in the remaining of this article. в /etc/dovecot/conf.d/10-auth.conf, set the parameters:

 disable_plaintext_auth = yes auth_mechanisms = plain #!include auth-system.conf.ext !include auth-sql.conf.ext

в /etc/dovecot/conf.d/auth-sql.conf.ext, set the following parameters:

 passdb {   driver = sql   args = /etc/dovecot/dovecot-sql.conf.ext } userdb {   driver = static   args = uid=mail gid=mail home=/var/mail/%d/%n }

where we just taught dovecot that users have their emails in /var/mail/domainname/username and to look up passwords from the database we just created. Now we still need to teach dovecot how exactly to use the database. За да направите това, put the following into /etc/dovecot/dovecot-sql.conf.ext:

 driver = mysql connect = host=localhost dbname=mailserver user=mailuser password=mailuserpass default_pass_scheme = SHA512-CRYPT password_query = SELECT email as user, password FROM virtual_users WHERE email='%u';

We now fix permissions on config files

 chown -R mail:dovecot /etc/dovecot chmod -R o-rwx /etc/dovecot

Almost there! We just need to edit a couple files more. в /etc/dovecot/conf.d/10-master.conf, set the following parameters:

 service imap-login {   inet_listener imap {     #port = 143     port = 0   }   inet_listener imaps {     port = 993     ssl = yes   } }  service pop3-login {  inet_listener pop3 {     #port = 110     port = 0   }   inet_listener pop3s {     #port = 995     #ssl = yes     port = 0   } }  service lmtp {   unix_listener /var/spool/postfix/private/dovecot-lmtp {     mode = 0666     group = postfix     user = postfix   }   user = mail }  service auth {   unix_listener auth-userdb {     mode = 0600     user = mail     #group =    }    # Postfix smtp-auth   unix_listener /var/spool/postfix/private/auth {     mode = 0666     user = postfix     group = postfix   }    # Auth process is run as this user.   #user = $  default_internal_user   user = dovecot }  service auth-worker {   user = mail }

Note that we set ports for all services but imaps to 0, which effectively disables them. Тогава, в /etc/dovecot/conf.d/15-lda.conf, specify an email address for the postmaster:

 postmaster_address = [email protected]

Last but not least, we need to generate a pair of public and private key for the server, which we will use both in dovecot and postfix:

 openssl req -new -newkey rsa:4096 -x509 -days 365 -nodes -out "/etc/ssl/certs/cloud.crt" -keyout "/etc/ssl/private/cloud.key"

Make sure that you specify your the Fully Qualified Domain Name (FQDN) of the server, in our case:

 Common Name (e.g. server FQDN or YOUR name) []

Ако не го направите, our clients may complain that the server name in the SSL certificate does not match the name of the server they are connecting to. We tell dovecot to use these keys by setting the following parameters in /etc/dovecot/conf.d/10-ssl.conf:

 ssl = required ssl_cert = </etc/ssl/certs/cloud.crt ssl_key = </etc/ssl/private/cloud.key

That's it! Now on to testing the postfix and dovecot servers!

 service dovecot restart service postfix restart

From the server itself, try to send an email to a local user:

 telnet localhost 25 EHLO MAIL FROM:[email protected] rcpt to:[email protected] data Subject: Hallo!  This is a test, to check if is ready to be an MX!  Cheers, Roudy . QUIT

The server should accept our email with a message like

 250 2.0.0 Ok: queued as 58D54101DB

Check the logs in /var/log/mail.log if everything went fine. There should be line saying something like

 ноември 14 07:57:06 cloud dovecot: lmtp(4375, [email protected]): ... saved mail to INBOX

So far so good? Good. Сега, let's try the same from a different machine, like the computer we are using to set up the server. We'll talk to the server using encryption (TLS) this time:

 openssl s_client -connect -starttls smtp EHLO MAIL FROM:[email protected] rcpt to:[email protected]

to which the server should respond

 554 5.7.1 <[email protected]>: Relay access denied

That's good: had the server accepted the mail, it would have meant that we set up postfix as an open relay for all the spammers of the world and beyhond to use. Instead of the 'Relay access denied' message, you may instead get the message

 554 5.7.1 Service unavailable; Client host [] blocked using;

This means that you are trying to contact the server from an IP адрес that is considered as a spammer's address. I got this message while trying to connect to the server through my regular Internet Service Provider (ISP). To fix this issue, you can try to connect from another host, maybe another server you have access to through SSH. алтернативно, you can reconfigure Postfix's not to use Spamhaus's RBL, reload postfix, and verify that the above test works. In both cases, it's important that you find a solution that works for you because we'll test other things in a minute. If you chose to reconfigure Postfix not to use RBLs, don't forget to put the RBLs back in and to reload postfix after finishing the article to avoid getting more spam than necessary.

Now let's try to send a valid email by SMTP on port 25, which regular mail servers use to talk to each other:

 openssl s_client -connect -starttls smtp EHLO MAIL FROM:[email protected] rcpt to:[email protected]

to which the server should respond

 Client host rejected: Greylisted, see

which shows that postgrey is working as it should. What postgrey does it to reject emails with a temporary error if the sender has never been seen before. The technical rules of email require email servers to try to deliver the email again. After five minutes, postgrey will accept the email. Legit email servers around the world will try repeatidly to redeliver the email to us, but most spammers won't. Така, wait for 5 минути, try to send the email again using the command above, and verify that postfix now accepts the email.

Afterwards, we'll check that we can fetch the two emails that we just sent ourselves by talking IMAP to dovecot:

 openssl s_client -crlf -connect 1 login [email protected] "mypassword" 2 LIST "" "*" 3 SELECT INBOX 4 UID fetch 1:1 (UID RFC822.SIZE FLAGS BODY.PEEK[]) 5 LOGOUT

where you should replace mypassword with the password you set for this email account. If that works, we basically have a functional email server which can receive our incoming emails, and from which we get retreive these emails from our devices (PC/laptop, таблетки, phones, ...). But we can't give it our emails to send unless we send them from the server itself. We'll now allow postfix to forward our emails, but only upon successful authentification, that is after it could make sure that the email comes from someone who has a valid account on the server. За да направите това, we'll open a special, SSL-only, SASL-authentified email submission service. Set the following parameters in /etc/postfix/

 submission inet n       -       -       -       -       smtpd   -o syslog_name=postfix/submission   -o smtpd_tls_security_level=encrypt   -o smtpd_sasl_auth_enable=yes   -o smtpd_client_restrictions=permit_sasl_authenticated,reject   -o smtpd_sasl_type=dovecot   -o smtpd_sasl_path=private/auth   -o smtpd_sasl_security_options=noanonymous   -o smtpd_recipient_restrictions=permit_sasl_authenticated,reject_non_fqdn_recipient,reject_unauth_destination

and reload postfix

 service postfix reload

Сега, let's try to use this service from a different machine than than the server, to verify postfix will now relay our emails and nobody else's:

 openssl s_client -connect -starttls smtp EHLO

Notice the '250-AUTH PLAIN' capabilities advertized by server, which doesn't appear when we connect to port 25.

 MAIL FROM:[email protected] rcpt to:[email protected] 554 5.7.1 <[email protected]>: Relay access denied QUIT

That's good, postfix won't relay our emails if he doesn't know us. So let's authentify ourselves first. За да направите това, we first need to generate an authentification string:

 echo -ne '[email protected]'|base64

and let's try to send emails through the server again:

 openssl s_client -connect -starttls smtp EHLO AUTH PLAIN DGplYW5AMTk4NGNsb3VQLm5ldAA4bmFmNGNvNG5jOA== MAIL FROM:[email protected] rcpt to:[email protected]

which postfix should now accept. To complete the test, let's verify that our virtual aliases work by sending an email to [email protected] and making sure it goes to [email protected]:

 telnet 25 EHLO MAIL FROM:[email protected] rcpt to:[email protected] data Subject: Virtual alias test  Dear postmaster, Long time no hear! I hope your MX is working smoothly and securely. Yours sincerely, Roudy . QUIT

Let's check the mail made it all the way to the right inbox:

 openssl s_client -crlf -connect 1 login [email protected] "mypassword" 2 LIST "" "*" 3 SELECT INBOX * 2 EXISTS * 2 RECENT 4 LOGOUT

В този момент, we have a functional email server, both for incoming and outgoing mails. We can set up our devices to use it.

PS: did you remember to try sending an email to an account hosted by the server through port 25 again, to verify that you are not longer blocked by postgrey?

Prevent SPAM from reaching your INBOX

For the sake of SPAM filtering, we already have Realtime BlackLists (RBLs) and greylisting (postgrey) in place. We'll now take our spam fighting capabilities up a notch by adding adaptative spam filtering. This means we'll add artificial intelligence to our email server, so that it can learn from experience what is spam and what is not. We will use dspam for that.

 apt-get install dspam dovecot-antispam postfix-pcre dovecot-sieve

dovecot-antispam is a package that allows dovecot to retrain the spam filter if we find an email that is misclassified by dspam. в основата си, all we need to do is to move emails in or out of the Junk/Spam folder. dovecot-antispam will then take care of calling dspam to retrain the filter. As for postfix-pcre and dovecot-sieve, we will use them respectively to pass incoming emails through the spam filter and to automatically move spam to the user's Junk/Spam folder.

в /etc/dspam/dspam.conf, set the following parameters to these values:

 TrustedDeliveryAgent "/usr/sbin/sendmail" UntrustedDeliveryAgent "/usr/lib/dovecot/deliver -d %u" Tokenizer osb IgnoreHeader X-Spam-Status IgnoreHeader X-Spam-Scanned IgnoreHeader X-Virus-Scanner-Result IgnoreHeader X-Virus-Scanned IgnoreHeader X-DKIM IgnoreHeader DKIM-Signature IgnoreHeader DomainKey-Signature IgnoreHeader X-Google-Dkim-Signature ParseToHeaders on ChangeModeOnParse off ChangeUserOnParse full ServerPID               /var/run/dspam/ ServerDomainSocketPath  "/var/run/dspam/dspam.sock" ClientHost      /var/run/dspam/dspam.sock

Тогава, в /etc/dspam/default.prefs, change the following parameters to:

 spamAction=deliver         # { quarantine | tag | deliver } -> default:quarantine signatureLocation=headers  # { message | headers } -> default:message showFactors=on

Now we need to connect dspam to postfix and dovecot by adding these two lines at the end of /etc/postfix/

 dspam     unix  -       n       n       -       10      pipe   flags=Ru user=dspam argv=/usr/bin/dspam --deliver=innocent,spam --user $  recipient -i -f $  sender -- $  recipient dovecot   unix  -       n       n       -       -       pipe   flags=DRhu user=mail:mail argv=/usr/lib/dovecot/deliver -f $  {sender} -d $  {recipient}

Now we will tell postfix to filter every new email that gets submitted to the server on port 25 (normal SMTP traffic) through dspam, except if the email is submitted from the server itself (permit_mynetworks). Note that the emails we submit to postfix with SASL authentication won't be filtered through dspam either, as we set up a separate submission service for those in the previous section. Edit /etc/postfix/ to change the smtpd_client_restrictions to the following:

 smtpd_client_restrictions = permit_mynetworks, reject_rbl_client, check_policy_service inet:, check_client_access pcre:/etc/postfix/dspam_filter_access

At the end of the file, also also add:

 # For DSPAM, only scan one mail at a time dspam_destination_recipient_limit = 1

We now need to specify the filter we defined. в основата си, we will tell postfix to send all emails (/./) to dspam through a unix socket. Create a new file /etc/postfix/dspam_filter_access and put the following line into it:

 /./   FILTER dspam:unix:/run/dspam/dspam.sock

That's it for the postfix part. Now let's set up dovecot for spam filtering. в /etc/dovecot/conf.d/20-imap.conf, edit the imap mail_plugins plugins parameter such that:

 mail_plugins = $  mail_plugins antispam

and add a section for lmtp:

 protocol lmtp { # Space separated list of plugins to load (default is global mail_plugins).   mail_plugins = $  mail_plugins sieve }

We now configure the dovecot-antispam plugin. Edit /etc/dovecot/conf.d/90-plugin.conf to add the following content to the plugin section:

 plugin {   ...   # Antispam (DSPAM)   antispam_backend = dspam   antispam_allow_append_to_spam = YES   antispam_spam = Junk;Spam   antispam_trash = Trash;trash   antispam_signature = X-DSPAM-Signature   antispam_signature_missing = error   antispam_dspam_binary = /usr/bin/dspam   antispam_dspam_args = --user;%u;--deliver=;--source=error   antispam_dspam_spam = --class=spam   antispam_dspam_notspam = --class=innocent   antispam_dspam_result_header = X-DSPAM-Result }

and in /etc/dovecot/conf.d/90-sieve.conf, specify a default sieve script which will apply to all users of the server:

 sieve_default = /etc/dovecot/default.sieve

What is sieve and why do we need a default script for all users? Sieve lets us automatize tasks on the IMAP server. В нашия случай, we won't all emails identified as spam to be put in the Junk folder instead of in the Inbox. We would like this to be the default behavior for all users on the server; that's why we just set this script as default script. Let's create this script now, by creating a new file /etc/dovecot/default.sieve with the following content:

 require ["regex", "fileinto", "imap4flags"]; # Catch mail tagged as Spam, except Spam retrained and delivered to the mailbox if allof (header :regex "X-DSPAM-Result" "^(Spam|Virus|Bl[ao]cklisted)$  ",           not header :contains "X-DSPAM-Reclassified" "Innocent") {   # Mark as read   # setflag "\Seen";   # Move into the Junk folder   fileinto "Junk";   # Stop processing here   stop; }

Now we need to compile this script so that dovecot can run it. We also need to give it appropriate permissions.

 cd /etc/dovecot sievec . chown mail.dovecot default.siev* chmod 0640 default.sieve chmod 0750 default.svbin

накрая, we need to fix permissions on two postfix config files that dspam needs to read from:

 chmod 0644 /etc/postfix/ /etc/postfix/

That's it! Let's restart dovecot and postfix

 service dovecot restart service postfix restart

and test the antispam, by contacting the server from a remote host (e.g. the computer we are using to set the server):

 openssl s_client -connect -starttls smtp EHLO MAIL FROM:[email protected] rcpt to:[email protected] DATA Subject: DSPAM test  Hi Roudy, how'd you like to eat some ham tonight? Yours, J . QUIT

Let's check if the mail arrived:

 openssl s_client -crlf -connect 1 login [email protected] "mypassword" 2 LIST "" "*" 3 SELECT INBOX 4 UID fetch 3:3 (UID RFC822.SIZE FLAGS BODY.PEEK[])

Which should return something the email with a collection of flag set by SPAM which look like this:

 X-DSPAM-Result: Innocent X-DSPAM-Processed: Sun Oct  5 16:25:48 2014 X-DSPAM-Confidence: 1.0000 X-DSPAM-Probability: 0.0023 X-DSPAM-Signature: 5431710c178911166011737 X-DSPAM-Factors: 27,         Received*Postfix+with, 0.40000,         Received*with+#+id, 0.40000,         like+#+#+#+ham, 0.40000,         some+#+tonight, 0.40000,         Received*certificate+requested, 0.40000,         Received*client+certificate, 0.40000,         Received*for+roudy, 0.40000,         Received*Sun+#+#+#+16, 0.40000,         Received*Sun+#+Oct, 0.40000,         Received*roudy+#+#+#+Oct, 0.40000,         eat+some, 0.40000,         Received*5+#+#+16, 0.40000,         Received*, 0.40000,         Roudy+#+#+#+to, 0.40000,         Received*Oct+#+16, 0.40000,         to+#+#+ham, 0.40000,         Received*No+#+#+requested, 0.40000,         Received*, 0.40000,         Received*256+256, 0.40000,         like+#+#+some, 0.40000,         Received*ESMTPS+id, 0.40000,         how'd+#+#+to, 0.40000,         tonight+Yours, 0.40000,         Received*with+cipher, 0.40000 5 LOGOUT

Good! You now have adaptive spam filtering set up for the users of your server. Разбира се, each user will need to train the filter in the first few weeks. To train a message as spam, just move it to a folder called "Spam" or "Junk" using any of your devices (PC, tablet, phone). Otherwise it'll be trained as ham.

Make sure the emails you send get through spam filters

Our goal in this section will be to make our mail server appear as clean as possible to the world and to make it harder for spammers to send emails in our name. As a side-effect, this will help us get our emails through the spam filters of other mail servers.

Sender Policy Framework

Sender Policy Framework (SPF) is a record that your add to your zone which declares which mail servers on the whole internet can send emails for your domain name. Setting it up is very easy, use the SPF wizard at to generate your SPF record, and then add it to your zone as a TXT record. It will look like this:       300 IN  TXT     v=spf1 mx -all

Reverse PTR

We discussed this point earlier in this article, it's a good idea that you set up the reverse DNS for your server correctly, so that doing a reverse lookup on the IP address of your server returns the actual name of your server.


When we activate OpenDKIM, postfix will sign every outgoing email using a cryptographic key. We will then deposit that key in our zone, on the DNS. По този начин, every mail server in the world will be able to verify if the email actually came from us, or if it was forged by a spammer. Let's install opendkim:

 apt-get install opendkim opendkim-tools

And set it up by editing /etc/opendkim.conf so that it looks like this:

 ## ## opendkim.conf -- configuration file for OpenDKIM filter ## Canonicalization        relaxed/relaxed ExternalIgnoreList      refile:/etc/opendkim/TrustedHosts InternalHosts           refile:/etc/opendkim/TrustedHosts KeyTable                refile:/etc/opendkim/KeyTable LogWhy                  Yes MinimumKeyBits          1024 Mode                    sv PidFile                 /var/run/opendkim/ SigningTable            refile:/etc/opendkim/SigningTable Socket                  inet:[email protected] Syslog                  Yes SyslogSuccess           Yes TemporaryDirectory      /var/tmp UMask                   022 UserID                  opendkim:opendkim

We'll need a couple of additional files which we will store in /etc/opendkim:

 mkdir -pv /etc/opendkim/ cd /etc/opendkim/

Let's create a new file /etc/opendkim/TrustedHosts with the following content

and a new file called /etc/opendkim/KeyTable with the following content


This tells OpenDKIM that we want to use an encryption key named 'cloudkey' whose contents can be found in /etc/opendkim/mail.private. We will create another file named /etc/opendkim/SigningTable and add the following line:

 * cloudkey

which tells OpenDKIM that every emails of the domain should be signed using the key 'cloudkey'. If we have other domains which we want to sign, we can add them here too.

The next step is to generate that key and fix permissions on OpenDKIM's config files.

 opendkim-genkey -r -s mail [-t] chown -Rv opendkim:opendkim /etc/opendkim chmod 0600 /etc/opendkim/* chmod 0700 /etc/opendkim

Първо, it's a good idea to use the -T which will signal to other mail servers that you are just in testing mode, and that they shouldn't discard emails based on your OpenDKIM signature (yet). You can get your OpenDKIM key from the mail.txt file:

 cat mail.txt

and then add it to your zone file as TXT record, which should look like this     300     IN TXT  v=DKIM1; k=rsa; p=MIGfMA0GCSqG...

накрая, we need to tell postfix to sign outgoing emails. At the end of /etc/postfix/, add:

 # Now for OpenDKIM: we'll sign all outgoing emails smtpd_milters = inet: non_smtpd_milters = $  smtpd_milters milter_default_action = accept

And reload the corresponding services

 service postfix reload service opendkim restart

Now let's test if our OpenDKIM public key can be found and matches the private key:

 opendkim-testkey -d -s mail -k mail.private -vvv

which should return

 opendkim-testkey: key OK

For this, you may need to wait a bit until the name server has reloaded the zone (on Linode, this happens every 15min). You can use dig to check if the zone was reloaded yet.

If this works, let's test if other servers can validate our OpenDKIM signatures and SPF record. Да го направя, we can use Brandon Checkett's email test. To send an email to a test address given to us on Brandon's webpage, we can run the following command on the server

 mail -s CloudCheck [email protected]

On Brandon's webpage, you should then see result = pass in the 'DKIM Signature' section, и Result: pass in the 'SPF Information' section. If our emails pass this test, just regenerate an OpenDKIM key without the -T ключ, upload the new key to the zone file, and retest to still if it still passes the tests. Ако е така, congrats! You just successfully set up OpenDKIM and SPF on your server!

Host calendars, Контакти, files with Owncloud and set up a webmail with Roundcube

Now that we have a top-notch email server, let's add to it the possibility to store your contacts, календари, and files in the cloud. These are services that the Owncloud provides out of the box. While we're at it, we'll also set up a webmail, so you can check email even if you're travelling without electronics, or in case your phone and laptop run out of battery.

Installing Owncloud is straighforward and is well described тук. On Debian, it boils down to adding the owncloud repository to your apt sources, downloading owncloud's release key and adding it to your apt keyring, and then installing owncloud itself using apt-get:

 echo 'deb /' >> /etc/apt/sources.list.d/owncloud.list wget apt-key add - < Release.key  apt-get update apt-get install apache2 owncloud roundcube

When prompted for it, избирам dbconfig and then say you want roundcube to use mysql. Тогава, provide the mysql корен парола and set a good password for the roundcube mysql user. Тогава, edit the roundcube config file /etc/roundcube/ so that logging in on roundcube will default to using your IMAP server:

 $  rcmail_config['default_host'] = 'ssl://localhost'; $  rcmail_config['default_port'] = 993;

Now we will set up the apache2 webserver with SSL so that we can talk to Owncloud and Roundcube using encryption for our passwords and data. Let's turn on Apache's ssl module:

 a2enmod ssl

and edit /etc/apache2/ports.conf to set the following parameters:

 NameVirtualHost *:80 Listen 80 ServerName  <IfModule mod_ssl.c>     # If you add NameVirtualHost *:443 here, you will also have to change     # the VirtualHost statement in /etc/apache2/sites-available/default-ssl     # да се <VirtualHost *: 443>     # Сървър Name Indication for SSL named virtual hosts is currently not     # supported by MSIE on Windows XP.     NameVirtualHost *:443     Listen 443 </IfModule>  <IfModule mod_gnutls.c>     Listen 443 </IfModule>

We'll set up a default уебсайт for encrypted connections to the webserver as при /var/www. Edit /etc/apache2/sites-available/default-ssl:

 <VirtualHost _default_:443>         ServerAdmin [email protected]          DocumentRoot /var/www         ServerName         [...]         <Directory /var/www/owncloud>           Deny from all         </Directory>         [...]         SSLCertificateFile    /etc/ssl/certs/cloud.crt         SSLCertificateKeyFile /etc/ssl/private/cloud.key         [...] </VirtualHost>

and let's also set up a website for unencrypted connections to при /var/www. Edit /etc/apache2/sites-available/default:

 <VirtualHost _default_:443>         DocumentRoot /var/www         ServerName         [...]         <Directory /var/www/owncloud>           Deny from all         </Directory> </VirtualHost>

По този начин, we can serve pages for by putting them in /var/www. The 'Deny from all' directive prevents access to Owncloud through we will set it up to access it through вместо.

We will now set up the webmail (roundcube) so that it will be accessed through Edit /etc/apache2/sites-available/roundcube to have the following content:

 <IfModule mod_ssl.c> <VirtualHost *: 443>         ServerAdmin [email protected]          DocumentRoot /var/lib/roundcube         # The host name under which you'd like to access the webmail         ServerName         <Directory />                 Options FollowSymLinks                 AllowOverride None         </Directory>          ErrorLog $  {APACHE_LOG_DIR}/error.log          # Possible values include: debug, info, notice, warn, грешка, crit,         # alert, emerg.         LogLevel warn          CustomLog $  {APACHE_LOG_DIR}/ssl_access.log combined          #   SSL Engine Switch:         #   Enable/Disable SSL for this virtual host.         SSLEngine on          # do not allow unsecured connections         # SSLRequireSSL         SSLCipherSuite HIGH:MEDIUM          #   A self-signed (snakeoil) certificate can be created by installing         #   the ssl-cert package. виждам         #   /usr/share/doc/apache2.2-common/README.Debian.gz for more info.         #   If both key and certificate are stored in the same file, only the         #   SSLCertificateFile directive is needed.         SSLCertificateFile    /etc/ssl/certs/cloud.crt         SSLCertificateKeyFile /etc/ssl/private/cloud.key          # Those aliases do not work properly with several hosts on your apache server         # Uncomment them to use it or adapt them to your configuration         Alias /program/js/tiny_mce/ /usr/share/tinymce/www/          # Access to tinymce files         <Directory "/usr/share/tinymce/www/">                 Options Indexes MultiViews FollowSymLinks                 AllowOverride None                 Order allow,deny                 allow from all         </Directory>          <Directory /var/lib/roundcube/>                 Options +FollowSymLinks                 # This is needed to parse /var/lib/roundcube/.htaccess. See its                 # content before setting AllowOverride to None.                 AllowOverride All                 order allow,deny                 allow from all         </Directory>          # Protecting basic directories:         <Directory /var/lib/roundcube/config>                 Options -FollowSymLinks                 AllowOverride None         </Directory>          <Directory /var/lib/roundcube/temp>                 Options -FollowSymLinks                 AllowOverride None                 Order allow,deny                 Deny from all         </Directory>          <Directory /var/lib/roundcube/logs>                 Options -FollowSymLinks                 AllowOverride None                 Order allow,deny                 Deny from all         </Directory>          <FilesMatch "\.(cgi|shtml|phtml|php)$  ">                 SSLOptions +StdEnvVars         </FilesMatch>         <Directory /usr/lib/cgi-bin>                 SSLOptions +StdEnvVars         </Directory>         #   SSL Protocol Adjustments:         #   The safe and default but still SSL/TLS standard compliant shutdown         #   approach is that mod_ssl sends the close notify alert but doesn't wait for         #   the close notify alert from client. When you need a different shutdown         #   approach you can use one of the following variables:         #   o ssl-unclean-shutdown:         #     This forces an unclean shutdown when the connection is closed, i.e. не         #     SSL close notify alert is send or allowed to received.  This violates         #     the SSL/TLS standard but is needed for some brain-dead browsers. употреба         #     this when you receive I/O errors because of the standard approach where         #     mod_ssl sends the close notify alert.         #   o ssl-accurate-shutdown:         #     This forces an accurate shutdown when the connection is closed, i.e. а         #     SSL close notify alert is send and mod_ssl waits for the close notify         #     alert of the client. This is 100% SSL/TLS standard compliant, но в         #     practice often causes hanging connections with brain-dead browsers. употреба         #     this only for browsers where you know that their SSL implementation         #     works correctly.         #   Notice: Most problems of broken clients are also related to the HTTP         #   keep-alive facility, so you usually additionally want to disable         #   keep-alive for those clients, също. Use variable "nokeepalive" for this.         #   по същия начин, one has to force some clients to use HTTP/1.0 to workaround         #   their broken HTTP/1.1 implementation. Use variables "downgrade-1.0" and         #   "force-response-1.0" for this.         BrowserMatch "MSIE [2-6]" \                 nokeepalive ssl-unclean-shutdown \                 downgrade-1.0 force-response-1.0         # MSIE 7 and newer should be able to use keepalive         BrowserMatch "MSIE [17-9]" ssl-unclean-shutdown </VirtualHost> </IfModule>

and declare the server in your DNS, for instance:       300     IN      CNAME

Now let's enable these three websites

 a2ensite default default-ssl roundcube service apache2 restart

and the webmail, accessible under, should basically work. Log in using the full email (e.g. [email protected]) and the password you set in mailserver DB at the beginning of this article. The first time you connect, the browser will warn you that the certificate was not signed by a certification authority. That's fine, just add an exception.

Last but not least, we will create a virtual host for owncloud by putting the following content in /etc/apache2/sites-available/owncloud:

 <IfModule mod_ssl.c> <VirtualHost *: 443>         ServerAdmin [email protected]          DocumentRoot /var/www/owncloud         ServerName         <Directory />                 Options FollowSymLinks                 AllowOverride None         </Directory>         <Directory /var/www/owncloud>                 Options Indexes FollowSymLinks MultiViews                 AllowOverride All                 Order allow,deny                 allow from all         </Directory>          ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/         <Directory "/usr/lib/cgi-bin">                 AllowOverride None                 Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch                 Order allow,deny                 Allow from all         </Directory>          ErrorLog $  {APACHE_LOG_DIR}/error.log          # Possible values include: debug, info, notice, warn, грешка, crit,         # alert, emerg.         LogLevel warn          CustomLog $  {APACHE_LOG_DIR}/ssl_access.log combined          #   SSL Engine Switch:         #   Enable/Disable SSL for this virtual host.         SSLEngine on          # do not allow unsecured connections         # SSLRequireSSL         SSLCipherSuite HIGH:MEDIUM         SSLCertificateFile    /etc/ssl/certs/cloud.crt         SSLCertificateKeyFile /etc/ssl/private/cloud.key          <FilesMatch "\.(cgi|shtml|phtml|php)$  ">                 SSLOptions +StdEnvVars         </FilesMatch>         <Directory /usr/lib/cgi-bin>                 SSLOptions +StdEnvVars         </Directory>          BrowserMatch "MSIE [2-6]" \                 nokeepalive ssl-unclean-shutdown \                 downgrade-1.0 force-response-1.0         # MSIE 7 and newer should be able to use keepalive         BrowserMatch "MSIE [17-9]" ssl-unclean-shutdown </VirtualHost> </IfModule>

and activate owncloud by running

 a2ensite owncloud service apache2 reload

Then go ahead an configure owncloud by connecting to in a web browswer.

That's it! Now you've got your own Google Drive, Calendar, Contacts, Dropbox, and Gmail! Enjoy your freshly recovered privacy! 🙂

Sync your devices to the cloud

To sync your emails, you can just use your favorite email client: the standard email program on Android or iOS, k9mail, or Thunderbird on your PC. Or you can also use the webmail we set up.

How to sync your calendar and contacts with the cloud is described in the doc of owncloud. On Android, I'm using the CalDAV-Sync and CardDAV-Sync apps which act as bridges between the Android calendar and contacts apps of the phone and the owncloud server.

For files, there is an Android app called Owncloud to access your files from your phone and automatically upload pictures and videos you take to your cloud. Accessing your files on the your Mac/PC is easy and well described in the Owncloud documentation.

Last tips

During the first few weeks, it's a good idea to monitor /var/log/syslog и /var/log/mail.log on a daily basis and make sure everything everything is running smoothly. It's important to do so before you invite others (friends, семейство, ...) to be hosted on your server; you might loose their trust in self-hosting for good if they trust you with their data and the server suddently becomes unavailable.

To add another email user, just add a row to the virtual_users table of the mailserver DB.

To add a domain, just add a row to the virtual_domains table. Then update /etc/opendkim/SigningTable to get outgoing emails signed, upload the OpenDKIM key to the zone, and reload OpenDKIM.

Owncloud has its own user DB which can be managed by logging in in Owncloud as administrator.

накрая, it's important to think in advance of a solution in case your server becomes temporarily unavailable. Например, where would your mails go until your server returns? One solution would be to find a friend who can act as your backup MX, while you act as his backup MX (see the relay_domains и relay_recipient_maps setting in Postfix's file). по същия начин, what if your server is compromised and a malicious individual erases all your files there? For that, it's important to think of a regular backup system. Linode offers backups as an option. On, I set up a basic but sufficient automatic backup system using on crontabs and scp.

Един Отговор

Оставете коментар

Този сайт използва Akismet да се намали спамът. Научете как се обработва коментар данни.

Ние използваме бисквитки, за да ви даде най-доброто онлайн достъп. Със съгласието си вие приемате използването на бисквитки, в съответствие с нашата политика за бисквитка.