For a mailserver you need an MTA
This could be Postfix, sendmail, exim or qmail (these are the one's who come to mind first.)
Now you can send and receive mail.
Your clients however use an MUA (Mail User Agent)
This could be outlook, thunderbird or a webbased client.
These MUA's need to connect to the server and log in using a username/password combination.
Most used protocol for this is POP or IMAP
So you need an imap/pop server.
This could be dovecot, courier imap or cyrus imap. (also these come to mind first)
These two are the basics for a mailserver.
Now for the authentication, username and password storage.
You could use ldap, pam or a database like postgreSQL or MySQL for this.
For a webmail client, you need a webserver.
Webservers you can use are apache, nginx or lighttp (also these comes to mind first.)
Webmail client are Roundcube webmail, Squirrelmail and Horde (again these come to mind first)
For spam filtering, i would use another machine which accept mail and wring it through the filters and deliver the clean mail to the actual mailserver.
So to go short.
I use for my mailservers a postfix with dovecot solution with a mysql backend to store the users.
Also i use postfixadmin to configure my domains and users.
Setting up an mailserver is not something you must take lightly.
My advise, test test and test.
Install postfix with dovecot and postfixadmin and try if it works.
Then do it again, reinstall everything.
Then add a webmail client to the mix.
There are full solutions like iRedmail.
this one uses also postfix dovecot and the like.
But be carefull not to become an open relay!
Also on the internet there are a lot of howto's how to install a postfix dovecot solution.
Try these as well, get to know the system.
Do not simply cut and paste, but try to understand what things do.
In the HOWTO section of this forum there is also a setup guide for a mailserver.
It has however not included a webbased gui for managing your users.
Good luck.
Also try exim instead of postfix.
and try a other webserver then apache.
Maybe you like exim more or not.
The only way to find out is to try them.
I would however stay away from qmail and sendmail.
Sendmail is a beast to configure.
Qmail is old, but some user do not want something else, it is all a mather of taste and liking.
Same goes for the pop/imap server.
I used to use courier, but switched to dovecot, because for me it was easier to understand and configure.
But again all a matter of taste.
regards
Johan