Tag Archives: rootserver

More performance for Roundcube/SquirrelMail webmailer with imapproxy

There is an easy way out there to improve performance on webmail clients like SquirrelMail and Rouncube.
Roundcube and similar clients opens a new IMAP connection on each request, e.g. opening the next e-Mail.
A caching proxy like imapproxy is reported to increase performance from 0.5 seconds up to 3 seconds per Action.

Installation on debian is very simple.
Just install the imapproxy package via aptitude install imapproxy.
In our setup the webmail client and imap server is running on different servers.

in /etc/imapproxy.conf we just change the configuration of our IMAP Server
server_hostname mail.myhostname.com

the following important options could stay default but are worth mentioning:

listen_port 1143 - Port imapproxy is listening on
server_port 143 - Port of the imap server host)

After starting imapproxy we only have to change roundcube configuration in config/main.inc.php:

$rcmail_config['default_host'] = 'localhost';
$rcmail_config['default_port'] = 1143;

and that’s it.
Happy proxying :)

Sources:
http://trac.roundcube.net/wiki/Howto_Performance
https://maze.io/2011/03/25/speed-up-roundcube-webmail/

Related Posts:

graylog2 logo

Upgrading Graylog2 from 0.9.5p2 to 0.9.6 with data migration

graylog2 logoThe Graylog2 “party-gorilla” team around Lennart Koopmann just released the final version of graylog2 0.9.6. It includes major changes and new features, such as:

  • ElasticSearch as new message store
  • new analytics shell, a terminal like browser gui for running queries
  • faster long term graphs
  • internal message queing systems to absorb and resist load spikes
  • new stream rules and filters
  • a lot of small improvements and bugfixes

The following migration guide describes the manual upgrade process on debian in detail:

Download new 0.9.6 release, for example into /opt and unpack it
https://github.com/downloads/Graylog2/graylog2-server/graylog2-server-0.9.6.tar.gz
tar xzfv graylog2-server-0.9.6.tar.gz

Download elasticsearch, most recent version from here and unpack it, for example:
wget https://github.com/downloads/elasticsearch/elasticsearch/elasticsearch-0.18.6.tar.gz
tar xzfv elasticsearch-0.18.6.tar.gz

configure basic elasticsearch values, see elasticsearch configuration for details, for example:
network.host: 192.168.164.100
path:
logs: /var/log/elasticsearch
data: /var/data/elasticsearch
cluster:
name: graylog2

Download elasticsearch-servicewrapper (tanuki-wrapper) into your elasticserach/bin installation directory and unpack it there
wget https://github.com/elasticsearch/elasticsearch-servicewrapper/zipball/master
mv master elasticsearch-servicewrapper.zip && unzip elasticsearch-servicewrapper.zip
mv elasticsearch-elasticsearch-servicewrapper-*/* . && rm -rf elasticsearch-elasticsearch-servicewrapper-*

Start elasticsearch instance bia servicewrapper in /bin/service
./elasticsearch start
Starting ElasticSearch...
Waiting for ElasticSearch.......
running: PID:14816

Check if your elasticsearch instance started successfully. The logfile (defaut here in /var/log/elasticsearch/graylog2.log) should show something like this:
[2011-12-23 22:21:03,711][INFO ][node ] [Celestial Madonna] {0.18.6}[14818]: initializing ...
[2011-12-23 22:21:03,716][INFO ][plugins ] [Celestial Madonna] loaded [], sites []
[2011-12-23 22:21:05,299][INFO ][node ] [Celestial Madonna] {0.18.6}[14818]: initialized
[2011-12-23 22:21:05,299][INFO ][node ] [Celestial Madonna] {0.18.6}[14818]: starting ...
[2011-12-23 22:21:05,352][INFO ][transport ] [Celestial Madonna] bound_address {inet[/192.168.164.100:9300]}, publish_address {inet[/192.168.164.100:9300]}
[2011-12-23 22:21:08,385][INFO ][cluster.service ] [Celestial Madonna] new_master [Celestial Madonna][WiNh0iYwQyeipER3PuXZSg][inet[/192.168.164.100:9300]], reason: zen-disco-join (elected_as_master)
[2011-12-23 22:21:08,408][INFO ][discovery ] [Celestial Madonna] graylog2/WiNh0iYwQyeipER3PuXZSg
[2011-12-23 22:21:08,415][INFO ][http ] [Celestial Madonna] bound_address {inet[/192.168.164.100:9200]}, publish_address {inet[/192.168.164.100:9200]}
[2011-12-23 22:21:08,416][INFO ][node ] [Celestial Madonna] {0.18.6}[14818]: started
[2011-12-23 22:21:08,419][INFO ][gateway ] [Celestial Madonna] recovered [0] indices into cluster_state

Get the MongoDB to ElasticSearch migrator script from joschi
git clone https://github.com/joschi/graylog2-mongo-es-migrator.git

you need at least to adapt to config/migrator.yml with your mongodb username, password and host, usage:
After you've downloaded the script you should edit the configuration file in config/migrator.yml. If the ElasticSearch and MongoDB servers are running on the same system (on localhost) and MongoDB doesn't need authentication you can keep the file as is.

Get required dependencies with Bundler:
bundle install

prepare a new graylog2.conf in /tmp/graylog2.conf with new settings (or just copy the new config and adapt your settings). The new default configuration settings are:
elasticsearch_url = http://localhost:9200/
elasticsearch_index_name = graylog2
force_syslog_rdns = false
mq_batch_size = 4000
mq_poll_freq = 1
mq_max_size = 0

be brave and do a “hole-in-one” migration from the migrator directory (the following line works depending from where your graylog2 servers resides):
ps aux | grep graylog2-server.jar | awk {'print $2'} | xargs kill; ruby migrator.rb;cp /tmp/graylog2.conf /etc/graylog2.conf/; cd ../graylog2-server-0.9.6/bin/;./graylog2ctl start

Get the new web interface from github
git clone https://github.com/Graylog2/graylog2-web-interface.git

get required dependencies for the graylog2 web-interface
cd graylog2-web-interface; bundle install

Configure new web-interface (copy old configs, adapt new parameters, create new indexer.yml file with correct elasticsearch settings), be sure to check
indexer.yml
mongoid.yml
general.yml
email.yml

Start the bundled server or adapt your apache/passenger settings

Login to your web-interface, check if latest events are dropping in.

Write a “thank you party-gorilla” tweet to the @graylog2 twitter account

Related Posts:

Installing Apache Solr on Debian Squeeze 6.0 and tomcat7

Apache solr logoDue to the lack of an official Apache Solr Package in Debian Squeeze 6.0 this post will briefly describe the installation of tomcat7 together with Apache (Lucene) Solr from Source.

Install tomcat7 package:

apt-get install tomcat7

Download Apache Solr from here, for example:

cd /opt/ && wget http://apache.lehtivihrea.org//lucene/solr/3.4.0/apache-solr-3.4.0.tgz

unpack the package

tar xzfv apache-solr-3.4.0.tgz

copy the solr file to tomcat webapps directory

cp apache-solr-3.4.0/dist/apache-solr-3.4.0.war /var/lib/tomcat7/webapps/solr.war

copy example configs and files to tomcat solr directory

cp -fr example/solr/ /var/lib/tomcat7/

change owner to tomcat

chown -R tomcat7:tomcat7 /var/lib/tomcat7/solr/

restart tomcat

/etc/init.d/tomcat restart

If everything went well you should be able to see the Solr Admin page on http://yourhost.com:8080/solr/admin.
Have Fun!

Related Posts:

Installing Icinga 1.0 with IDOUtils and NConf

Today the icinga team released the first stable version of their open source monitoring tool – Icinga 1.0.
The team behind icinga formed up earlier this year and forked nagios due to several reasons.
In this small howto i’d like to describe the installation of Icinga in cooperation with IDOUtils and NConf for easier configuration of multiple hosts and services on a fresh debian system:

  1. Install some prequisites:

    apt-get install apache2 build-essential libgd2-xpm-dev libjpeg62 libjpeg62-dev libpng12 libpng12-dev mysql-server mysql-client libdbi0 libdbi0-dev libdbd-mysql libdbi-perl libdbd-mysql-perl

  2. As root, add an user and group for icinga:

    /usr/sbin/useradd -m icinga && /usr/sbin/groupadd icinga

  3. Download icinga-core 1.0 (not icinga-web) from the icinga download site
  4. Unpack the downloaded archive, for example in a new directory called /usr/local/icinga with the command

    tar xzfv icinga-1.0.tar.gz

  5. Enter the new directory and configure icinga with
    ./configure –enable-idoutils

  6. General options should look like this:

    General Options:
    ————————-
    Icinga executable: icinga
    Icinga user/group: icinga,icinga
    Command user/group: icinga,icinga
    Embedded Perl: no
    Event Broker: yes
    Build IDOUtils: yes
    Install ${prefix}: /usr/local/icinga
    Lock file: ${prefix}/var/icinga.lock
    Check result directory: ${prefix}/var/spool/checkresults
    Init directory: /etc/init.d
    Apache conf.d directory: /etc/apache2/conf.d
    Mail program: /usr/bin/mail
    Host OS: linux-gnu

  7. Compile the icinga source code with

    make all

  8. If everything went fine, just install icinga completely with

    make fullinstall

  9. Time for some customization:
    comment out all cfg_file options from line 30 to line 36 of icinga.cfg, they´ll be provided by NConf later:

    # You can specify individual object config files as shown below:

    #cfg_file=/usr/local/icinga/etc/objects/commands.cfg
    #cfg_file=/usr/local/icinga/etc/objects/contacts.cfg
    #cfg_file=/usr/local/icinga/etc/objects/timeperiods.cfg
    #cfg_file=/usr/local/icinga/etc/objects/templates.cfg

    # Definitions for monitoring the local (Linux) host
    #cfg_file=/usr/local/icinga/etc/objects/localhost.cfg

    # Definitions for monitoring a Windows machine
    #cfg_file=/usr/local/icinga/etc/objects/windows.cfg

    # Definitions for monitoring a router/switch
    #cfg_file=/usr/local/icinga/etc/objects/switch.cfg

    # Definitions for monitoring a network printer
    #cfg_file=/usr/local/icinga/etc/objects/printer.cfg

  10. Add two new cfg_dir options for NConf:

    cfg_dir=/usr/local/icinga/etc/objects/global
    cfg_dir=/usr/local/icinga/etc/objects/Default_collector/

  11. uncomment broker_module in line 251:

    broker_module=/usr/local/icinga/bin/idomod.o config_file=/usr/local/icinga/etc/idomod.cfg

  12. In /usr/local/icinga/etc directrory, create IDOUtils sampe configuration:

    cp idomod.cfg-sample idomod.cfg
    cp ido2db.cfg-sample ido2db.cfg

  13. Generate MySQL Database and User for icinga

    # mysql -u root -p
    mysql> CREATE DATABASE icinga;
    GRANT USAGE ON \*.\* TO ‘icinga’@'localhost’ IDENTIFIED BY ‘MYSUPERHEROPASSWORD’ WITH MAX_QUERIES_PER_HOUR 0 MAX_CONNECTIONS_PER_HOUR 0 MAX_UPDATES_PER_HOUR 0;

    GRANT ALL PRIVILEGES ON icinga.\* TO ‘icinga’@'localhost’;FLUSH PRIVILEGES ;
    quit

  14. Import corresponding database scheme for MySQL:

    # cd /path/to/icinga-src/module/idoutils/db
    # mysql -u root -p icinga < mysql.sql

  15. Customize database credentials in /usr/local/icinga/etc/ido2db.cfg
    and take care of the following options:

    db_servertype=mysql
    db_port=3306
    db_user=icinga
    db_pass=MYSUPERHEROPASSWORD

  16. Change .htaccess password for the icingaadmin: htpasswd -c /usr/local/icinga/etc/htpasswd.users icingaadmin
  17. Copy sample configuration for apache:

    cp /usr/local/icinga/icinga-1.0/sample-config/httpd.conf /etc/apache2/conf.d/icinga.conf

  18. Download latest NConf release from the NConf download page
  19. Unpack the archive in one of your web directories, e.g. in /var/www/nconf

    tar xzfv nconf-1.2.6-0.tgz

  20. Make the new directory read/writeable to your webserver:

    chown -R www-data:www-data nconf/

  21. Create new file nconf.conf in /etc/apache2/conf.d:

    #nconf.conf
    Alias /nconf "/var/www/virtual/nconf"

    # SSLRequireSSL
    Options None
    AllowOverride None
    Order allow,deny
    Allow from all
    # Order deny,allow
    # Deny from all
    # Allow from 127.0.0.1
    AuthName "Icinga Access"
    AuthType Basic
    AuthUserFile /usr/local/icinga/etc/htpasswd.users
    Require valid-user

  22. Reload apache
  23. /etc/init.d/apache2 reload

  24. Create database and user for NConf:

    # mysql -u root -p
    mysql> CREATE DATABASE nconf;
    GRANT USAGE ON \*.\* TO ‘nconf’@'localhost’ IDENTIFIED BY ‘MYSUPERHEROPASSWORD’ WITH MAX_QUERIES_PER_HOUR 0 MAX_CONNECTIONS_PER_HOUR 0 MAX_UPDATES_PER_HOUR 0;
    GRANT ALL PRIVILEGES ON nconf.\* TO ‘nconf’@'localhost’;
    FLUSH PRIVILEGES ;
    quit

  25. Complete installation via NConf Web Gui, for example on
  26. Put /usr/local/icinga/bin/icinga as NAGIOS_BIN
  27. Configure your hosts and services in the NConf WebGui
  28. make sure to delete the host and service notification state "[n]" editing your contacts, otherwise you won’t get any notification.
  29. Edit the notify-host-by-email/notify-sercice-by-email command in misccommands section in order to point to the corresponding binary for the mail command, in my case /usr/bin/mail
  30. After finishing host and services configuration, hit the Generate Nagios config Button.
  31. Your nagios configuration is now saved in the /output directory of your NConf installation.
  32. customize NConf deploy script in /var/www/virtualnconf/ADD-ONS/deploy_local.sh, for example:

    #!/bin/bash

    OUTPUT_DIR="/var/www/virtual/nconf/output/"
    NAGIOS_DIR="/usr/local/icinga/etc/objects/"
    TEMP_DIR=${NAGIOS_DIR}"import/"
    CONF_ARCHIVE="NagiosConfig.tgz"

    if [ ! -e ${TEMP_DIR} ] ; then
    mkdir -p ${TEMP_DIR}
    fi

    if [ ${OUTPUT_DIR}${CONF_ARCHIVE} -nt ${TEMP_DIR}${CONF_ARCHIVE} ] ; then
    cp -p ${OUTPUT_DIR}${CONF_ARCHIVE} ${TEMP_DIR}${CONF_ARCHIVE}
    tar -xf ${TEMP_DIR}${CONF_ARCHIVE} -C ${NAGIOS_DIR}
    /etc/init.d/icinga reload
    fi

    exit

  33. Configure cronjob to refresh your configuration regulary, if you want to

    */10 * * * * /var/www/virtual/nconf/ADD-ONS/deploy_local.sh

  34. Start IDOUtils:

    /etc/init.d/ido2db start

  35. Start Icinga:

    /etc/init.d/icinga start

  36. Check icinga.log logfile in /usr/local/icinga/var
  37. Be Happy!
    1. For more information on Icinga and IDOUtils check out the documentation.
      More information on NConf is available in the NConf documentation.

Related Posts:

Freier Monitoring Service von pingdom

Seit gestern bietet der schwedische IT Dienstleister Pingdom seinen Monitoringdienst in eingeschränktem Umfang kostenlos an.

Dabei kann man beispielsweise seine Website mit einem einfachen (oder custom) HTTP Check überwachen, Netzwerk Checks über UDP, TCP oder einfachen Ping einrichten oder auch Dienste wie POP3, IMAP und SMTP prüfen lassen. In der kostenlosen Version kann man jedoch nur einen Host/IP überwachen. Allerdings kann man Meldungen dann an mehrere Kontakte verschicken, die wahlweise per EMail und/oder SMS über den Status benachrichtigt werden (20 SMS pro Monat sind inklusive).
Der Clou an der Sache ist, dass Pingdom Monitoring Hubs an verschiedenen Standorten auf der ganzen Welt betreibt, und so werden die Dienste beispielsweise aus Montreal (Kanada), London (UK), Dallas (Texas), Stockholm und Amsterdam überwacht!
Die Statistiken und Reports sind schön aufbereitet und es gibt sogar eine IPhone App!

Links:
Pingdom Homepage

Related Posts:

Google gewährt Einblick in Container Rechenzentrum

Auf dem Google Data Center Efficiency Summit in Mountain View hat Google erstmals offiziell bestätigt, dass ihr RZ Design auf RZ Containern basiert. Ein Google Mitarbeiter (Jimmy Clidaras) erklärte mit Hilfe einer Präsentation die Besonderheiten und Möglichkeiten dieser seit 2005 im Einsatz befindlichen Architektur. Jeder der bis zu 12 Meter langen Container enthält dabei bis zu 1160 Server und verbraucht bis zu 250KW Strom. Der gezeigte Container Hangar enthält 45 Container, zum Teil sogar auf einem 2. Stock!
Über die Präsentation gibt es ein über 5 minütiges Video auf youtube, weitere Videos der Konferenz sollen noch folgen:

Auch schon offiziell ist die Bauweise der Google Webserver, welche mich doch stark an die RootServer bei 1&1 erinnern:

Bei Datacenterknowledge.com gibts einige interessante Berichte über IT Architektur bei google:
Google’s Custom Webserver revealed
Google unveils container Data Center
Efficient UPS Aids Google’s Extreme PUE
Inside A Google Data Center

(via datacenterknowledge.com)

Related Posts:

RAID Monitoring mit Nagios

Vor kurzem schrieb ich über einen Festplatten Ausfall in einem Software Raid.
Dieser Ausfall wurde einem Nagios Überwachungssystem gemeldet, mit welchem wir unsere Server überwachen. Als RAID Systeme kommen “normale” Linux Software Raids mit md devices und ein Server mit einem 3ware Hardware Raid (3ware Inc 8000-series) zum Einsatz.

Für das 3Ware Raid verwenden wir das check_3ware_raid von nagiosexchange.org. Dafür muss wie im Artikel beschrieben noch sudoers um “nagios ALL=(ALL) NOPASSWD: /path/to/check_3ware_raid” erweitert werden. Der Check selbst wird dann beispielsweise im nrpe Client über ein Wrapper Skript ausgeführt, welches ein “sudo /path/to/check_3ware_raid” ausführt.
Zur Ausführung benötigt man noch das 3ware Command Line Interface (CLI). Für Debian bekommt man das einfach über debian-unoffical mirror, der noch zu den apt-Sourcen hinzugefügt werden muss:

# For getting 3dm2-package for 3ware
deb http://ftp.debian-unofficial.org/debian sid main contrib non-free restricted

Nach einem

apt-get update && apt-get install 3ware-3dm2-binary 3ware-cli-binary

befinden sich die benötigten Binaries auf eurem Server und die Funktion lässt sich mit dem Aufruf von

tw_cli /c0 show

Unit UnitType Status %Cmpl Stripe Size(GB) Cache AVerify IgnECC
——————————————————————————
u0 RAID-1 OK – – 698.637 ON – -

Port Status Unit Size Blocks Serial
—————————————————————
p0 OK u0 698.63 GB 1465149168 5QD2DXQ8
p1 OK u0 698.63 GB 1465149168 5QD0XX6A

testen.

Software Raids werden mit dem einfachen check_md_raid überwacht. Dazu wird der Check einfach als command oder im nrpe Client hinzugefügt und ausgeführt. Allerdings schreit er schon mit Critical, wenn nur ein Device Critical ist, und dies kann man auch nicht konfigurieren oder auf bestimmte md devices eingrenzen.

Related Posts: