Tag Archives: server

Bei einem unserer Hetzner Server ist eine Platte defekt. Der Hetzner Support möchte nun vor dem Tausch einen S.M.A.R.T. Status Output der Festplatte haben.
Normalerweise geht das einfach mit smartctl -l error -d ata /dev/sda.

Da das Device aber ein Hardware RAID ist, verstecken sich hinter /dev/sda eben zwei oder mehr physikalische Disks.

Über einen Umweg kann man aber trotzdem mit smartctl die S.M.A.R.T. Daten der Disk auslesen:

smartctl -a /dev/twa0 -d 3ware,1

/dev/twa0 steht hier für den Controller selbst
-d 3ware,1 gibt die Infos der Disk ID 1 aus (siehe auch tw_cli /c0 show)

Ein Output sieht im Fehlerfall dann beispielsweise so aus:

smartctl 5.41 2011-06-09 r3365 [x86_64-linux-2.6.32-5-xen-amd64] (local build)
Copyright (C) 2002-11 by Bruce Allen, http://smartmontools.sourceforge.net

=== START OF INFORMATION SECTION ===
Model Family: Seagate Barracuda 7200.11
Device Model: ST31500341AS
Serial Number: 9VS256GQ
LU WWN Device Id: 5 000c50 0150484fb
Firmware Version: CC1H
User Capacity: 1,500,301,910,016 bytes [1.50 TB]
Sector Size: 512 bytes logical/physical
Device is: In smartctl database [for details use: -P show]
ATA Version is: 8
ATA Standard is: ATA-8-ACS revision 4
Local Time is: Tue Jan 8 19:34:32 2013 CET
SMART support is: Available – device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: FAILED!
Drive failure expected in less than 24 hours. SAVE ALL DATA.
See vendor-specific Attribute list for failed Attributes.

General SMART Values:
Offline data collection status: (0×82) Offline data collection activity
was completed without error.
Auto Offline Data Collection: Enabled.
Self-test execution status: ( 0) The previous self-test routine completed
without error or no self-test has ever
been run.
Total time to complete Offline
data collection: ( 617) seconds.
Offline data collection
capabilities: (0x7b) SMART execute Offline immediate.
Auto Offline data collection on/off support.
Suspend Offline collection upon new
command.
Offline surface scan supported.
Self-test supported.
Conveyance Self-test supported.
Selective Self-test supported.
SMART capabilities: (0×0003) Saves SMART data before entering
power-saving mode.
Supports SMART auto save timer.
Error logging capability: (0×01) Error logging supported.
General Purpose Logging supported.
Short self-test routine
recommended polling time: ( 1) minutes.
Extended self-test routine
recommended polling time: ( 255) minutes.
Conveyance self-test routine
recommended polling time: ( 2) minutes.
SCT capabilities: (0x103f) SCT Status supported.
SCT Error Recovery Control supported.
SCT Feature Control supported.
SCT Data Table supported.

SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE
1 Raw_Read_Error_Rate 0x000f 119 099 006 Pre-fail Always – 209614914
3 Spin_Up_Time 0×0003 100 100 000 Pre-fail Always – 0
4 Start_Stop_Count 0×0032 100 100 020 Old_age Always – 9
5 Reallocated_Sector_Ct 0×0033 001 001 036 Pre-fail Always FAILING_NOW 4095
7 Seek_Error_Rate 0x000f 091 060 030 Pre-fail Always – 1398653228
9 Power_On_Hours 0×0032 068 068 000 Old_age Always – 28546
10 Spin_Retry_Count 0×0013 100 100 097 Pre-fail Always – 0
12 Power_Cycle_Count 0×0032 100 100 020 Old_age Always – 9
184 End-to-End_Error 0×0032 100 100 099 Old_age Always – 0
187 Reported_Uncorrect 0×0032 042 042 000 Old_age Always – 58
188 Command_Timeout 0×0032 100 099 000 Old_age Always – 12885098499
189 High_Fly_Writes 0x003a 001 001 000 Old_age Always – 274
190 Airflow_Temperature_Cel 0×0022 048 033 045 Old_age Always In_the_past 52 (0 188 54 47)
194 Temperature_Celsius 0×0022 052 067 000 Old_age Always – 52 (0 20 0 0)
195 Hardware_ECC_Recovered 0x001a 042 025 000 Old_age Always – 209614914
197 Current_Pending_Sector 0×0012 100 100 000 Old_age Always – 0
198 Offline_Uncorrectable 0×0010 100 100 000 Old_age Offline – 0
199 UDMA_CRC_Error_Count 0x003e 200 200 000 Old_age Always – 0
240 Head_Flying_Hours 0×0000 100 253 000 Old_age Offline – 12652973682562
241 Total_LBAs_Written 0×0000 100 253 000 Old_age Offline – 1787571001
242 Total_LBAs_Read 0×0000 100 253 000 Old_age Offline – 2802815673

SMART Error Log Version: 1
ATA Error Count: 58 (device log contains only the most recent five errors)
CR = Command Register [HEX]
FR = Features Register [HEX]
SC = Sector Count Register [HEX]
SN = Sector Number Register [HEX]
CL = Cylinder Low Register [HEX]
CH = Cylinder High Register [HEX]
DH = Device/Head Register [HEX]
DC = Device Command Register [HEX]
ER = Error register [HEX]
ST = Status register [HEX]
Powered_Up_Time is measured from power on, and printed as
DDd+hh:mm:SS.sss where DD=days, hh=hours, mm=minutes,
SS=sec, and sss=millisec. It “wraps” after 49.710 days.

Error 58 occurred at disk power-on lifetime: 28298 hours (1179 days + 2 hours)
When the command that caused the error occurred, the device was active or idle.

After command completion occurred, registers were:
ER ST SC SN CL CH DH
— – — – — – –
40 51 00 ff ff ff 0f Error: UNC at LBA = 0x0fffffff = 268435455

Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
— – — – — – — – —————- ——————–
60 00 80 ff ff ff 4f 00 48d+22:29:37.025 READ FPDMA QUEUED
61 00 80 80 6e 0b 45 00 48d+22:29:36.732 WRITE FPDMA QUEUED
61 00 80 00 6e 0b 45 00 48d+22:29:36.731 WRITE FPDMA QUEUED
61 00 80 80 6d 0b 45 00 48d+22:29:36.731 WRITE FPDMA QUEUED
61 00 80 00 6d 0b 45 00 48d+22:29:36.730 WRITE FPDMA QUEUED

Error 57 occurred at disk power-on lifetime: 28298 hours (1179 days + 2 hours)
When the command that caused the error occurred, the device was active or idle.

After command completion occurred, registers were:
ER ST SC SN CL CH DH
— – — – — – –
40 51 00 ff ff ff 0f Error: WP at LBA = 0x0fffffff = 268435455

Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
— – — – — – — – —————- ——————–
61 00 80 80 7c fa 44 00 48d+22:29:14.883 WRITE FPDMA QUEUED
61 00 80 00 7c fa 44 00 48d+22:29:14.882 WRITE FPDMA QUEUED
61 00 80 80 7b fa 44 00 48d+22:29:14.882 WRITE FPDMA QUEUED
61 00 80 00 7b fa 44 00 48d+22:29:14.880 WRITE FPDMA QUEUED
61 00 80 80 7a fa 44 00 48d+22:29:14.879 WRITE FPDMA QUEUED

Error 56 occurred at disk power-on lifetime: 28298 hours (1179 days + 2 hours)
When the command that caused the error occurred, the device was active or idle.

After command completion occurred, registers were:
ER ST SC SN CL CH DH
— – — – — – –
40 51 00 ff ff ff 0f Error: UNC at LBA = 0x0fffffff = 268435455

Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
— – — – — – — – —————- ——————–
60 00 80 ff ff ff 4f 00 48d+22:28:52.552 READ FPDMA QUEUED
60 00 80 ff ff ff 4f 00 48d+22:28:52.552 READ FPDMA QUEUED
60 00 80 ff ff ff 4f 00 48d+22:28:52.541 READ FPDMA QUEUED
60 00 80 ff ff ff 4f 00 48d+22:28:52.539 READ FPDMA QUEUED
60 00 80 ff ff ff 4f 00 48d+22:28:52.538 READ FPDMA QUEUED

Error 55 occurred at disk power-on lifetime: 28298 hours (1179 days + 2 hours)
When the command that caused the error occurred, the device was active or idle.

After command completion occurred, registers were:
ER ST SC SN CL CH DH
— – — – — – –
40 51 00 ff ff ff 0f Error: UNC at LBA = 0x0fffffff = 268435455

Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
— – — – — – — – —————- ——————–
60 00 80 ff ff ff 4f 00 48d+22:28:20.253 READ FPDMA QUEUED
61 00 18 ff ff ff 4f 00 48d+22:28:20.252 WRITE FPDMA QUEUED
61 00 08 61 18 8c 44 00 48d+22:28:20.252 WRITE FPDMA QUEUED
61 00 10 11 36 48 44 00 48d+22:28:20.252 WRITE FPDMA QUEUED
61 00 01 80 35 48 44 00 48d+22:28:20.252 WRITE FPDMA QUEUED

Error 54 occurred at disk power-on lifetime: 28298 hours (1179 days + 2 hours)
When the command that caused the error occurred, the device was active or idle.

After command completion occurred, registers were:
ER ST SC SN CL CH DH
— – — – — – –
40 51 00 ff ff ff 0f Error: UNC at LBA = 0x0fffffff = 268435455

Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
— – — – — – — – —————- ——————–
60 00 01 ff ff ff 4f 00 48d+22:27:27.107 READ FPDMA QUEUED
60 00 01 ff ff ff 4f 00 48d+22:27:27.094 READ FPDMA QUEUED
e7 00 00 00 00 00 00 00 48d+22:27:26.248 FLUSH CACHE
61 00 08 f1 9b 7d 40 00 48d+22:27:26.237 WRITE FPDMA QUEUED
61 00 01 00 9a 7d 40 00 48d+22:27:26.236 WRITE FPDMA QUEUED

SMART Self-test log structure revision number 1
No self-tests have been logged. [To run self-tests, use: smartctl -t]

SMART Selective self-test log data structure revision number 1
SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS
1 0 0 Not_testing
2 0 0 Not_testing
3 0 0 Not_testing
4 0 0 Not_testing
5 0 0 Not_testing
Selective self-test flags (0×0):
After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

Nach dem Plattentausch kann man das Raid dann mittels tw_cli maint rebuild c0 u0 p1 neu aufbauen.
c0 steht für Controller 0, u0 für Unit 0 und p1 für das physical drive 1.

Upgrading Graylog2 from 0.9.5p2 to 0.9.6 with data migration

4 Replies

The Graylog2 “party-gorilla” team around Lennart Koopmann just released the final version of graylog2 0.9.6. It includes major changes and new features, such as:

ElasticSearch as new message store
new analytics shell, a terminal like browser gui for running queries
faster long term graphs
internal message queing systems to absorb and resist load spikes
new stream rules and filters
a lot of small improvements and bugfixes

The following migration guide describes the manual upgrade process on debian in detail:

Download new 0.9.6 release, for example into /opt and unpack it
https://github.com/downloads/Graylog2/graylog2-server/graylog2-server-0.9.6.tar.gz tar xzfv graylog2-server-0.9.6.tar.gz

Download elasticsearch, most recent version from here and unpack it, for example:
wget https://github.com/downloads/elasticsearch/elasticsearch/elasticsearch-0.18.6.tar.gz tar xzfv elasticsearch-0.18.6.tar.gz

configure basic elasticsearch values, see elasticsearch configuration for details, for example:
network.host: 192.168.164.100 path: logs: /var/log/elasticsearch data: /var/data/elasticsearch cluster: name: graylog2

Download elasticsearch-servicewrapper (tanuki-wrapper) into your elasticserach/bin installation directory and unpack it there
wget https://github.com/elasticsearch/elasticsearch-servicewrapper/zipball/master mv master elasticsearch-servicewrapper.zip && unzip elasticsearch-servicewrapper.zip mv elasticsearch-elasticsearch-servicewrapper-*/* . && rm -rf elasticsearch-elasticsearch-servicewrapper-*

Start elasticsearch instance bia servicewrapper in /bin/service
./elasticsearch start Starting ElasticSearch... Waiting for ElasticSearch....... running: PID:14816

Check if your elasticsearch instance started successfully. The logfile (defaut here in /var/log/elasticsearch/graylog2.log) should show something like this:
[2011-12-23 22:21:03,711][INFO ][node ] [Celestial Madonna] {0.18.6}[14818]: initializing ... [2011-12-23 22:21:03,716][INFO ][plugins ] [Celestial Madonna] loaded [], sites [] [2011-12-23 22:21:05,299][INFO ][node ] [Celestial Madonna] {0.18.6}[14818]: initialized [2011-12-23 22:21:05,299][INFO ][node ] [Celestial Madonna] {0.18.6}[14818]: starting ... [2011-12-23 22:21:05,352][INFO ][transport ] [Celestial Madonna] bound_address {inet[/192.168.164.100:9300]}, publish_address {inet[/192.168.164.100:9300]} [2011-12-23 22:21:08,385][INFO ][cluster.service ] [Celestial Madonna] new_master [Celestial Madonna][WiNh0iYwQyeipER3PuXZSg][inet[/192.168.164.100:9300]], reason: zen-disco-join (elected_as_master) [2011-12-23 22:21:08,408][INFO ][discovery ] [Celestial Madonna] graylog2/WiNh0iYwQyeipER3PuXZSg [2011-12-23 22:21:08,415][INFO ][http ] [Celestial Madonna] bound_address {inet[/192.168.164.100:9200]}, publish_address {inet[/192.168.164.100:9200]} [2011-12-23 22:21:08,416][INFO ][node ] [Celestial Madonna] {0.18.6}[14818]: started [2011-12-23 22:21:08,419][INFO ][gateway ] [Celestial Madonna] recovered [0] indices into cluster_state

Get the MongoDB to ElasticSearch migrator script from joschi
git clone https://github.com/joschi/graylog2-mongo-es-migrator.git
you need at least to adapt to config/migrator.yml with your mongodb username, password and host, usage:
After you've downloaded the script you should edit the configuration file in config/migrator.yml. If the ElasticSearch and MongoDB servers are running on the same system (on localhost) and MongoDB doesn't need authentication you can keep the file as is.

Get required dependencies with Bundler:
bundle install

prepare a new graylog2.conf in /tmp/graylog2.conf with new settings (or just copy the new config and adapt your settings). The new default configuration settings are:
elasticsearch_url = http://localhost:9200/ elasticsearch_index_name = graylog2 force_syslog_rdns = false mq_batch_size = 4000 mq_poll_freq = 1 mq_max_size = 0

be brave and do a “hole-in-one” migration from the migrator directory (the following line works depending from where your graylog2 servers resides):
ps aux | grep graylog2-server.jar | awk {'print $2'} | xargs kill; ruby migrator.rb;cp /tmp/graylog2.conf /etc/graylog2.conf/; cd ../graylog2-server-0.9.6/bin/;./graylog2ctl start

Get the new web interface from github
git clone https://github.com/Graylog2/graylog2-web-interface.git

get required dependencies for the graylog2 web-interface
cd graylog2-web-interface; bundle install

Configure new web-interface (copy old configs, adapt new parameters, create new indexer.yml file with correct elasticsearch settings), be sure to check
indexer.yml mongoid.yml general.yml email.yml

Start the bundled server or adapt your apache/passenger settings

Write a “thank you party-gorilla” tweet to the @graylog2 twitter account

OpenX Performance optimization with Amazon S3 and CloudFront as CDN

3 Replies

On one of my current projects called diginights.com we are heavily using the popular and free Ad server solution OpenX for banner delivery. To optimize its performance and spread the http connections over multiple domains to boost the page speed we decided to outsource the ad images to a Content Delivery Network (CDN). We chose the Amazon Web Services for a first test due to the easy installation and configuration of their cloud services.
The two services to enable in your Amazon Web Management Console are Amazon S3 (storage solution) and CloudFront (CDN distribution). We sync the images between our root servers and Amazon S3 with the help of the fuse integrated and free filesystem s3fs (FuseOverAmazon).

First of all you need to get an Amazon S3 account on http://aws.amazon.com/s3/ and set up a new S3 Bucket (beware that every bucket name needs to be unique) in a region that fits your needs, create a CloudFront Distribution for the Bucket and wait until its enabled. The images below should be enough to help you with setting up AWS:

: Creating an Amazon S3 Bucket

: Create Amazon Cloudfront Distribution

: Deployment of Amazon Cloudfront

Please note the *.cloudfront.net. You’ll need it later for OpenX configuration. You could also set up a CNAME in your DNS system, configure this CNAME in the CloudFront Console and deliver the ad images via this CNAME from OpenX.

Installation of s3fs on our Debian Linux servers is fairly easy:

Grab the recent version of s3fs from its download page and extract it somewhere (eg. /usr/src)

Install installation prequisites via

apt-get install build-essential libfuse-dev fuse-utils libcurl4-openssl-dev libxml2-dev mime-support
Compile and install s3fs
- cd s3fs-1.16/
- ./configure –prefix=/usr
- make
- make install (as root)
Put your security credentials either in /root/.passwd-s3fs or the system-wide file /etc/passwd-s3fs. Please take care of the file permissions. The syntax in the file is simple accessKeyId:secretAccessKey ( or bucketName:accessKeyId:secretAccessKey for multiple buckets)

Enable fuse module, if it isn’t already in use:

modprobe fuse
You should then be able to mount your bucket to your local filesystem:

s3fs bucketName mountpoint
Put a similar entry in the fstab of your servers:

s3fs#bucketName /space/amazons3 fuse default_acl=public-read,allow_other 0 0

public-read is necessary to make sure that copied files are available online for every user.
That’s it for the s3fs part. Now you have to reconfigure your OpenX Installation

Change the Local Directory under Global Settings -> Banner Storage Settings to the directory where your S3 mountpoint is located.

Change the Image Store URL for HTTP & HTTPS under Global Settings -> Banner Delivery Settings to your *.cloudfront.net Domain (Or your own CNAME URL).

Save the settings, sit back and relax! But don’t forget to check your AWS account balance

Probleme bei Dovecot Migration mit Sieve Plugin

6 Replies

Nach Installation der neusten Dovecot Version (1.2.4-2 via debian-testing) kommt es bei Verwendung des alten CMUSieve Plugins zu folgendem Problem im Mail.log:

Fatal: Plugin cmusieve not found from directory /usr/lib/dovecot/modules/lda

Nachrichten werden dann nicht mehr zugestellt.

Warum tuts auf einmal nicht mehr? CMUSieve ist eigentlich deprecated und man möchte das für Dovecot 1.2 geschriebene Dovecot Sieve plugin verwende.
Dazu muss man folgenden Eintrag ändern.

protocol lda {
mail_plugin_dir = /usr/lib/dovecot/modules/lda
mail_plugins = sieve
sieve_extensions = +imapflags
log_path = /var/log/dovecot/deliver.log
info_log_path = /var/log/dovecot/deliver-info.log

In der “mail_plugins” Zeile ersetzt man also einfach “cmusieve” durch “sieve”.
Folgende Änderungen muss man noch beachten:

# The imapflags extension is now called imap4flags. The CMUSieve implementation is based on an old draft specification that is not completely compatible. Particularly, the mark and unmark commands were removed from the new specification. For backwards compatibility, support for the old imapflags extension can be enabled using the sieve_extensions setting (as explained above). This is disabled by default.

# The notify extension is now called enotify. Sieve scripts need to be adjusted to incorporate this change: unlike imapflags, no backwards compatibility is provided currently.

Hier muss man also noch ein wenig aufpassen und ggf. seine Sieve Tools, wie beispielsweise den pysieved auf die aktuellste Version aktualisieren.

Seriennummer von AIX und HP Maschinen/Linux remote auslesen

1 Reply

Immer wieder hilfreich ist, wenn man Seriennummern von AIX Maschinen und HP Servern, die unter Linux lauffen, remote auslesen kann.
Dieser Post dient als kleine Erinnerung:

Auslesen unter AIX:

lsconf | grep -i serial

Auslesen unter SLES10 / HP Server:

dmidecode | grep -i serial

Google gewährt Einblick in Container Rechenzentrum

1 Reply

Auf dem Google Data Center Efficiency Summit in Mountain View hat Google erstmals offiziell bestätigt, dass ihr RZ Design auf RZ Containern basiert. Ein Google Mitarbeiter (Jimmy Clidaras) erklärte mit Hilfe einer Präsentation die Besonderheiten und Möglichkeiten dieser seit 2005 im Einsatz befindlichen Architektur. Jeder der bis zu 12 Meter langen Container enthält dabei bis zu 1160 Server und verbraucht bis zu 250KW Strom. Der gezeigte Container Hangar enthält 45 Container, zum Teil sogar auf einem 2. Stock!
Über die Präsentation gibt es ein über 5 minütiges Video auf youtube, weitere Videos der Konferenz sollen noch folgen:

Auch schon offiziell ist die Bauweise der Google Webserver, welche mich doch stark an die RootServer bei 1&1 erinnern:

Bei Datacenterknowledge.com gibts einige interessante Berichte über IT Architektur bei google:
Google’s Custom Webserver revealed
Google unveils container Data Center
Efficient UPS Aids Google’s Extreme PUE
Inside A Google Data Center

(via datacenterknowledge.com)

andreas-lehr.com

IT stuff, linux, biking, movies and more

Tag Archives: server

More performance for Roundcube/SquirrelMail webmailer with imapproxy

Related Posts:

Harddisk SMART Status auslesen bei 3ware RAID

Related Posts:

Upgrading Graylog2 from 0.9.5p2 to 0.9.6 with data migration

Related Posts:

Probleme bei Dovecot Migration mit Sieve Plugin

Related Posts:

Seriennummer von AIX und HP Maschinen/Linux remote auslesen

Related Posts:

Google gewährt Einblick in Container Rechenzentrum

Related Posts: