Introduction

Thruk is a multibackend monitoring webinterface which currently supports Nagios, Icinga and Shinken as backend using the Livestatus API. It is designed to be a dropin replacement and covers almost 100% of the original features plus adds additional enhancements for large installations and increased usability.

Main Features / Advantages

  • multiple backends

  • faster, no need to parse status.dat for every request

  • less cpu usage

  • display live data, no delay between core and gui

  • independant from monitoring core, can be installed on remote host

  • clusterable, can be clustered over hosts

  • change status.cgi filter inline

  • extended logfile search

  • better logging, uses log4perl (who submitted which command?)

  • valid HTML code

  • no frames ( can be reenabled for better addon integration)

  • easy to extend with plugins

  • 100% Perl (using Catalyst Framework)

  • pagination

  • multiple themes included

  • Excel export for status and logfiles

  • Adjustable side menu

  • Full expanded plugin commandline for easy testing

  • Save searches in personal bookmarks

  • Config Tool included

  • Mobile interface included

  • SLA Reports in PDF format

  • Recurring Downtimes

  • Fully Featured Dashboard

How it works

Thruk is written in Perl using the Catalyst Framework. Backend monitoring systems will be connected with the Monitoring::Livestatus Perl Module. Thruk itself is running as a fastcgi process. Availability will be calculated with Monitoring::Availability. Authentication is provided by the Apache webserver (For example with mod_auth_basic, mod_auth_ldap, mod_auth_mysql, …).

Thruk Architecture

Thruk Architecture

Reasons to choose Thruk

There are a couple of cool things in Thruk. A few of them are listed here.

Multi Site Support

Thruk connects to as many cores as you want. You could even combine Nagios, Icinga and Shinken instances into one big combined view. That way you get the advantage of several independent locations and still have an overview about all your installations. Sites can be put into groups for better arrangement.

Multi Site

SLA Reports

The reporting plugin creates comprehensive SLA reports in HTML and PDF format which may be send by mail at regular intervals. Besides that you may turn any page in Thruk into a regular E-Mail report.

SLA Reporting

Business Process Integration

Using the Business Process addon allows modeling your important applications and business processes. Online editor and notifications included.

Business Process

Excel Export and Bookmarks

The Bookmarks feature allows you to save your searches and add them as your personal menu item. Besides the bookmarks, you can adjust the menu easily with the menu_local.conf for all users.

The Excel export creates real Excel files which can be send by mail or used to extract hostnames and other information.

Bookmarks and Excel Export

Config Tool

The Config Tool makes is very easy to change object configuration of your monitoring solution. It also allows you to make quick changes to your Thruk configuration, as well as managing user access via htpasswd files or adjusting your cgi.cfg.

Config Tool

Sending Multiple Commands

The new status pages makes it very convenient to send multiple commands at once. It is even possible to send host and service commands at the same time. When rescheduling hosts and services, Thruk will wait until your check is finished and display the result as soon as the check has come back.

Multiple Commands

Easy Filtering

Remember the days when you had to guess numbers in the url to filter hosts or services. With Thruk it’s possible to quickly change your display filter. You can combine multiple filter to create whatever views you like. An Ajax search supports you, so you don’t have to guess host or service names, plus you get the power of regular expressions.

Easy Filtering

PNP4Nagios Graphs

When your action_url contains /pnp4nagios/, there will be automatically a graph displayed for your host and service. This gives you a quick view about the performance history. The image is then linked to PNP4Nagios to get detailed information.

PNP4Nagios

Multiple Lines of Plugin Output

When your check returns multiple lines of plugin output. Thruk marks the output in blue and a click on it displays the complete output. This is especially useful for check_multi checks. In addition to that, the comments and downtimes also have a small popup with their data. So you don’t have to open the host/service page just to see who set a comment and when there is a downtime.

Multiple Lines of Plugin Output

Mine Map

The Mine Map is the perfect tool to get a quick overview. It is especially useful if you have a lot of common services across your hosts. Otherwise use hostgroups or servicegroups for nice results. Normal filtering is possible too.

Mine Map

Mobile Interface

The Mobile interface gives you access to the most important things and allows you to quickly view and acknowledge problems.

Mobile Interface

Installation

There are several ways of installing Thruk.

Labs Consol Repository

The Labs Repository provides packages of latest Thruk releases (including daily development builds) and other Tools like Mod-Gearman. Just follow the guide on the labs page for your distribution.

OMD

An easy way of installing Thruk is using OMD from omdistro.org. There are Debian, Ubuntu, Centos and Suse Packages containing preconfigured latest versions of Nagios and Thruk. The package also includes Icinga, Shinken, pnp4nagios, check_mk and Nagvis. OMD is the recommended way of installing new Nagios / Thruk setups when you want to install several addons at once.

Use Packages

Standalone installation from a binary package is another easy way to get Thruk running.

Download packages from http://www.thruk.org/files/pkg/

All packages have the following filesystem structure:

  /etc/thruk                       Thruks config
  /etc/httpd/conf.d/thruk.conf     Apache config
  /usr/share/thruk                 shared files
  /usr/lib/thruk/perl5             Perl libraries
  /var/cache/thruk                 temporary files
  /var/lib/thruk                   stored user settings

After installation, Thruk is available at http://your-host/thruk/ and has a default user thrukadmin with password thrukadmin configured. You may need to change backend configuration. This should be done in the thruk_local.conf where all settings can be overridden.

Debian / Ubuntu

  #> dpkg -i thruk_1.80_debian6_amd64.deb

In case of dependency errors, run apt-get -f install and try the dpkg -i… again.

Centos / Redhat

  #> yum install --nogpgcheck thruk-1.80-1.rhel6.x86_64.rpm

You may need to include an external repository for mod_fastcgi module. Epel or Rpmforge should do it.

SLES

  #> zypper install thruk-1.80-1.sles11.x86_64.rpm

You may need the SLES sdk dvd for additional dependencies.

Install from Source

Installation Steps

Requirements
Important
Experienced Users Only

Source installation is for experienced users only. Using packages/repositories should be the preferred solution in almost all situations. Especially for production environments.

In order to install the Thruk Monitoring Webinterface from source you will need the following:

  • Perl

  • Git Client

  • compiler tools: automake, make, g++, gcc

  • Apache Webserver (optional for fastcgi only)

Create New User

refer to your systems manual on how to add new user. This guide uses the following:

  user:  thruk
  group: thruk
Important all following steps should be done by the thruk user.
Install Local::Lib

quick guide:

  %> su - thruk
  %> wget %http://search.cpan.org/CPAN/authors/id/A/AP/APEIRON/local-lib-1.008004.tar.gz
  %> tar zxf local-lib-1.008004.tar.gz
  %> cd local-lib-1.008004
  %> perl Makefile.PL --bootstrap && make install
Tip
proxy configuration

if you need a proxy configuration, you should answer <no> at this question:

Would you like me to configure as much as possible automatically? [yes]

Setting a proper http_proxy/ftp_proxy environment should work for the automatic configuration.

Add the following line to the thruk users .profile or .bashrc

eval $(perl -I$HOME/perl5/lib/perl5 -Mlocal::lib)

logout and login again and verify that your perl env looks like this:

 %> env | grep perl
 PERL5LIB=/home/thruk/perl5/lib/perl5:/home/thruk/perl5/lib/perl5/x86_64-linux-gnu-thread-multi
 MODULEBUILDRC=/home/thruk/perl5/.modulebuildrc
 PATH=/home/thruk/perl5/bin:/usr/local/bin:/usr/bin:/bin:/usr/games
 PERL_MM_OPT=INSTALL_BASE=/home/thruk/perl5
Install Module::Install
  perl -MCPAN -e 'install Module::Install::Catalyst'
Git Clone Thruk
    git clone https://github.com/sni/Thruk.git
    cd Thruk
    perl Makefile.PL

press <enter> here:

  ==> Auto-install the 35 mandatory module(s) from CPAN? [y]

then run make

    make

This may take a while, as there are probably several modules missing.

Tip
yes

you can use /usr/bin/yes to automatically answer all questions with yes. (yes | make)

Press <y> at this questions:

  Do you want to build the XS Stash module? [y]
  Do you want to use the XS Stash by default? [y]

run perl Makefile.PL again to see if all dependencies are now installed properly.

    perl Makefile.PL
Configuration

Copy thruk.conf to thruk_local.conf and adjust the livestatus settings to your needs.

  %> cp thruk.conf thruk_local.conf

The thruk.conf will be overwritten with new defaults on updates. The thruk_local.conf contains the local overrides and will never be overwritten.

Edit cgi.conf and adjust settings to your needs.

See the [Thruk Configuration] section for detailed explanation of configuration options.

Start Server

After running these steps successfully, you should be able to test your installation:

    ./script/thruk_server.pl

Open your browser and open http://<your_host>:3000

Apache Configuration

Tip
apache is preconfigured when using packages

Apache configuration is not needed if you use OMD or packages. Apache is already preconfigured in that case.

Integration in the Apache webserver is done by fastcgi. There are two fastcgi modules for Apache at the moment. Choose the one which fits best into your environment. If unsure, use mod_fcgid. The main difference is, that mod_fcgid starts the fastcgi process upon the first request whereas in mod_fastcgi you have to start the fastcgi process by yourself.

Do not use the port 3000 thruk server in production, it’s only for testing and development. It cannot handle authentication and is slower than the fastcgi variants.

Tip
lighthttpd

Francois Ponsard wrote an article on how to integrate Thruk in Lighthttpd: http://www.dahwa.fr/dotclear/index.php?post/2011/03/15/Thruk-in-Lighttpd

mod_fcgid

Important Thruk runs with the webserver user in this scenario, make sure the webserver user has access to all Thruk files and perl modules.
Apache configuration with mod_fcgid
LoadModule fcgid_module /usr/lib/apache2/modules/mod_fcgid.so
<VirtualHost *:80>
    ServerName   thruk.company.local

    DocumentRoot /home/thruk/Thruk/root/
    CustomLog    /home/thruk/Thruk/logs/access.log combined
    ErrorLog     /home/thruk/Thruk/logs/error.log

    <Directory /home/thruk/Thruk/root/>
        Options FollowSymLinks
        AllowOverride All
        order allow,deny
        allow from all
    </Directory>

    AliasMatch /thruk/(.*\.cgi|.*\.html)  /home/thruk/Thruk/script/thruk_fastcgi.pl/thruk/$1
    <Location /thruk>
        Options ExecCGI
        Order allow,deny
        Allow from all
        AuthName "Monitoring Access"
        AuthType Basic
        AuthUserFile /home/thruk/Thruk/htpasswd.users
        Require valid-user
    </Location>

    <IfModule mod_fcgid.c>
      AddHandler fcgid-script .pl
      MaxRequestsPerProcess 100
    </IfModule>

</VirtualHost>
Tip
Example
Another more complicated example can be found on github: apache_fcgid.conf. This file is used for the official Thruk packages.

mod_fastcgi

start your fcgi server:

    %>./script/thruk_fastcgi.pl -n 5 \
                -l /tmp/thruk_fastcgi.socket \
                -p /tmp/thruk_fastcgi.pid

you may want to copy the init.d script to /etc/init.d and adjust its paths:

    %> sudo cp ./script/thruk_fastcgi_server.sh /etc/init.d/thruk_fastcgi_server
    %> vi /etc/init.d/thruk_fastcgi_server
    %> sudo chown root: /etc/init.d/thruk_fastcgi_server

Or create a custom init.d script (additional modules required) with:

    %> ./script/thruk_create.pl FastCGI::ExternalServer l=/tmp/thruk_fastcgi.socket n=5 p=/tmp/thruk_fastcgi.pid

use this apache example configuration:

  • replace /home/thruk/Thruk with your installation path

  • replace your-web-host.local with your hostname

  • create a /home/thruk/Thruk/htpasswd.users with htpasswd2

  • make sure the /home/thruk/Thruk/logs/ directory exists

Apache configuration within existing vhost
<VirtualHost *:80>
    # ... existing configuration

    # thruk configuration
    <Directory /home/thruk/Thruk/root/>
        order allow,deny
        allow from all
        Options FollowSymLinks
        AllowOverride All
    </Directory>
    <Directory /home/thruk/Thruk/plugins/>
        order allow,deny
        allow from all
        Options FollowSymLinks
        AllowOverride All
    </Directory>

    Alias /thruk/ /home/thruk/Thruk/root/thruk/

    # authorization
    <Location "/thruk">
        AuthName "Monitoring Access"
        AuthType Basic
        AuthUserFile /home/thruk/Thruk/htpasswd.users
        Order Allow,Deny
        Allow from all
        require valid-user
    </Location>

    # Load fastcgi module unless already loaded
    LoadModule fastcgi_module /usr/lib/apache2/modules/mod_fastcgi.so

    # fastcgi configuration
    FastCGIExternalServer /tmp/thruk_fastcgi.fcgi -socket /tmp/thruk_fastcgi.socket -idle-timeout 120

    # Load rewrite module unless already loaded
    LoadModule rewrite_module /usr/lib/apache2/modules/mod_rewrite.so

    # rewrite configuration
    RewriteEngine On
    RewriteCond %{DOCUMENT_ROOT}%{REQUEST_FILENAME} !-f
    RewriteCond %{REQUEST_FILENAME} !-f
    RewriteRule ^/thruk(.*)$ /tmp/thruk_fastcgi.fcgi/thruk$1 [E=HTTP_AUTHORIZATION:%{HTTP:Authorization},QSA,L]

</VirtualHost>
Apache configuration with own vhost
<VirtualHost *:80>
    ServerName   thruk.your-host.local

    DocumentRoot /home/thruk/Thruk/root/
    CustomLog    /home/thruk/Thruk/logs/access.log combined
    ErrorLog     /home/thruk/Thruk/logs/error.log

    <Directory />
        order deny,allow
        deny from all
    </Directory>

    <Directory /home/thruk/Thruk/root/>
        Options FollowSymLinks
        AllowOverride All
        order allow,deny
        allow from all
    </Directory>
    <Directory /home/thruk/Thruk/plugins/>
        order allow,deny
        allow from all
        Options FollowSymLinks
        AllowOverride All
    </Directory>

    # authorization
    <Location "/">
        AuthName "Monitoring Access"
        AuthType Basic
        AuthUserFile /home/thruk/Thruk/htpasswd.users
        Order Allow,Deny
        Allow from all
        require valid-user
    </Location>

    # Load fastcgi module unless already loaded
    LoadModule fastcgi_module /usr/lib/apache2/modules/mod_fastcgi.so

    # fastcgi configuration
    FastCGIExternalServer /tmp/thruk_fastcgi.fcgi -socket /tmp/thruk_fastcgi.socket -idle-timeout 120

    # Load rewrite module unless already loaded
    LoadModule rewrite_module /usr/lib/apache2/modules/mod_rewrite.so

    # rewrite configuration
    RewriteEngine On
    RewriteCond %{DOCUMENT_ROOT}%{REQUEST_FILENAME} !-f
    RewriteCond %{REQUEST_FILENAME} !-f
    RewriteRule ^/(.*)$ /tmp/thruk_fastcgi.fcgi/$1 [E=HTTP_AUTHORIZATION:%{HTTP:Authorization},QSA,L]
</VirtualHost>

Backend Configuration

You may connect multiple backends into one Thruk interface. Just specify multiple peer items.

Livestatus

Livestatus is a addon for Nagios or Icinga which provides real time status data. Contrary to database addons it adds very little overhead to the Nagios core which makes livestatus based gui fast and real time.

Tip
Shinken

Shinken has livestatus support built-in and does not require additional modules.

Refer to http://mathias-kettner.de/checkmk_livestatus.html#H1:%20Setting%20up%20and%20using%20Livestatus on how to install livestatus onto your monitoring box. The Lab Consol Repository has livestatus rpm packages. For debian based Linux systems just use the provided packages of your distribution.

Basically you have to build the NEB module and add it to your nagios/icinga.cfg.

 broker_module=/opt/local/livestatus/livestatus.o /tmp/live.sock

Then use the socket as peer parameter in your backend configuration.

Thruk Backends

Thruk Backends Manager

Livestatus Architecture

Livestatus Architecture

HTTP

A HTTP connection is possible between multiple Thruk installations. A HTTP connection not only makes live data available, it also offers a secure way to manage object configuration from remote. To authenticate the central Thruk installation, you need to provide the secret key from the slave instance. This setup is perfect for managing several independent installations from a central point of view.

Tip
use SSL

It’s strongly recommended to use HTTPS for remote connections.

    # package installation
    %> cat /var/lib/thruk/secret.key

    # omd site
    OMD[site]:~$ cat var/thruk/secret.key
HTTP Architecture

HTTP Architecture

MongoDB

The MongoDB is not a real backend for status data, but it can be used as a logfile cache. If unsure whether to use MongoDB or MySQL, use MySQL which is newer and faster. The MongoDB logfile cache may be deprecated in the future.

ex.:

logcache = mongodb://localhost:27017/thruk_logs

Details are explained in the Logfile Cache section.

MySQL

The MySQL Database is not a real backend for status data, but it can be used as a logfile cache. If unsure whether to use MongoDB or MySQL, use MySQL which is newer and faster. The MongoDB logfile cache may be deprecated in the future.

ex.:

logcache = mysql://username:password@localhost:3306/thruk_logs

Details are explained in the Logfile Cache section.

Thruk Configuration

Configuration is managed mainly in these configuration files.

  * thruk.conf          # shipped default Thruk config
  * thruk_local.conf    # local override of Thruks settings
  * cgi.cfg             # Nagios cgi.cfg
  * log4perl.conf       # Logging configuration
  * menu.conf           # Thruks default side navigation
  * menu_local.conf     # local override for the navigation
Table of Contents

Config Files

thruk.conf

The thruk.conf contains the shipped defaults. You should not edit this file directly. Overwrite your settings in your thruk_local.conf instead.

thruk_local.conf

The thruk_local.conf is a copy of the thruk.conf. Both files are used by thruk. The idea is, that the thruk.conf is provided with the packaged defaults and the thruk_local.conf is to override these settings where needed.

General Settings

title_prefix

set the title prefix for all urls this piece of text will be prepended to all page titles.

ex.:

title_prefix = Prod

use_bookmark_titles

Sets the page title to the name of a bookmark if the url matches. Private bookmarks are processed first and the name of the first match is used.

ex.:

use_bookmark_titles = 1

use_dynamic_titles

Uses more descriptive page titles. Bookmark titles take precedence and can be overriden by adding the title parameter on most pages

ex.:

use_dynamic_titles = 1

url_prefix

Changes the usual url path for Thruk. Don’t change it unless you plan to run multiple Thruk instances on the same webserver. You will have to change your fastcgi configuration too.

ex.:

url_prefix = /

use_timezone

Changes the timezone from the systems default to this timezone. Only set this if you have trouble with displaying the right timestamps.

ex.:

use_timezone = CET

mobile_agent

Specify user agents which will be redirected to the mobile plugin (if enabled).

ex.:

mobile_agent=iPhone,Android,IEMobile

default_theme

Default theme to use for all users. Must be a valid sub directory in the themes folder.

ex.:

default_theme = Thruk

first_day_of_week

Set first day of week. Used in reports. Sunday: 0 Monday: 1

ex.:

first_day_of_week = 0

report_use_temp_files

Large reports will use temp files to avoid extreme memory usage. With report_use_temp_files you may set the report duration in days which will trigger the use of temp files. Default is 14days, so for example the last31days report will use temp files, the thisweek not. Can be disabled by setting to 0.

ex.:

report_use_temp_files = 14

start_page

This link is used as startpage and points usually to the main.html with displays version information and general links.

ex.:

start_page = /thruk/main.html

This link is used whenever you click on one of the main logos. By default those logos are the Thruk logos and the link will take you to the Thruk homepage. Replace this with where you want your home location to be.

ex.:

home_link = http://www.thruk.org

This link is used in the side navigation menu as link to the documentation. Replace with your documentation location. Set it to a blank value if you don’t want a documentation link in the menu at all.

ex.:

documentation_link = /thruk/docs/

Customizable link for the problems link in side menu. Can be useful to reflect your companies process of error handling.

ex.:

all_problems_link = /thruk/cgi-bin/status.cgi?...

List of allowed patterns, where links inside frames can be set to. You can link to /thruk/frame.html?link=http://wiki.my-company.com/page/blah Your wiki will then be displayed with the Thruk navigation frame. Useful for other addons, so they don’t have to display a own navigation.

ex.:

allowed_frame_links = http://intranet.my-company.com
allowed_frame_links = https://wiki.my-company.com

Authorization Settings

can_submit_commands

Set this if a contact should be allowed to send commands unless defined for the contact itself. This is the default value for all contacts unless the user has a can_submit_commands setting in your monitoring configuration.

ex.:

can_submit_commands = 1

command_disabled

Use this to disabled specific commands. Can be use multiple times to disabled multiple commands. The number can be found in the cmd_typ cgi parameter from links to the command page. You may use ranges here.

ex.:

command_disabled = 14
command_disabled = 35
command_disabled = 17-34,50-65

make_auth_user_lowercase

Convert authenticated username to lowercase.

ex.:

make_auth_user_lowercase = 1

make_auth_user_uppercase

Convert authenticated username to uppercase.

ex.:

make_auth_user_uppercase = 1

use_strict_host_authorization

When set to a true value, every contact will only see the hosts where he is contact for plus the services where he is contact for. When disabled, a host contact will see all services for this host regardless of whether he is a service contact or not.

ex.:

use_strict_host_authorization = 1

Path Settings

cgi_cfg

The path to your cgi.cfg. See cgi.cfg for details.

ex.:

cgi_cfg = cgi.cfg

log4perl_conf

The path to your log4perl configuration file.

ex.:

log4perl_conf = ./log4perl.conf

plugin_path

Path to your plugins directory. Can be used to specify different location for you Thruk plugins. Don’t forget to set appropriate apache alias or rewrite rules when changing the plugin path. Otherwise the static content from plugins is not accessible.

Example redirect rule for apache:

AliasMatch /thruk/plugins/(.*?)/(.*)$ /plugins/plugins-enabled/$1/root/$2

ex.:

plugin_path = ./plugins

themes_path

Path to your themes directory. Can be used to specify different location for you Thruk themes. Don’t forget to set appropriate apache alias or rewrite rules when changing the themes path. Otherwise the static content from your themes may not accessible.

Alias /thruk/themes/ /themes/themes-enabled/

ex.:

themes_path = ./themes

var_path

Path to the var directory. Thruk stores user specific date here.

ex.:

var_path = ./var

tmp_path

Path to a temporary directory. Defaults to /tmp if not set and usually this is a good place.

ex.:

tmp_path = /tmp

ssi_path

The path to your ssi (server side includes) files. See Server Side Includes for details.

ex.:

ssi_path = ssi/

user_template_path

Specify a additional directory for user supplied templates. This makes it easy to override thruks own templates. Template search order is:

  • users template path

  • themes template path

  • plugins template path

  • thruks template path

ex.:

user_template_path = ./my_templates

logo_path_prefix

Changes the path to your logo images. Default is $url_prefix+thruk/themes/$current_theme/images/logos/ and therefor relative to the current selected theme. You could set a fixed path here. Like usual, paths starting with a / will be absolute from your webserver root directory. Paths starting without a / will be relative to the cgi directory.

ex.:

logo_path_prefix = /icons/

physical_logo_path

Location of your logos in your filesystem. This directory should be mapped to your logo_path_prefix directory where logo_path_prefix is the path relative to your webserver root directory and physical_logo_path is the corresponding filesystem path.

ex.:

physical_logo_path = /usr/share/icons/

mode_file

Mode used when creating or saving files.

ex.:

mode_file = 0660

mode_dir

Mode used when creating folders

ex.:

mode_dir = 0770

resource_file

Set a general resource file. Make sure it does not contain any passwords or any other data which should not be displayed. Instead of using a general resource_file you could define one file per peer in your peer config.

ex.:

resource_file = /etc/nagios3/resource.cfg

Search Settings

Use the old or the classic search from the navigation. The new search supports regular expressions and searches in many attributes. For example: plugin_output, groups, names, descriptions

ex.:

use_new_search = 1

Enables the ajax search field. There will be suggestions while typing into the search field.

ex.:

use_ajax_search = 1

ajax_search_hosts

Enables the suggestion of hosts in the ajax search field. Depending on the number of hosts, this can make the search slow.

ex.:

ajax_search_hosts = 1

ajax_search_hostgroups

Enables the suggestion of hostgroups in the ajax search field.

ex.:

ajax_search_hostgroups = 1

ajax_search_services

Enables the suggestion of services in the ajax search field. Depending on the number of services, this can make the search slow.

ex.:

ajax_search_services = 1

ajax_search_servicegroups

Enables the suggestion of servicegroups in the ajax search field.

ex.:

ajax_search_servicegroups = 1

ajax_search_timeperiods

Enables the suggestion of timeperiods in the ajax search field. Timeperiods will only displayed when filtering by check- or notification period.

ex.:

ajax_search_timeperiods = 1

Paging Settings

use_pager

Using the pager will make huge pages much faster as most people don’t want a services page with 100.000 services displayed. Can be disabled if you don’t need it.

ex.:

use_pager = 1

paging_steps

Define the selectable paging steps. Use the * to set the default selected value.

ex.:

paging_steps = *100, 500, 1000, all

group_paging_overview

Just like the paging_steps, but only for the groups overview page.

ex.:

group_paging_overview =  *3,  10, 100, all

group_paging_summary

Just like the paging_steps, but only for the groups summary page.

ex.:

group_paging_summary = *10, 50, 100, all

group_paging_grid

Just like the paging_steps, but only for the groups grip page.

ex.:

group_paging_grid = *5,  10, 50,  all

Display Settings

host_action_icon

Change path to your host action icons. You may use relative paths to specify completely different location. You also may want to use action_pnp.png when using pnp. Icon can be overridden by a custom variable _ACTION_ICON.

ex.:

host_action_icon = action.gif

service_action_icon

Change path to your service action icons. You may use relative paths to specify completely different location. You also may want to use action_pnp.png when using pnp. Icon can be overridden by a custom variable _ACTION_ICON.

ex.:

service_action_icon = action.gif

initial_menu_state

Set initial menu state.

closed  => 0
open    => 1

ex.:

<initial_menu_state>
  General        = 1
  Current_Status = 1
  Reports        = 1
  System         = 1
  Bookmarks      = 1
</initial_menu_state>

use_frames

Set whether you want to use a framed navigation or not. With using frames it’s sometimes easier to include addons. See allowed_frame_links option for how to integrate addons.

ex.:

use_frames = 0

use_new_command_box

Show the new split command box on the host / service details page.

ex.:

use_new_command_box = 1

bug_email_rcpt

what email address bug reports will be sent to

ex.: bug_email_rcpt = bugs@thruk.org

datetime_format

Default timeformat. Use POSIX format.

ex.:

datetime_format = %Y-%m-%d  %H:%M:%S

datetime_format_long

Default long timeformat.

ex.:

datetime_format_long = %a %b %e %H:%M:%S %Z %Y

datetime_format_log

Default log timeformat.

ex.:

datetime_format_log = %B %d, %Y  %H

Default trends timeformat.

ex.:

datetime_format_trends = %a %b %e %H:%M:%S %Y

datetime_format_today

Default timeformat for todays date. Can be useful if you want a shorter date format for today.

ex.:

datetime_format_today = %H:%M:%S

info_popup_event_type

On which event should the comments / downtime or longpluginoutput popup show up. Valid values are onclick or onmouseover.

ex.:

info_popup_event_type = onmouseover

info_popup_options

Options for the popup window used for long pluginoutput, downtimes and comments. See http://www.bosrup.com/web/overlib/?Command_Reference for what options are available

ex.:

info_popup_options = STICKY,CLOSECLICK,HAUTO,MOUSEOFF

show_notification_number

Display the current number of notification after the current / max attempts on the status details page.

ex.:

show_notification_number = 0

show_backends_in_table

Display the backend/site name in the status table. This is useful if you have same hosts or services on different backends and need to know which one returns an error. Valid values are:

1 - show site name at the end
2 - put site name in front

ex.:

show_backends_in_table = 0

show_config_edit_buttons

Show links to config tool for each host / service. You need to have the config tool plugin enabled and you need proper permissions for the link to appear.

ex.:

show_config_edit_buttons = 1

show_full_commandline

Display the full command line for host / service checks . Be warned, the command line could contain passwords and other confidential data. In order to replace the user macros for commands, you have to set the resource_file in your peer config or the general resource_file option.

  • 0 = off, don’t show the command line at all

  • 1 = show them for contacts with the role: authorized_for_configuration_information

  • 2 = show them for everyone

ex.:

show_full_commandline = 0

show_full_commandline_source

Usually the source of your expanded check_command should be the check_command attribute of your host / service. But under certain circumstances you might want to use display expanded commands from a custom variable. In this case, set show_full_commandline_source to _CUST_VAR_NAME.

ex.:

show_full_commandline_source = check_command

show_logout_button

Show additional logout button next to the top right preferences button. (works only together with cookie authentication)

ex.:

show_logout_button=1

show_long_plugin_output

When a plugin returns more than one line of output, the output can be displayed directly in the status table, as popup or not at all. Choose between popup, inline and off

ex.:

show_long_plugin_output = popup

status_color_background

Color complete status line with status colour or just the status itself.

ex.:

status_color_background=0

show_modified_attributes

Show if a host / service has modified attributes.

ex.:

show_modified_attributes = 1

show_host_attempts

Show check attempts for hosts too. The default is to show them on the problems page only. Use this value to force a value.

ex.:

show_host_attempts = 1

perf_bar_mode

This option enables a performance bar inside the status/host list which create a graph from the performance data of the plugin output. Available options are match, first, all, worst and off.

match: try to set graph which matches the output
all: graph all performance values available
first: graph only the first performance value
worst: graph only the graph for the worst state
off: graph no value at all

ex.:

perf_bar_mode = match

perf_bar_pnp_popup

Show pnp popup if performance data are available and pnp is used as graph engine.

ex.:

perf_bar_pnp_popup = 1

shown_inline_pnp

Show inline pnp graph if available. If a service or host has a pnp4nagios action or notes url set. Thruk will show a inline graph on the extinfo page. This works for /pnp4nagios/ urls and /pnp/.

ex.:

shown_inline_pnp = 1

graph_word

graph_word is a regexp used to display any graph on the details page. if a service or host has a graph url in action url (or notes url) set it can be displayed by specifying a regular expression that always appears in this url. You can specify multiple graph_words.

sample service configuration for graphite:

  define service{
    service_description   Load
    host_name             localhost
    use                   generic-service
    action_url            http://YOURGRAPHITE/render/?lineMode=connected&width=586&height=308&_salt=1355923874.899&target=cactiStyle($HOSTNAME$.$SERVICEDESC$.*)&xFormat=%25H%3A%25M&tz=Europe/Paris
    check_command         check_load
  }

ex.:

graph_word = /pnp4nagios/   # for pnp4nagios
graph_word = /render/       # for graphite

Info : Quotes are supported in the action_url statement, you may want to use it for special graphite function (http://graphite.readthedocs.org/en/1.0/functions.html) Do not escape double quotes here, otherwise graph won’t work.

show_custom_vars

Show custom vars in host / service ext info. List variable names to display in the host and service extinfo details page. Can be specified more than once to define multiple variables. You may use html in your variables. Use * as wildcard, ex.: _VAR*

ex.:

show_custom_vars = _VAR1

show_error_reports

Show link to bug reports when internal errors occur. Set to server to log js error server side.

ex.: show_error_reports = 1

skip_js_errors

don’t report some known harmless javascript errors

ex.: skip_js_errors = cluetip is not a function

strict_passive_mode

Normally passive checks would be marked as disabled. With this option set, disabled checks will only be displayed as disabled if their last result was active. Otherwise they would be marked as passive checks. This option also changes the passive icon only to be shown when the last check was passive, otherwise the disabled icon will be displayed.

ex.:

strict_passive_mode = 1

hide_passive_icon

Normally passive checks whould be displayed with a passive icon if their last result is passive. With this option, passive icon will be hidden in status details.

ex.: hide_passive_icon = 0

sitepanel

The sitepanel is used to display multiple backends/sites at a glance. With than 10 or more sites, the list of backends will be combined into the compact site panel which just displays the totals of available / down / disabled sites. The compact panel will also automatically be used if you use sections. With more than 50 backends, the collapsed will be selected in auto mode. Set sitepanel to list/compact/collapsed/auto/off to change the default behaviour.

ex.:

sitepanel = auto

apache_status

You can integrate the output of apache status into Thruk. The following list of apache status pages will be accessible from the performance info page. Make sure the page is accessible from Thruk, credentials will be passed through. So both, basic authentication or ip based authentication would be possible. Read more about Apaches mod_status here: http://httpd.apache.org/docs/current/mod/mod_status.html

ex.:

<apache_status>
  Site        http://127.0.0.1:5000/server-status
  System      http://127.0.0.1/server-status
  Other       http://127.0.0.1/server-status
</apache_status>

Connection Settings

check_local_states

Get the status for remote backends from local instances. This can increase performance when using multiple remote sites. It is enabled by default when using more than one site. You have to define hostchecks in any local backend (using unix sockets) with a name or alias of the address of your remote backends. To explicitly connect a host to a specific backend, set state_host in that peer configuration.

ex.:

check_local_states = 1

backend_debug

Set logging of backend in verbose mode. This only makes sense when debug logging is activated.

ex.:

backend_debug = 1

connection_pool_size

Use connection pool when accessing multiple sites. Increases the performance because backends will be queried parallel but uses around 10mb of memory per pool member. Disabled when set to 0, number of concurrent connections otherwise.

ex.:

connection_pool_size = 5

logcache

Enables caching logfiles for faster access and less memory usage for the nagios process. Cache supports MongoDB and Mysql only. Prefered type is Mysql. Format is a Mysql/MongoDB connection string like mysql://hostname:port/db. Using a cache dramatically decreases cpu and memory usage of Thruk and Nagios when accessing logfiles, for example when creating reports.

ex.:

logcache = mysql://user:password@localhost:3306/thruk_log_cache
logcache = mongodb://localhost:27017/thruk_log_cache

delay_pages_after_backend_reload

Delay the page delivery until the backends uptime is at least this amount of seconds. Displaying pages soon after backend restarts may display wrong results and all services are pending. Enable this if you experience problems with pending services after reloading your backend. Should be obsolete with Livestatus versions greater than 1.2 ex.: setting this to 10 would start serving pages 10 seconds after the backend reload

ex.:

delay_pages_after_backend_reload = 10

ssl_verify_hostnames

Can be set to enable / disable hostname verification for https connections. For example for the cookie login or the http backend.

ex.:

ssl_verify_hostnames = 1

Specifies the url where non-authenticated users will be redirected too.

ex.:

cookie_auth_login_url = thruk/cgi-bin/login.cgi

Specifies the url against the cookie auth provider will verify its credentials.

ex.:

cookie_auth_restricted_url = http://localhost/thruk/cgi-bin/restricted.cgi

Specifies the timeout for idle sessions.

ex.:

cookie_auth_session_timeout       = 86400

Specifies the amount of seconds in which subsequent requests won’t verify authentication again.

ex.:

cookie_auth_session_cache_timeout = 5

Cookie domain is usually set automatically. Use this option to override the default value. Domains have to contain at least two periods. Useful for single sign on environments.

ex.:

cookie_auth_domain = .domain.com

Hook script which is called on every successful login. The REMOTE_USER environment variable will be set to the username of the current logged in user. Useful to do magic stuff on each login.

ex.:

cookie_auth_login_hook = ./bin/hook.sh

Command Settings

cmd_defaults

Set the default checked state for command options.

ex.:

<cmd_defaults>
  ahas                   = 0  # For Hosts Too
  broadcast_notification = 0  # Broadcast
  force_check            = 0  # Forced Check
  force_notification     = 0  # Forced Notification
  send_notification      = 1  # Send Notification
  sticky_ack             = 1  # Sticky Acknowledgement
  persistent_comments    = 1  # Persistent Comments
  persistent_ack         = 0  # Persistent Acknowledgement Comments
  ptc                    = 0  # For Child Hosts Too
  use_expire             = 0  # Use expire time ( for cores which support it)
</cmd_defaults>

force_sticky_ack

Forces acknowledgments to be sticky.

ex.:

force_sticky_ack = 1

force_send_notification

Forces sending a notification for acknowledgments.

ex.:

force_send_notification = 1

force_persistent_ack

Forces comments on acknowledgments to be persistent.

ex.:

force_persistent_ack = 1

force_persistent_comments

Forces normal comments to be persistent.

ex.:

force_persistent_comments = 1

downtime_duration

Default duration of new downtimes in seconds. Default is 2 hours.

ex.:

downtime_duration = 7200

expire_ack_duration

Default duration of acknowledgements with expire date. Default is one day.

ex.:

expire_ack_duration = 86400

cmd_quick_status

Configure which commands should be available as quick status commands.

ex.:

<cmd_quick_status>
  reschedule             = 0  # Reschedule next check
  downtime               = 0  # Add/remove downtimes
  comment                = 0  # Add/remove comments
  acknowledgement        = 0  # Add/remove acknowledgements
  active_checks          = 0  # Enable/disable active checks
  notifications          = 0  # Enable/disable notifications
  submit_result          = 0  # Submit passive check result
  reset_attributes       = 0  # Reset modified attributes
</cmd_quick_status>

command_reschedule_alias

When you want to reschedule passive checks for which the result is fetched by an agent (For example check_mk or some scenarios of check_multi). You usually want to reschedule the agent instead of the passive check.

The command reschedule alias can be used to translate the reschedule command from the passive service to the active agent service.

command_reschedule_alias = pattern;master_service_description
Notes
  • The pattern will be tested against the service description and the command_name of the passive check.

  • The resulting service name be on the same host and the contact must be authorized for that service too.

  • The pattern must be a valid perl regular expression.

  • Duplicates will be removed. So if you reschedule 10 services which result in the same master service will only trigger one reschedule.

  • Only passive services will be translated

In this example, all passive check_mk checks will trigger the active agent check and therefor allow you to reschedule passive checks directly from the problems page.

ex.:

command_reschedule_alias = ^check_mk\-(?!inventory);Check_MK

use_feature_recurring_downtime

Use recurring downtime, shows recurring downtime links.

ex.:

use_feature_recurring_downtime = 1

use_feature_histogram

Use histograms, shows histogram links.

ex.:

use_feature_histogram = 1

use_service_description

Use service’s description instead of display name.

ex.:

use_service_description = 1

Use trends, shows trend links.

ex.:

use_feature_trends = 1

use_wait_feature

Waiting is a livestatus feature. When enabled, Thruk will wait after rescheduling hosts/services checks until the check has been really executed up to a maximum of 10 seconds. Adjust the time waiting with the wait_timeout option.

ex.:

use_wait_feature = 1

wait_timeout

Amount of seconds to wait until a rescheduled check finishes. Thruk will wait this amount and display the result immediately.

ex.:

wait_timeout = 10

Cron Settings

cron_file

Specify a file which is then completely under the control of Thruk. It will be used to store cronjobs, ex. for reports. The file has to be writable by Thruk.

ex.:

cron_file = /tmp/thruk_cron.tmp

cron_pre_edit_cmd

The pre edit cmd can be used to do run a command just before Thruk will edit the crontab.

ex.:

cron_pre_edit_cmd  = /usr/bin/crontab -l > /tmp/thruk_cron.tmp

cron_post_edit_cmd

The post edit cmd is necessary for OMD where you need to reload the crontab after editing or for replacing the users cron with the edited file.

ex.:

cron_post_edit_cmd = crontab /tmp/thruk_cron.tmp

thruk_bin

Path to your thruk executable. Will be used in cronjobs.

ex.:

thruk_bin = /usr/bin/thruk

Icinga Settings

enable_icinga_features

This one activates all icinga specific features. If not set, Thruk will try to auto-detect your backends. Currently auto detection will only work within OMD. Don’t enable it unless all your backends are icinga.

ex.:

enable_icinga_features = 1

Shinken Settings

enable_shinken_features

This one activates all problem/impact and criticity features. Currently it will only work with shinken backends. Don’t enable it unless all your backends are shinken. If not set, it will be automatically enabled when using only shinken backends.

ex.:

enable_shinken_features = 1

priorities

Set the names of the priority (criticity in shinken). Currently this will only work with shinken backends.

ex.:

<priorities>
  5   = Business Critical
  4   = Top Production
  3   = Production
  2   = Standard
  1   = Testing
  0   = Development
</priorities>

Other Settings

no_external_job_forks

Normally reports will be generated in an external process to avoid timeouts on long running reports. Use this switch to turn external jobs off and generate reports directly. Make sure they are finished within 40seconds which is the default fcgi timeout.

ex.:

no_external_job_forks = 1

thruk_init

Path to your thruk init script. Will be used to restart thruk.

ex.:

thruk_init = /etc/init.d/thruk

Path used for cookies. Do not change unless you have weird url rewrites which breaks setting cookies.

ex.:

cookie_path = /

Component Thruk::Backend

Enter your backend connection settings here. See Backend Configuration for details.

peer

  • name name for this connection

  • type type of this connection. Can be livestatus, http or mongodb

  • hidden should this peer be hidden initially ( can be reenabled via gui switch ) Only useful with more than one backend. Setting this option removes this backend from any report or cli action unless backends are specified explicitly.

  • display Set to 0 if you want to hide this backend from the default view. This can be changed by the site panel again. In opposite to the hidden flag, backends set to display=0 are still used for automatic actions and reports.

  • groups if set, only contacts from these groups have access. You may add multiple groups separated by comma. Users without the right contactgroup don’t even see that there is a backend. Note that this implies one extra backend request per page. (deprecated, please do not use)

  • section to group backends/sites by different sections, enter a section.

  • state_host Set state_host explicitly. See See check_local_states for details.

  • options

    • peer address of this connection.

    • resource_file resource_file for this peer (used for macro replacement)

  • configtool

    • core_type Give the config parser a hint about your config. Can be nagios, icinga or shinken.

    • core_conf Path to your nagios.cfg / icinga.cfg. Read all object directories and files from this config file.

    • obj_check_cmd Commandline to verify the config.

    • obj_reload_cmd Commandline to reload the config.

    • obj_readonly Filename pattern to define readonly objects. For example for generated config files. The same effect can be achieved by adding "# thruk: readonly" on top of a object config file.

    • obj_dir Path to your objects. Enables the objects editor. Reads all *.cfg from this folder and all subfolders. (only needed when not using core_conf)

    • obj_file Path to a single objects file. Enables the objects editor. Both obj_dir and obj_file can be specified more than once. (only needed when not using core_conf)

    • obj_exclude Specify some exception pattern for the obj_dir. (only needed when not using core_conf)

    • git_base_dir Override global git_base_dir value.

ex.:

<Component Thruk::Backend>
  <peer>
      name   = Local Nagios
      type   = livestatus
      hidden = 1             # makes this backend hidden by default
      groups = admins,locals # makes this backend only visible to the
                             # admin and the locals contactgroup
      <options>
          peer       = /tmp/livestatus.socket
      </options>
      <configtool>
          core_conf  = /etc/nagios/nagios.cfg
          obj_check  = /etc/init.d/nagios checkconfig
          obj_reload = /etc/init.d/nagios reload
      </configtool>
  </peer>
  <peer>
      name   = External Icinga
      type   = livestatus
      <options>
          peer   = 172.16.0.2:9999
     </options>
  </peer>
  <peer>
      name   = External Shinken
      type = livestatus
      <options>
          peer   = 172.16.0.3:50000
     </options>
  </peer>
</Component>

Component Thruk::Plugin::ConfigTool

Enable config tool by setting path to different components config files. Users with the roles authorized_for_configuration_information and authorized_for_system_commands will then have access to the config tool. You don’t have to restart Thruk when changing the config with the config tool.

ex.:

<Component Thruk::Plugin::ConfigTool>
  show_plugin_syntax_helper = 1
  thruk                     = .../thruk_local.conf
  cgi.cfg                   = .../cgi.cfg
  htpasswd                  = .../htpasswd
  pre_obj_save_cmd          = .../hook.sh
  post_obj_save_cmd         = .../hook.sh
  git_base_dir              = /etc/nagios/conf.d/custom/
</Component>

show_plugin_syntax_helper

Enable/Disable the plugin syntax helper. When enabled, Thruk will run the plugins with "plugin -h" to get the help information.

thruk

Path to your thruk_local.conf. Enables adjusting Thruks config if set.

cgi.cfg

Path to your cgi.cfg. Enables adjusting the cgi.cfg if set.

htpasswd

Path to your htpasswd. Enables user management based an Apaches basic auth with htpasswd.

pre_obj_save_cmd

Run a command before saving object config files. Save will be canceled if the command returns non-zero exit code. Command will be run with pre as first argument and the root config folder as second argument. REMOTE_USER will be set in the environment.

post_obj_save_cmd

Run a command after saving object config files. Command will be run with post as first argument and the root config folder as second argument. REMOTE_USER will be set in the environment.

git_base_dir

If not all config folders are managed by git, then you can specify a directory which is used to display the history. Should point to a directory which is part of a git repository. This config setting is only used for the history support in the configuration tool. The git_base_dir may be overriden by an per backend value.

Component Thruk::Plugin::Panorama

The Panorama plugin is a nice, fully customizable dashboard allowing you to build your own panorama views.

ex.:

<Component Thruk::Plugin::Panorama>
  state_provider  = server
  default_view    = /var/lib/thruk/default_panorama_view
  readonly        = 0
</Component>

state_provider

Sets the way states between reloads are preserved. Valid options are cookie or server. Cookie store is useful for demo systems where lot people share the same account.

default_view

Can be either the string of an exported view or path to a file with exported view.

readonly

Make panorama dashboard readonly. The user cannot add, remove or change panels and tabs. Commands may still be send according to the user permissions.

Component Thruk::Plugin::Reports2

The Reports2 plugin creates sla reports in html or pdf format. You probably have to download the wkhtmltopdf binary in a recent version.

ex.:

<Component Thruk::Plugin::Reports2>
  report_nice_level = 5
  report_base_url   = http://host.local/thruk/cgi-bin/
  wkhtmltopdf       = /usr/bin/wkhtmltopdf
  pnp_export        = plugins/plugins-enabled/reports2/script/pnp_export.sh
  report_from_email = User Name <example@mail.com>
</Component>

report_nice_level

Execute regular scheduled reports with this nice level.

report_base_url

Url used to replace relative links in html reports.

wkhtmltopdf

wkhtmltopdf is used to convert html pages into pdf. You will need at least version 0.11. Get wkhtmltopdf from http://code.google.com/p/wkhtmltopdf/

pnp_export

pnp_export defines a script which exports a PNP4nagios graph into a local file which then can be included in reports. PNP4nagios images are either exported localy by executing php or remotly fetched with wget. This works automatically in OMD, but only for local sites of course. To make this work without OMD, set the variable PNP_ETC and PNP_INDEX in the ~/.thruk file of your webserver. The PNP4nagios url is taken from the action url or from the notes url.

ex.:

export PNP_ETC="/etc/pnp4nagios"
export PNP_INDEX="/usr/share/pnp4nagios/htdocs/index.php"

Remote PNP4nagios graphs will be fetched by wget and usually the graphs are password protected, so you may want to set a different wget command to specify a username and password. This can to be set by the PNP_WGET variable in the ~/.thruk file of the webserver user. You may force to use wget by setting PNP_URL_PREFIX.

ex.:

export PNP_WGET="wget -q --user=admin --password=secret"
export PNP_URL_PREFIX="http://demo.thruk.org"

report_from_email

Set the from address used in e-mail reports. Format is "User Name <example@mail.com>"

Component Thruk::Plugin::Statusmap

The Statusmap plugin gives an overview over your network.

ex.:

<Component Thruk::Plugin::Statusmap>
  statusmap_default_groupby = address
  statusmap_default_type    = table
</Component>

statusmap_default_type

You may change the default map type of the statusmap here. Valid types are: table and circle

statusmap_default_groupby

And the statusmap default group by which has to be one of: parent, address, domain, hostgroup, servicegroup

Component Thruk::Plugin::Minemap

The Minemap plugin gives an overview of your hosts and services.

ex.:

<Component Thruk::Plugin::Minemap>
  minemap_default_link = /thruk/cgi-bin/minemap.cgi
</Component>

You may change the default minemap link here.

Component Thruk::Plugin::BP

The Business Process plugin gives the possibility to create and model your business processes. In order to edit and create new business processes you will need the roles:

  • authorized_for_configuration_information

  • authorized_for_system_commands

Configuration Example:

<Component Thruk::Plugin::BP>
  spool_dir              = /var/nagios/rw/check_results
  objects_save_file      = /etc/nagios/conf.d/bp_generated.cfg
  objects_templates_file = /etc/nagios/conf.d/thruk_bp_templates.cfg
  objects_reload_cmd     = /etc/init.d/nagios reload
  #pre_obj_save_cmd      =
  #post_obj_save_cmd     =
  refresh_interval       = 1
</Component>

spool_dir

Results will be send back by using the spool folder. This folder should point to the check_result_path of your core.

objects_save_file

Save objects to this file. Content will be overwritten.

objects_templates_file

User maintained file containing templates used for business process services.

objects_reload_cmd

Command to apply changes to the objects_save_file.

pre_obj_save_cmd

Run this hook command before saving a business process.

post_obj_save_cmd

Run this hook command after saving a business process.

refresh_interval

Refresh interval defines how often business processes will be recalculated and refreshed. (in minutes)

User & Group Specific Overrides

Both, the Users and the Groups directive override default settings for single users or groups. In theory it’s possible to override each and every config setting from the thruk.conf but you will get funny results if you override fundamental settings like backends, paths and such things.

Groups

The groups directive overrides specific config settings for one group only. Group overrides are applied in alphabetical order. For example, if you have a user in group Admins and in Superadmins, he will get all overrides from both groups, but the Admins overrides will be superseded by the Superadmins.

ex.:

    # disable all commands, except reschedule for all users
    command_disabled = 0-6,8-95,97-999

    <Group admins>
        # enable commands for admins again
        command_disabled =
    </Group>

Users

The users directive overrides specific config settings for one user only.

ex.:

    <User guest>
        # override single configuration item
        show_error_reports  = 0
        can_submit_commands = 0

        # also available for nested components
        <Component Thruk::Plugin::Panorama>
          state_provider  = server
          default_view    = /etc/thruk/default_panorama_view_guest
        </Component>
    </User>

cgi.cfg

The cgi.cfg is mainly the same as in Nagios or Icinga but not all values are used.

show_context_help

Displays an icon with context specific help on most pages.

ex.:

show_context_help = 1

use_authentication

Determines whether to use authentication or not. If enabled, Thruk will use the REMOTE_USER from the apache environment as authenticated user name. So the authentication has to be done by the webserver. Authorization is done by Thruk.

ex.:

use_authentication = 1

use_ssl_authentication

If enabled, the authenticated username is taken from the SSL_CLIENT_S_DN_CN environment instead of the remote user.

ex.:

use_ssl_authentication = 1

default_user_name

The default user will be used if no username has been provided by the webserver. You don’t have to set a default user.

ex.:

default_user_name = thrukadmin

authorized_for_system_information

List of usernames who have access to the system and process information pages. You may use wildcards here.

ex.:

authorized_for_system_information = thrukadmin

authorized_for_configuration_information

List of usernames who have access to the configuration information pages. You may use wildcards here.

ex.:

authorized_for_configuration_information = thrukadmin

authorized_for_system_commands

List of usernames who are allowed to send system commands. You may use wildcards here.

ex.:

authorized_for_system_commands = thrukadmin

authorized_for_all_services

List of usernames who are authorized to view all services. You may use wildcards here.

ex.:

authorized_for_all_services = thrukadmin

authorized_for_all_hosts

List of usernames who are authorized to view all hosts. You may use wildcards here.

ex.:

authorized_for_all_hosts = thrukadmin

authorized_for_all_service_commands

List of usernames who are authorized to send commands for all services. You may use wildcards here.

ex.:

authorized_for_all_service_commands = thrukadmin

authorized_for_all_host_commands

List of usernames who are authorized to send commands for all hosts. You may use wildcards here.

ex.:

authorized_for_all_host_commands = thrukadmin

refresh_rate

Number of seconds after which most pages are refreshed automatically.

ex.:

refresh_rate = 90

escape_html_tags

Determines whether html output from plugins is escaped or not.

ex.:

escape_html_tags = 1

action_url_target

Sets the target of the action url links.

ex.:

action_url_target = _blank

notes_url_target

Sets the target of the notes url links.

ex.:

notes_url_target = _blank

lock_author_names

If enabled, user are not allowed to change the author name for commands.

ex.:

lock_author_names = 1

Logfiles

Logging configuration takes place in the log4perl.conf. You can do whatever is possible in log4perl. Refer to the log4perl manual for detailed information: http://log4perl.sourceforge.net/releases/Log-Log4perl/docs/html/Log/Log4perl/Config.html There is a log4perl.conf.example file shipped with the package.

ex.:

log4perl.logger=ALL, ErrorLog, DebugLog
log4perl.appender.ErrorLog=Log::Dispatch::File
log4perl.appender.ErrorLog.filename=/home/thruk/Thruk/logs/error.log
log4perl.appender.ErrorLog.mode=append
log4perl.appender.ErrorLog.Threshold=ERROR
log4perl.appender.ErrorLog.layout=Log::Log4perl::Layout::PatternLayout
log4perl.appender.ErrorLog.layout.ConversionPattern=[%d][%H][%p][%c] %m%n
Tip
command logging

if you want to log all sent commands, just set the loglevel to INFO.

ex.: log4perl.appender.ErrorLog.Threshold=INFO

Logfile Cache

Accessing logfiles can be time and memory consuming, especially when having remote sites connected to Thruk. Livestatus caches logfile data used for queries. This increases the size of the nagios process, slows down the monitoring and increases the overall load on your monitoring host. Therefor Thruk may use a logfile cache to increase performance of logfile access and reports.

If unsure whether to use MongoDB or MySQL, use MySQL which is newer and faster. The MongoDB logfile cache may be deprecated in the future.

For maximum performance, the Logfile Cache Database should be installed on the same server as Thruk itself (or at least in the same network).

Logfile Cache Architecture

Logfile Cache Architecture

Configuration

You will need to specify your mysql connection configuration in your thruk_local.conf. Make sure the user and database exists. The tables will then be created automatically.

thruk_local.conf:

  logcache = mysql://username:password@localhost:3306/thruk_logs
Tip
create mysql user and tables
#> mysql -u root -p
mysql> CREATE USER 'thruk'@'localhost' IDENTIFIED BY '****';
mysql> CREATE DATABASE IF NOT EXISTS `thruk`;
mysql> GRANT ALL PRIVILEGES ON `thruk`.* TO 'thruk'@'localhost';

Actions

Import

It’s a good idea to import the logfiles after enabling the logfilecache:

Initial import:

    %> thruk -a logcacheimport --local
    ...
    OK - imported 20374 log items from 1 site successfully in 4.94s (4122/s)

Be careful, the logcacheimport deletes all current cached data, so only do that once.

Depending on the size, this may take a while.

Update

Logfile Cache Update

    %> thruk -a logcacheupdate
    OK - imported 3 log items from 3 sites successfully in 0.13s (23/s)

The logcacheupdate imports the delta since the last update. That’s pretty fast and done automatically whenever you access logfiles via Thruk. However, if you access the eventlog page only once in a while, it would be a good idea to update the logcache periodically by a cronjob.

Import from Logfile Archive Files

The MySQL logcache supports import from files too. Duplicates are not imported twice, so you can do that safely as often as you like.

    %> thruk -a logcacheupdate --local /var/log/nagios/archive/nagios-09-07-2012-00.log
    OK - imported 2 log items from 1 site successfully in 0.01s (247/s)

Authentication Update

The cache needs authentication information which has to be updated whenever contacts have changed.

Update logfile caches authentication information:

    %> thruk -a logcacheauthupdate
    OK - updated 71 log items from 3 sites successfully in 0.25s (286/s)

Cleanup

You may want to remove old logfile entries from time to time. This is possible with the logcacheclean command. Optionally you can define the number of days which should be kept in the cache.

    %> thruk -a logcacheclean=730
    OK - removed 18046 log items from 1 site successfully in 0.08s (218816/s)

Statistics

There is also a command to check the current usage statistics of your logfile cache:

    %> thruk -a logcachestats
    Backend              Index Size      Data Size       Items
    Devel                 14.4 MB        136.4 MB       238104

Maintainance

Using the logfile cache needs some extra maintainance to work smoothly, the following cronjobs should cover that. When not using OMD, you will have to adjust the paths.

  # update logcache data
  */5 * * * * $OMD_ROOT/bin/thruk -a logcacheupdate     >>$OMD_ROOT/var/log/thruk.log 2>&1

  # update authentication data in our logcache
  0   2 * * * $OMD_ROOT/bin/thruk -a logcacheauthupdate >>$OMD_ROOT/var/log/thruk.log 2>&1

  # optimize and repair tables
  0  20 * * * $OMD_ROOT/bin/thruk -a logcacheoptimize   >>$OMD_ROOT/var/log/thruk.log 2>&1

  # clean logfiles from cache after two years
  0   1 * * * $OMD_ROOT/bin/thruk -a logcacheclean=730  >>$OMD_ROOT/var/log/thruk.log 2>&1

Menu Configuration

Navigation configuration is split in two files

  • menu.conf

  • menu_local.conf

The menu.conf contains the default side menu configuration. Do not change it as this file will be overwritten with every update. If you want to create a complete custom navigation, just copy this file to menu_local.conf and adjust it to your needs. See the next chapter on how to add just add a few links without having to copy the complete file.

The menu_local.conf contains the user configuration for the side menu. See the FAQ for some examples.

The file itself is perl syntax, so you can do whatever perl can do. Make sure you verify the syntax after changing the file. The changes will be used immediately. So maybe you want to test your changes on a test instance first.

Tip
syntax check for menu configuration
%>perl -wc menu_local.conf
menu_local.conf syntax OK

If you just want to add a few entries, create a empty menu_local.conf and put these lines into it:

do '/usr/share/thruk/menu.conf';
insert_item('General', { 'href' => 'http://labs.consol.de', 'name' => 'Labs', target => '_blank' });

Server Side Includes

You can place static include files or executables in the ssi folder. The static files will be place right after the body or just before the end of the body.

Important This has nothing to do with the Apache module mod_include and just implements the nagios way of SSIs.

There are two global includes, which will be included in every cgi page:

  • ssi/common-footer.ssi

  • ssi/common-header.ssi

and there are page specific includes, which will only be placed in the specific page:

  • ssi/status-footer.ssi

  • ssi/status-header.ssi

Files will be statically placed inside the output of the normal cgi output. Executables will be executed and the output will then be used as content.

CGI Parameters

Most of the CGI Parameters can be changed with buttons, but there are some which are only accessible by adding them directly to the url.

List of Parameters

columns

Pages: status overview

Description: Sets the number of columns

Example: /thruk/cgi-bin/status.cgi?hostgroup=all&style=overview&columns=5

debug

Pages: availability reports

Description: Creates a file with debug information regarding this report

Example: /thruk/cgi-bin/avail.cgi?host=HOSTNAME&debug=1

hidetop

Pages: status pages

Description: Show/Hide the status totals header.

Example: /thruk/cgi-bin/status.cgi?hidetop=1

minimal

Pages: status page

Description: Hides almost all user interface buttons. Useful for monitor screens where only the current problems should be displayed.

Example: /thruk/cgi-bin/status.cgi?host=all&servicestatustypes=28&minimal=1

nav

Pages: all pages

Description: Show/Hide the navigation. Only useful in non-frames mode.

Example: /thruk/cgi-bin/tac.cgi?nav=0

noexternalforks

Pages: all pages

Description: Skip external page generation. Used primarily in reports.

Example: /thruk/cgi-bin/avail.cgi?noexternalforks

noheader

Pages: status pages

Description: Show/Hide the status totals header.

Example: /thruk/cgi-bin/status.cgi?noheader

refresh

Pages: all pages

Description: Sets refresh timer to a custom value.

Example: /thruk/cgi-bin/status.cgi?refresh=30

JSON Exporting

There are several pages which offer json exports.

List of Pages

Status Data

Example: /thruk/cgi-bin/status.cgi?view_mode=json&host=all

Parameters: all usual status page parameters and filter. Use the gui filter and then json export link for maximum convenience.

It is possible to set the columns by using the columns parameter:

Example: /thruk/cgi-bin/status.cgi?view_mode=json&host=all&columns=name,status

Availability Data

Example: /thruk/cgi-bin/avail.cgi?view_mode=json&host=all

Parameters:

    t1=1357575984                       # start timestamp
    t2=1357662384                       # end timestamp
    host=hostname                       # hostname
    assumestateretention=yes            # assume state retention or not
    assumeinitialstates=yes             # assume initial state or not
    assumestatesduringnotrunning=yes    # assume state during core not running or not
    initialassumedhoststate=0           # initial assumed host state
    initialassumedservicestate=0        # initial assumed service state
    full_log_entries                    # include full logs
    showscheduleddowntime=
    rpttimeperiod=

See Monitoring::Availability for a detailed description of all parameters.

Business Process Data

Example: /thruk/cgi-bin/bp.cgi?view_mode=json&bp=id

Parameters: optional business process id

Example: /thruk/cgi-bin/bp.cgi?view_mode=json&bp=1

SLA Reporting

How it works

SLA Reporting consists of several steps.

  • report configuration

  • report template

  • converting html report into pdf

  • optionally send report by mail

Report Configuration

You can create and adjust report settings on the Reporting page. Make sure the plugin is enabled. After selecting a template for your report you have to set specific types of input fields on the second page. If you make you report public, everyone can view and refresh the report.

Report Templates

All templates used for reports can be found in plugins/plugins-enabled/reports2/templates/reports. You may adjust or override all templates to your needs and put them into a reports subfolder of the user_template_path.

  | reports     contains the selectable reports
  |- comp       contains components used in reports
  |- locale     contains localization settings
  |- style      contains css used to make reports pretty

See http://thruk.org/api/Thruk/Utils/Reports/Render.html for details on available functions and examples.

The Templates itself are written in Template::Toolkit which is perfectly documented at http://template-toolkit.org/docs/manual/index.html

Converting

The generated HTML report will be converted into a pdf file using wkhtml2pdf together with xvfb. Both tools must be installed on Thruks host. There is a binary download for wkhtml2pdf on http://code.google.com/p/wkhtmltopdf/. Use at least the 0.11 version or later.

Sending Reports By Mail

It’s quite easy to setup sla reports by mail. All you need is to create send option in the report configuration. It is even possible to send one report at multiple times.

Customized Reports

Reports can be customized with some HTML and CSS knowledge. Reports in general have several blocks which specify the steps done to produce the report.

  • styles

  • edit

  • prepare

  • render

  • mail

Styles

Customizing css styles of your reports is easy. You need to create a new file reports/style/custom.tt below your user_template_path.

For example:

<style type="text/css">
/* override css styles */
BODY {
    font-family: arial, sans-serif;
    font-size: 11px;
    line-height: 18px;
    font-size: 16px;
}
</style>

<script language="javascript" type="text/javascript">
/* add complete new elements by javascript */
  jQuery(document).ready(function() {
    jQuery('<img src="[% url_prefix %]thruk/themes/Thruk/images/logo_thruk.png">').insertBefore('DIV.page.cover DIV.titletext');
  });
</script>

If thats not enough, you still can change each and every template used to create the reports.

Edit Block

The edit block contains configuration items used for this report. They create the edit page when someone chooses to create or edit a report of this type. Basically this block contains all required and optional fields used for this report.

For example:

[% IF block == 'edit' %]
  [%
    required_fields = [
        { 'host'       => [ 'Host',        'hosts',         ''              , '', 1 ] },
        { 'timeperiod' => [ 'Timeperiod',  'rptimeperiod',  'last12months'          ] },
    ];
  %]
[% END %]

The key in that hash defines the parameter by which that value can be accessed later. Each field has to point to an array with at least 3 values. The required flag is optional.

  • Name

  • Type

  • Default value

  • Required Flag

Prepare Block

The prepare block is used to gather data used in the report. Usually it’s been used to get availability data via livestatus or fetch events from logfiles.

For example:

[% IF block == 'prepare' %]
  [%
    set_unavailable_states(param.unavailable);
    calculate_availability();
  %]
[% END %]

You may use all functions available from the Render Helper

Render Block

The render block defines the layout of the report.

For example:

[% IF block == 'render' %]
  [% PROCESS 'reports/locale/en.tt' %]
  [%
    title        = 'Hello World Report'
    subtitle     = r.name
    coverdetails = [ 'Report Timeperiod:', get_report_timeperiod(start, end, reportDateFormat),
                     'Host:', param.host,
                   ]
  %]

  [% WRAPPER 'reports/comp/report.tt' %]
    [% WRAPPER 'reports/comp/page_cover.tt' %][% END %]
    [% WRAPPER 'reports/comp/page_standard.tt' %]
      <b>hello world</b>
    [% END %]
  [% END %]
[% END %]

This report would contain 2 pages. First you wrap everything into the report.tt which defines the basic report layout and contains the stylesheets. Then you have to define some settings used by the cover page. r is a reference to the report object, in this example we just use that as subtitle. coverdetails is a list of name / value pairs used in the details list on the cover page. Finally we wrap a hello world in a standard page. The content could contain standard HTML tags like tables and images as well.

Mail Block

Finally we define a mail block which is used for the mail layout if a report is sent by e-mail.

For example:

[% IF block == 'mail' +%]
SUBJECT: Report: [% r.name %]

Your report is attached.

[% IF r.desc %]Description: [%+ r.desc %][% END %]
[%+ END %]

We use the report name r.name as subject and put the report description r.desc in the body.

Custom Perl Render Helper

You may create a Perl Module named Thruk::Utils::Reports::CustomRender which will be automatically made available to the stash, so functions from this module can be used in the reporting templates.

Business Process

Thruk comes with a business process engine which allows you to design, operate and illustrate your business intelligence models. It is possible to span business prosesses across multiple sites, but the calculation will happen all in one single local site. Thruk needs to able to access the check_results folder to submit results and a config file to create hosts and services for your objects.

In edit mode nodes can be changed by right-clicking them and select Edit Node. General options, like using hard or soft states can be changed in the General Tab of any node.

How it works

Business Process Workflow

Business Process Architecture

  • Thruks Business Process Engine is usually by cron, which is automatically added if you create your first business process.

  • Business Process Config and Runtime Status are JSON files which are used to store objects and status

  • Results are submited to Nagios by using the check_results folder.

  • Livestatus will be used to fetch input data for the calculation itself.

Setup

If you are not using OMD, you will have to do some manual steps:

  • copy the <Component Thruk::Plugin::BP> block from your /etc/thruk/thruk.conf into your thruk_local.conf in order to make some local adjustments.

  • copy the Thruk BP Templates from either github or /usr/share/thruk/thruk_templates.cfg into your nagios conf.d folder, ex .:/etc/nagios/conf.d/thruk_bp_templates.cfg

  • change the objects_templates_file to that location.

  • create an empty file thruk_bp_generated.cfg, ex.: /etc/nagios/conf.d/thruk_bp_generated.cfg

  • change objects_save_file to that location.

  • make that file editable for the webserver/thruk user. ex.: chown www-data: /etc/nagios/conf.d/thruk_bp_generated.cfg

  • change the spool_dir to ex.: /var/nagios/rw/check_results or whatever your nagios check_result_path points too.

  • make sure your webserver/thruk user is allowed to write to that folder, ex.: chgrp www-data /var/nagios/rw/check_results && chmod g+rwx /var/nagios/rw/check_results

  • set objects_reload_cmd to "sudo -u nagios /etc/init.d/nagios reload". You will need to add sudo permissions to get this working. ex.: put "thruk ALL=(ALL) NOPASSWD: /etc/init.d/nagios" in your /etc/sudoers.

Integration in Nagios

Thruk will create Hosts and Services for selected nodes of your business model. That way it is possible to send notifications, even for sub steps of the business process. The business process itself will be created as Nagios Host object and all selected nodes will be created as Service objects.

To make that work, you will need 2 files:

Both file locations can be changed in your thruk_local.conf. The templates file is user maintained, while the objects file will be overwritten by Thruk whenever you change a business process. Also make sure Thruk is allowed to write the objects file.

In order to make integration as simple as possible, you can only choose from different templates which can be edited by the Thruk Config Tool or manually in the objects_templates_file file.

The name of the business process will be used as hostname, so make sure it does not clash with existing hosts by using a uniq prefix or something similar.

Tip
notifications

As each business process will result in at least one host and one service. It is recommended to setup notifications on the service, because the host will almost always be ok and only change the state if some internal error happens.

Creating New Business Processes

In order to change or create business processes, you need these roles from the cgi.cfg.

  • authorized_for_configuration_information

  • authorized_for_system_commands

Then simply model your business process from the web gui.

Configuration

Configuration settings are covered by the configuration sections.

CLI Tool & Scripting

It is possible to do comprehensive scripting with Thruk. For example set downtimes automatically, schedule reports or change the objects of your monitoring core configuration.

See http://thruk.org/api/Thruk/Utils/CLI.html for details and examples.

There are also some scripting examples in the examples folder.

Integration With 3rd Party Software

Configuration Generators

If your Nagios configuration is generated by a 3rd party config generator, you can export some additional information which is then displayed in the configuration section of each host or service. Just export the special custom variables: _SRC, _SRC2 or _SRC3. 3rd Party Config Generator