Introduction

Thruk is a multibackend monitoring webinterface which currently supports Nagios, Icinga and Shinken as backend using the Livestatus API. It is designed to be a dropin replacement and covers almost all of the original features plus adds additional enhancements for large installations.

Main Features / Advantages

  • multiple backends

  • faster, no need to parse status.dat for every request

  • less cpu usage

  • display live data, no delay between core and gui

  • independant from monitoring core, can be installed on remote host

  • clusterable, can be clustered over hosts

  • change status.cgi filter inline

  • extended logfile search

  • better logging, uses log4perl (who submitted which command?)

  • valid HTML code

  • no frames ( can be reenabled for better addon integration)

  • easy to extend with plugins

  • 100% Perl (using Catalyst Framework)

  • paging ( can be disabled and customized )

  • multiple themes included

  • Excel export for status and logfiles

  • Adjustable side menu

  • Full expanded plugin commandline for easy testing

  • Save searches like bookmarks

  • Config Tool included

  • Mobile interface included

  • SLA Reports in PDF format

  • Recurring Downtimes

  • Fully Featured Dashboard

How it works

Thruk is written in Perl using the Catalyst Framework. Backend monitoring systems will be connected with the Monitoring::Livestatus Perl Module. Thruk itself is running as a fastcgi process. Availability will be calculated with Monitoring::Availability. Authentication is provided by the Apache webserver (For example with mod_auth).

Thruk Architecture

Thruk Architecture

What makes Thruk cool

There are a couple of cool things in Thruk. A few of them are listed here.

Excel Export and Bookmarks

The Bookmarks feature allows you to save your searches and add them as your personal menu item. Besides the bookmarks, you can adjust the menu easiely with the menu_local.conf for all users.

The Excel export creates real Excel files which can be send by mail or used to extract hostnames.

Bookmarks and Excel Export

Config Tool

The Config Tool is great to make quick changes to your Thruk configuration. It is also possible to manage access with htpasswd files as well as adjusting your cgi.cfg.

Config Tool

Sending Multiple Commands

The new status pages make it very convenient to send multiple commands at once. It is even possible to send host and service commands at the same time. When rescheduling hosts and services, Thruk will wait until your check is finished and display the result as soon as the check is over.

Multiple Commands

Easy Filtering

Remember the days when you had to guess numbers in the url to filter hosts or services. With Thruk it’s possible to quickly change your display filter. You can combine multiple filter to create whatever views you like. An Ajax search supports you, so you don’t have to guess host or service names.

Easy Filtering

PNP4Nagios Graphs

When your action_url contains /pnp4nagios/, there will be automatically a graph displayed for your host and service. This gives you a quick view about the performance history. The image is then linked to PNP4Nagios to get detailed information.

PNP4Nagios

Multiple Lines of Plugin Output

When your check returns multiple lines of plugin output. Thruk marks the output in blue and a click on it displays the complete output. This is especially usefull for check_multi checks. In addition to that, the comments and downtimes also have a small popup with their data. So you don’t have to open the host/service page just to see who set a comment and when there is a downtime.

Multiple Lines of Plugin Output

Mine Map

The Mine Map is the perfect tool to get a quick overview. It is especially usefull if you have a lot of common services across your hosts. Otherwise use hostgroups or servicegroups for nice results. Normal filtering is possible too.

Mine Map

Mobile Interface

The Mobile interface gives you access to the most important things and allows you to quickly view and acknowledge problems.

Mobile Interface

Installation

There are several ways of installing Thruk.

Labs Consol Repository

The Labs Repository provides packages of latest Thruk releases (including daily development builds) and other Tools like Mod-Gearman. Just follow the guide on the labs page for your distribution.

Use OMD

An easy way of installing thruk is using OMD from omdistro.org. There are Debian, Ubuntu, Centos and Suse Packages containing preconfigured latest versions of Nagios and Thruk. The package also includes icinga, shinken, pnp4nagios, check_mk and nagvis. OMD is the recommended way of installing new Nagios / Thruk setups when you want to install several addons at once.

Use Packages

Standalone installation from a binary package is another easy way to get Thruk running.

Download packages from http://www.thruk.org/files/pkg/

All packages have the following filesystem structure:

  /etc/thruk                       thruks config
  /etc/httpd/conf.d/thruk.conf     apache config
  /usr/share/thruk                 shared files
  /usr/lib/thruk/perl5             perl librarys
  /var/cache/thruk                 temporary files
  /var/lib/thruk                   stored user settings

After installation, Thruk is available at http://your-host/thruk/ and has a default user thrukadmin with password thrukadmin configured. You may need to change backend configuration. This should be done in the thruk_local.conf where all settings can be overriden.

Debian / Ubuntu

  #> dpkg -i thruk_1.58_debian6_amd64.deb

In case of dependency errors, run apt-get -f install and try the dpkg -i… again.

Centos / Redhat

  #> yum install --nogpgcheck thruk-1.58-1.rhel6.x86_64.rpm

You may need to include an external repository for mod_fastcgi module. Epel or Rpmforge should do it.

SLES

  #> zypper install thruk-1.58-1.sles11.x86_64.rpm

You may need the SLES sdk dvd for additional dependencies.

Install from Source

Requirements

Important
Experienced Users Only

Source installation is for experienced users only. Using packages/repositories should be the prefered solution in almost all situations. Especially for productional usage.

In order to install the Thruk Monitoring Webinterface from source you will need the following:

  • Perl

  • Git Client

  • compiler tools: automake, make, g++, gcc

  • Apache Webserver (optional for fastcgi only)

Create New User

refer to your systems manual on howto add new user. This guide uses the following:

  user:  thruk
  group: thruk
Important all following steps should be done by the thruk user.

Install Local::Lib

quick guide:

  %> su - thruk
  %> wget %http://search.cpan.org/CPAN/authors/id/A/AP/APEIRON/local-lib-1.008004.tar.gz
  %> tar zxf local-lib-1.008004.tar.gz
  %> cd local-lib-1.008004
  %> perl Makefile.PL --bootstrap && make install
Tip
proxy configuration

if you need a proxy configuration, you should answer <no> at this question:

Would you like me to configure as much as possible automatically? [yes]

Setting a proper http_proxy/ftp_proxy environment should work for the automatic configuration.

Add the following line to the thruk users .profile or .bashrc

eval $(perl -I$HOME/perl5/lib/perl5 -Mlocal::lib)

logout and login again and verify that your perl env looks like this:

 %> env | grep perl
 PERL5LIB=/home/thruk/perl5/lib/perl5:/home/thruk/perl5/lib/perl5/x86_64-linux-gnu-thread-multi
 MODULEBUILDRC=/home/thruk/perl5/.modulebuildrc
 PATH=/home/thruk/perl5/bin:/usr/local/bin:/usr/bin:/bin:/usr/games
 PERL_MM_OPT=INSTALL_BASE=/home/thruk/perl5

Install Module::Install

  perl -MCPAN -e 'install Module::Install::Catalyst'

Git Clone Thruk

    git clone https://github.com/sni/Thruk.git
    cd Thruk
    perl Makefile.PL

press <enter> here:

  ==> Auto-install the 35 mandatory module(s) from CPAN? [y]

then run make

    make

This may take a while, as there are probably several modules missing.

Tip
yes

you can use /usr/bin/yes to automatically answer all questions with yes. (yes | make)

Press <y> at this questions:

  Do you want to build the XS Stash module? [y]
  Do you want to use the XS Stash by default? [y]

run perl Makefile.PL again to see if all dependecies are now installed properly.

    perl Makefile.PL

Install Livestatus

Refer to http://mathias-kettner.de/checkmk_livestatus.html#H1:%20Setting%20up%20and%20using%20Livestatus on how to install livestatus onto your monitoring box.

Basically you have to build the NDO addon and add it to your nagios/icinga.cfg.

 broker_module=/opt/local/livestatus/livestatus.o /tmp/live.sock

Configuration

Copy thruk.conf to thruk_local.conf and adjust the livestatus settings to your needs.

  %> cp thruk.conf thruk_local.conf

The thruk.conf will be overwritten with new defaults on updates. The thruk_local.conf contains the local overrides and will never be overwritten.

Edit cgi.conf and adjust settings to your needs.

See the [Configuration] section for detailed explaination of configuration options.

Start Server

After running these steps successfully, you should be able to test your installation:

    ./script/thruk_server.pl

Open your browser and open http://<your_host>:3000

Apache Configuration

Integration in the Apache webserver is done by fastcgi. There are two fastcgi modules for apache at the moment. Choose the one which fits best into your environment. If unsure, use mod_fcgid. The main difference is, that mod_fcgid starts the fastcgi process upon the first request whereas in mod_fastcgi you have to start the fastcgi process by yourself.

Do not use the port 3000 thruk server in production, it’s only for testing and development. It cannot handle authentication and is slower than the fastcgi variants.

Tip
lighthttpd

Francois Ponsard wrote an article on how to integrate Thruk in Lighthttpd: http://www.dahwa.fr/dotclear/index.php?post/2011/03/15/Thruk-in-Lighttpd

mod_fastcgi

start your fcgi server:

    %>./script/thruk_fastcgi.pl -n 5 \
                -l /tmp/thruk_fastcgi.socket \
                -p /tmp/thruk_fastcgi.pid

you may want to copy the init.d script to /etc/init.d and adjust its paths:

    %> sudo cp ./script/thruk_fastcgi_server.sh /etc/init.d/thruk_fastcgi_server
    %> vi /etc/init.d/thruk_fastcgi_server
    %> sudo chown root: /etc/init.d/thruk_fastcgi_server

Or create a custom init.d script (additional modules required) with:

    %> ./script/thruk_create.pl FastCGI::ExternalServer l=/tmp/thruk_fastcgi.socket n=5 p=/tmp/thruk_fastcgi.pid

Apache Configuration

use this apache example configuration:

  • replace /home/thruk/Thruk with your installation path

  • replace your-web-host.local with your hostname

  • create a /home/thruk/Thruk/htpasswd.users with htpasswd2

  • make sure the /home/thruk/Thruk/logs/ directory exists

Apache configuration within existing vhost
<VirtualHost *:80>
    # ... existing configuration

    # thruk configuration
    <Directory /home/thruk/Thruk/root/>
        order allow,deny
        allow from all
        Options FollowSymLinks
        AllowOverride All
    </Directory>

    Alias /thruk/ /home/thruk/Thruk/root/thruk/

    # authorization
    <Location "/thruk">
        AuthName "Monitoring Access"
        AuthType Basic
        AuthUserFile /home/thruk/Thruk/htpasswd.users
        Order Allow,Deny
        Allow from all
        require valid-user
    </Location>

    # Load fastcgi module unless already loaded
    LoadModule fastcgi_module /usr/lib/apache2/modules/mod_fastcgi.so

    # fastcgi configuration
    FastCGIExternalServer /tmp/thruk_fastcgi.fcgi -socket /tmp/thruk_fastcgi.socket -idle-timeout 120

    # Load rewrite module unless already loaded
    LoadModule rewrite_module /usr/lib/apache2/modules/mod_rewrite.so

    # rewrite configuration
    RewriteEngine On
    RewriteCond %{REQUEST_FILENAME} !-f
    RewriteRule ^/thruk(.*)$ /tmp/thruk_fastcgi.fcgi/thruk$1 [E=HTTP_AUTHORIZATION:%{HTTP:Authorization},QSA,L]

</VirtualHost>
Apache configuration with own vhost
<VirtualHost *:80>
    ServerName   thruk.your-host.local

    DocumentRoot /home/thruk/Thruk/root/
    CustomLog    /home/thruk/Thruk/logs/access.log combined
    ErrorLog     /home/thruk/Thruk/logs/error.log

    <Directory />
        order deny,allow
        deny from all
    </Directory>

    <Directory /home/thruk/Thruk/root/>
        Options FollowSymLinks
        AllowOverride All
        order allow,deny
        allow from all
    </Directory>

    # authorization
    <Location "/">
        AuthName "Monitoring Access"
        AuthType Basic
        AuthUserFile /home/thruk/Thruk/htpasswd.users
        Order Allow,Deny
        Allow from all
        require valid-user
    </Location>

    # Load fastcgi module unless already loaded
    LoadModule fastcgi_module /usr/lib/apache2/modules/mod_fastcgi.so

    # fastcgi configuration
    FastCGIExternalServer /tmp/thruk_fastcgi.fcgi -socket /tmp/thruk_fastcgi.socket -idle-timeout 120

    # Load rewrite module unless already loaded
    LoadModule rewrite_module /usr/lib/apache2/modules/mod_rewrite.so

    # rewrite configuration
    RewriteEngine On
    RewriteCond %{REQUEST_FILENAME} !-f
    RewriteRule ^/(.*)$ /tmp/thruk_fastcgi.fcgi/$1 [E=HTTP_AUTHORIZATION:%{HTTP:Authorization},QSA,L]
</VirtualHost>

mod_fcgid

Important Thruk runs with the webserver user in this scenario, make sure the webserver user has access to all Thruk files and perl modules.
Apache configuration with mod_fcgid
LoadModule fcgid_module /usr/lib/apache2/modules/mod_fcgid.so
<VirtualHost *:80>
    ServerName   thruk.company.local

    DocumentRoot /home/thruk/Thruk/root/
    CustomLog    /home/thruk/Thruk/logs/access.log combined
    ErrorLog     /home/thruk/Thruk/logs/error.log

    <Directory /home/thruk/Thruk/root/>
        Options FollowSymLinks
        AllowOverride All
        order allow,deny
        allow from all
    </Directory>

    AliasMatch /thruk/(.*\.cgi|.*\.html)  /home/thruk/Thruk/script/thruk_fastcgi.pl/thruk/$1
    <Location /thruk>
        Options ExecCGI
        Order allow,deny
        Allow from all
        AuthName "Monitoring Access"
        AuthType Basic
        AuthUserFile /home/thruk/Thruk/htpasswd.users
        Require valid-user
    </Location>

    <IfModule mod_fcgid.c>
      AddHandler fcgid-script .pl
      MaxRequestsPerProcess 100
    </IfModule>

</VirtualHost>

Configuration

Configuration is managed mainly in these configuration files.

  • thruk.conf

  • thruk_local.conf

  • cgi.cfg

  • log4perl.conf

  • menu.conf

  • menu_local.conf

thruk.conf

The thruk.conf contains the shipped defaults. You should not edit this file directly. Overwrite your settings in your thruk_local.conf instead.

thruk_local.conf

The thruk_local.conf is a copy of the thruk.conf. Both files are used by thruk. The idea is, that the thruk.conf is provided with the packaged defaults and the thruk_local.conf is to override these settings where needed.

use_timezone

Changes the timezone from the systems default to this timezone. Only set this if you have trouble with displaying the right timestamps.

ex.:

use_timezone = CET

title_prefix

set the title prefix for all urls this piece of text will be prepended to all page titles.

ex.:

title_prefix = Prod

url_prefix

Changes the usual url path for Thruk. Don’t change it unless you plan to run multiple Thruk instances on the same webserver. You will have to change your fastcgi configuration too.

ex.:

url_prefix = /

logo_path_prefix

Changes the path to your logo images. Default is $url_prefix+thruk/themes/$current_theme/images/logos/ and therefor relative to the current selected theme. You could set a fixed path here. Like usual, paths starting with a / will be absolute from your webserver root directory. Paths starting without a / will be relative to the cgi directory.

ex.:

logo_path_prefix = /icons/

physical_logo_path

Location of your logos in your filesystem. This directory should be mapped to your logo_path_prefix directory where logo_path_prefix is the path relative to your webserver root directory and physical_logo_path is the corresponding filesystem path.

ex.:

physical_logo_path = /usr/share/icons/

host_action_icon

Change path to your host action icons. You may use relative paths to specify completely different location. You also may want to use action_pnp.png when using pnp. Icon can be overriden by a custom variable _ACTION_ICON.

ex.:

host_action_icon = action.gif

service_action_icon

Change path to your service action icons. You may use relative paths to specify completely different location. You also may want to use action_pnp.png when using pnp. Icon can be overriden by a custom variable _ACTION_ICON.

ex.:

service_action_icon = action.gif

use_strict_host_authorization

When set to a true value, every contact will only see the hosts where he is contact for plus the services where he is contact for. When disabled, a host contact will see all services for this host regardless of wheter he is a service contact or not.

ex.:

use_strict_host_authorization = 1

make_auth_user_lowercase

Convert authenticated username to lowercase.

ex.:

make_auth_user_lowercase = 1

make_auth_user_uppercase

Convert authenticated username to uppercase.

ex.:

make_auth_user_uppercase = 1

start_page

This link is used as startpage and points usually to the main.html with displays version information and general links.

ex.:

start_page = /thruk/main.html

This link is used in the side navigation menu as link to the documentation. Replace it with whatever your documentation is located. Set it to a blank value if you don’t want a documentation link in the menu at all.

ex.:

documentation_link = /thruk/docs/

Customizable link for the problems link in side menu. Can be useful to reflect your companys process of error handling.

ex.:

all_problems_link = /thruk/cgi-bin/status.cgi?...

List of allowed patterns, where links inside frames can be set to. You can link to /thruk/frame.html?link=http://wiki.my-company.com/page/blah Your wiki will then be displayed with the Thruk navigation frame. Useful for other addons, so they don’t have to display a own navigation.

ex.:

allowed_frame_links = http://intranet.my-company.com
allowed_frame_links = https://wiki.my-company.com

initial_menu_state

Set initial menu state.

closed  => 0
open    => 1

ex.:

<initial_menu_state>
  General        = 1
  Current_Status = 1
  Reports        = 1
  System         = 1
  Bookmarks      = 1
</initial_menu_state>

cgi_cfg

The path to your cgi.cfg. See cgi.cfg for details.

ex.:

cgi_cfg = cgi.cfg

log4perl_conf

The path to your log4perl configuration file.

ex.:

log4perl_conf = ./log4perl.conf

plugin_path

Path to your plugins directory. Can be used to specify different location for you Thruk plugins. Don’t forget to set appropriate apache alias or rewrite rules when changing the plugin path. Otherwise the static content from plugins is not accessible.

Example redirect rule for apache:

AliasMatch /thruk/plugins/(.*?)/(.*)$ /plugins/plugins-enabled/$1/root/$2

ex.:

plugin_path = ./plugins

themes_path

Path to your themes directory. Can be used to specify different location for you Thruk themes. Don’t forget to set appropriate apache alias or rewrite rules when changing the themes path. Otherwise the static content from your themes may not accessible.

Alias /thruk/themes/ /themes/themes-enabled/

ex.:

themes_path = ./themes

var_path

Path to the var directory. Thruk stores user specific date here.

ex.:

var_path = ./var

tmp_path

Path to a temporary directory. Defaults to /tmp if not set and usually this is a good place.

ex.:

tmp_path = /tmp

ssi_path

The path to your ssi (server side includes) files. See Server Side Includes for details.

ex.:

ssi_path = ssi/

user_template_path

Specify a additional directory for user supplied templates. This makes it easy to override thruks own templates. Template search order is:

  • users template path

  • plugins template path

  • themes template path

  • thruks template path

ex.:

user_template_path = ./my_templates

delay_pages_after_backend_reload

Delay the page delivery until the backends uptime is at least this amount of seconds. Displaying pages soon after backend restarts may display wrong results and all services are pending. Enable this if you experience problems with pending services after reloading your backend. Should be obsolete with Livestatus versions greater than 1.2 ex.: setting this to 10 would start serving pages 10 seconds after the backend reload

ex.:

delay_pages_after_backend_reload = 10

use_frames

Set whether you want to use a framed navigation or not. With using frames it’s sometimes easier to include addons. See allowed_frame_links option for how to integrate addons.

ex.:

use_frames = 0

strict_passive_mode

Normally passive checks would be marked as disabled. With this option set, disabled checks will only be displayed as disabled if their last result was active. Otherwise they would be marked as passive checks. This option also changes the passive icon only to be shown when the last check was passive, otherwise the disabled icon will be displayed.

ex.:

strict_passive_mode = 1

Use the old or the classic search from the navigation. The new search supports regular expressions and searches in many attributes. For example: plugin_output, groups, names, descriptions

ex.:

use_new_search = 1

use_new_command_box

Show the new split command box on the host / service details page.

ex.:

use_new_command_box = 1

Enables the ajax search field. There will be suggestions while typing into the search field.

ex.:

use_ajax_search = 1

ajax_search_hosts

Enables the suggestion of hosts in the ajax search field. Depending on the number of hosts, this can make the search slow.

ex.:

ajax_search_hosts = 1

ajax_search_hostgroups

Enables the suggestion of hostgroups in the ajax search field.

ex.:

ajax_search_hostgroups = 1

ajax_search_services

Enables the suggestion of services in the ajax search field. Depending on the number of services, this can make the search slow.

ex.:

ajax_search_services = 1

ajax_search_servicegroups

Enables the suggestion of servicegroups in the ajax search field.

ex.:

ajax_search_servicegroups = 1

ajax_search_timeperiods

Enables the suggestion of timeperiods in the ajax search field. Timeperiods will only displayed when filtering by check- or notification period.

ex.:

ajax_search_timeperiods = 1

default_theme

Default theme to use for all users. Must be a valid subdirectory in the themes folder.

ex.:

default_theme = Thruk

Path used for cookies. Do not change unless you have weird url rewrites which breaks setting cookies.

ex.:

cookie_path = /

use_pager

Using the pager will make huge pages much faster as most people don’t want a services page with 100.000 services displayed. Can be disabled if you don’t need it.

ex.:

use_pager = 1

paging_steps

Define the selectable paging steps. Use the * to set the default selected value.

ex.:

paging_steps = *100, 500, 1000, all

group_paging_overview

Just like the paging_steps, but only for the groups overview page.

ex.:

group_paging_overview =  *3,  10, 100, all

group_paging_summary

Just like the paging_steps, but only for the groups summary page.

ex.:

group_paging_summary = *10, 50, 100, all

group_paging_grid

Just like the paging_steps, but only for the groups grip page.

ex.:

group_paging_grid = *5,  10, 50,  all

can_submit_commands

Set this if a contact should be allowed to send commands unless defined for the contact itself. This is the default value for all contacts unless the user has a can_submit_commands setting in your monitoring configuration.

ex.:

can_submit_commands = 1

command_disabled

Use this to disabled specific commands. Can be use multiple times to disabled multiple commands. The number can be found in the cmd_typ cgi parameter from links to the command page. You may use ranges here.

ex.:

command_disabled = 14
command_disabled = 35
command_disabled = 17-34,50-65

cmd_defaults

Set the default checked state for command options.

ex.:

<cmd_defaults>
  ahas                   = 0  # For Hosts Too
  broadcast_notification = 0  # Broadcast
  force_check            = 0  # Forced Check
  force_notification     = 0  # Forced Notification
  send_notification      = 1  # Send Notification
  sticky_ack             = 1  # Sticky Acknowledgement
  persistent_comments    = 1  # Persistent Comments
  persistent_ack         = 0  # Persistent Acknowledgement Comments
  ptc                    = 0  # For Child Hosts Too
  use_expire             = 0  # Use expire time ( for cores which support it)
</cmd_defaults>

downtime_duration

Default duration of new downtimes in seconds. Default is 2 hours.

ex.:

downtime_duration = 7200

expire_ack_duration

Default duration of acknowledgements with expire date. Default is one day.

ex.:

expire_ack_duration = 86400

cmd_quick_status

Configure which commands should be available as quick status commands.

ex.:

<cmd_quick_status>
  reschedule             = 0  # Reschedule next check
  downtime               = 0  # Add/remove downtimes
  comment                = 0  # Add/remove comments
  acknowledgement        = 0  # Add/remove acknowledgements
  active_checks          = 0  # Enable/disable active checks
  notifications          = 0  # Enable/disable notifications
  submit_result          = 0  # Submit passive check result
  reset_attributes       = 0  # Reset modified attributes
</cmd_quick_status>

command_reschedule_alias

When you want to reschedule passive checks for which the result is fetched by an agent (For example check_mk or some scenarios of check_multi). You usually want to reschedule the agent instead of the passive check.

The command reschedule alias can be used to translate the reschedule command from the passive service to the active agent service.

command_reschedule_alias = pattern;master_service_description
Notes
  • The pattern will be tested against the service description and the command_name of the passive check.

  • The resulting service name be on the same host and the contact must be authorized for that service too.

  • The pattern must be a valid perl regular expression.

  • Duplicates will be removed. So if you reschedule 10 services which result in the same master service will only trigger one reschedule.

  • Only passive services will be translated

In this example, all passive check_mk checks will trigger the active agent check and therefor allow you to reschedule passive checks directly from the problems page.

ex.:

command_reschedule_alias = ^check_mk\-(?!inventory);Check_MK

datetime_format

Default timeformat. Use POSIX format.

ex.:

datetime_format = %Y-%m-%d  %H:%M:%S

datetime_format_long

Default long timeformat.

ex.:

datetime_format_long = %a %b %e %H:%M:%S %Z %Y

datetime_format_log

Default log timeformat.

ex.:

datetime_format_log = %B %d, %Y  %H

Default trends timeformat.

ex.:

datetime_format_trends = %a %b %e %H:%M:%S %Y

datetime_format_today

Default timeformat for todays date. Can be useful if you want a shorter date format for today.

ex.:

datetime_format_today = %H:%M:%S

show_long_plugin_output

When a plugin returns more than one line of output, the output can be displayed directly in the status table, as popup or not at all. Choose between popup, inline and off

ex.:

show_long_plugin_output = popup

show_modified_attributes

Show if a host / service has modified attributes.

ex.:

show_modified_attributes = 1

info_popup_event_type

On which event should the comments / downtime or longpluginout popup show up. Valid values are onclick or onmouseover.

ex.:

info_popup_event_type = onmouseover

info_popup_options

Options for the popup window used for long pluginoutput, downtimes and comments. See http://www.bosrup.com/web/overlib/?Command_Reference for what options are available

ex.:

info_popup_options = STICKY,CLOSECLICK,HAUTO,MOUSEOFF

show_notification_number

Display the current number of notification after the current / max attempts on the status details page.

ex.:

show_notification_number = 0

show_backends_in_table

Display the backend/site name in the status table. This is useful if you have same hosts or services on different backends and need to know which one returns an error. Valid values are:

1 - show site name at the end
2 - put site name in front

ex.:

show_backends_in_table = 0

show_config_edit_buttons

Show links to config tool for each host / service. You need to have the config tool plugin enabled and you need proper permissions for the link to appear.

ex.:

show_config_edit_buttons = 1

show_full_commandline

Display the full command line for host / service checks . Be warned, the command line could contain passwords and other confidential data. In order to replace the user macros for commands, you have to set the resource_file in your peer config or the general resource_file option.

  • 0 = off, don’t show the command line at all

  • 1 = show them for contacts with the role: authorized_for_configuration_information

  • 2 = show them for everyone

ex.:

show_full_commandline = 0

resource_file

Set a general resource file. Make sure it does not contain any passwords or any other data which should not be displayed. Instead of using a general resource_file you could define one file per peer in your peer config.

ex.:

resource_file = /etc/nagios3/resource.cfg

perf_bar_mode

This option enables a performance bar inside the status/host list which create a graph from the performance data of the plugin output. Available options are match, first, all, worst and off.

match: try to set graph which matches the output
all: graph all performance values available
first: graph only the first performance value
worst: graph only the graph for the worst state
off: graph no value at all

ex.:

perf_bar_mode = match

shown_inline_pnp

Show inline pnp graph if available. If a service or host has a pnp4nagios action or notes url set. Thruk will show a inline graph on the extinfo page. This works for /pnp4nagios/ urls and /pnp/.

ex.:

shown_inline_pnp = 1

show_custom_vars

Show custom vars in host / service ext info. List variable names to display in the host and service extinfo details page. Can be specified more than once to define multiple variables. You may use html in your variables. Use * as wildcard, ex.: _VAR*

ex.:

show_custom_vars = _VAR1

statusmap_default_type

You may change the default map type of the statusmap here. Valid types are: table and circle

ex.:

statusmap_default_type = table

statusmap_default_groupby

And the statusmap default group by which has to be one of: parent, address, domain, hostgroup, servicegroup

ex.:

statusmap_default_groupby = address

use_wait_feature

Waiting is a livestatus feature. When enabled, Thruk will wait after rescheduling hosts/services checks until the check has been really executed up to a maximum of 10 seconds. Adjust the time waiting with the wait_timeout option.

ex.:

use_wait_feature = 1

wait_timeout

Amount of seconds to wait until a rescheduled check finishes. Thruk will wait this amount and display the result immediately.

ex.:

wait_timeout = 10

mobile_agent

Specify user agents which will be redirected to the mobile plugin (if enabled).

ex.:

mobile_agent=iPhone,Android,IEMobile

show_error_reports

Show link to bug reports when internal errors occur.

ex.: show_error_reports = 1

skip_js_errors

don’t report some known harmless javascript errors

ex.: skip_js_errors = cluetip is not a function

no_external_job_forks

Normally reports will be generated in an external process to avoid timeouts on long running reports. Use this switch to turn external jobs off and generate reports directly. Make sure they are finished within 40seconds which is the default fcgi timeout.

ex.:

no_external_job_forks = 1

cron_file

Specifiy a file which is then completly under the control of Thruk. It will be used to store cronjobs, ex. for reports. The file has to be writable by Thruk.

ex.:

cron_file = /tmp/thruk_cron.tmp

cron_pre_edit_cmd

The pre edit cmd can be used to do run a command just before Thruk will edit the crontab.

ex.:

cron_pre_edit_cmd  = /usr/bin/crontab -l > /tmp/thruk_cron.tmp

cron_post_edit_cmd

The post edit cmd is necessary for OMD where you need to reload the crontab after editing or for replacing the users cron with the edited file.

ex.:

cron_post_edit_cmd = crontab /tmp/thruk_cron.tmp

thruk_bin

Path to your thruk executable. Will be used in cronjobs.

ex.:

thruk_bin = /usr/bin/thruk

thruk_init

Path to your thruk init script. Will be used to restart thruk.

ex.:

thruk_init = /etc/init.d/thruk

report_nice_level

Execute regular scheduled reports with this nice level.

ex.:

report_nice_level = 5

report_base_url

Url used to replace relative links in html reports.

ex.:

report_base_url = http://host.local/thruk/cgi-bin/

first_day_of_week

Set first day of week. Used in reports. Sunday: 0 Monday: 1

ex.:

first_day_of_week = 0

logcache

Enables caching logfiles for faster access. Cache supports MongoDB only. Format is a MongoDB connection string like hostname:port/db. Using a cache dramatically decreases cpu and memory usage of Thruk when accessing logfiles, for example when creating reports.

ex.:

logcache = localhost:27017/thruk_log_cache

Specifies the url where non-authenticated users will be redirected too.

ex.:

cookie_auth_login_url = thruk/cgi-bin/login.cgi

Specifies the url against the cookie auth provider will verifys the credentials.

ex.:

cookie_auth_restricted_url = http://localhost/thruk/cgi-bin/restricted.cgi

Specifies the timeout for idle sessions.

ex.:

cookie_auth_session_timeout       = 86400

Specifies the amount of seconds in which subsequent requests won’t verify authentication again.

ex.:

cookie_auth_session_cache_timeout = 5

enable_shinken_features

This one activates all problem/impact and criticity features. Currently it will only work with shinken backends. Dont enable it unless all your backends are shinken. If not set, it will be automatically enabled when using only shinken backends.

ex.:

enable_shinken_features = 1

priorities

Set the names of the priority (criticity in shinken). Currently this will only work with shinken backends.

ex.:

<priorities>
  5   = Business Critical
  4   = Top Production
  3   = Production
  2   = Standard
  1   = Testing
  0   = Development
</priorities>

enable_icinga_features

This one activates all icinga specific features. If not set, Thruk will try to auto-detect your backends. Currently autodetection will only work within OMD. Dont enable it unless all your backends are icinga.

ex.:

enable_icinga_features = 1

check_local_states

Get the status for remote backends from local instances. This can increase performance when using multiple remote sites. It is enabled by default when using more than one site. You have to define hostchecks in any local backend (using unix sockets) with a name or alias of the address of your remote backends.

ex.:

check_local_states = 1

backend_debug

Set logging of backend in verbose mode. This only makes sense when debug logging is activated.

ex.:

backend_debug = 1

connection_pool_size

Use connection pool when accessing multiple sites. Increases the performance because backends will be queried parallel but uses around 50mb of memory per pool member. Disabled when set to 0. (experimental feature)

ex.:

connection_pool_size = 0

Component Thruk::Backend

Enter your backend connection settings here. At the moment only livestatus is supported. The port is the port from the xinetd.conf. You can enter local unix sockets too.

  • peer

    • name name for this connection

    • type type of this connection. Only livestatus possible at the moment

    • hidden should this peer be hidden initially ( can be reenabled via gui switch ) Only useful with more than one backend.

    • groups if set, only contacts from these groups have access. You may add multiple groups seperated by comma. Users without the right contactgroup don’t even see that there is a backend. Note that this implies one extra backend request per page.

    • section to group backends/sites by different sections, enter a section.

    • options

      • peer address of this connection.

      • resource_file resource_file for this peer (used for macro replacement)

    • configtool

      • core_type Give the config parser a hint about your config. Can be nagios, icinga or shinken.

      • core_conf Path to your nagios.cfg / icinga.cfg. Read all object directories and files from this config file.

      • obj_check_cmd Commandline to verify the config.

      • obj_reload_cmd Commandline to reload the config.

      • obj_readonly Filename pattern to define readonly objects. For example for generated config files.

      • obj_dir Path to your objects. Enables the objects editor. Reads all *.cfg from this folder and all subfolders. (only needed when not using core_conf)

      • obj_file Path to a single objects file. Enables the objects editor. Both obj_dir and obj_file can be specified more than once. (only needed when not using core_conf)

      • obj_exclude Specify some exection pattern for the obj_dir. (only needed when not using core_conf)

ex.:

<Component Thruk::Backend>
  <peer>
      name   = Local Nagios
      type   = livestatus
      hidden = 1             # makes this backend hidden by default
      groups = admins,locals # makes this backend only visible to the
                             # admin and the locals contactgroup
      <options>
          peer       = /tmp/livestatus.socket
      </options>
      <configtool>
          core_conf  = /etc/nagios/nagios.cfg
          obj_check  = /etc/init.d/nagios checkconfig
          obj_reload = /etc/init.d/nagios reload
      </configtool>
  </peer>
  <peer>
      name   = External Icinga
      type   = livestatus
      <options>
          peer   = 172.16.0.2:9999
     </options>
  </peer>
  <peer>
      name   = External Shinken
      type = livestatus
      <options>
          peer   = 172.16.0.3:50000
     </options>
  </peer>
</Component>

Component Thruk::Plugin::ConfigTool

Enable config tool by setting path to different components config files. Users with the roles authorized_for_configuration_information and authorized_for_system_commands will then have access to the config tool. You don’t have to restart Thruk when changing the config with the config tool.

  • general options for the config tool

    • show_plugin_syntax_helper Enable/Disable the plugin syntax helper. When enabled, Thruk will run the plugins with "plugin -h" to get the help information.

  • you can manage different types of files with the config tool:

    • thruk Path to your thruk_local.conf. Enables adjusting Thruks config if set.

    • cgi.cfg Path to your cgi.cfg. Enables adjusting the cgi.cfg if set.

    • htpasswd Path to your htpasswd. Enables user management based an apaches basic auth with htpasswd.

ex.:

<Component Thruk::Plugin::ConfigTool>
  show_plugin_syntax_helper = 1
  thruk                     = .../thruk_local.conf
  cgi.cfg                   = .../cgi.cfg
  htpasswd                  = .../htpasswd
</Component>

Component Thruk::Plugin::Panorama

The Panorama plugin is a nice, fully customizable dashboard allowing you to build your own panorama views.

  • general options for the config tool

    • state_provider Sets the way states between reloads are preserved. Valid options are cookie or server. Cookie store is usefull for demo systems where lot people share the same account.

    • default_view Can be either the string of an exported view or path to a file with exported view.

ex.:

<Component Thruk::Plugin::Panorama>
  state_provider  = server
  default_view    = /var/lib/thruk/default_panorama_view
</Component>

cgi.cfg

The cgi.cfg is mainly the same as in Nagios or Icinga but not all values are used.

show_context_help

Displays an icon with context specific help on most pages.

ex.:

show_context_help = 1

use_authentication

Determines whether to use authentication or not. If enabled, Thruk will use the REMOTE_USER from the apache environment as authenticated user name. So the authentication has to be done by the webserver. Authorization is done by Thruk.

ex.:

use_authentication = 1

use_ssl_authentication

If enabled, the authenticated username is taken from the SSL_CLIENT_S_DN_CN environment instead of the remote user.

ex.:

use_ssl_authentication = 1

default_user_name

The default user will be used if no username has been provided by the webserver. You don’t have to set a default user.

ex.:

default_user_name = thrukadmin

authorized_for_system_information

List of usernames who have access to the system and process information pages. You may use wildcards here.

ex.:

authorized_for_system_information = thrukadmin

authorized_for_configuration_information

List of usernames who have access to the configuration information pages. You may use wildcards here.

ex.:

authorized_for_configuration_information = thrukadmin

authorized_for_system_commands

List of usernames who are allowed to send system commands. You may use wildcards here.

ex.:

authorized_for_system_commands = thrukadmin

authorized_for_all_services

List of usernames who are authorized to view all services. You may use wildcards here.

ex.:

authorized_for_all_services = thrukadmin

authorized_for_all_hosts

List of usernames who are authorized to view all hosts. You may use wildcards here.

ex.:

authorized_for_all_hosts = thrukadmin

authorized_for_all_service_commands

List of usernames who are authorized to send commands for all services. You may use wildcards here.

ex.:

authorized_for_all_service_commands = thrukadmin

authorized_for_all_host_commands

List of usernames who are authorized to send commands for all hosts. You may use wildcards here.

ex.:

authorized_for_all_host_commands = thrukadmin

refresh_rate

Number of seconds after which most pages are refreshed automatically.

ex.:

refresh_rate = 90

escape_html_tags

Determines whether html output from plugins is escaped or not.

ex.:

escape_html_tags = 1

action_url_target

Sets the target of the action url links.

ex.:

action_url_target = _blank

notes_url_target

Sets the target of the notes url links.

ex.:

notes_url_target = _blank

lock_author_names

If enabled, user are not allowed to change the author name for commands.

ex.:

lock_author_names = 1

log4perl.conf

This file contains the log4perl configuration. You can configure whatever is possible in log4perl. Refer to the log4perl manual for detailed information: http://log4perl.sourceforge.net/releases/Log-Log4perl/docs/html/Log/Log4perl/Config.html There is a log4perl.conf.example file shiped with the package.

ex.:

log4perl.logger=ALL, ErrorLog, DebugLog
log4perl.appender.ErrorLog=Log::Dispatch::File
log4perl.appender.ErrorLog.filename=/home/thruk/Thruk/logs/error.log
log4perl.appender.ErrorLog.mode=append
log4perl.appender.ErrorLog.Threshold=ERROR
log4perl.appender.ErrorLog.layout=Log::Log4perl::Layout::PatternLayout
log4perl.appender.ErrorLog.layout.ConversionPattern=[%d][%H][%p][%c] %m%n
Tip
command logging

if you want to log all sent commands, just set the loglevel to INFO.

ex.: log4perl.appender.ErrorLog.Threshold=INFO

menu.conf

This file contains the default side menu configuration. Do not change it as this file will be overwritten with every update. If you want to create a complete custom navigation, just copy this file to menu_local.conf and adjust it to your needs. See menu_local.conf on how to add just add a few links without having to copy the complete file.

menu_local.conf

This file contains the user configuration for the side menu. See the FAQ for some examples.

The file itself is perl syntax, so you can do whatever perl can do. Make sure you verify the syntax after changing the file. The changes will be used immediately. So maybe you want to test your changes on a test instance first.

Tip
syntax check for menu configuration
%>perl -wc menu_local.conf
menu_local.conf syntax OK

If you just want to add a few entrys, create a empty menu_local.conf and put these lines into it:

do '/usr/share/thruk/menu.conf';
insert_item('General', { 'href' => 'http://labs.consol.de', 'name' => 'Labs', target => '_blank' });

Server Side Includes

You can place static include files or executables in the ssi folder. The static files will be place right after the body or just before the end of the body.

Important This has nothing to do with the Apache module mod_include and just implements the nagios way of SSIs.

There are two global includes, which will be included in every cgi page:

  • ssi/common-footer.ssi

  • ssi/common-header.ssi

and there are page specific includes, which will only be placed in the specific page:

  • ssi/status-footer.ssi

  • ssi/status-header.ssi

Files will be statically placed inside the output of the normal cgi output. Executables will be executed and the output will then be used as content.

Examples

Here is a list of common tasks and configurations.

CGI Parameters

Most of the CGI Parameters can be changed with buttons, but there are some which are only accessible by adding them directly to the url.

hidetop

Pages: status pages

Description: Show/Hide the status totals header.

Example: /thruk/cgi-bin/status.cgi?hidetop=1

nav

Pages: all pages

Description: Show/Hide the navigation. Only usefull in non-frames mode.

Example: /thruk/cgi-bin/tac.cgi?nav=0

noheader

Pages: status pages

Description: Show/Hide the status totals header.

Example: /thruk/cgi-bin/status.cgi?noheader

minimal

Pages: status page

Description: Hides almost any user interface buttons. Usefull for monitor screens where only the current problems should be displayed.

Example: /thruk/cgi-bin/status.cgi?host=all&servicestatustypes=28&minimal=1

refresh

Pages: all pages

Description: Sets refresh timer to a custom value.

Example: /thruk/cgi-bin/status.cgi?refresh=30

SLA Reporting

SLA Reporting consists of several parts.

  • report configuration

  • master pdf

  • template (contains mail text and pdf content)

Report Configuration

You can create and adjust report setttings on the Reporting page. Make sure the plugin is enabled. After selecting a template for your report you have to set specific types of input fields on the second page. If you make you report public, eveyone can view and refresh the report.

Master PDF

The master pdf is used as source for the pdf report and used together with the values filled in from the template to generate the pdf file. You may adjust the shipped sla.pdf to your needs and put it into a pdf subfolder of the user_template_path.

Template

The template defines the dynamic content of the sla report. You may copy the sla.tt file into a pdf subfolder of the user_template_path and adjust it to your needs.

See http://thruk.org/api/Thruk/Utils/PDF.html for details on available functions and examples.

Sending Reports By Mail

It’s quite easy to setup sla reports by mail. All you need is to create send option in the report configuration. It is even possible to send one report at multiple times.

CLI Tool & Scripting

It is possible to do comprehensive scripting with Thruk. For example set downtimes automatically, schedule reports or change the objects of your monitoring core configuration.

See http://thruk.org/api/Thruk/Utils/CLI.html for details and examples.