Thruk Users Guide
=================

== Introduction

Thruk is a multibackend monitoring webinterface which currently
supports Nagios, Icinga and Shinken as backend using the Livestatus
API. It is designed to be a 'dropin' replacement and covers almost
all of the original features plus adds additional enhancements for
large installations.

Main Features / Advantages

  * multiple backends
  * faster, no need to parse status.dat for every request
  * less cpu usage
  * display live data, no delay between core and gui
  * independant from monitoring core, can be installed on remote host
  * clusterable, can be clustered over hosts
  * change status.cgi filter inline
  * extended logfile search
  * better logging, uses log4perl (who submitted which command?)
  * valid HTML code
  * no frames ( can be reenabled for better addon integration)
  * easy to extend with plugins
  * 100% Perl (using Catalyst Framework)
  * paging ( can be disabled and customized )
  * multiple themes included
  * Excel export for status and logfiles
  * Adjustable side menu
  * Full expanded plugin commandline for easy testing
  * Save searches like bookmarks
  * Config Tool included
  * Mobile interface included
  * SLA Reports in PDF format
  * Recurring Downtimes
  * Fully Featured Dashboard



== How it works

Thruk is written in Perl using the
http://www.catalystframework.org[Catalyst Framework]. Backend
monitoring systems will be connected with the
http://search.cpan.org/dist/Monitoring-Livestatus[Monitoring::Livestatus]
Perl Module. Thruk itself is running as a fastcgi process.
Availability will be calculated with
http://search.cpan.org/dist/Monitoring-Availability[Monitoring::Availability].
Authentication is provided by the Apache webserver (For example with
mod_auth).

.Thruk Architecture
image:source/arch_preview.png[Thruk Architecture]


== What makes Thruk cool

There are a couple of cool things in Thruk. A few of them are listed
here.

=== Excel Export and Bookmarks

The Bookmarks feature allows you to save your searches and add them as
your personal menu item. Besides the bookmarks, you can adjust the
menu easiely with the menu_local.conf for all users.

The Excel export creates real Excel files which can be send by mail or
used to extract hostnames.

image:source/Bookmarks_and_excel_export.png[Bookmarks and Excel Export]


=== Config Tool

The Config Tool is great to make quick changes to your Thruk
configuration. It is also possible to manage access with htpasswd
files as well as adjusting your cgi.cfg.

image:source/Config_Tool.png[Config Tool]


=== Sending Multiple Commands

The new status pages make it very convenient to send multiple commands
at once. It is even possible to send host and service commands at the
same time. When rescheduling hosts and services, Thruk will wait until
your check is finished and display the result as soon as the check is
over.

image:source/Reschedule.png[Multiple Commands]


=== Easy Filtering

Remember the days when you had to guess numbers in the url to filter
hosts or services. With Thruk it's possible to quickly change your
display filter. You can combine multiple filter to create whatever
views you like. An Ajax search supports you, so you don't have to
guess host or service names.

image:source/Filter.png[Easy Filtering]


=== PNP4Nagios Graphs

When your action_url contains /pnp4nagios/, there will be automatically
a graph displayed for your host and service. This gives you a quick
view about the performance history. The image is then linked to
PNP4Nagios to get detailed information.

image:source/PNP4nagios.png[PNP4Nagios]


=== Multiple Lines of Plugin Output

When your check returns multiple lines of plugin output. Thruk marks
the output in blue and a click on it displays the complete output.
This is especially usefull for check_multi checks. In addition to
that, the comments and downtimes also have a small popup with their
data. So you don't have to open the host/service page just to see who
set a comment and when there is a downtime.

image:source/PluginOutput.png[Multiple Lines of Plugin Output]


=== Mine Map

The Mine Map is the perfect tool to get a quick overview. It is
especially usefull if you have a lot of common services across your
hosts. Otherwise use hostgroups or servicegroups for nice results.
Normal filtering is possible too.

image:source/MineMap.png[Mine Map]


=== Mobile Interface

The Mobile interface gives you access to the most important things
and allows you to quickly view and acknowledge problems.

image:source/Mobile.png[Mobile Interface]





== Installation

There are several ways of installing Thruk.

=== Labs Consol Repository
The https://labs.consol.de/repo/[Labs Repository] provides packages of
latest Thruk releases (including daily development builds) and other
Tools like Mod-Gearman. Just follow the guide on the labs page for
your distribution.


=== Use OMD
An easy way of installing thruk is using OMD from
http://omdistro.org[omdistro.org]. There are Debian, Ubuntu, Centos
and Suse Packages containing preconfigured latest versions of Nagios
and Thruk. The package also includes icinga, shinken, pnp4nagios,
check_mk and nagvis. OMD is the recommended way of installing new
Nagios / Thruk setups when you want to install several addons at once.


=== Use Packages
Standalone installation from a binary package is another easy way
to get Thruk running.

Download packages from http://www.thruk.org/files/pkg/

All packages have the following filesystem structure:

-------
  /etc/thruk                       thruks config
  /etc/httpd/conf.d/thruk.conf     apache config
  /usr/share/thruk                 shared files
  /usr/lib/thruk/perl5             perl librarys
  /var/cache/thruk                 temporary files
  /var/lib/thruk                   stored user settings
-------

After installation, Thruk is available at http://your-host/thruk/ and
has a default user 'thrukadmin' with password 'thrukadmin' configured.
You may need to change backend configuration. This should be done in
the thruk_local.conf where all settings can be overriden.



=== Debian / Ubuntu

-------
  #> dpkg -i thruk_1.56_debian6_amd64.deb
-------

In case of dependency errors, run 'apt-get -f install' and try the
'dpkg -i...' again.


=== Centos / Redhat

-------
  #> yum install --nogpgcheck thruk-1.56-1.rhel6.x86_64.rpm
-------

You may need to include an external repository for mod_fastcgi module.
Epel or Rpmforge should do it.


=== SLES

-------
  #> zypper install thruk-1.56-1.sles11.x86_64.rpm
-------

You may need the SLES sdk dvd for additional dependencies.



=== Install from Source
==== Requirements

[IMPORTANT]
.Experienced Users Only
=======
Source installation is for experienced users only. Using
packages/repositories should be the prefered solution in almost all
situations. Especially for productional usage.
=======


In order to install the Thruk Monitoring Webinterface from source you
will need the following:

- Perl
- Git Client
- compiler tools: automake, make, g++, gcc
- Apache Webserver (optional for fastcgi only)

==== Create New User
refer to your systems manual on howto add new user.
This guide uses the following:

-------
  user:  thruk
  group: thruk
-------


IMPORTANT: all following steps should be done by the thruk user.

==== Install Local::Lib


Follow the steps on http://search.cpan.org/perldoc?local::lib#The_bootstrapping_technique

quick guide:

-------
  %> su - thruk
  %> wget %http://search.cpan.org/CPAN/authors/id/A/AP/APEIRON/local-lib-1.008004.tar.gz
  %> tar zxf local-lib-1.008004.tar.gz
  %> cd local-lib-1.008004
  %> perl Makefile.PL --bootstrap && make install
-------

[TIP]
.proxy configuration
=======
if you need a proxy configuration, you should answer <no> at this question:

 Would you like me to configure as much as possible automatically? [yes]

Setting a proper http_proxy/ftp_proxy environment should work for the automatic
configuration.
=======


Add the following line to the thruk users .profile or .bashrc

 eval $(perl -I$HOME/perl5/lib/perl5 -Mlocal::lib)

logout and login again and verify that your perl env looks like this:

-------
 %> env | grep perl
 PERL5LIB=/home/thruk/perl5/lib/perl5:/home/thruk/perl5/lib/perl5/x86_64-linux-gnu-thread-multi
 MODULEBUILDRC=/home/thruk/perl5/.modulebuildrc
 PATH=/home/thruk/perl5/bin:/usr/local/bin:/usr/bin:/bin:/usr/games
 PERL_MM_OPT=INSTALL_BASE=/home/thruk/perl5
-------


==== Install Module::Install
-------
  perl -MCPAN -e 'install Module::Install::Catalyst'
-------


==== Git Clone Thruk

-------
    git clone https://github.com/sni/Thruk.git
    cd Thruk
    perl Makefile.PL
-------

press <enter> here:
-------
  ==> Auto-install the 35 mandatory module(s) from CPAN? [y]
-------

then run make
-------
    make
-------

This may take a while, as there are probably several modules missing.


[TIP]
.yes
=======
you can use /usr/bin/yes to automatically answer all questions with
yes. (yes | make)
=======


Press <y> at this questions:
-------
  Do you want to build the XS Stash module? [y]
  Do you want to use the XS Stash by default? [y]
-------

run perl Makefile.PL again to see if all dependecies are now installed properly.

-------
    perl Makefile.PL
-------


==== Install Livestatus

Refer to http://mathias-kettner.de/checkmk_livestatus.html#H1:%20Setting%20up%20and%20using%20Livestatus
on how to install livestatus onto your monitoring box.

Basically you have to build the NDO addon and add it to your nagios/icinga.cfg.

-------
 broker_module=/opt/local/livestatus/livestatus.o /tmp/live.sock
-------


==== Configuration

Copy 'thruk.conf' to 'thruk_local.conf' and adjust the livestatus settings to your needs.
-------
  %> cp thruk.conf thruk_local.conf
-------
The thruk.conf will be overwritten with new defaults on updates. The
thruk_local.conf contains the local overrides and will never be
overwritten.


Edit 'cgi.conf' and adjust settings to your needs.

See the <<Configuration>> section for detailed explaination of configuration options.



==== Start Server
After running these steps successfully, you should be able to test your
installation:

-------
    ./script/thruk_server.pl
-------

Open your browser and open http://<your_host>:3000


== Apache Configuration

Integration in the Apache webserver is done by fastcgi.  There are two
fastcgi modules for apache at the moment. Choose the one which fits
best into your environment. If unsure, use <<_mod_fcgid,mod_fcgid>>.
The main difference is, that mod_fcgid starts the fastcgi process upon
the first request whereas in mod_fastcgi you have to start the fastcgi
process by yourself.

Do not use the port 3000 thruk server in production, it's only for
testing and development. It cannot handle authentication and is slower
than the fastcgi variants.

[TIP]
.lighthttpd
=======
Francois Ponsard wrote an article on how to integrate Thruk in Lighthttpd:
http://www.dahwa.fr/dotclear/index.php?post/2011/03/15/Thruk-in-Lighttpd
=======

=== mod_fastcgi

start your fcgi server:

-------
    %>./script/thruk_fastcgi.pl -n 5 \
                -l /tmp/thruk_fastcgi.socket \
                -p /tmp/thruk_fastcgi.pid
-------

you may want to copy the init.d script to /etc/init.d and adjust its paths:
-------
    %> sudo cp ./script/thruk_fastcgi_server.sh /etc/init.d/thruk_fastcgi_server
    %> vi /etc/init.d/thruk_fastcgi_server
    %> sudo chown root: /etc/init.d/thruk_fastcgi_server
-------

Or create a custom init.d script (additional modules required) with:
-------
    %> ./script/thruk_create.pl FastCGI::ExternalServer l=/tmp/thruk_fastcgi.socket n=5 p=/tmp/thruk_fastcgi.pid
-------



==== Apache Configuration

use this apache example configuration:

- replace /home/thruk/Thruk with your installation path
- replace your-web-host.local with your hostname
- create a /home/thruk/Thruk/htpasswd.users with htpasswd2
- make sure the /home/thruk/Thruk/logs/ directory exists


.Apache configuration within existing vhost
-------
<VirtualHost *:80>
    # ... existing configuration

    # thruk configuration
    <Directory /home/thruk/Thruk/root/>
        order allow,deny
        allow from all
        Options FollowSymLinks
        AllowOverride All
    </Directory>

    Alias /thruk/ /home/thruk/Thruk/root/thruk/

    # authorization
    <Location "/thruk">
        AuthName "Monitoring Access"
        AuthType Basic
        AuthUserFile /home/thruk/Thruk/htpasswd.users
        Order Allow,Deny
        Allow from all
        require valid-user
    </Location>

    # Load fastcgi module unless already loaded
    LoadModule fastcgi_module /usr/lib/apache2/modules/mod_fastcgi.so

    # fastcgi configuration
    FastCGIExternalServer /tmp/thruk_fastcgi.fcgi -socket /tmp/thruk_fastcgi.socket -idle-timeout 120

    # Load rewrite module unless already loaded
    LoadModule rewrite_module /usr/lib/apache2/modules/mod_rewrite.so

    # rewrite configuration
    RewriteEngine On
    RewriteCond %{REQUEST_FILENAME} !-f
    RewriteRule ^/thruk(.*)$ /tmp/thruk_fastcgi.fcgi/thruk$1 [E=HTTP_AUTHORIZATION:%{HTTP:Authorization},QSA,L]

</VirtualHost>
-------


.Apache configuration with own vhost
-------
<VirtualHost *:80>
    ServerName   thruk.your-host.local

    DocumentRoot /home/thruk/Thruk/root/
    CustomLog    /home/thruk/Thruk/logs/access.log combined
    ErrorLog     /home/thruk/Thruk/logs/error.log

    <Directory />
        order deny,allow
        deny from all
    </Directory>

    <Directory /home/thruk/Thruk/root/>
        Options FollowSymLinks
        AllowOverride All
        order allow,deny
        allow from all
    </Directory>

    # authorization
    <Location "/">
        AuthName "Monitoring Access"
        AuthType Basic
        AuthUserFile /home/thruk/Thruk/htpasswd.users
        Order Allow,Deny
        Allow from all
        require valid-user
    </Location>

    # Load fastcgi module unless already loaded
    LoadModule fastcgi_module /usr/lib/apache2/modules/mod_fastcgi.so

    # fastcgi configuration
    FastCGIExternalServer /tmp/thruk_fastcgi.fcgi -socket /tmp/thruk_fastcgi.socket -idle-timeout 120

    # Load rewrite module unless already loaded
    LoadModule rewrite_module /usr/lib/apache2/modules/mod_rewrite.so

    # rewrite configuration
    RewriteEngine On
    RewriteCond %{REQUEST_FILENAME} !-f
    RewriteRule ^/(.*)$ /tmp/thruk_fastcgi.fcgi/$1 [E=HTTP_AUTHORIZATION:%{HTTP:Authorization},QSA,L]
</VirtualHost>
-------

=== mod_fcgid

IMPORTANT: Thruk runs with the webserver user in this scenario, make
sure the webserver user has access to all Thruk files and perl modules.

.Apache configuration with mod_fcgid
-------
LoadModule fcgid_module /usr/lib/apache2/modules/mod_fcgid.so
<VirtualHost *:80>
    ServerName   thruk.company.local

    DocumentRoot /home/thruk/Thruk/root/
    CustomLog    /home/thruk/Thruk/logs/access.log combined
    ErrorLog     /home/thruk/Thruk/logs/error.log

    <Directory /home/thruk/Thruk/root/>
        Options FollowSymLinks
        AllowOverride All
        order allow,deny
        allow from all
    </Directory>

    AliasMatch /thruk/(.*\.cgi|.*\.html)  /home/thruk/Thruk/script/thruk_fastcgi.pl/thruk/$1
    <Location /thruk>
        Options ExecCGI
        Order allow,deny
        Allow from all
        AuthName "Monitoring Access"
        AuthType Basic
        AuthUserFile /home/thruk/Thruk/htpasswd.users
        Require valid-user
    </Location>

    <IfModule mod_fcgid.c>
      AddHandler fcgid-script .pl
      MaxRequestsPerProcess 100
    </IfModule>

</VirtualHost>
-------


== Configuration

Configuration is managed mainly in these configuration files.

  * thruk.conf
  * thruk_local.conf
  * cgi.cfg
  * log4perl.conf
  * menu.conf
  * menu_local.conf


=== thruk.conf

The thruk.conf contains the shipped defaults. You should not edit this
file directly. Overwrite your settings in your thruk_local.conf
instead.


=== thruk_local.conf

The thruk_local.conf is a copy of the thruk.conf. Both files are used by thruk.
The idea is, that the thruk.conf is provided with the packaged defaults and the
thruk_local.conf is to override these settings where needed.


==== use_timezone

Changes the timezone from the systems default to this timezone.
Only set this if you have trouble with displaying the right timestamps.

ex.:

  use_timezone = CET


==== title_prefix

set the title prefix for all urls
this piece of text will be prepended to
all page titles.

ex.:

  title_prefix = Prod


==== url_prefix

Changes the usual url path for Thruk. Don't change it unless you plan
to run multiple Thruk instances on the same webserver. You will have
to change your fastcgi configuration too.

ex.:

  url_prefix = /


==== logo_path_prefix

Changes the path to your logo images. Default is
$url_prefix+'thruk/themes/'+$current_theme+'/images/logos/' and
therefor relative to the current selected theme. You could set a fixed
path here.
Like usual, paths starting with a / will be absolute from your
webserver root directory. Paths starting without a / will be relative
to the cgi directory.

ex.:

  logo_path_prefix = /icons/


==== physical_logo_path

Location of your logos in your filesystem. This directory should be
mapped to your 'logo_path_prefix' directory where 'logo_path_prefix' is
the path relative to your webserver root directory and 'physical_logo_path' is
the corresponding filesystem path.

ex.:

  physical_logo_path = /usr/share/icons/


==== host_action_icon

Change path to your host action icons. You may use
relative paths to specify completely different location.
You also may want to use 'action_pnp.png' when using pnp.
Icon can be overriden by a custom variable '_ACTION_ICON'.

ex.:

  host_action_icon = action.gif


==== service_action_icon

Change path to your service action icons. You may use
relative paths to specify completely different location.
You also may want to use 'action_pnp.png' when using pnp.
Icon can be overriden by a custom variable '_ACTION_ICON'.

ex.:

  service_action_icon = action.gif


==== use_strict_host_authorization

When set to a true value, every contact will only see the hosts where
he is contact for plus the services where he is contact for. When
disabled, a host contact will see all services for this host
regardless of wheter he is a service contact or not.

ex.:

  use_strict_host_authorization = 1


==== make_auth_user_lowercase

Convert authenticated username to lowercase.

ex.:

  make_auth_user_lowercase = 1


==== make_auth_user_uppercase

Convert authenticated username to uppercase.

ex.:

  make_auth_user_uppercase = 1


==== start_page

This link is used as startpage and points usually to the main.html
with displays version information and general links.

ex.:

  start_page = /thruk/main.html


==== documentation_link

This link is used in the side navigation menu as link to the documentation.
Replace it with whatever your documentation is located. Set it to a blank value
if you don't want a documentation link in the menu at all.

ex.:

  documentation_link = /thruk/docs/


==== all_problems_link

Customizable link for the 'problems' link in side menu. Can be useful
to reflect your companys process of error handling.

ex.:

  all_problems_link = /thruk/cgi-bin/status.cgi?...


==== allowed_frame_links

List of allowed patterns, where links inside frames can be set to.
You can link to /thruk/frame.html?link=http://wiki.my-company.com/page/blah
Your wiki will then be displayed with the Thruk navigation frame.
Useful for other addons, so they don't have to display a own
navigation.

ex.:

  allowed_frame_links = http://intranet.my-company.com
  allowed_frame_links = https://wiki.my-company.com


==== initial_menu_state

Set initial menu state.

  closed  => 0
  open    => 1

ex.:

  <initial_menu_state>
    General        = 1
    Current_Status = 1
    Reports        = 1
    System         = 1
    Bookmarks      = 1
  </initial_menu_state>


==== cgi_cfg

The path to your cgi.cfg. See <<_cgi_cfg_2,cgi.cfg>> for details.

ex.:

  cgi_cfg = cgi.cfg


==== log4perl_conf

The path to your log4perl configuration file.

ex.:

  log4perl_conf = ./log4perl.conf


==== plugin_path

Path to your plugins directory. Can be used to specify different
location for you Thruk plugins. Don't forget to set appropriate apache
alias or rewrite rules when changing the plugin path. Otherwise the
static content from plugins is not accessible.

Example redirect rule for apache:

+++++
 AliasMatch /thruk/plugins/(.*?)/(.*)$ <your-plugin-dir>/plugins/plugins-enabled/$1/root/$2
+++++

ex.:

  plugin_path = ./plugins


==== themes_path

Path to your themes directory. Can be used to specify different
location for you Thruk themes. Don't forget to set appropriate apache
alias or rewrite rules when changing the themes path. Otherwise the
static content from your themes may not accessible.

+++++
 Alias /thruk/themes/ <your-themes-dir>/themes/themes-enabled/
+++++

ex.:

  themes_path = ./themes


==== var_path

Path to the var directory. Thruk stores user specific date here.

ex.:

  var_path = ./var


==== tmp_path

Path to a temporary directory. Defaults to /tmp if not set and usually
this is a good place.

ex.:

  tmp_path = /tmp


==== ssi_path

The path to your ssi (server side includes) files. See
<<_server_side_includes,Server Side Includes>> for details.

ex.:

  ssi_path = ssi/


==== user_template_path

Specify a additional directory for user supplied templates. This makes
it easy to override thruks own templates.
Template search order is:

 * users template path
 * plugins template path
 * themes template path
 * thruks template path

ex.:

  user_template_path = ./my_templates


==== delay_pages_after_backend_reload

Delay the page delivery until the backends uptime is at least this
amount of seconds. Displaying pages soon after backend restarts
may display wrong results and all services are pending. Enable this if
you experience problems with pending services after reloading your
backend.
Should be obsolete with Livestatus versions greater than 1.2
ex.: setting this to 10 would start serving pages 10 seconds
after the backend reload

ex.:

  delay_pages_after_backend_reload = 10


==== use_frames

Set whether you want to use a framed navigation or not. With using frames it's
sometimes easier to include addons.
See allowed_frame_links option for how to integrate addons.

ex.:

  use_frames = 0


==== strict_passive_mode

Normally passive checks would be marked as disabled. With this
option set, disabled checks will only be displayed as disabled if their
last result was active. Otherwise they would be marked as passive
checks. This option also changes the passive icon only to be shown
when the last check was passive, otherwise the disabled icon will be
displayed.

ex.:

  strict_passive_mode = 1


==== use_new_search

Use the old or the classic search from the navigation. The new search
supports regular expressions and searches in many attributes. For
example: plugin_output, groups, names, descriptions

ex.:

  use_new_search = 1


==== use_new_command_box
Show the new split command box on the host / service details page.

ex.:

  use_new_command_box = 1


==== use_ajax_search

Enables the ajax search field. There will be suggestions while typing
into the search field.

ex.:

  use_ajax_search = 1


==== ajax_search_hosts

Enables the suggestion of hosts in the ajax search field.
Depending on the number of hosts, this can make the search slow.

ex.:

  ajax_search_hosts = 1


==== ajax_search_hostgroups

Enables the suggestion of hostgroups in the ajax search field.

ex.:

  ajax_search_hostgroups = 1


==== ajax_search_services

Enables the suggestion of services in the ajax search field.
Depending on the number of services, this can make the search slow.

ex.:

  ajax_search_services = 1


==== ajax_search_servicegroups

Enables the suggestion of servicegroups in the ajax search field.

ex.:

  ajax_search_servicegroups = 1


==== ajax_search_timeperiods

Enables the suggestion of timeperiods in the ajax search field.
Timeperiods will only displayed when filtering by check- or
notification period.

ex.:

  ajax_search_timeperiods = 1


==== default_theme

Default theme to use for all users. Must be a valid subdirectory in
the themes folder.

ex.:

  default_theme = Thruk


==== cookie_path

Path used for cookies. Do not change unless you have weird url
rewrites which breaks setting cookies.

ex.:

  cookie_path = /


==== use_pager
Using the pager will make huge pages much faster as most people don't want a
services page with 100.000 services displayed. Can be disabled if you don't
need it.

ex.:

  use_pager = 1


==== paging_steps
Define the selectable paging steps. Use the * to set the default
selected value.

ex.:

  paging_steps = *100, 500, 1000, all


==== group_paging_overview
Just like the paging_steps, but only for the groups overview page.

ex.:

  group_paging_overview =  *3,  10, 100, all


==== group_paging_summary
Just like the paging_steps, but only for the groups summary page.

ex.:

  group_paging_summary = *10, 50, 100, all


==== group_paging_grid
Just like the paging_steps, but only for the groups grip page.

ex.:

  group_paging_grid = *5,  10, 50,  all


==== can_submit_commands
Set this if a contact should be allowed to send commands unless defined for the
contact itself. This is the default value for all contacts unless the user has
a can_submit_commands setting in your monitoring configuration.

ex.:

  can_submit_commands = 1


==== command_disabled
Use this to disabled specific commands. Can be use multiple times to disabled
multiple commands. The number can be found in the 'cmd_typ' cgi parameter from
links to the command page.
You may use ranges here.

ex.:

  command_disabled = 14
  command_disabled = 35
  command_disabled = 17-34,50-65


==== cmd_defaults
Set the default checked state for command options.

ex.:

  <cmd_defaults>
    ahas                   = 0  # For Hosts Too
    broadcast_notification = 0  # Broadcast
    force_check            = 0  # Forced Check
    force_notification     = 0  # Forced Notification
    send_notification      = 1  # Send Notification
    sticky_ack             = 1  # Sticky Acknowledgement
    persistent_comments    = 1  # Persistent Comments
    persistent_ack         = 0  # Persistent Acknowledgement Comments
    ptc                    = 0  # For Child Hosts Too
    use_expire             = 0  # Use expire time ( for cores which support it)
  </cmd_defaults>


==== downtime_duration
Default duration of new downtimes in seconds. Default is 2 hours.

ex.:

  downtime_duration = 7200


==== expire_ack_duration
Default duration of acknowledgements with expire date. Default is one
day.

ex.:

  expire_ack_duration = 86400



==== cmd_quick_status
Configure which commands should be available as quick status commands.

ex.:

  <cmd_quick_status>
    reschedule             = 0  # Reschedule next check
    downtime               = 0  # Add/remove downtimes
    comment                = 0  # Add/remove comments
    acknowledgement        = 0  # Add/remove acknowledgements
    active_checks          = 0  # Enable/disable active checks
    notifications          = 0  # Enable/disable notifications
    submit_result          = 0  # Submit passive check result
    reset_attributes       = 0  # Reset modified attributes
  </cmd_quick_status>



==== command_reschedule_alias
When you want to reschedule passive checks for which the result is fetched by
an agent (For example check_mk or some scenarios of check_multi). You
usually want to reschedule the agent instead of the passive check.

The command reschedule alias can be used to translate the reschedule command
from the passive service to the active agent service.

  command_reschedule_alias = pattern;master_service_description

.Notes
* The pattern will be tested against the service description and the command_name
  of the passive check.
* The resulting service name be on the same host and the contact must
  be authorized for that service too.
* The pattern must be a valid perl regular expression.
* Duplicates  will be removed. So if you reschedule 10 services which result in
  the same master service will only trigger one reschedule.
* Only passive services will be translated

In this example, all passive check_mk checks will trigger the active agent check
and therefor allow you to reschedule passive checks directly from the problems
page.

ex.:

  command_reschedule_alias = ^check_mk\-(?!inventory);Check_MK


==== datetime_format
Default timeformat. Use POSIX format.

ex.:

  datetime_format = %Y-%m-%d  %H:%M:%S


==== datetime_format_long
Default long timeformat.

ex.:

  datetime_format_long = %a %b %e %H:%M:%S %Z %Y


==== datetime_format_log
Default log timeformat.

ex.:

  datetime_format_log = %B %d, %Y  %H


==== datetime_format_trends
Default trends timeformat.

ex.:

  datetime_format_trends = %a %b %e %H:%M:%S %Y


==== datetime_format_today
Default timeformat for todays date. Can be useful if you want a
shorter date format for today.

ex.:

  datetime_format_today = %H:%M:%S


==== show_long_plugin_output

When a plugin returns more than one line of output, the
output can be displayed directly in the status table, as
popup or not at all.
Choose between popup, inline and off

ex.:

  show_long_plugin_output = popup


==== show_modified_attributes

Show if a host / service has modified attributes.

ex.:

  show_modified_attributes = 1


==== info_popup_event_type
On which event should the comments / downtime or longpluginout popup
show up. Valid values are onclick or onmouseover.

ex.:

  info_popup_event_type = onmouseover


==== info_popup_options
Options for the popup window used for long pluginoutput, downtimes and
comments.
See http://www.bosrup.com/web/overlib/?Command_Reference for
what options are available

ex.:

  info_popup_options = STICKY,CLOSECLICK,HAUTO,MOUSEOFF


==== show_notification_number
Display the current number of notification after the current / max
attempts on the status details page.

ex.:

  show_notification_number = 0


==== show_backends_in_table
Display the backend/site name in the status table.
This is useful if you have same hosts or services on different
backends and need to know which one returns an error.
Valid values are:

 1 - show site name at the end
 2 - put site name in front

ex.:

  show_backends_in_table = 0


==== show_config_edit_buttons
Show links to config tool for each host / service.
You need to have the config tool plugin enabled
and you need proper permissions for the link to appear.

ex.:

  show_config_edit_buttons = 1


==== show_full_commandline
Display the full command line for host / service checks .
Be warned, the command line could contain passwords and other confidential data.
In order to replace the user macros for commands, you have to set the
'resource_file' in your peer config or the general 'resource_file' option.

 * 0 = off, don't show the command line at all
 * 1 = show them for contacts with the role: authorized_for_configuration_information
 * 2 = show them for everyone

ex.:

  show_full_commandline = 0


==== resource_file
Set a general resource file. Make sure it does not contain any
passwords or any other data which should not be displayed. Instead of
using a general 'resource_file' you could define one file per peer in your
peer config.

ex.:

  resource_file = /etc/nagios3/resource.cfg


==== perf_bar_mode
This option enables a performance bar inside the status/host list which
create a graph from the performance data of the plugin output. Available
options are 'match', 'first', 'all', 'worst' and 'off'.

 match: try to set graph which matches the output
 all: graph all performance values available
 first: graph only the first performance value
 worst: graph only the graph for the worst state
 off: graph no value at all

ex.:

  perf_bar_mode = match


==== shown_inline_pnp
Show inline pnp graph if available. If a service or host has a
pnp4nagios action or notes url set. Thruk will show a inline graph on the
extinfo page.
This works for /pnp4nagios/ urls and /pnp/.

ex.:

  shown_inline_pnp = 1



==== show_custom_vars
Show custom vars in host / service ext info. List variable names to
display in the host and service extinfo details page. Can be specified
more than once to define multiple variables. You may use html in your
variables. Use * as wildcard, ex.: _VAR*

ex.:

  show_custom_vars = _VAR1



==== statusmap_default_type
You may change the default map type of the statusmap here. Valid
types are: 'table' and 'circle'

ex.:

  statusmap_default_type = table


==== statusmap_default_groupby
And the statusmap default group by which has to be one of:
'parent', 'address', 'domain', 'hostgroup', 'servicegroup'

ex.:

  statusmap_default_groupby = address


==== use_wait_feature
Waiting is a livestatus feature. When enabled, Thruk will wait
after rescheduling hosts/services checks until the
check has been really executed up to a maximum of 10 seconds. Adjust
the time waiting with the 'wait_timeout' option.

ex.:

  use_wait_feature = 1


==== wait_timeout
Amount of seconds to wait until a rescheduled check finishes. Thruk
will wait this amount and display the result immediately.

ex.:

  wait_timeout = 10


==== mobile_agent
Specify user agents which will be redirected to the mobile plugin (if
enabled).

ex.:

  mobile_agent=iPhone,Android,IEMobile


==== show_error_reports
Show link to bug reports when internal errors occur.

ex.:
  show_error_reports = 1


==== skip_js_errors
don't report some known harmless javascript errors

ex.:
  skip_js_errors  = cluetip is not a function


==== no_external_job_forks
Normally reports will be generated in an external process to avoid
timeouts on long running reports. Use this switch to turn external
jobs off and generate reports directly. Make sure they are finished
within 40seconds which is the default fcgi timeout.

ex.:

  no_external_job_forks = 1


==== cron_file
Specifiy a file which is then completly under the control of Thruk.
It will be used to store cronjobs, ex. for reports. The file has to be
writable by Thruk.

ex.:

  cron_file = /tmp/thruk_cron.tmp


==== cron_pre_edit_cmd
The pre edit cmd can be used to do run a command just before Thruk
will edit the crontab.

ex.:

  cron_pre_edit_cmd  = /usr/bin/crontab -l > /tmp/thruk_cron.tmp


==== cron_post_edit_cmd
The post edit cmd is necessary for OMD where you need to reload the
crontab after editing or for replacing the users cron with the edited
file.

ex.:

  cron_post_edit_cmd = crontab /tmp/thruk_cron.tmp


==== thruk_bin
Path to your thruk executable. Will be used in cronjobs.

ex.:

  thruk_bin = /usr/bin/thruk


==== thruk_init
Path to your thruk init script. Will be used to restart thruk.

ex.:

  thruk_init = /etc/init.d/thruk


==== report_nice_level
Execute regular scheduled reports with this nice level.

ex.:

  report_nice_level = 5


==== report_base_url
Url used to replace relative links in html reports.

ex.:

  report_base_url = http://host.local/thruk/cgi-bin/


==== logcache
Enables caching logfiles for faster access. Cache supports MongoDB
only. Format is a MongoDB connection string like 'hostname:port/db'.
Using a cache dramatically decreases cpu and memory usage of Thruk
when accessing logfiles, for example when creating reports.

ex.:

  logcache = localhost:27017/thruk_log_cache


==== cookie_auth_login_url
Specifies the url where non-authenticated users will be redirected
too.

ex.:

  cookie_auth_login_url = thruk/cgi-bin/login.cgi


==== cookie_auth_restricted_url
Specifies the url against the cookie auth provider will verifys the
credentials.

ex.:

  cookie_auth_restricted_url = http://localhost/thruk/cgi-bin/restricted.cgi


==== cookie_auth_session_timeout
Specifies the timeout for idle sessions.

ex.:

  cookie_auth_session_timeout       = 86400


==== cookie_auth_session_cache_timeout
Specifies the amount of seconds in which subsequent requests won't
verify authentication again.

ex.:

  cookie_auth_session_cache_timeout = 5


==== enable_shinken_features
This one activates all problem/impact and criticity features.
Currently it will only work with shinken backends. Dont enable it
unless all your backends are shinken.
If not set, it will be automatically enabled when using only
shinken backends.

ex.:

  enable_shinken_features = 1


==== priorities
Set the names of the priority (criticity in shinken). Currently this
will only work with shinken backends.

ex.:

  <priorities>
    5   = Business Critical
    4   = Top Production
    3   = Production
    2   = Standard
    1   = Testing
    0   = Development
  </priorities>


==== enable_icinga_features
This one activates all icinga specific features.
If not set, Thruk will try to auto-detect your backends.
Currently autodetection will only work within OMD. Dont enable it
unless all your backends are icinga.

ex.:

  enable_icinga_features = 1


==== check_local_states
Get the status for remote backends from local instances. This can
increase performance when using multiple remote sites. It is enabled
by default when using more than one site. You have to define
hostchecks in any local backend (using unix sockets) with a name or
alias of the address of your remote backends.

ex.:

  check_local_states = 1


==== backend_debug
Set logging of backend in verbose mode. This only
makes sense when debug logging is activated.

ex.:

  backend_debug = 1


==== Component Thruk::Backend
Enter your backend connection settings here. At the moment only
livestatus is supported. The port is the port from the
xinetd.conf. You can enter local unix sockets too.

  * peer
         ** name     name for this connection
         ** type     type of this connection. Only 'livestatus' possible at the
                     moment
         ** hidden   should this peer be hidden initially ( can be reenabled
                     via gui switch ) Only useful with more than one
                     backend.
         ** groups   if set, only contacts from these groups have access. You
                     may add multiple groups seperated by comma. Users
                     without the right contactgroup don't even see
                     that there is a backend. Note that this implies
                     one extra backend request per page.
         ** section  to group backends/sites by different sections, enter a
                     section.
         ** options
            *** peer            address of this connection.
            *** resource_file   resource_file for this peer (used for macro replacement)
         ** configtool
            *** core_type       Give the config parser a hint about
                                your config. Can be 'nagios', 'icinga'
                                or 'shinken'.
            *** core_conf       Path to your nagios.cfg / icinga.cfg. Read all
                                object directories and files from this config
                                file.
            *** obj_check_cmd   Commandline to verify the config.
            *** obj_reload_cmd  Commandline to reload the config.
            *** obj_readonly    Filename pattern to define readonly objects.
                                For example for generated config files.
            *** obj_dir         Path to your objects. Enables the objects editor.
                                Reads all *.cfg from this folder and all subfolders.
                                (only needed when not using 'core_conf')
            *** obj_file        Path to a single objects file. Enables the objects
                                editor. Both 'obj_dir' and 'obj_file' can be
                                specified more than once.
                                (only needed when not using 'core_conf')
            *** obj_exclude     Specify some exection pattern for the obj_dir.
                                (only needed when not using 'core_conf')

ex.:

  <Component Thruk::Backend>
    <peer>
        name   = Local Nagios
        type   = livestatus
        hidden = 1             # makes this backend hidden by default
        groups = admins,locals # makes this backend only visible to the
                               # admin and the locals contactgroup
        <options>
            peer       = /tmp/livestatus.socket
        </options>
        <configtool>
            core_conf  = /etc/nagios/nagios.cfg
            obj_check  = /etc/init.d/nagios checkconfig
            obj_reload = /etc/init.d/nagios reload
        </configtool>
    </peer>
    <peer>
        name   = External Icinga
        type   = livestatus
        <options>
            peer   = 172.16.0.2:9999
       </options>
    </peer>
    <peer>
        name   = External Shinken
        type = livestatus
        <options>
            peer   = 172.16.0.3:50000
       </options>
    </peer>
  </Component>



==== Component Thruk::Plugin::ConfigTool

Enable config tool by setting path to different components config
files. Users with the roles 'authorized_for_configuration_information'
and 'authorized_for_system_commands' will then have access to the
config tool.
You don't have to restart Thruk when changing the config with the
config tool.

  * general options for the config tool
      - show_plugin_syntax_helper    Enable/Disable the plugin syntax
                                     helper. When enabled, Thruk will
                                     run the plugins with "plugin -h"
                                     to get the help information.

  * you can manage different types of files with the config tool:
      - thruk               Path to your thruk_local.conf. Enables
                            adjusting Thruks config if set.
      - cgi.cfg             Path to your cgi.cfg. Enables adjusting the
                            cgi.cfg if set.
      - htpasswd            Path to your htpasswd. Enables user management
                            based an apaches basic auth with htpasswd.

ex.:

  <Component Thruk::Plugin::ConfigTool>
    show_plugin_syntax_helper = 1
    thruk                     = .../thruk_local.conf
    cgi.cfg                   = .../cgi.cfg
    htpasswd                  = .../htpasswd
  </Component>


==== Component Thruk::Plugin::Panorama

The 'Panorama' plugin is a nice, fully customizable dashboard allowing
you to build your own panorama views.

  * general options for the config tool
      - state_provider    Sets the way states between reloads are
                          preserved.
                          Valid options are 'cookie' or 'server'.
                          Cookie store is usefull for demo systems
                          where lot people share the same account.
      - default_view      Can be either the string of an exported
                          view or path to a file with exported view.

ex.:

  <Component Thruk::Plugin::Panorama>
    state_provider  = server
    default_view    = /var/lib/thruk/default_panorama_view
  </Component>



=== cgi.cfg

The cgi.cfg is mainly the same as in Nagios or Icinga but not all
values are used.


==== show_context_help
Displays an icon with context specific help on most pages.

ex.:

  show_context_help = 1


==== use_authentication
Determines whether to use authentication or not. If enabled, Thruk will
use the REMOTE_USER from the apache environment as authenticated user
name. So the authentication has to be done by the webserver.
Authorization is done by Thruk.

ex.:

  use_authentication = 1


==== use_ssl_authentication
If enabled, the authenticated username is taken from the
SSL_CLIENT_S_DN_CN environment instead of the remote user.

ex.:

  use_ssl_authentication = 1


==== default_user_name
The default user will be used if no username has been provided by the
webserver. You don't have to set a default user.

ex.:

  default_user_name = thrukadmin


==== authorized_for_system_information
List of usernames who have access to the system and process
information pages.
You may use wildcards here.

ex.:

  authorized_for_system_information = thrukadmin


==== authorized_for_configuration_information
List of usernames who have access to the configuration
information pages.
You may use wildcards here.

ex.:

  authorized_for_configuration_information = thrukadmin


==== authorized_for_system_commands
List of usernames who are allowed to send system commands.
You may use wildcards here.

ex.:

  authorized_for_system_commands = thrukadmin


==== authorized_for_all_services
List of usernames who are authorized to view all services.
You may use wildcards here.

ex.:

  authorized_for_all_services = thrukadmin


==== authorized_for_all_hosts
List of usernames who are authorized to view all hosts.
You may use wildcards here.

ex.:

  authorized_for_all_hosts = thrukadmin


==== authorized_for_all_service_commands
List of usernames who are authorized to send commands for all
services.
You may use wildcards here.

ex.:

  authorized_for_all_service_commands = thrukadmin


==== authorized_for_all_host_commands
List of usernames who are authorized to send commands for all
hosts.
You may use wildcards here.

ex.:

  authorized_for_all_host_commands = thrukadmin


==== refresh_rate
Number of seconds after which most pages are refreshed automatically.

ex.:

  refresh_rate = 90


==== escape_html_tags
Determines whether html output from plugins is escaped or not.

ex.:

  escape_html_tags = 1


==== action_url_target
Sets the target of the action url links.

ex.:

  action_url_target = _blank


==== notes_url_target
Sets the target of the notes url links.

ex.:

  notes_url_target = _blank


==== lock_author_names
If enabled, user are not allowed to change the author name for
commands.

ex.:

  lock_author_names = 1



=== log4perl.conf

This file contains the log4perl configuration. You can configure whatever is
possible in log4perl. Refer to the log4perl manual for detailed
information:
http://log4perl.sourceforge.net/releases/Log-Log4perl/docs/html/Log/Log4perl/Config.html
There is a log4perl.conf.example file shiped with the package.

ex.:

 log4perl.logger=ALL, ErrorLog, DebugLog
 log4perl.appender.ErrorLog=Log::Dispatch::File
 log4perl.appender.ErrorLog.filename=/home/thruk/Thruk/logs/error.log
 log4perl.appender.ErrorLog.mode=append
 log4perl.appender.ErrorLog.Threshold=ERROR
 log4perl.appender.ErrorLog.layout=Log::Log4perl::Layout::PatternLayout
 log4perl.appender.ErrorLog.layout.ConversionPattern=[%d][%H][%p][%c] %m%n


[TIP]
.command logging
=======
if you want to log all sent commands, just set the loglevel to INFO.

ex.: log4perl.appender.ErrorLog.Threshold=INFO
=======




=== menu.conf

This file contains the default side menu configuration. Do not change
it as this file will be overwritten with every update. If you want to
create a complete custom navigation, just copy this file to
menu_local.conf and adjust it to your needs. See 'menu_local.conf' on
how to add just add a few links without having to copy the complete
file.



=== menu_local.conf

This file contains the user configuration for the side menu. See the
http://www.thruk.org/faq.html#_how_to_change_the_side_menu[FAQ] for
some examples.

The file itself is perl syntax, so you can do whatever perl can do.
Make sure you verify the syntax after changing the file. The changes
will be used immediately. So maybe you want to test your changes
on a test instance first.

[TIP]
.syntax check for menu configuration
=======
 %>perl -wc menu_local.conf
 menu_local.conf syntax OK
=======


If you just want to add a few entrys, create a empty menu_local.conf
and put these lines into it:

-------
do '/usr/share/thruk/menu.conf';
insert_item('General', { 'href' => 'http://labs.consol.de', 'name' => 'Labs', target => '_blank' });
-------




=== Server Side Includes

You can place static include files or executables in the ssi folder.
The static files will be place right after the body or just before the
end of the body.

IMPORTANT: This has *nothing to do* with the Apache module 'mod_include'
and just implements the nagios way of SSIs.

There are two global includes, which will be included in every cgi
page:

* ssi/common-footer.ssi
* ssi/common-header.ssi

and there are page specific includes, which will only be placed in the
specific page:

* ssi/status-footer.ssi
* ssi/status-header.ssi

Files will be statically placed inside the output of the normal cgi
output. Executables will be executed and the output will then be used
as content.



== Examples

Here is a list of common tasks and configurations.

=== CGI Parameters

Most of the CGI Parameters can be changed with buttons, but there are
some which are only accessible by adding them directly to the url.


==== hidetop

Pages: status pages

Description: Show/Hide the status totals header.

Example: /thruk/cgi-bin/status.cgi?hidetop=1


==== nav

Pages: all pages

Description: Show/Hide the navigation. Only usefull in non-frames mode.

Example: /thruk/cgi-bin/tac.cgi?nav=0


==== noheader

Pages: status pages

Description: Show/Hide the status totals header.

Example: /thruk/cgi-bin/status.cgi?noheader


==== minimal

Pages: status page

Description: Hides almost any user interface buttons. Usefull for
monitor screens where only the current problems should be displayed.

Example: /thruk/cgi-bin/status.cgi?host=all&servicestatustypes=28&minimal=1


==== refresh

Pages: all pages

Description: Sets refresh timer to a custom value.

Example: /thruk/cgi-bin/status.cgi?refresh=30



=== SLA Reporting

SLA Reporting consists of several parts.

 - report configuration
 - master pdf
 - template (contains mail text and pdf content)

==== Report Configuration

You can create and adjust report setttings on the 'Reporting' page.
Make sure the plugin is enabled. After selecting a template for your
report you have to set specific types of input fields on the second
page. If you make you report public, eveyone can view and refresh the
report.

==== Master PDF

The master pdf is used as source for the pdf report and used together
with the values filled in from the 'template' to generate the pdf
file. You may adjust the shipped 'sla.pdf' to your needs and put it
into a 'pdf' subfolder of the 'user_template_path'.

==== Template

The template defines the dynamic content of the sla report. You may
copy the 'sla.tt' file into a 'pdf' subfolder of the
'user_template_path' and adjust it to your needs.

See http://thruk.org/api/Thruk/Utils/PDF.html for details on
available functions and examples.

==== Sending Reports By Mail

It's quite easy to setup sla reports by mail. All you need is to
create send option in the report configuration. It is even possible to
send one report at multiple times.



=== CLI Tool & Scripting

It is possible to do comprehensive scripting with Thruk. For example
set downtimes automatically, schedule reports or change the objects
of your monitoring core configuration.

See http://thruk.org/api/Thruk/Utils/CLI.html for details and examples.
