Introduction
Thruk is an multibackend monitoring web interface which currently supports Nagios and Icinga using the Livestatus NEB module. It is designed to be a "drop-in" replacement and targets covering 100% of the original features plus additional enhancements for large installations.
Main Features / Advantages
-
multiple backends
-
faster, no need to parse status.dat for every request
-
less cpu usage
-
display live data, no delay between core and gui
-
independant from monitoring core, can be installed on remote host
-
clusterable, can be clustered over hosts
-
change status.cgi filter inline
-
extended logfile search
-
better logging, uses log4perl (who submitted which command?)
-
valid HTML code
-
no frames ( can be reenabled for better addon integration)
-
easy to extend with plugins
-
100% Perl (using Catalyst Framework)
-
paging ( can be disabled and customized )
-
multiple themes included
How it works
Thruk is written in Perl using the Catalyst Framework. Backend monitoring systems will be connected with the Monitoring::Livestatus Perl Module. Thruk itself is running as a fastcgi process. Availability will be calculated with Monitoring::Availability. Authentication is provided by the Apache webserver (For example with mod_auth).
Installation
The clean and proper way to install Thruk is to create a new local user and install the missing perl modules with local::lib.
Use Package
Installation from a package is easy and should be painless:
|
you may need to install libgd and libxpm. |
%> arch=$(perl -e 'use Config; print $Config{archname}') %> wget http://www.thruk.org/files/Thruk-0.60-$arch.tar.gz %> tar zxf Thruk-0.60-$arch.tar.gz %> cd Thruk-0.60 %> ./script/thruk_server.pl
For the FastCGI installation follow the steps at fastcgi installation.
Quick Installation Guide for Debian
1. install debian netinst - minimal inst, create user thruk 2. add "contrib non-free" to /etc/apt/sources.list 3. #> apt-get install libapache2-mod-fastcgi libfile-copy-recursive-perl libmodule-install-perl \ make gcc git git-core libgd2-xpm-dev 4. #> su - thruk 5. %> cd /tmp 6. %> wget http://search.cpan.org/CPAN/authors/id/A/AP/APEIRON/local-lib-1.004009.tar.gz 7. %> tar zxvf local-lib-1.004009.tar.gz 8. %> cd local-lib-1.004009 9. %> perl Makefile.PL --bootstrap 10. %> make install 11. %> echo 'eval $(perl -I$HOME/perl5/lib/perl5 -Mlocal::lib)' >>~/.bashrc 12. %> . ~/.bashrc 13. %> cd && git clone http://github.com/sni/Thruk.git 14. %> cd Thruk 15. %> perl Makefile.PL && make 16. %> cp thruk.conf thruk_local.conf && vi thruk_local.conf 17. %> vi cgi.cfg 18. %> ./script/thruk_server.pl
Install from Source
Requirements
In order to install the Thruk Monitoring Webinterface from source for Nagios/Icinga you will need the following:
-
Perl
-
Git Client
-
compiler tools: automake, make, g++, gcc
-
Apache Webserver (optional for fastcgi only)
Create New User
refer to your systems manual on howto add new user. This guide uses the following:
user: thruk group: thruk
|
all following steps should be done by the thruk user. |
Install Local::Lib
Follow the steps on http://search.cpan.org/perldoc?local::lib#The_bootstrapping_technique
quick guide:
%> su - thruk %> wget http://search.cpan.org/CPAN/authors/id/A/AP/APEIRON/local-lib-1.004009.tar.gz %> tar zxf local-lib-1.004009.tar.gz %> cd local-lib-1.004009 %> perl Makefile.PL --bootstrap && make install
|
proxy configuration
if you need a proxy configuration, you should answer <no> at this question: Would you like me to configure as much as possible automatically? [yes] Setting a proper http_proxy/ftp_proxy environment should work for the automatic configuration. |
Add the following line to the thruk users .profile or .bashrc
eval $(perl -I$HOME/perl5/lib/perl5 -Mlocal::lib)
logout and login again and verify that your perl env looks like this:
%> env | grep perl PERL5LIB=/home/thruk/perl5/lib/perl5:/home/thruk/perl5/lib/perl5/x86_64-linux-gnu-thread-multi MODULEBUILDRC=/home/thruk/perl5/.modulebuildrc PATH=/home/thruk/perl5/bin:/usr/local/bin:/usr/bin:/bin:/usr/games PERL_MM_OPT=INSTALL_BASE=/home/thruk/perl5
Install Module::Install
perl -MCPAN -e "install Module::Install::Catalyst"
Git Clone Thruk
git clone http://github.com/sni/Thruk.git cd Thruk perl Makefile.PL
press <enter> here:
==> Auto-install the 35 mandatory module(s) from CPAN? [y]
then run make
make
This may take a while, as there are probably several modules missing.
Press <y> at this questions:
Do you want to build the XS Stash module? [y] Do you want to use the XS Stash by default? [y]
Tell me if someone knows a way to answer this questions automatically.
run perl Makefile.PL again to see if all dependecies are now installed properly.
perl Makefile.PL
Install Livestatus
Refer to http://mathias-kettner.de/checkmk_livestatus.html#H1:%20Setting%20up%20and%20using%20Livestatus on how to install livestatus onto your monitoring box.
Basically you have to build the NDO addon and add it to your nagios/icinga.cfg.
broker_module=/opt/local/livestatus/livestatus.o /tmp/live.sock
Configuration
Copy "thruk.conf" to "thruk_local.conf" and adjust the livestatus settings to your needs.
%> cp thruk.conf thruk_local.conf
The thruk.conf will be overwritten with new defaults on updates. The thruk_local.conf contains the local overrides and will never be overwritten.
Edit "cgi.conf" and adjust settings to your needs.
See the [Configuration] section for detailed explaination of configuration options.
Start Server
After running these steps successfully, you should be able to test your installation:
./script/thruk_server.pl
Open your browser and open http://<your_host>:3000
FastCGI
There are two fast cgi modules for apache at the moment. Choose the one which fits best into your environment.
mod_fastcgi
start your fcgi server:
%>./script/thruk_fastcgi.pl -n 5 \ -l /tmp/thruk_fastcgi.socket \ -p /tmp/thruk_fastcgi.pid
you may want to copy the init.d script to /etc/init.d and adjust its paths:
%> sudo cp ./script/thruk_fastcgi_server.sh /etc/init.d/thruk_fastcgi_server %> vi /etc/init.d/thruk_fastcgi_server %> sudo chown root: /etc/init.d/thruk_fastcgi_server
Or create a custom init.d script (additional modules required) with:
%> ./script/thruk_create.pl FastCGI::ExternalServer l=/tmp/thruk_fastcgi.socket n=5 p=/tmp/thruk_fastcgi.pid
Apache Configuration
use this apache example configuration:
-
replace /home/thruk/Thruk with your installation path
-
replace your-web-host.local with your hostname
-
create a /home/thruk/Thruk/htpasswd.users with htpasswd2
-
make sure the /home/thruk/Thruk/logs/ directory exists
<VirtualHost *:80> # ... existing configuration # thruk configuration <Directory /home/thruk/Thruk/root/> order allow,deny allow from all </Directory> Alias /thruk/ /home/thruk/Thruk/root/thruk/ <Location "/thruk"> AuthName "Monitoring Access" AuthType Basic AuthUserFile /home/thruk/Thruk/htpasswd.users Order Allow,Deny Allow from all require valid-user </Location> # Load fastcgi module unless already loaded LoadModule fastcgi_module /usr/lib/apache2/modules/mod_fastcgi.so # fastcgi configuration FastCGIExternalServer /tmp/thruk_fastcgi.fcgi -socket /tmp/thruk_fastcgi.socket -idle-timeout 120 # Load rewrite module unless already loaded LoadModule rewrite_module /usr/lib/apache2/modules/mod_rewrite.so # rewrite configuration RewriteEngine On RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^/thruk(.*)$ /tmp/thruk_fastcgi.fcgi/thruk$1 [E=HTTP_AUTHORIZATION:%{HTTP:Authorization},QSA,L] </VirtualHost>
<VirtualHost *:80> ServerName thruk.your-host.local DocumentRoot /home/thruk/Thruk/root/ CustomLog /home/thruk/Thruk/logs/access.log combined ErrorLog /home/thruk/Thruk/logs/error.log <Directory /> order deny,allow deny from all </Directory> <Directory /home/thruk/Thruk/root/> order allow,deny allow from all </Directory> # authorization <Location "/"> AuthName "Monitoring Access" AuthType Basic AuthUserFile /home/thruk/Thruk/htpasswd.users Order Allow,Deny Allow from all require valid-user </Location> # Load fastcgi module unless already loaded LoadModule fastcgi_module /usr/lib/apache2/modules/mod_fastcgi.so # fastcgi configuration FastCGIExternalServer /tmp/thruk_fastcgi.fcgi -socket /tmp/thruk_fastcgi.socket -idle-timeout 120 # Load rewrite module unless already loaded LoadModule rewrite_module /usr/lib/apache2/modules/mod_rewrite.so # rewrite configuration RewriteEngine On RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^/(.*)$ /tmp/thruk_fastcgi.fcgi/$1 [E=HTTP_AUTHORIZATION:%{HTTP:Authorization},QSA,L] </VirtualHost>
mod_fcgid
|
Thruk runs with the webserver user in this scenario, make sure the webserver user has access to all files and perl modules. |
LoadModule fcgid_module /usr/lib/apache2/modules/mod_fcgid.so <VirtualHost *:80> ServerName thruk.company.local DocumentRoot /home/thruk/Thruk/root/ CustomLog /home/thruk/Thruk/logs/access.log combined ErrorLog /home/thruk/Thruk/logs/error.log <Directory /home/thruk/Thruk/root/> order allow,deny allow from all </Directory> AliasMatch /thruk/(.*\.cgi|.*\.html) /home/thruk/Thruk/script/thruk_fastcgi.pl/thruk/$1 <Location /thruk> Options ExecCGI Order allow,deny Allow from all AuthName "Monitoring Access" AuthType Basic AuthUserFile /home/thruk/Thruk/htpasswd.users Require valid-user </Location> <IfModule mod_fcgid.c> AddHandler fcgid-script .pl MaxRequestsPerProcess 100 </IfModule> </VirtualHost>
Configuration
Configuration is managed mainly in these configuration files.
-
thruk.conf
-
thruk_local.conf
-
cgi.cfg
-
log4perl.conf
thruk.conf
The thruk.conf contains the shipped defaults. You should not edit this file directly. Overwrite your settings in your thruk_local.conf instead.
thruk_local.conf
The thruk_local.conf is a copy of the thruk.conf. Both files are used by thruk. The idea is, that the thruk.conf is provided with the packaged defaults and the thruk_local.conf is to override these settings where needed.
use_strict_host_authorization
When set to a true value, every contact will only see the hosts where he is contact for plus the services where he is contact for. When disabled, a host contact will see all services for this host regardless of wheter he is a service contact or not.
ex.:
use_strict_host_authorization = 1
start_page
This link is used as startpage and points usually to the main.html with displays version information and general links.
ex.:
start_page = /thruk/main.html
documentation_link
This link is used in the side navigation menu as link to the documentation. Replace it with whatever your documentation is located. Set it to a blank value if you don’t want a documentation link in the menu at all.
ex.:
documentation_link = /thruk/docs/
all_problems_link
Customizable link for the "problems" link in side menu. Can be useful to reflect your companys process of error handling.
ex.:
all_problems_link = /thruk/cgi-bin/status.cgi?...
allowed_frame_links
List of allowed patterns, where links inside frames can be set to. You can link to /thruk/frame.html?link=http://wiki.my-company.com/page/blah Your wiki will then be displayed with the Thruk navigation frame. Useful for other addons, so they don’t have to display a own navigation.
ex.:
allowed_frame_links = http://intranet.my-company.com allowed_frame_links = https://wiki.my-company.com
cgi_cfg
The path to your cgi.cfg. See [cgi.cfg] for details.
ex.:
cgi_cfg = cgi.cfg
delay_pages_after_backend_reload
Delay the page delivery until the backends uptime is at least this amount of seconds. Displaying pages soon after backend restarts may display wrong results and all services are pending. Enable this if you experience problems with pending services after reloading your backend ex.: setting this to 10 would start serving pages 10 seconds after the backend reload
ex.:
delay_pages_after_backend_reload = 10
use_frames
Set whether you want to use a framed navigation or not. With using frames it’s sometimes easier to include addons. See allowed_frame_links option for how to integrate addons.
ex.:
use_frames = 0
use_new_search
Use the old or the classic search from the navigation. The new search supports regular expressions and searches in many attributes. For example: plugin_output, groups, names, descriptions
ex.:
use_new_search = 1
use_ajax_search
Enables the ajax search field. There will be suggestions while typing into the search field.
ex.:
use_ajax_search = 1
ajax_search_hosts
Enables the suggestion of hosts in the ajax search field. Depending on the number of hosts, this can make the search slow.
ex.:
ajax_search_hosts = 1
ajax_search_hostgroups
Enables the suggestion of hostgroups in the ajax search field.
ex.:
ajax_search_hostgroups = 1
ajax_search_services
Enables the suggestion of services in the ajax search field. Depending on the number of services, this can make the search slow.
ex.:
ajax_search_services = 1
ajax_search_servicegroups
Enables the suggestion of servicegroups in the ajax search field.
ex.:
ajax_search_servicegroups = 1
default_theme
Default theme to use for all users. Must be a valid subdirectory in the themes folder.
ex.:
default_theme = Classic
use_pager
Using the pager will make huge pages much faster as most people don’t want a services page with 100.000 services displayed. Can be disabled if you don’t need it.
ex.:
use_pager = 1
default_page_size
Set the default value for paging. Only used when paging is enabled. Should be a value of the paging_steps.
ex.:
default_page_size = 100
paging_steps
Define the single selectable values for paging sizes.
ex.:
paging_steps = 100 paging_steps = 500 paging_steps = 1000 paging_steps = 5000 paging_steps = all
can_submit_commands
Set this if a contact should be allowed to send commands unless defined for the contact itself. This is the default value for all contacts unless the user has a can_submit_commands setting in your monitoring configuration.
ex.:
can_submit_commands = 1
command_disabled
Use this to disabled specific commands. Can be use multiple times to disabled multiple commands. Use the number from the cmd_typ cgi parameter.
ex.:
command_disabled = 14 command_disabled = 35
cmd_defaults
Set the default checked state for command options.
ex.:
<cmd_defaults> ahas = 0 # For Hosts Too broadcast_notification = 0 # Broadcast force_check = 0 # Forced Check force_notification = 0 # Forced Notification send_notification = 1 # Send Notification sticky_ack = 1 # Sticky Acknowledgement persistent_comments = 1 # Persistent Comments persistent_ack = 0 # Persistent Acknowledgement Comments ptc = 0 # For Child Hosts Too </cmd_defaults>
cmd_quick_status
Configure which commands should be available as quick status commands.
ex.:
<cmd_quick_status> reschedule = 0 # Reschedule next check downtime = 0 # Add/remove downtimes comment = 0 # Add/remove comments acknowledgement = 0 # Add/remove acknowledgements active_checks = 0 # Enable/disable active checks notifications = 0 # Enable/disable notifications submit_result = 0 # Submit passive check result </cmd_quick_status>
datetime_format
Default timeformat. Use POSIX format.
ex.:
datetime_format = %Y-%m-%d %H:%M:%S
datetime_format_long
Default long timeformat.
ex.:
datetime_format_long = %a %b %e %H:%M:%S %Z %Y
datetime_format_log
Default log timeformat.
ex.:
datetime_format_log = %B %d, %Y %H
datetime_format_trends
Default trends timeformat.
ex.:
datetime_format_trends = %a %b %e %H:%M:%S %Y
datetime_format_today
Default timeformat for todays date. Can be useful if you want a shorter date format for today.
ex.:
datetime_format_today = %H:%M:%S
info_popup_event_type
On which event should the comments / downtime or longpluginout popup show up. Valid values are onclick or onmouseover.
ex.:
info_popup_event_type = onmouseover
info_popup_options
Options for the popup window used for long pluginoutput, downtimes and comments. See http://www.bosrup.com/web/overlib/?Command_Reference for what options are available
ex.:
info_popup_options = STICKY,CLOSECLICK,HAUTO,MOUSEOFF
Component Monitoring::Livestatus
Enter your livestatus connection settings here. The port is the port from the xinetd.conf. You can enter local unix sockets too.
-
keepalive keeps open the connection to livestatus.
-
use_threads access multiple backends with threads. Usually it’s faster without threads except the backends have a very poor connection
-
peer enter the peer configuration. You can enter a name for each connection.
-
connect_timeout timeout for new connections
-
query_timeout timeout for queries
ex.:
<Component Monitoring::Livestatus> keepalive = 1 use_threads = 0 <peer> name = Local Nagios peer = /tmp/livestatus.socket </peer> <peer> name = External Nagios peer = 172.16.0.1:9999 </peer> <peer> name = External Icinga peer = 172.16.0.2:9999 </peer> </Component>
cgi.cfg
The cgi.cfg is mainly the same as in Nagios or Icinga itself stripped down some unneeded values.
show_context_help
Displays an icon with context specific help on most pages.
ex.:
show_context_help = 1
use_authentication
Determines whether to use authentication or not. If enabled, Thruk will use the REMOTE_USER from the apache environment as authenticated user name. So the authentication has to be done by the webserver. Authorization is done by Thruk.
ex.:
use_authentication = 1
use_ssl_authentication
If enabled, the authenticated username is taken from the SSL_CLIENT_S_DN_CN environment instead of the remote user.
ex.:
use_ssl_authentication = 1
default_user_name
The default user will be used if no username has been provided by the webserver. You don’t have to set a default user.
ex.:
default_user_name = thrukadmin
authorized_for_system_information
List of usernames who have access to the system and process information pages. You may use wildcards here.
ex.:
authorized_for_system_information = thrukadmin
authorized_for_configuration_information
List of usernames who have access to the configuration information pages. You may use wildcards here.
ex.:
authorized_for_configuration_information = thrukadmin
authorized_for_system_commands
List of usernames who are allowed to send system commands. You may use wildcards here.
ex.:
authorized_for_system_commands = thrukadmin
authorized_for_all_services
List of usernames who are authorized to view all services. You may use wildcards here.
ex.:
authorized_for_all_services = thrukadmin
authorized_for_all_hosts
List of usernames who are authorized to view all hosts. You may use wildcards here.
ex.:
authorized_for_all_hosts = thrukadmin
authorized_for_all_service_commands
List of usernames who are authorized to send commands for all services. You may use wildcards here.
ex.:
authorized_for_all_service_commands = thrukadmin
authorized_for_all_host_commands
List of usernames who are authorized to send commands for all hosts. You may use wildcards here.
ex.:
authorized_for_all_host_commands = thrukadmin
refresh_rate
Number of seconds after which most pages are refreshed automatically.
ex.:
refresh_rate = 90
escape_html_tags
Determines whether html output from plugins is escaped or not.
ex.:
escape_html_tags = 1
action_url_target
Sets the target of the action url links.
ex.:
action_url_target = _blank
notes_url_target
Sets the target of the notes url links.
ex.:
notes_url_target = _blank
lock_author_names
If enabled, user are not allowed to change the author name for commands.
ex.:
lock_author_names = 1
log4perl.conf
This file contains the log4perl configuration. You can configure whatever is possible in log4perl. Refer to the log4perl manual for detailed information: http://log4perl.sourceforge.net/releases/Log-Log4perl/docs/html/Log/Log4perl/Config.html There is a log4perl.conf.example file shiped with the package.
ex.:
log4perl.logger=ALL, ErrorLog, DebugLog log4perl.appender.ErrorLog=Log::Dispatch::File log4perl.appender.ErrorLog.filename=/home/thruk/Thruk/logs/error.log log4perl.appender.ErrorLog.mode=append log4perl.appender.ErrorLog.Threshold=ERROR log4perl.appender.ErrorLog.layout=Log::Log4perl::Layout::PatternLayout log4perl.appender.ErrorLog.layout.ConversionPattern=[%d][%H][%p][%c] %m%n
|
command logging
if you want to log all sent commands, just set the loglevel to INFO. ex.: log4perl.appender.ErrorLog.Threshold=INFO |
Server Side Includes
You can place static include files or executables in the ssi folder. The static files will be place right after the body or just before the end of the body.
There are two global includes, which will be included in every cgi page: - ssi/global-footer.ssi - ssi/global-header.ssi
and there are page specific includes, which will only be placed in the specific page: ssi/status-footer.ssi ssi/status-header.ssi
Files will be statically placed inside the output of the normal cgi output. Executables will be executed and the output will then be used as content.
Examples
Here is a list of common tasks and configurations.
Remote Livestatus with SSL
Generate ssl key for the remote machine:
%> FILENAME=remote-livestatus %> openssl genrsa -out $FILENAME.key 1024 %> openssl req -new -key $FILENAME.key -x509 -days 3653 -out $FILENAME.crt %> cat $FILENAME.key $FILENAME.crt >$FILENAME.pem %> chmod 600 $FILENAME.key $FILENAME.pem
Generate ssl key for thruk. Use the same command with a different FILENAME.
%> FILENAME=thruk-host
Now start socat on both machines:
%> /usr/bin/socat \ -L/tmp/liveststatus-socat.lock \ openssl-listen:9999,reuseaddr,fork,cert=/path_to_your/certs/remote-livestatus.pem,cafile=/path_to_your/certs/thruk-host.crt,keepalive,nodelay \ UNIX-CLIENT:/usr/local/nagios/var/live.sock,keepalive
Socat will then listen on port 9999 and connect to the local unix socket from livestatus. Change the paths to your needs.
Start another socat on the thruk host: .Thruk Machine
%> /usr/bin/socat \ -L/tmp/liveststatus-socat.lock \ UNIX-LISTEN:/home/thruk/var/remote.sock,fork,keepalive,mode=0660,group=thruk \ openssl-connect:remote-host:9999,cert=/path_to_your/certs/thruk-host.pem,cafile=/path_to_your/certs/remote-livestatus.crt
Socat will then connect to the remote host on port 9999 and listen on the local unix socket. The tcp connection is encrypted. You can now add the /home/thruk/var/remote.sock to your thruk_local.conf.