If you set up ELK (elastic search, logstash, kibana) using a guide, you may get the following error:
“Unable to fetch mapping. Do you have indices matching the pattern”
This indicates that Kibana can’t find log data in elastic search. When the logstash service starts correctly, it creates this database (one per day, apparently).
Apparently if it encounters errors on some log file, it will just give up, rather than indexing the log files it can access.
You can see this by doing this:
tail -n 50 /var/log/logstash/logstash.log
For instance, I had to fix several errors:
{:timestamp=>"2015-12-21T15:07:01.931000+0000", :message=>"Error: No config files found: /etc/ash/conf.d/*\nCan you make sure this path is a logstash config file?"}
{:timestamp=>"2015-12-21T15:07:01.950000+0000", :message=>"You may be interested in the '--const' flag which you can\nuse to validate logstash's configuration before you choose\nto restartnning system."}
This meant I used a guide intended for an older version of logstash – the “host” entry changed to “hosts”:
{:timestamp=>"2015-12-21T15:10:24.956000+0000", :message=>"Error: The setting `host` in pluginsticsearch` is obsolete and is no longer available. Please use the 'hosts' setting instead. Yo specify multiple entries separated by comma in 'host:port' format. If you have any questions this, you are invited to visit https://discuss.elastic.co/c/logstash and ask."}
{:timestamp=>"2015-12-21T15:10:24.968000+0000", :message=>"You may be interested in the '--const' flag which you can\nuse to validate logstash's configuration before you choose\nto restartnning system."}
And this meant logstash couldn’t read all the files I gave it – the solution was to add the logstash user to the adm group:
{:timestamp=>”2015-12-21T17:15:52.282000+0000″, :message=>”failed to open /var/log/auth.log: Permission denied – /var/log/auth.log”, :level=>:warn}
{:timestamp=>”2015-12-21T17:15:52.296000+0000″, :message=>”failed to open /var/log/kern.log: Permission denied – /var/log/kern.log”, :level=>:warn}