Splunk

This is more common with smaller implementations, but in some cases a splunk instance might be running on a multi-homed \ multi-IP box. Say you have several web pages or another application on the same box. Maybe you have some management tools for the box loading that take up a specific IP. If there is a conflict, you may need to bind Splunk to a specific IP.

Why does Splunk allow you to setup Multiple Indexes? One should be enough right?

There are several reasons for creating multiple indexes.

The general rule of thumb in security is if you don't use it, disable it. Why give a potential attacker yet another door to try?

In the Splunk realm, there are often times where installations such as light forwarders, which need no real interaction other than to throw data at the indexers, we can turn off the management port 8089. There is no real reason to keep it open. Changes can still be made using a deployment server and no interaction is needed with that forwarder.

In local/server.conf add the following:

Ever do a search and get extra fields that you don't want to see in your reports?  Here's how to get rid of those fields.

add "| fields - <fieldname>" to the search.  Field names can also be separated by a comma.

index=main | fields - _raw, _time

This can work if you have fields missing and you want to add them to the report.

index=main | fields count, name, src_ip

And it works in conjunction with each other by adding a |.

How do I reset my forgotten admin password in Splunk?

Resetting the Admin Password in Splunk requires file system access.

Backup the $SPLUNK_HOME/etc/passwd file and then delete it.
$ mv $SPLUNK_HOME/etc/passwd $SPLUNK_HOME/etc/passwd.bak

Restart Splunk.
$ $SPLUNK_HOME/bin/splunk restart

Browse to the login page and login with the default user admin and default password changeme.

Note: be sure to cut off external access during this time and change the password ASAP to avoid compromising situations.

Splunk throws the “Max concurrent searches reached.” error to warn you that the system will become overloaded soon.  This is a common error for dashboards and saved scheduled searches.  This can also be a pain if there are a large amount of concurrent users searching.  Here are a few ways to fix this issue.