Missing User Profile Instance in IONIX

This may be helpful if you logged in and received the following message.

Cannot find instance ‘ICS_UserProfile::ICS-PROFILE-default-profile’

Quick and Dirty:

This means that somewhere along the way a user profile has been removed that is associated with the account that is being used to login with.  This can also mean that the repository has been corrupted.  To fix this, go to <SAM Home>/local/repos/icf and restore the last .rps file.

DNS Lookup on Fields in Splunk

You have your search, but it comes up with a bunch of IPs in the results. What are the host names? How do I translate this into usable data?
<Search> | lookup dnslookup clientip as <IP Field> OUTPUT clienthost as <Resolved Hostname>

So this could be:
<Search> | lookup dnslookup clientip as src_ip OUTPUT clienthost as My_Source_Host

Logging Spikes Alert for Splunk

We recently broke our license limits in Splunk. After going back to find out why, we found that we were being attacked and the firewalls were going crazy spitting out logs. We also found that we were able to find misconfigured devices based on log spikes. We typically log at let’s say 500kbps with a maximum of 700 kbps throughout the day. Taking this I can create a search to find the current rate of indexing. I then take that search and create an alert that will let me know if we spike over 800 kbps so I can go back and find out why we are logging so much.

Export to CSV in Splunk

Typically users can easily click on the export button and export data to .csv. For larger data sets, specifically anything with more than 10,000 lines, Splunk hides the export button and will not allow users to export the search results. This is a built-in safety feature that protects Splunk's performance and is designed to prevent a crash.

Bulk Device Removal in SMARTS\IONIX

So one of your data centers closed and moved across the country. This made all the names and IPs change. Now you have a bunch of down devices that will never come back up. Here is how to remove multiple devices from EMC IONIX.

For this example we will assume you already have a list of devices to remove. To find out how to pull a list of non-responsive devices from the Pending List for go here export-pending-list.

To remove the devices use the following command:

Export Pending List from EMC IONIX

You have a list of new devices sitting in the queue to discover that never seems to change. Why not? Are the devices configured correctly? Did SMARTS\IONIX have problems during discovery?

Here is how to export the device list along with various other information to troubleshoot discovery problems or tell sys admins which devices they need to fix.

"<InCharge Home>/IP/smarts/bin/dmctl -s <DOMAIN_NAME> invoke AD_Pending::AD-Pending getPending"

Help! I need to roll my EMC IONIX Audit Files!

Have you run into problems with large .audit files in EMC SMARTS or IONIX?

For just the .audit log files, you can run the following command or schedule it to run at a specified interval. This would be run for each instance, including IP, NPM, and ICOI.

"<InCharge Home>/<Component>/smarts/bin/dmctl -s <DOMAIN> exec roll_sam_auditlog"

Typical SAM install in /opt/Incharge with a DOMAIN named INCHARGE-SAM the command would be:
"/opt/InCharge/SAM/smarts/bin/dmctl -s INCHARGE-SAM exec roll_sam_auditlog"

SPLUNK Top Hits by Country Report

Getting a lot of foreign computers knocking on your door causing network latency?  This search will show the top countries of origin.  This can be useful in determining firewall rules, to know your enemy, and just for fun information.  To setup SPLUNK to run this search you will need to install the geoip package.  amMap or Google Maps are great plugins if you want a more graphical representation.