I am very happy to share with you today the DataGravity Content Pack for Log Insight. It is working its way through VMware Certification but it should be out soon and so I wanted to share with you what it is (update – it is now available).
We should start with what is a content pack? Log Insight is a log consolidation engine that provides machine learning and high performance searching which all help to make it easy to work with logs. But Log Insight doesn’t know much about DataGravity logs so that is where a content pack comes in. Once it is installed in Log Insight, there is Content Pack specific information that can help you understand and work with the event logs. There are many content packs (CP) available – in my lab I use the View, Exchange, Microsoft and Active Directory CPs along with vSphere. Of course, I also use the DataGravity CP too so I am happy to share it with you.
So lets get going!
You need to have a few things in order to make this all work.
- Log Insight 2.5 or later
- DataGravity array firmware 2.0 or later
- DataGravity Content Pack (it is in the Marketplace)
- You should configure your DataGravity array to forward its logs to Log Insight. I like to have it done before I install LI so that as soon as LI is running I see events. Start in System Mgmt, then select Settings \ Syslog Destinations as seen below and follow that with adding your own syslog destination.
You would install the DataGravity CP from the Marketplace by clicking on it.
BTW, if you are not sure how to get to this screen, you point at the top right corner, and select Content Packs from the drop down list.
Once you are in Content Packs change to the Marketplace.
Once you install you will see the screen below.
You can see in the screen above lots of information about the DataGravity CP. Ignore the version number as I suspect it might just say 1.0 on release.
So the CP is now loaded. Time to explore.
There is a number of dashboards (four), each with a collection of widgets.
If you change to the Dashboards view, and select DataGravity you will likely see the screen below. It is the first screen that normally you will see. It is the Overview screen.
There is some important things you may already know but I would like to mention. The little i in each of the widgets means Information. If you click on it there will be a popup of information on that widget. If you see something displayed in the widget you would like to know more about you can click on the magnifying glass and it will move you to the query of the widget in Interactive Analytics. On the various dashboards you will see a filter line at the top of the screen. You can use that to filter what is seen in the dashboard and its widgets.
So in this Overview dashboard you can see all of the array events in the top left, and any actives of export which means creating / publishing mount points in the top right. In the bottom left you see DiscoveryPoint activities. This is like when the system, or a user, takes a DiscoveryPoint, and all of the associated actions with it. In the bottom right you see the number of arrays you have. But this is a little confusing as the array has two controllers, and each can send out traffic.
What is this dashboard good for? It provides a good overview of the array. How busy is it?
This is something where the widgets should normally be empty! You can see my screen below.
If you see any errors in the first or the third widgets you should call DataGravity support. If you see errors in the second, like I do above, you should use the magnifying glass to see what is going on. You may, or may not, need to call support. If you are not sure then call.
What is this dashboard good for? Generally it should show mostly nothing. And if it does you call for help and tell support what you see. But if you see nothing likely you have nothing to worry about.
This dashboard is composed of user related information.
So we see at the top of this screen the ability to filter on time. But below that we can use extracted fields to do more precise filtering. When you do filtering here it impacts all of the widgets.
- Storage Admin Activities – The first widget – top left – is about showing the number of events that occur when people are working on the array and are members of the Storage Admin role. It will report the username in the form of domain\user_name. I would expect many of our customers to not require that role and so it will be blank.
- Root Activities – As well, in the second widget, the number of events of the root user, should also normally be blank. People will not use the root account to work – after all how would you know who did what if the root account was used – don’t we all need accountability in this day and age?
- Count of Super User events – In the third widget, you see the count of Super User events and it is sorted by user. As you can see above all of the super user activity was done by the two users. If you were to use the magnifying glass you would see the all of the activities behind this widget.
- Download events – In the fourth widget you can see the download activities. Meaning when someone downloads a file. You will also see who did the download.
- Count of Audit User – in the fifth widget you see the events for the audit user. This will show the events that a member of the audit role does. It will show the name as domain\user_name.
- Preview events – in the sixth widget you see all of the events where people previewed a file.
- DiscoveryPoint Activity – in the seventh widget you can see DiscoveryPoint traffic based on people.
What is this dashboard good for? It helps you keep track of your users. For most customers root accounts are not supposed to be used so that widget should be blank. Do you have one user who is downloading a lot of stuff – maybe good to know who and what. The DataGravity Audit Role is not read only so keep track of what the user in that role has done is handy.
This dashboard is about watching for content alerts about tags.
If you select the magnifying glass on this one you will see something like below.
Now you see the events that went into the query that was the widget. Notice in each event that there is a URL? That will take you to the search that generated this content alert.
How do we create these content alerts? You first create a search that is public, and then you create a Content Alert. You can do this for your important tags, and then create widgets and dashboards to highlight as I have done in this one widget. Consider it an example.
What is this dashboard good for? It is a sample of how to do a widget for content alerts. So you could have a content alert for when unencrypted pictures show up on your array. This will be alerted when that happens. That would need you to do an Alert to pair with the widget.
There are a number of RegEx created extracted fields. You can use them to create filters of your own. Here is a list of them.
- dg_component – this is used when a sub component creates the event. Not always present.
- dg_role – super user, storage admin, end user, audit user, are the roles and you can use this field to filter on any or all of them.
- dg_severity – this is roughly equivalent to syslog Levels.
- dg_status – success or fail of DataGravity actions.
- dg_user – domain and user name when they are associated with an event.
There are three alerts that have been included with the Content Pack as examples. They are:
- Platform Errors – things worth calling support over.
- Alert on Super User activity on the array – easy to change to root user if you like but handy to be alerted when they log in if you are a very compliant show.
- Audit User Logged in – generate an email when an audit user logs in – handy as Audit users are not read only.
You will need to configure and enable them. You can find them under Manage Alerts after selecting the red bell.
I do recommend getting the Platform Error one configured and enabled so you can be proactive as necessary about dealing with support issues.
Improving the DataGravity Content Pack
If you have ideas on how to improve the DataGravity CP please let me know below or email dglabs at datagravity.com. Any suggestions would be quite appreciated.
You have seen a lot of what you can do with the DataGravity CP and Log Insight. The cool bit is that you can even do your own thing that no one else has thought of! You can also make suggestions on how to improve the CP. Remember the widgets and dashboards in this CP are just the start. Not only will we update them, but you can too.
BTW, I will update this as soon as the CP is approved and available via the Marketplace which hopefully should be very soon.
- 8/19/15 – Updated for the fact that the CP is in the Marketplace
- 8/19/15 – I thought I should add the info on how to configure the DataGravity array to do log forwarding.
- 8/17/15 – GA
As always, questions or comments welcome,
=== END ===