Climbing the DIKW Pyramid: Applying Data, Information, Knowledge, Wisdom principles at the University of Leeds

Tim Banks
Faculty IT Manager
University of Leeds

One of the many areas of knowledge that the EDUCAUSE conference  helped me to develop was the importance of metrics and monitoring. All good metrics are based upon accurate data, but data isn’t useful on its own or in isolation. Here is one concrete example of how my attendance at EDUCAUSE 2015 has helped to shape my professional development and bring benefits to my institution.

The Information Technology Infrastructure Library (ITIL) framework makes reference to the DIKW pyramid (Data, Information, Knowledge, Wisdom) as can be seen below. Wisdom is based on sound knowledge, which in turn comes from useful information, which is based on accurate data.

final blog image

 

 

 

 

 

Let’s take an example of a typical automated monitoring system. An example of each level of the DIKW pyramid is as follows:

Data
09/01 18:29:45: Message from InterMapper 5.8.1

Event: Critical
Name: website-host.leeds.ac.uk Nagios Plugin
Document: Unix: Webhosting
Address: 129.11.1.1
Probe Type: Nagios Plugin
Condition: CRITICAL – Socket timeout after 10 seconds

Time since last reported down: 39 days, 3 hours, 12 minutes, 47 seconds Device’s up time: N/A

Information
This alert relates to one of our website servers.
This is not normal behaviour.

Knowledge
There is a planned network upgrade in one of our datacentres between 18:00 – 19:00 which is expected to cause network outages.
The server is part of a clustered pair with only one node affected, so service to end users will not be interrupted.

Wisdom
No action is required.

Most systems will generate endless data records. With some careful filtering of the data, it is possible to automatically generate ‘Information’. However, in most cases, ‘Knowledge’ (and in all cases ‘Wisdom’) will need some level of human intervention.

My team have recently started using the University of Leeds IT Service Management system (ServiceNow) and as part of this move, we have updated all of our automated monitoring systems so they now report into one shared email account. Previously,  they were going to various individual and shared email accounts, so we didn’t have a single view of everything. This single shared email account is our data store in the DIKW model. We have then applied a number of rules to identify the subset of alerts from the general notifications. We have defined alerts are something which we have defined as requiring human intervention. This takes us to the information level. These alerts are automatically entered into our Service Management system as incidents, where they are reviewed by a human and acted on as appropriate.

The ultimate goal is to use the configuration management database (CMDB) and change management records to try and automate some of the ‘Knowledge’ layer. e.g. Approved change X will affect the network between 07:00 and 07:30 on 5th May in Data Centre 1 in which server Y is located, so ignore any warnings from this server on this date between these times.

Accurate monitoring is the basis of building meaningful metrics. You cannot generate a useful metric on the ‘number of unplanned service outages in the last six months’ based on data alone. By ensuring that we have a model which allows us to record useful knowledge based on the raw data, we will be able to build some accurate and meaningful metrics.

The sessions I attended on data monitoring and metrics, in particular the one by led by the Consortium for the Establishment of Information Technology Performance Standards (CEITPS), really helped to define this approach and stopped me from falling into the trap of generating endless metrics (of little value) based on data alone. Hearing from other institutions that are further ahead on this journey than us and having the benefit of their advice on what approach to take and what pitfalls to avoid has been invaluable. I am also part of a small group at the University who are responsible for defining the institution-wide IT configuration management standards for recording and managing IT assets. Again, I will be bringing information and knowledge from EDUCAUSE sessions to these discussions.

Leave a Reply

Your email address will not be published. Required fields are marked *