Small Things about Big Data

Common questions that bother most IT decision makers today are:

• Do I need a Big Data solution?
• How will my organization benefit from it?
• Should I implement one now or wait for a more opportune time?
• If I implement one, what is the scale I should look at?

Before answering these questions, it might be helpful to understand some “Small things about Big Data.”

An obvious requirement of any IT solution is the ability to provide reports on a regular basis. These reports represent anything from financial health, project status, and customer data to audit & security related information.  

Most times, organizations tend to generate reports during low peak periods to ensure their infrastructure is highly utilized and gives better ROI.  However, some of these reports if available at more regular intervals in a 24 hour period will benefit the organization much more than just generating them once in a 24 hour period.  This is nothing but Analytics.

Any workload that analyzes static or streaming data to produce reports useful for business & IT are essentially called “Analytic Workloads.”  

As evident from this case study at Coca Cola, timely delivery of reports can make a huge difference to an organization’s business and strategy.

http://www-01.ibm.com/common/ssi/cgi-bin/ssialias?subtype=AB&infotype=PM&appname=STGE_TS_ZU_USEN&htmlfid=TSC03243USEN&attachment=TSC03243USEN.PDF

               

Therefore, for organizations to understand if they need to ride the Big Data wave or not, they first should identify the analytical pieces that form their IT backbone.  Important questions here would be:

1. What are the critical reports generated?
2. When are these reports generated? (time of day)
3. Are they based upon accurate and timely data?
4. If the information from these reports can be generated at a higher frequency or on-demand, will it enable quicker decision making or faster go-to market strategy?
5. If the answer to 4 is, “Yes” for certain reports, what stops the organization from generating these reports at a higher frequency? 

If the answer to question 3 is “No,” the first step is to address data quality with the business.  This gets into data structure and data governance.  It also requires cooperation with the business as the data owners.

Moving down the list, most often, the organizations I work with say the answer to question 5 lies in availability of IT resources needed to generate these reports.   A common reason to generate reports in the night (or during low peak window) is to avoid performance issues during peak load and to use resources effectively.

To address this, I see the need for a thought process shift to view analytics as an investment that can help bring higher efficiency to an organization’s business, thereby staying ahead of the competition.  Business cases that are developed around the opportunities these insights will bring to the business tend to be the most successful ones.  They justify investing in the right tools to facilitate this versus trying to limp along with analytics workloads taking second priority over other workloads.  

From the standpoint of balancing optimization of resources with business value potential, my experience has been the best way to get better and timely analytical reports is to create a separate set of analytics workloads that can work in real-time using faster systems & storage.   Compute and Storage systems have matured in terms of performance to an extent where one can run real-time reports without impacting the performance of online workloads.  And if the data analyzed contain untapped insights, the business value usually justifies using this approach.

Having identified analytics workloads and the need, the next step is to understand “Why Big Data?”  For any organization that benefits from real-time data sources like social media, complaint records, and service requests etc., Big Data is the way to go. These data sources helps organizations arrive at quick decisions and react quickly to the market or customer needs.

A common example is a customer recording their experience in a public domain which can either help or harm an organization’s reputation. In such cases, when real-time data are made available to the customer service team, they can quickly step-in, understand the situation and help manage the expectation with the respective customer.   However, there are many more.  Customers use these insights to target marketing to customers more effectively, identify patent infringements, identify trademark misuse, improve product design, gauge overall product and brand sentiment, etc…  There are countless strong business uses.

The CIO can initiate the Big Data discussion with the business in business terms.  Often this is a way for the CIO to earn a “seat at the table” with the other business executives.  But, it’s good to be ready by thinking about some practical “small things about big data” before you do.

Article by Subramaniam Meenakshisundaram, a guest contributor and IBM Executive Consultant in the Systems Group Lab Services organization. 

Many thanks Subbu for your guest column!