Small Things about Big Data

Common questions that bother most IT decision makers today are:

• Do I need a Big Data solution?
• How will my organization benefit from it?
• Should I implement one now or wait for a more opportune time?
• If I implement one, what is the scale I should look at?

Before answering these questions, it might be helpful to understand some “Small things about Big Data.”

An obvious requirement of any IT solution is the ability to provide reports on a regular basis. These reports represent anything from financial health, project status, and customer data to audit & security related information.  

Most times, organizations tend to generate reports during low peak periods to ensure their infrastructure is highly utilized and gives better ROI.  However, some of these reports if available at more regular intervals in a 24 hour period will benefit the organization much more than just generating them once in a 24 hour period.  This is nothing but Analytics.

Any workload that analyzes static or streaming data to produce reports useful for business & IT are essentially called “Analytic Workloads.”  

As evident from this case study at Coca Cola, timely delivery of reports can make a huge difference to an organization’s business and strategy.

http://www-01.ibm.com/common/ssi/cgi-bin/ssialias?subtype=AB&infotype=PM&appname=STGE_TS_ZU_USEN&htmlfid=TSC03243USEN&attachment=TSC03243USEN.PDF

               

Therefore, for organizations to understand if they need to ride the Big Data wave or not, they first should identify the analytical pieces that form their IT backbone.  Important questions here would be:

1. What are the critical reports generated?
2. When are these reports generated? (time of day)
3. Are they based upon accurate and timely data?
4. If the information from these reports can be generated at a higher frequency or on-demand, will it enable quicker decision making or faster go-to market strategy?
5. If the answer to 4 is, “Yes” for certain reports, what stops the organization from generating these reports at a higher frequency? 

If the answer to question 3 is “No,” the first step is to address data quality with the business.  This gets into data structure and data governance.  It also requires cooperation with the business as the data owners.

Moving down the list, most often, the organizations I work with say the answer to question 5 lies in availability of IT resources needed to generate these reports.   A common reason to generate reports in the night (or during low peak window) is to avoid performance issues during peak load and to use resources effectively.

To address this, I see the need for a thought process shift to view analytics as an investment that can help bring higher efficiency to an organization’s business, thereby staying ahead of the competition.  Business cases that are developed around the opportunities these insights will bring to the business tend to be the most successful ones.  They justify investing in the right tools to facilitate this versus trying to limp along with analytics workloads taking second priority over other workloads.  

From the standpoint of balancing optimization of resources with business value potential, my experience has been the best way to get better and timely analytical reports is to create a separate set of analytics workloads that can work in real-time using faster systems & storage.   Compute and Storage systems have matured in terms of performance to an extent where one can run real-time reports without impacting the performance of online workloads.  And if the data analyzed contain untapped insights, the business value usually justifies using this approach.

Having identified analytics workloads and the need, the next step is to understand “Why Big Data?”  For any organization that benefits from real-time data sources like social media, complaint records, and service requests etc., Big Data is the way to go. These data sources helps organizations arrive at quick decisions and react quickly to the market or customer needs.

A common example is a customer recording their experience in a public domain which can either help or harm an organization’s reputation. In such cases, when real-time data are made available to the customer service team, they can quickly step-in, understand the situation and help manage the expectation with the respective customer.   However, there are many more.  Customers use these insights to target marketing to customers more effectively, identify patent infringements, identify trademark misuse, improve product design, gauge overall product and brand sentiment, etc…  There are countless strong business uses.

The CIO can initiate the Big Data discussion with the business in business terms.  Often this is a way for the CIO to earn a “seat at the table” with the other business executives.  But, it’s good to be ready by thinking about some practical “small things about big data” before you do.

Article by Subramaniam Meenakshisundaram, a guest contributor and IBM Executive Consultant in the Systems Group Lab Services organization. 

Many thanks Subbu for your guest column!

When do you need better IT Governance?

It’s rare to find an IT organization with more people and funding than they have things to do.  Usually it’s the opposite…far more things to do than time or money.  Numerous individuals and groups have great ideas that involve IT, creating an extensive backlog of projects and tasks which aren’t always prioritized across the organization.  Add to that people experiencing unplanned technical difficulties seeking IT’s help plus executives who expect any of their bright ideas to receive IT’s immediate attention and the IT department can be a very busy, intense place to work.  Sometimes despite all of IT’s hard work and personal heroics, business leaders still see IT as more of an impediment than partner to the business.  

Lots of things can cause this but the governance model frequently is a major one.  

Governance is about establishing a common understanding of

1.       Who has authority to make what decisions or undertake what actions

2.       Who must be consulted before making a decision or taking an action

3.       How are decisions made

4.       Who needs to be informed of decisions

5.       Who is accountable

6.       How are conflicts resolved

How do you know if your governance model is a culprit?   

Look for the following symptoms:

1.       Collisions occur frequently.  The colloquialism is “people are stepping on each other’s toes."  Multiple people think they have the authority to decide or do things, resulting in contention or redundant efforts.  Few companies and employees can afford duplication of effort.  And, quite frankly, many people get a bit ticked off if they are working on something only to discover someone else is already doing it or has already done it.  The result?  In addition to people wasting their time, often it's accompanied by a little workplace drama.

2.       Circular ownership occurs frequently.  The colloquialism here is “things are falling on the floor.”  In this case, people assume other people own something but those people also assume someone else owns it.  Consequently, no one ones it and important things don’t get done due to the gap in responsibility.  The expectations based upon incorrect assumptions can be the source of frustration because people recognized these tasks needed to be done and unjustly blamed others when they don't get done.  This also introduces counter-productive drama sometimes.

3.       The staff spends more than 10% of their time running around in react-mode.  A good question to ask is, “What percentage of the day do my employees spend doing what they planned to do each day versus having to fight the day’s fires?”  In a well-governed organization, the average day runs according to plan versus a non-stop merry-go-round of emergencies and urgent tasks.

4.       There’s a history of making poor decisions.

Some of the common sub-optimal areas of governance are portfolio management, project management, offering management, strategy and planning, financial accounting, audit, risk, and business continuity.  Having a plan or procedures is not enough.  There has to be uniform understanding of and respect for roles, accountability, rules of engagement, decision-making processes, and communication.  This must occur not only within the IT organization but between IT and the business.

Stay tuned for ideas for creating a harmonious governance model between IT and the business.

And a shout-out of thanks to my colleague Gisbert for giving me some ideas for this article.

Changing the IT Provider Relationship

Last time I wrote about four profiles that categorize the relationship between business and IT – called the “IT Provider Relationship” profiles.  If you have no idea what I’m talking about, you might want to read the last article and then come back here.

It’s common to see business leaders wanting IT to be a partner or enabler to the business yet treat the IT organization as a commodity or utility.  The tough question when this gap exists is, “How do we fix it?”

I wish there was a magical phrase you could utter or a magic wand you could wave to alter that relationship instantly.  In reality, it takes time, consistent commitment and effort.  However, here are some suggestions that have worked with clients that you might want to consider:

1.  Get the business people to recognize the current and desired IT provider relationship profiles for your organization.  Having a common vocabulary and some semi-scientific data do wonders for facilitating conversation.  It also helps secure the business leadership’s buy-in that this is something important which deserves focus.  Just like in a strained marital relationship, if only one party wants to work on things, progress is usually difficult to achieve.

2.  If you’ve made it to step 2 then you have business buy-in.  Congratulations!  But, act quickly.  Business leaders sometimes have short attention spans.  If you have their attention and awareness that the IT Provider relationship needs work, then do not fritter away the opportunity.

3.  Establish regular communications between business and IT leaders about things that matter to the business and the associated IT implications.  Some of my clients do this twice a year.  Others do it quarterly.  I wouldn’t recommend doing it any more frequently than quarterly because that’s kind of like watching grass grow.  I wouldn’t recommend meeting any less than twice a year because remember step 2…short attention spans. 

As a side comment, it is imperative that this begin with the executive teams.  Leaders need to model the way.  Your staff will see it as “programme du jour” if executives expect their underlings to make all the changes whilst their own behavior remains constant.  Keep in mind that most of the people who keep the IT shop running have probably been around for a lot longer than you and have watched bright-eyed, eager CIOs with “great ideas” come and go.  They know they have outlasted many a CIO.  They often see themselves as having a better understanding of how things really work in the company and many feel they care about the business’ longevity more than the leadership team.  You don’t want your team seeing you and your leadership team as “suits” primarily focused on their own resume enhancement.

4.  Build a practical IT transformation plan that has business buy-in, connection to business objectives, assigned ownership, identified dependencies, realistic schedules, and prioritization based upon the support for business objectives.  Don’t have more work-streams going at once than your organization can handle.

5.  Build an effective business / IT governance model.  Many business leaders think their IT team moves at a sloth-like pace but often that’s because the IT staff keeps being diverted to do firefighting or to handle impromptu, under-developed, executive requests.  Anytime an organization spends more than 20% of their time fighting fires and handling ad hoc requests, they are in desperate need of improved governance.  If they spend more than 5% of their time doing this, they are in need of improved governance…it’s just not at the full-out “desperate” state yet.   

Governance includes the organization’s guiding principles for decision-making and accountability.

• Who has the authority to make what decisions?
• Who will be held accountable for what?
• How are decisions made?
• Who has to be consulted before making a decision?
• Who has to be informed of decisions and when?
• What’s the dispute resolution process?

6. Set IT key performance indicators (KPIs) that are meaningful to the business.  Don’t set targets that are just IT focused or things that IT knows it will achieve.  And if you are gathering statistics, then review them and use them to make fact-based decisions.

7. Focus on delivering solid utility-based IT services.  If you deliver inconsistent or inadequate service quality, you haven’t earned the right to move to a partner relationship.  Service quality improvement needs to occur in concert with business / IT governance because often service quality issues arise from a firefighting / ad hoc culture that impedes IT’s ability to fix broken processes.  Sometimes fixing service quality issues involves organizing IT into dedicated teams focused on building new solutions, implementing those solutions, and maintaining them. 

As a side note, sending your whole staff to ITIL training does not magically fix service quality.  If the executive leadership team is not fulfilling their critical role in organizational and process change, then sending everyone off for ITIL certification is just going to frustrate the staff and reduce their confidence in the leadership team.  Please see the side note for step 3 above.

8. Improve your architecture skills for the full span of Enterprise Architecture (EA) and start to develop a vocabulary and repository for enterprise architecture elements.  You’ve already taken some of the initial steps in establishing EA if you’ve made it this far, but you just haven’t called it that yet…and therefore probably haven't gotten hung-up on an academic approach that would earn you an A+ in an Ivy League business school but be impossible to implement in the real world.

9. Establish a communication process to communicate regularly to key stakeholders.  Communicate successes in business terms.  Hopefully this is easy to do because by now you’re measuring things that are meaningful to the business.  This also gives a regular structure for having other levels of IT leadership communicate regularly with their business counterparts.

10. Establish an “innovation factory” where a few business and IT people look at industry and technology trends and then brainstorm on ways to use technology for business competitive advantage.  Or, get started with a simpler approach of having IT people job-shadow business people to see the impact of their handiwork on the end client.

I’d like to label this list as “10 easy steps to improve the IT / Business relationship.”  However, to accomplish all 10 items takes at least 2 years.  That doesn’t mean that you won’t see business benefits until then.  Most clients start to see benefits within the first quarter or two of committed execution on their transformation plan.  The key is to get started and be practical.  Two years from now you can either have made progress or be the same place you are now…or possibly worse…since doing nothing while others move forward turns into a relative step backwards.