Big Data

Have We Reached The End of Innovation?

Bill Schmarzo By Bill Schmarzo Chief Technology Officer, "Dean of Big Data" March 3, 2016

A recent article in BusinessWeek (January 19, 2016) titled “Better Living Through Robots” indicates that productivity-enhancing technology innovation has reached its end. Experts such as Tyler Cowen, an economist at George Mason University and the author of “The Great Stagnation”, believe that America has “eaten all the low-hanging fruit” for realizing the economic impact from new technology innovations.

Wow, guess it’s time to pack in all this excitement about big data and data science and get ready for a lifetime of “stasis in the way offices work, stasis in the way retailers work, and stasis in the way factories produce goods.”

Well, I am no historian (though I’m old enough now to have lived through quite a bit of history), but I believe that this article misses some very key points about how big data and data science will have dramatic impact on the productivity of humans worldwide.

Productivity Impact Requirements

There are a few areas of focus required for organizations of all types and sizes to start realizing significant (5x to 10x) productivity and operational improvements from big data and data science.

#1) Focus on the organization’s key business processes and initiatives. There are almost a countless opportunities for organizations to leverage the growing wealth of internal and external data (structured and unstructured) with data science and the growing body of big data technologies (Hadoop, Spark, YARN, Hive, HBase, HAWQ) to drive material business value. But one cannot just throw technology at the organization and hope that something sticks (“hope” is only a business strategy in cosmetics). One must begin with an end in mind (to quote Stephen Covey), and the best “end in mind” is the “end in mind” that can deliver the most business value to the organization and its customers.

All organizations have a wealth of opportunities to couple new sources of customer, product and operational data with data science to deliver significant business value by optimizing key business processes, uncovering new monetization opportunities and delivering a more compelling and effective customer engagement.

#2) Focus on making humans more effective. While much of the hype with productivity gains seems to be about replacing humans with technology (ATM’s, self-service checkout, toll fast passes, Skynet, etc.), there are many, many more opportunities to make humans more effective.

Big Data can dramatically improve human decision-making, especially at the front lines of the organization including teachers, physicians, clinicians, nurses, technicians, police officers, paramedics, fire fighters, coaches, counselors, tutors, store managers, call center reps, sales teams, financial agents, brokers, advisors, scientists, researchers, etc.

In order to make the humans (or key business stakeholders) more effective is to focus on the decisions that these humans need to make in support of the organization’s key business processes or initiatives (see Figure 1).

Figure 1: Role of Decisions in Making Humans More Effective

Figure 1: Role of Decisions in Making Humans More Effective

#3) Focus on driving organizational adoption. Organizations will have to learn not only to value data, but also even more importantly, to value the insights (and recommendations) that can be derived out of the data. This will challenge traditional “legacy” thinking that only the best ideas come from senior management and HIPPOs.

Organizations need to embrace a culture of creative thinking and experimentation; where all ideas are worthy of consideration and that the best ideas will come out of objective, fact-based testing.

Big Data Benefiting Mankind

The result will be a magnitude improvement in the creation of “smart things” such as smart cities, hospitals, schools, airports, grocery stores, roads, government, etc.

For example, let’s consider all the excitement about the “smart” city. Creating a “smart city” starts by understanding the city’s key business initiative or business objective (i.e., the business initiative is the “what” we want to accomplish). For example, “Improve traffic flow” might be the targeted business initiative. Next, let’s identify and understand the decisions that city management (our key business stakeholder in this example) needs to make to support the “Improve traffic flow” business initiative. Decisions that city management would need to make to support the “Improve traffic flow” business initiative include:

  • Traffic flow decisions: New roads? New lanes? New turn lanes? New bike lanes? Location of lights and stop signs? Timing of lights? Pedestrian crossings? Railroad crossings? Bus stops?
  • Road repair and maintenance decisions: Fixing potholes? Repaving choppy street surfaces? What materials to use in repaving street surfaces and fixing potholes? When to fix potholes and repave streets? What equipment will I need to fix potholes and repave streets? Repainting cross walks? Repainting street lines?
  • Construction permits decisions: Types of construction permits? Impact of proposed construction permits on traffic flow? Length of time to complete the construction work? Day and hours of operations for new business and commercial buildings? Number of employees for new businesses and commercial buildings? Number and location of additional parking spots for new businesses and commercial buildings?
  • Events management decisions: Traffic (number of cars and pedestrians) to attend proposed event? Impact of events on normal traffic flow? Date, time and duration of events? Location of events? Criteria for approving/rejecting events? How many additional police officers to support event traffic? How much to charge for the event license?
  • Parks decisions: Location of parks? Size of parks? Hours for park operations? When to close parks? Park equipment maintenance?
  • Schools decisions: Location of new schools? Size of new schools? Hours of operations? Location of stoplights and stop signs? Additional bike lanes? School crossing locations? Safety personnel?

The Best Is Yet To Come

Again, I am no historian, but I see a wide range of client organizations working to combine customer, product and operational data with data science to make their employee and customers more effective. It may not be the big bang of discovering electricity and replacing all of those candle makers, but even something like electricity realized its “society changing potential” one use case at a time. Yep, that is exactly what is happening with big data…one use case at a time.

Bill Schmarzo

About Bill Schmarzo


Chief Technology Officer, "Dean of Big Data"

The moniker “Dean of Big Data” may have been applied in a light-hearted spirit, but Bill’s expertise around data analytics is no joke. After being deeply immersed in the world of big data for over 20 years, he shows no signs of coming up for air.

Bill speaks frequently on the use of big data, with an engaging style that has gained him many accolades. He’s presented most recently at STRATA, The Data Science Summit and TDWI, and has written several white papers and articles about the application of big data and advanced analytics to drive an organization’s key business initiatives. Prior to joining Consulting as part of EMC Global Services, Bill co-authored with Ralph Kimball a series of articles on analytic applications, and was on the faculty of TDWI teaching a course on designing analytic applications.

Bill created the EMC Big Data Vision Workshop methodology that links an organization’s strategic business initiatives with supporting data and analytic requirements, and thus helps organizations wrap their heads around this complex subject.

Bill sets the strategy and defines offerings and capabilities for the Enterprise Information Management and Analytics within EMC Consulting, Global Services. Prior to this, he was the Vice President of Advertiser Analytics at Yahoo at the dawn of the online Big Data revolution.

Bill is the author of "Big Data: Understanding How Data Powers Big Business" published by Wiley.

Read More

Share this Story
Join the Conversation

Our Team becomes stronger with every person who adds to the conversation. So please join the conversation. Comment on our posts and share!

Leave a Reply

Your email address will not be published. Required fields are marked *