Organisations in Business and Social Sectors are at different levels of adoption and maturity in analytics, BI and Information systems. If we look at technology S-Curves of Analytics, BI and Big Data, businesses & public sector companies are scattered around both technology S curves:
As the forces of digitalisation and datafication progress further, the world of Information haves and have not’s will try to catch up with each other and those who do not have a baggage of past or are agile enough may be faster in experimenting with new open source technology and innovative engagement models.
If I quote from my chapter in the recently published Gartner E-Book (, January 2014), we are talking about a new world of disrupting Traditional Business Models where:
Information is an asset to be used for business growth and competitive advantage
Data ownership moves from the business to the consumer or citizen
Next to business analytics, there will be personal analytics and Planet analytics
“Privacy will be a part of technology by design.”
In such a connected world ‘aspiring creativity’,
‘Ability to Innovate through Analytics & Information would be crucial. Data & Analytics Architecture, Systems, processes and service models in the last 20 years have been mainly focussed around structured data.
The need of operational intelligence, real time analysis of data as it gets generated in extended ecosystem of Big Data will continue to impact the architecture as old architecture does not support it. Clearly, a New BI & Data Architecture is needed to support operational management, creative investigation, Big Data, Predictive analytics.
’Scalable Data Access’ and ‘Efficient Query execution’ at optimal cost in an acceptable time (say minutes, and not days) is more an area of innovation in big data era than it was earlier. Projects like ‘Optique’ are trying to develop a platform with a generic architecture that can be adapted to any domain that requires scalable data access and efficient query execution.
Organizing for Innovation, Why cross-organizational boundaries?
Moving on from Architecture, how we build the information organization in big Data and analytics era is equally important. We would need to consider all aspects – Process, Management, Organisation and Culture that create the necessary linkages between strategy, technology, business and extended eco-system for data driven innovation.
The traditional models have typically followed either one or a combination of the following:
• On premise : in-house, outsourced, mixed
• Cloud – Private, Public, Hybrid
• Iterative, Business Value led
In the new world of Big Data & Analytics, crossing organisational boundaries would be necessary for several reasons:
Big Data is in extended ecosystem – customers, suppliers, and partners, Internet of Things.
The possibilities of creating value from Internal Data assets is high but much higher value can be created by combining Data assets from various companies and sources.
People’s ability to raise questions is limited by what they think are the solutions possible! When open innovation is applied to big data, it increases people’s ability to ask big questions.
Extended reach and access to talent. Innovative, smart people and organisations
We have already started seeing Open innovation, Crowdsourcing being tried by various companies that came up with innovation challenges or accelerators with Industry partners tapping innovation networks.
What are the bottlenecks in organisations adopting collaborative innovation: Culture, Procurement process, mindset of leaders coming from big SI organizations who tend to rely more on established organizations, need for seeking compliance, leadership’s lack of focus on innovation.
I think the issue is with supplier eco system as well, there is a need of an agency that can bring processes that give the confidence to leaders that the risks, control procedures and security is managed with better rigor than closed innovation.
So, In summary, as new technologies unfold, the convergence of Big Data, machine learning, cloud and mobile will create new information architecture in the enterprise. One of the big issues highlighted are latency, inflexibility and lack of responsiveness of existing architecture and tools for business model innovation in a world impacted by digitalisation and datafication.
What is required is Building architectures, tools, applications and processes that get the data in the hands of business users for discovery of underlying patterns, creative manipulation, exploration of differentiating business value by creative experimentation & simulation of business model experiments. This means bringing advanced analytics and visualisation within the information delivery layer and closer to end-user.
In addition, several forces will drive new engagement models leveraging open innovation and crowdsourcing as ‘old regimes’ collapse paving way for innovation in sourcing practices and bring in mature processes for managing risk and rewards in open innovation.
I invite readers to continue the discussion; I would be keen to hear your views especially on some of the key questions:
• What is your take on the subject?
• What technical and business challenges are most important to overcome in the Big Data and Analytics era?
• What challenges do you see in adoption of Open innovation & Crowdsourcing in Analytics and Big Data projects?