Big data is a simple term used to describe the emergence of incredibly powerful ways to gather and analyse digital information to gain new insights about nearly every aspect of our world and lives. This is being made possible due to the increasingly digital and connected nature of our world; our ability to create, store and access digital information in real time; and the evolution of technology to apply creative and powerful analytical tools to the information, to create new, unexpected insights that promise to help solve our most intractable problems and highlight some of our greatest opportunities.
Progressive businesses and governments are already alive to the promise of big data and its impact on developing and emerging economies can be even more far reaching. India with its IT expertise has the opportunity to be a significant beneficiary. But like with any opportunity that promises a revolution comes responsibility, this one is no exception. The concern for data privacy is not speculative anymore; it needs to be amicably discussed amongst stakeholders as big data benefits all.
In 2011, the world created a staggering 1.8 zettabytes of digital information and this is expected to grow 50 times by 2020, as per an EMC sponsored Digital Universe study. In India, digital information will grow from 40,000 petabytes to 2.3 million petabytes, in this decade. Big data looks beyond sheer volume by bringing velocity, variety and value dimensions to this information. Utilising those zettabytes of structured and unstructured data and putting them to work can be significant in data monitoring and trend analysis like visualising epidemics, identifying cures, augmenting scientific discoveries, identifying economic volatility and trends, or enhancing and simplifying decision making.
Big data in real-world is not a next-generation outcome. It is already prevalent and is being leveraged right now. A recent relevant example of big data is the discovery of the Higgs Boson particle or the God particle, one of most significant discovery in our times. It is noted that scientists had to analyse more than 800 trillion proton-proton collisions to look for the God particle, which wouldnt have been possible without the advances in big data research and capabilities. T-Mobile USA used big data to study customer defections based on the analysis of its 33 million customer data records, Web logs, billing data and social media information. As a result of the insights gained, it reduced churn by half in a single quarter.
Similar examples exist across industries today. Even governments, like the UK and USA, have embraced big data. In March 2012, the US government and six federal agencies launched their own big data initiative backed by a $200 million investment calling it one of the most important public investments in technology since the rise of supercomputing and the internet.
A recent study undertaken by SAS and the Centre for Economics and Business Research, the UK-based think-tank, suggested that if the UK government capitalised on big data it could save 2 billion in fraud detection, create 2,000 new jobs and generate 3.6 billion in savings through better management of processes by, for example, integrating patient data to improve healthcare IT systems.
While businesses will leverage big data its true value can be much larger when we look beyond business towards the societal potential to solve human challenges. Such an opportunity can be indispensable to countries like India, which is characterised by its socio economic and geographical diversity. Take for example a basic societal foundation like education. If we can figure in real time basis the state of literacy among children basis their family background, gender, states they belong to, languages they speak and analyse the historical data on these factors, we could have easily figured out the hindrances towards inclusive education and find the ways to make the government literacy programmes a success.
Similarly, let us think of what big data technologies can do for the infrastructure sector. Data collected through different traffic conditions in different parts of the country can help us understanding exactly what we need to do to enhance the traffic flow. Just imagine a real time analysis of these data can not only save valuable time but can also avert accidents in roads and rail network; can offer superior solution to crisis services like police, fire and ambulance; daily commuters can avoid traffic jam and goods can reach way faster than it takes now.
The UIDAI/AADHAR project is potentially the single largest human data repository. The collected data can be analysed to get accurate insights on nutrition levels, mortality rates, sex ratios and disease ailments across age, geography and gender among other things. Add the dimension of historical data and the government could have strong insights that will aid in driving targeted programs and budgets to address these.
The government is moving in the right direction with investment in National Intelligence Grid (Natgrid) that will integrate the existing 21 databases with Central and state government agencies and other organisations in the public and private sector such as banks, insurance companies, stock exchanges, airlines, railways and telecom service providers among others.
At a time when the IT industry is battling global recessionary headwinds, big data presents a fantastic opportunity to deliver customer centric value and drive revenue. Additionally, the global shortfall for data scientists, a breed of individuals that can study huge amounts of data and deliver outcomes, presents an opportunity for Indias technical students and professionals.
It is important to view the promise of a big data as a revolution for a better future. The attitude of governments and businesses towards data will differentiate the progressive from the rest and define the impact they will have on their constituents. To India, big data presents the roadmap for it to achieve its vision of becoming a knowledge superpower.
The writer is president, EMC India & SAARC