Big Insurance Data: Drawing Lessons from Amazon, Google, and Facebook
There has been much written about the emergence of Big Data, both as a phenomenon and as a set of practices, infrastructure, and algorithms designed to allow modern computing to analyse ever-increasing data in fast and efficient ways. Celent uses the open source Hadoop solution to provide insight into what Big Data is, what it means to the insurance industry, and the kinds of solutions it provides to new and old insurance problems.
In a new report, Big Insurance Data: Drawing Lessons from Amazon, Google, and Facebook, Celent reviews how Big Data is now relevant and accessible to the insurance industry. In this report, Celent offers a discussion on the following topics:
- Grid computing or the use of many distributed, connected machines to undertake large-scale work.
- Distributed file systems, which store large amounts of data across many machines.
- MapReduce, which is an algorithm to split workloads over grid computing.
- Hadoop, which is a specific implementation of the MapReduce algorithm.
“The simple fact is that, with little more than a credit card, today you can rent an infrastructure from Amazon similar to that used by Yahoo to index and search the entire Internet,” says Craig Beattie, Analyst with Celent’s Insurance group and author of the report. “It is important that insurers understand just how much these frameworks do on their behalf and how easy it is to leverage this infrastructure to ask big questions of our data.”
The report draws together recent developments in cloud infrastructures, grid computing, and distributed computing to address issues regarding large volumes of structured and unstructured data. While there are many approaches to leveraging grid infrastructures, this report details one of the most popular approaches and perhaps one of the simpler approaches to implement. While the topic is necessarily technical in nature, the report addresses it in an accessible manner.
Celent is a research and advisory firm dedicated to helping financial institutions formulate comprehensive business and technology strategies. Celent publishes reports identifying trends and best practices in financial services technology and conducts consulting engagements for financial institutions looking to use technology to enhance existing business processes or launch new business strategies. With a team of internationally based analysts, Celent is uniquely positioned to offer strategic advice and market insights on a global basis. Celent is a member of the Oliver Wyman Group, which is a wholly-owned subsidiary of Marsh & McLennan Companies [NYSE: MMC].
Tel: +1 212 345 1366
Tel: +44 (0)782 448 3336
Tel.: +81 3 3500 3023
Table of Contents
An Overview of the Solution
An Overview of Grid Computing
Storing Data in Distributed Storage
Distributed Storage Isn’t Simple
Distributed File Systems
The MapReduce Approach
Build: The Map Then Reduce Approach
Distribution and Sharing
Example Applications of Hadoop
Analysing Social Data
Analysing Telematics Data
Hadoop Grids as a Data Analysis Resource
Hadoop Grids for Real Time Log Capture
Leveraging Celent’s Expertise
Support for Financial Institutions
Support for Vendors
Related Celent Research