Do You Really Want to Send Your Data FedEx?
Tuesday, 23rd April 2013
By Matt Benati
Moving data, and in particular, big data, is a specialist task. Matt Benati explains why it's essential that the transfer of large datasets be carried out speedily and efficiently - and how analytics rely on the freshest data for competitive advantage.
In this age of always-on, cloud-based connections, most of us never think much about how data gets from Point A to Point B. But movement of data is the core of what Attunity does, and when the data is actually "big data", Attunity's expertise is essential for realising the value of computing power.
FreePint conducted an interview with Matt Benati, VP Global Marketing for Attunity, to understand the impact of improved data movement on the success of big data projects. This extract focuses on the nub of the question: "Why does it matter how the data moves?"
FreePint: When I read about big data, the movement of data isn’t something I run into much. Most of the case studies and articles look at analytics and output. Tell me why data movement is an important part of the equation.
Matt Benati: If you think about the principles of big data, you’ll see that they rely on data of all kinds streaming continuously into the system - it might be HR data, financial data, inventory data. Before you can even get to the analytics, you have to think about how that data is going to get there efficiently and in a timely manner.
Analytics relies on three steps: 1. Build a model. 2. Train the model. And 3. Score the data. You’ve got to be bringing fresh data in all the time, moving it through an iterative process to train the model. And when the model’s trained, you have to keep bringing in that data to get the benefit.
Do you know how most big data is moved today?
MB: FedEx. Seriously. Data is transferred to a set of CDs, because there’s too much data to put on a single disk. The CDs are shipped to a data warehouse centre to upload the data to the cloud, and THEN you’re ready to train the model, score the data, do the analysis, and so on. Even if you do this as quickly as possible, the data is already a day old.
We call this the big data bottleneck.
FreePint: How does Attunity address this?
MB: Attunity’s heritage is in data availability. Without going into too much of the technical detail, we streamline the process of moving data. Traditionally, data transfer systems are an “all or nothing” proposition - you transfer the whole dataset, or none of the dataset.
What Attunity focuses on, however, is that only a small fraction of the data might have changed. After the initial data load, if you can transfer just the changed data rather than the whole dataset every time, it has a huge impact on the speed and efficiency of transfer. If traditional transfer is a commuter train, making several stops along the way, Attunity has created a bullet express train that goes from Point A to Point B rapidly, without stops.
FreePint Subscribers can read the full interview by logging in to view Transferring Big Data by Bullet Express Train.
Editor's Note: Big Data in Action
This article is part of the FreePint Topic Series: Big Data in Action, which includes articles, reports, webinars and resources published between April and June 2013. Learn more about the series here.
- Blog post title: Do You Really Want to Send Your Data FedEx?
- Link to this page
- View printable version
- Mixed Messages on Big Data
Thursday, 28th March 2013
- Big Data Creates New Opportunities for Collaboration
Thursday, 7th February 2013
- FreePint for Skills Upgrades: In an Era of Constant Change
Thursday, 14th March 2013
- Big Data, FreePint-Style
Monday, 18th February 2013
- Big Data = Big Opportunity
Tuesday, 20th November 2012
"I never before fully appreciated the role the information service plays in R&D productivity and innovation. This isn't something that benefits our department; this is something that benefits the whole of R&D."
You can make the impact of information on successful R&D visible and measurable, with Value Chain analysis.
Let us help you map Jinfo's Focus areas to your particular challenges with information strategy and operations.
- Gain confidence and trust through new vocabulary
Wednesday, 13th October 2021
- Manage your content portfolio for risk
Monday, 11th October 2021
- Is your content advisory board primed for strategy in 2022?
Thursday, 7th October 2021
- Centre of Excellence (step five - your evolution work plan)
Wednesday, 11th August 2021
- Centre of Excellence (step four - assess your team’s characteristics)
Tuesday, 22nd June 2021
- Centre of Excellence (step three - envision your Known Fors)
Thursday, 10th June 2021
- Community deck - teeing up value-based budget conversations for content licensing
Thursday, 30th September 2021
- Community deck - Centre of Excellence - make progress on your evolution, with your peers
Friday, 23rd July 2021
- Community deck - Centre of Excellence - examples of Pillars of Service and Known Fors
Wednesday, 23rd June 2021
- Investment philosophies for strategic portfolio management (Community) Thursday, 2nd December 2021
- Negotiation role-play for content licensing professionals (Community) Thursday, 7th October 2021
- Teeing up value-based budget conversations for content licensing (Community) Tuesday, 21st September 2021
- Value Chain (step five - putting the Value Chain together) (Webinar) Tuesday, 31st August 2021
- Value Chain (step four - measurement - selecting and collecting data) (Webinar) Monday, 30th August 2021
- Value Chain (step three - your contributions to the Value Chain) (Webinar) Friday, 27th August 2021
Our proven processes, resources and guidance will help your team make the shift from transaction centre to strategic asset.
Designed around the most common challenges and pain points for time- and resource-strapped information teams
A tailored overview of our research and active discussion with your Jinfo analyst.
Measure your starting point to articulate your strengths and set priorities for future improvements. Assessments gauge risk, capacity, value and more.
Read case studies, and start the conversation: