Are Dark Data the footprints of the future?

We live in a complex world full of data. There are multiple data sources such as e-commerce, mobile apps, B2B, industrial machines and sensors, WiFi, and people. Dark Data are those data that exist inside and outside your organization but you don't have access to. As a result, those Dark Data are not used in decision making.

Always connected

More than half of the world's population uses the internet. Each one of our steps through the network, even if it seems insignificant, leaves a digital footprint. Nowadays, wifi services are key, since they allow us to know how people move around a space and thus understand their habits. With this information, we can better understand user behaviour and the staff of that space can better adapt to these habits. In the same way, new business opportunities can be identified.

Millions of network-connected devices are digitizing our physical reality at every moment of every day, from security cameras inside shopping centers to weather sensors. These devices generate an astronomical amount of data of human and environmental behaviour - unprecedented until now. It is estimated that in the year 2020 there will be 50,000 million devices connected with 100,000 million sensors.

The great challenge is that many more data are generated than we are capable of capturing, processing and analyzing in a reasonable amount of time. This explains why more than 80% of this data remain inert and untapped in the darkness of the digital underworld.

But what would happen if we could use them? And if we could leverage our business intelligence to its fullest potential?

Take the example of the retail chain Walmart, one of the companies that has best been able to extract value from data. In 2005, Walmart was able to predict the impact of Hurricane Katrina and stock up on provisions better than the United States government itself. How? By identifying which products their customers would need most based on the threat level reported on the news each day.

In the tourism sector, if you know that Germans usually have breakfast at 8:15 in the morning while on vacation, you can optimize your hotel's resources to have breakfast ready at that time. And if you notice that 80% of the clients coming to your establishment next week is of German origin, you can adjust the operational tasks of your staff accordingly in order to offer a better service. 

Similarly, if you know that 90% of your customers go out for dinner, it's a matter of understanding what they're looking for and identifying what your hotel can do to change this. Identifying behavioral habits like these makes it much easier to determine your conversion rate. You can also check which product requests you cannot meet; negotiate better prices with your suppliers if they are too high compared to competitors (failing to do this translates to low conversion rates), and to better understand how people are moving around your hotel so you can constantly improve and optimize your service offering.

Take the example of one of the biggest tour operators in Spain, Globalia. Most companies analyze and understand the information related to the hotels they are selling. But, what happens with the rest of the remaining availability queries? For example, why did other users not click to buy? 

Companies do not usually store this information, called dark data, as they occupy an almost unfeasible storage volume and require massive processing and analysis capabilities. Today, the biggest tour operator in Spain captures more than 200 million of its interactions. Dark data allows this company to understand the destinations their customers are seeking, the products that resonate and enables them to track conversions of products with the highest sales performance.

Currently, farmers and breeders heavily depend on taking samples and analyzing on external labs. This process takes at least one week until analytical results are returned and animal owner decides based on reported parameters. Animal pregnancy is one of the most interesting topics that are based on analytical results - finding out the perfect time for the animal to become pregnant can be easily detected based on certain hormones.

New platforms can analyze samples on premises and see reported results in minutes and with a powerful data middleware solution deploying by Datumize you can automate the whole process.

Large retailers can leverage this type of data to obtain insights on how much time their customers spend in their stores, which routes they follow, where they spend the most time (either choosing a product or waiting in a checkout line) or even how frequently they visit.

Marketing studies consumer behavior. A person is what he or she does, not what he or she says. Dark Data say more about intentions than facts. All marketing experts see dark data as a very useful tool that can help fine-tune their communication campaigns very effectively.


Dark Data is the new oil of the digital economy. But without an original software that can extract and refine it its value will remain lost in the darkness like tears in the rain.

Datumize Technology

The Datumize Company is the world first "Dark Data" software company that Captures 99% of all operational Data, Unlocking this data giving you access to analyze core, previously unknown "Dark Data".

Datumize allows companies from the Travel Industry as tour operators, airlines, and hotels to maximize revenues, optimize prices, discover new business opportunities and tendencies and understand customer behaviours.

Our innovative technology integrates with any system non intrusively to capture and processes in real time availability requests and the responses offered in all sales channels.

Some additional references you might be interested:


Stay connected with us via Linkedin and Twitter

written by Raquel Perez