How To Make The Most Of The Internet Of Things

David Cameron's announcement of an extra £45 million to develop the Internet of Things is great news – that's £73 million that the UK has earmarked in funding research in this space. The announcement highlights a growing understanding at the highest levels of government of the huge potential, both commercially and from a consumer perspective, presented by the vast volumes of data that connected objects are generating every day.

Some of these objects have been in existence for decades – manufacturing chains, aeroplanes, HVAC systems and access control systems, to name only a few – while some are newer, like smartphones, fitness trackers, intelligent scales, smart meters and medical implants.

What they all have in common though is that they generate big data and therefore present a potentially vast opportunity for many businesses. They also bring along a number of challenges.

Data from the Internet of Things is not big data just because of its volume – although connected objects can indeed cause volumes to explode quickly. Consider smart meters: instead of one reading per quarter, they will generate a reading every 15 minutes. That's approximately 10,000 more data points per subscriber. New generation aeroplanes, such as Boeing's 787 Dreamliner, create terabytes of sensor data per flight – to be multiplied by thousands of planes in airlines' fleets.

Of course, with information pouring into their systems as a result of the Internet of Things, IT departments have the challenge of figuring out how to deal with and use it to their advantage. "A billion is the new million," said one senior director describing the sheer proliferation of data we are about to experience.

To manage and ultimately leverage this data effectively for competitive advantage, a "total data management" approach is required. This needs to be capable of dealing not just with the vast volumes involved, but also of tying together historical information around consumption habits and invoicing patterns with new types of data generated by sensors, for example.

To effectively manage their big data, organisations will also need to make use of Hadoop or other open source processing frameworks which, when coupled with open source data management, data quality and integration software solutions, help to lower the technical barrier and make big data much more accessible.

Managing use cases

Data from the Internet of Things is driving new use cases across a range of industries. So-called "intelligent buildings" (which include intelligent elevators, intelligent HVAC, intelligent access control, etc.) optimise energy use based on actual occupancy. Fitness trackers can also help individuals better control their eating habits and sleep patterns.

Implementing such use cases requires advanced research on this new data. It entails exploration, intuition and trial and error by data scientists. Once models and algorithms have been defined, they need to be operationalised - implemented in real time. And reliability is essential; even if it's safer to ground an aeroplane by oversight than letting it fly, even if it's better to call an ambulance for a patient who doesn't need one than the opposite, these errors come with a cost. And worse, they can kill the credibility of the concept. Remember "The Boy Who Cried Wolf"?

The rise of the Internet of Things has security implications too. The big data generated will introduce new challenges for the security and quality of the data. Hadoop natively uses Kerberos for security, and users should look out for integration solutions that offer support for Kerberos, the number one security framework for Hadoop. Legacy is the major issue here – legacy integration engines require their own proprietary security methods. And big data without big data quality tends to lead to a big mess, as legacy data quality approaches simply don't work with Hadoop – instead, they require that data to be extracted from Hadoop for processing.

The Internet of Things is clearly a huge commercial opportunity. Analyst, IDC estimated the Internet of things technology and services spending at $4.8 trillion (£2.9 trillion) in 2012 and expects the market to be worth $8.9 trillion (£5.3 trillion) in 2020, with an annual growth rate of 7.9 per cent. But if businesses want to take full advantage of it, they need to understand not just the opportunity but the challenges and make sure they are fully prepared for a development that looks set to transform existing business models and drive huge commercial value long into the future.

Yves is the vice president of marketing at Talend