The Internet of Things Meets Big Data, with Chris Curran


The Internet of Things is an idea that’s been
around for maybe 10 or 15 years or so. And I think the first time I heard about it there
was some discussion of a European appliance manufacturer who had a refrigerator that was
connected to the network, the Internet. And I think the scenarios that were playing out
there were: “Okay, what if the refrigerator was smart and knew enough to adjust temperatures?”
Maybe it knew enough to send a request or a message to the manufacture saying “Hey,
the compressor is going bad.” That was the first time I had heard about this idea of
connecting a product to the network. A product that wasn’t a traditional computer; wasn’t
something that we thought was supposed to be connected to the network or interconnected.
And that started to open up a lot of people’s eyes I think to this idea. It may not have
been the first theoretical time that it was discussed but it was the first time that I
think popular media and culture started talking about this idea. But, for whatever reason that connected refrigerator
didn’t really take off. The idea really didn’t kind of explode. Then we started hearing about
the connected washing machine in the same kind of context. And I think that that idea
of the Internet of Things then started to refine a bit and we heard about machine-to-machine
communication. The idea that instead of human to machine, so sort of through a screen or
through a webpage or through our mobile phones or whatever, that machines would talk to one
another. So the idea of a stock quote generating a message that would be sent to another machine
that might think about “Okay now I may need to make a trade.” So the automated trading
world. Or a weather forecast that’s reported online triggering a message to another system
that’s going to adjust stock levels. So this idea that machines might talk to one another
without a human in between was sort of a refinement I think of the Internet of Things idea. And now over the last handful of years we’re
seeing more consumer facing ideas and concepts and products coming out that I think has regenerated
interest in this machine-to-machine communication. So now we’re hearing things about the connected
car and the connected thermostat and the smart home. So now that the consumer side of things
is heating up, we’re also asking ourselves from a business perspective: “What is the
Internet of Things for a business?” Maybe where you don’t have a consumer product or
a widget that you sell. Maybe you’re a services company or a bank, what does the Internet
of Things mean for a business like that, particularly a services company? And so for companies that aren’t as consumer
facing or product oriented, I think there’s more opportunity but more sort of clouds around
what the Internet of Things means for business. And I think that anything that we think about,
that we estimate, that we operate in, places we work in, people we work with, we think
about the effectiveness of those things, the effectiveness of our workplace, the effectiveness
of a warehouse, the effectiveness of routes that a truck takes or that a forklift makes.
And we make guesses about the right ways to design a workspace or design a logistics path,
design a warehouse. And the question is can we use the idea of sensors to collect data
about things that we were just guessing about before, that we were estimating, that we were
sort of using our gut to design? Can we collect real data about performance of people and
places and processes that can give us more insight into optimizing and evolving our designs
for the way our business works? And that’s what I call the Internet of Business Things.
It’s not the consumer facing stuff, it’s the business stuff. Most of the data we think about from an enterprise
perspective is sort of modular, it’s transactional, it’s a call center call or it’s a web transaction
or it’s a sale or it’s a quote or it’s a piece of data about a product. But the Internet
of Things will be creating streams of data. So one of the analogies is how social media
creates streams of data. So we’ve got tweets and we’ve got flows on our wall and such that
are more streams of information. And so think about a sensor that might be on a door in
a warehouse. Every time the door opens the sensor records a door opening, a time of day,
a day of the week. Maybe it creates some other ambient information – it records the temperature
in the facility or it records how long the door was open or whatever. So these are little
bits of data that are happening every time the door is open. And that door might be open
a lot of times during an hour or a day or a week or a month, but it’s creating these
streams of data. And it’s not like that one piece of data might be that interesting, but
it’s the trends and the patterns and things that we’re looking for in the data that may
be what we’re looking for. So I want to know how often is that room used
during the morning, the afternoon, certain days of the week, weeks of the month, months
of the year, to help me be more efficient about the way I staff people in that office
or the temperature or the cooling or the electricity that I feed to that part of my facility. So
that data that’s created, these streams of data are sets of data that I may not already
have a technology platform to process. I don’t know how to process streams maybe. So now
I have to think about a new architecture, a new way to process a stream of data, not
just a stream but maybe I have a hundred facilities that I have these sensors in and they’re creating
these streams of data. So now I’ve got a new challenge around data. One of the challenges starts to feed and link
into the big data topic. So as I’m collecting these individual sensor data points over hundreds
or thousands or tens of thousands of sensors every minute or every second, I’ve now got
these parallel streams of data I have to process. And so it’s unclear exactly how much data
I would want to keep for how long, but it’s certainly a new kind of data architecture,
and for some companies will be a big data problem because there’s going to be a lot
of this data for large companies for a lot of sensors and a lot of facilities or a lot
of products that they want to track or whatever it is that they want to get more insight into
by putting tags or sensors on them. So, to me this is one of the sort of subsets, subdomains
of big data around sensor data, streams of data that we’re going to have to think about
new architectures and new designs of and new technologies for capturing them, storing them,
processing them, abrogating them, analyzing them and so on. So I think it’s going to be
a real interesting evolution to see how the big data technologies and technologies in
general are emerging to deal with streams of sensor data.

28 thoughts on “The Internet of Things Meets Big Data, with Chris Curran

  1. Hey everyone, it's pretty difficult to start up a small youtube channel but I was hoping you would be willing to give my channel a shot. I spend a lot of time making educational videos in hopes that it will help students do better in their classes. You don't even have to subscribe if you don't want to but it would mean a great deal to me if you just took a look at my channel! Thanks so much in advance for your time and consideration!

  2. What's new and special about his door example?
    Wouldn't that just be implemented as plain ol database storage or file logging? (with the ability to overwrite itself after X days)
    Sounds like standard stuff to me.
    How would this be more like a "stream" of data? What's so new and special about it?

  3. I want my fridge to order my groceries and keep me stocked on my staples with a cue that I can reshape in my spare time.

  4. How is the reasonable privacy thing going for you? rebranding access to all data about us and pumping it up as a good thing is still an invasion of privacy.

  5. All this has some use, and many dangers. The fallacy is that through micro management and efficiency we can always solve big problems. Often this is just an infinite regress of a false premise. The junk economics of today, with their huge financial markets supposedly providing more and more indicators or sensors on the market. Yet the boom bust instability is very much still with us, because of the deep rooted flawed assumptions that no sensor or data is intelligent enough to comprehend.

  6. I think the phrase "consumer facing" is interesting.  When was it first used?  Is it a  meme yet? Or is it just "techno-biz-speak"? When this presenter talks of tens of thousands of sensors he's talking only of the largest of businesses; not my little one-man shop.

  7. He doesn't even approach the ethical questions which are essential to legitimate the disposition of such computer systems of Big Data in the products for consumers. We now already have much information about what happens to companies such as Facebook, Google, in relation to the way it relates to its users and collects data from them.

    Moreover, this hypothesis of empirically testing over and over the user products in their own homes through the Internet of Things, while being a plausible and very interesting way to obtain data to improve the quality of the products, seems to be a narrow way of dealing with product quality, which, as we clearly know, such technologies are often developed from a business perspective whose first priority (goal) is profit, and not the quality of their products. In other words, I'm stating that there exist more sophisticated ways to test and develop quality projects, the problem is that not often those solutions will it be the best way to obtain profit. I mean that this question (quality VS. profit) isn't even discussed here. "It must be The Internet of Things the solution (for Business of course)" is what he is clearly suggesting. But this suggestion just doesn't even stops a second to consider ethical questions that might be relevant in the XXI century.

    I'm not against the idea of further development of an Internet of Things, but I think that, it can't be uniquely a question of Business, since there must have a paralallel, ethical discussed of public rights, concerning the problematics of Big Data.

Leave a Reply

Your email address will not be published. Required fields are marked *