The Internet of Things

The Internet of Things (IoT) is a concept that has become more exciting over the last few years, with the prevalence of Internet-ready and connectivity-enabled embedded devices. Traditionally, embedded devices were closed systems, with a dedicated or purpose-built functionality in mind, and not necessarily intended for inclusion as a part of a larger system with more open functionality. The ability to network physical devices in your daily life—your car, your electricity meter, your thermostat, your fitness device, your watch, your shoes, your jacket, your coffee-maker, and yes, even your home itself—makes the possibility of providing richer and smarter experiences very real today.

Essentially, the IoT is the network of physical “things” containing embedded technology that can sense, act, process, and communicate within the network and with the external environment. The term IoT is used to encompass the hardware platforms, the communication protocols, the software architecture/layers, the infrastructure services, and the application services all together.

The Gartner report, Forecast: The Internet of Things, Worldwide, 2013,” predicted that the Internet of Things (IoT), which excludes PCs, tablets and smartphones, would grow to 26 billion units installed in 2020, compared to only 7.3 billion units for smartphones, tablets and PCs. It is becoming common to refer to IoT as the Internet of Everything, and the Business Intelligence report on “The Internet of Everything: 2015” shows how the number of devices is increasing in various domains.

number-of-devices-in-the-internet-of-everything

IoT is not specific to any domain or vertical. We’re seeing its impact in producing transformations and new experiences in different domains, including retail, sports, healthcare, home, transportation and cities. Every single physical object around us potentially has a part to play in this new ecosystem, and we are only limited by our imaginations in what these objects can do for us, when we put them together and enable them to interact with each other in new and interesting ways.

As a professor, I’ve been teaching embedded systems in the Electrical and Computer Engineering Department of Carnegie Mellon University in 2001. I’ve taught our junior and senior (capstone) courses, and have enjoyed the process of watching our students create beautiful things out of their imagination and their efforts. Being a huge sports fan, I’ve sought ways of infusing sports into my courses, and also sought ways of teaching through sports. Each of us has a favorite sport that we are passionate about watching and/or playing, and I’ve found that the problems that we perceive in sports—whether it’s improved concussion testing, better referee calls, improved parking at stadiums—are all problems that cry out for engineering solutions.

That’s what my new Internet of Things course is about. The Internet of Things (IoT) is about uniquely identifying/addressing various (and often dissimilar) physical objects, and finding new ways to network them together, in pursuit of a loftier objective. In most cases, these objects were not born to communicate with each other, and they were not designed/intended to work together. The beauty lies in how we put them together and induce them to serve a different, and compelling, objective. It’s about how the whole (IoT) is greater than the sum of its parts (the devices).

Think of all the objects on the field of play at a football stadium, from the seats, the spectators, the players, the ball, the pylons, the turf itself, and more. What if we could “embed” sensors into all of them, and what if all of these so-very-different objects were uniquely addressable and could actually communicate with each other? What game-changing (pun intended) possibilities would emerge out of this Internet of (Sports) Things? How can we use the fabric of this infrastructure (the devices, the data, the network, the humans) to improve the fan experience, to improve player safety, to enhance player recovery, to improve player performance on game-day, to improve the coaching on the sidelines, to improve the management of the stadium facilities, to improve spectator safety, and much, much more? Think of all of the objects inside a hospital or a health-care facility, from the sensors, the physical assets, the medical devices, and the patients themselves. What possibilities exist with the Internet of (Healthy) Things?

It’s more timely than ever, given that a number of organizations are exploring wearable and networked devices, both on the field and off the field, in various ways, such as:

It’s time to bring all of the recent hardware, sensor, cloud, infrastructure, and network developments together. It’s time to see what happens when you start out with dedicated, purpose-built smart objects and get them to network together, to see what sorts of applications may emerge. The device/system innovations that we produce should keep in mind a number of factors: battery power, the infrastructure required for operation, the cost of the infrastructure, the types of data to be collected, the comfort of the wearer, the ruggedness of the device, the non-invasiveness of the data-collection process, the utility of the data to the athlete/coach/fan, security, privacy, reliability, and much more.

I’ve been wanting to teach a course like this forever. I’m bringing to this course what I’ve learnt during my recent 6 adrenalin-filled years as CEO and Founder of YinzCam, and during my 15 amazing years as an Electrical and Computer Engineering Department professor at Carnegie Mellon. This course is not about the technology alone; it’s about the practical applications and innovations that the technology can enable in the sports industry, and the business avenues and markets that can result. We’re going to cover technologies such as Wi-Fi, GSM, CDMA, RFID, NFC, Bluetooth-LE, SPI, cloud computing, and we’re going to study current technologies being used in a variety of applications, including healthcare, shopping, inventory management, player performance, player safety, training, officiating, fan experience, and more. Most of all, as engineers, we believe in learning by doing. To that end, we’re going to have students propose, build, and demonstrate a working, useful IoT system for an application of their choice.

I’m like a kid in a candy store. I can’t wait to see what we create this semester.

Got iBeacon?

iBeacon was announced by Apple in 2013, as a part of iOS7, as a indoor-positioning system based on Bluetooth Low-Energy devices. The underlying intent was to reduce showrooming (shoppers going into stores to sample a product, but choosing to buy the product online instead of in-store), to increase foot traffic to stores, and to provide a compelling in-store experience via users’ smartphones. Working in conjunction with the Apple Store app, the beacon-based alerts provide shoppers inside 250+ Apple Stores with information on product deals, genius-bar appointments, and more.

Why yet another location-based service (LBS)? Yes, indeed, our smartphones have GPS-enabled maps, and while they suffice (somewhat) for getting to a building, they are not great for getting around/within a building. The accuracy of GPS degrades in urban areas. GPS provides a sense of location (a point and a radius around that point), but not proximity (“I’m close to parking lot 7”) or micro-location (within feet/meters, not miles). The significant distinction here is not just the notion of proximity/nearness, but also the fact that the range of the location can be as tight as a few metres.

What is a beacon? It’s a physical battery-powered device, often the size of a quarter, acting as a transmitter of information via an implementation of the Bluetooth Low-Energy (BLE) specification. The receiver (for this transmitter) is a smartphone app (think team/stadium/league app). It is possible to trigger actions on the smartphone app based on distance from the receiver, e.g., actions immediately next to the beacon, actions near the beacon, and actions far from the beacon. In the context of a stadium, it’s possible to install a beacon at a shelf of jerseys inside the stadium merchandise-store, and trigger messages in the team/stadium app as one walks past the store (“step into the store for our 10% discounts, only today”), as one walks into the store (“the jerseys are discounted an extra 15% for the next hour”), and then, as one stands right next to that shelf of jerseys (“the top-selling jersey of the day is #15–get yours now!”). In addition, every device since the iPhone 4S and iPad 3rd Gen is capable of being a BLE-enabled iBeacon receiver or transmitter, if configured properly.


Beacons and the Proximity-Aware Mobile User of the Future. Beacons present limitless opportunities to amplify the fan experience. Indeed, we are only limited by our imagination in the kinds of use-cases that are possible. It is possible to send custom alerts/notifications to fans within specific locations, it is possible to target content/experiences to locations, and to vary that experience by time of day. As with any technology, it is important to put the user first, and to understand what sorts of experiences would surprise, delight and reward the user, and to focus attention and energy on those experiences. 

It is important to remember that beacon-based experiences are possible only when fans turn location services on and Bluetooth on their device. In addition, it is important for any kind of beacon-based experience to be completely opt-in for the fan. It is just as important that, if the fan chooses to opt out of the experience, all of the experience and the associated data must be deactivated immediately.

Technologies for Officiating in Sports

There are several embedded-system technologies that have emerged over the last two decades, to improve or automate refereeing.

The Grant-Hicks tennis-court line-monitoring apparatus was one of the earliest electronic line-judge systems that aimed to provide automated line-calls in tennis matches. The system was built out of pressure sensors underneath a court surface, where the sensors were sensitive enough to detect the impact of a ball, and differentiate it from the impact of a player’s footfall. The Grant-Hicks system could make automated “in” and “out” calls relative to the boundary lines of a tennis court. In addition, the system could detect foot-faults through the use of directional microphones along with a timing circuit to detect the activation of the baseline “in” sensor prior to, or the during the time of, the player striking the ball. Using a piezoelectric sensor on the net, the system could also make automated net-cord legal-serve decisions. The Grant-Hicks system was perhaps an early IoT system—a fascinating combination of embedded pressure sensors in the court, a piezoelectric sensor at the net, and sound sensors for localization—put together to improve refereeing. While the system was used in the 1974 Men’s World Championship Tennis match in Dallas and in the 1975 Ladies’ Virginia Slims tour in Los Angeles, it was never commercialized. David Lyle also independently developed an electronic officiating system based on a combination of an electrically conductive tennis ball, a micro-computer network system for making and using automatic line-call decisions in tennis, along with an impact-detection apparatus for determining whether or not a tennis ball landed in or out of the court. The Lyle system was also never commercialized.

000831_cir_TENNIS_CLR_chCyclops was the first commercial electronic officiating system, introduced in the 1980s, and invented by Bill Carlton (an aeronautics engineer and the inventor of the plastic shuttlecock used in badminton). Known as the “magic eye” service-line machine, Cyclops consisted of a pair of transmitter and receiver boxes that were bolted into the sidelines of the tennis court, on either side of the net. The Cyclops transmitter box sent 5-6 infra-red beams, 1 cm over the ground, over to its counterpart receiver box on the other side of the net. The system was designed so that one infrared beam ran along the good side of the service-box line, while the other four infra-red beams ran on the fault side. When a ball is hit on/inside the service-line, the ball momentarily broke the first beam and turned off the 4 others. If the serve was long, it would break one of the 4 other beams instead, which produced a loud audible beep. The receiver unit was attached to a control box in the hands of the service-line umpire, who activated the box before a serve and deactivated it after the serve. Cyclops sometimes produced phantom beeps if the control button was pressed at the wrong moment. John McEnroe reportedly commented on Cyclops to an umpire, “I don’t want to sound paranoid, but that machine knows who I am.” In another famous incident, Ilie Nastase reportedly got down on his knees to speak to Cyclops about an “out” call during Wimbledon 1980.

Trinity was an electronic net-cord monitor that was so named because it aimed to help the three parties–the umpire, the players, and the person who sits at the net and might get hurt. It was introduced to reduce the high risk (due to serves as fast as 137mph) to a human who would otherwise need to manually monitor the net. Trinity was comprised of sensors that were placed at each end of the net and a cable that was fitted to an umpire-operated hand control. The umpire would press a button when the serving player tossed the ball and would release it after the ball crossed the net. A beep sounded if the ball touched the net cord.  Trinity was developed by the Brauer brothers, and piloted at ATP matches in 1995.

Hawk-Eye is a real-time ball-tracking system that was developed by Dr. Paul Hawkins as a technology to enhance the TV broadcast of cricket matches in 2001. It has since evolved into a tool that is now being used for officiating in a range of sports, including tennis, soccer, cricket, hurling, baseball, snooker and Australian rules football. Hawk-Eye relies on a system of 6-7 high-performance cameras placed high above and around the field-of-play, with the cameras tracking the players and the ball. The video data from the cameras is processed by the computers, and is used to synthesize a 3D representation of the ball trajectory, which is accurate to within 5 mm. The processing is also able to take into account player skid and the compression of the ball.

Goal-line technology for soccer has been a field of innovation in the past few years. Several contending technologies have emerged, ranging from Adidas’ Intelligent Ball with Cairos’ Goal-Line Technology system to the Hawk-Eye system.

8512_141207113128The goal of the Cairos system is a smart soccer-ball with sensors and a miniature RF transmitter (suspended ingeniously to be able to ensure that it withstands the forces of the ball being kicked, while nevertheless providing accurate data), along with sensors and a miniature RF transmitter in the shin-guards of players, and possibly the umpires, on the soccer pitch. The data from these miniature transmitters is received by a central base-station that, along with a central computer, provides a real-time, in-game data-aggregation and movement-analysis system for soccer. The system claims to provide data to within millimeter accuracy of position/location tracking 200+ times a second. The purpose of the system is to provide movement analysis, flight-path tracking of the ball, etc., in order to assist with refereeing, coaching and training. Because every player is tracked, it is also possible for the system to assist in analyzing combinations of players and plays.

Inspiring Innovations, One Prototype at a Time (Spring 2015)

What happens when you introduce a group of intelligent, curious computer-engineering students to the problems and the challenges of the sports industry? Pure magic. During the past 15 weeks of Spring 2015, Carnegie Mellon students taking the Internet of Things course have blown me away with a set of unique ideas that will impact a variety of aspects in sports. The students have worked in groups to build prototypes of innovations that could impact training (visual pacing-feedback system for runners, smart cleats for soccer), crowd flow and facility management (beacons), the fan experience at home (Oculus and Kinect-based virtual-reality systems), and the fan experience in the stands (smart beverage cup).

I am personally grateful to all of the sport teams and executives who have visited Carnegie Mellon, who have interacted with these students to provide feedback on the early prototypes, and who have provided the critical business and industry insight to sharpen the students’ thinking and focus.

Here are the inspiring innovations of the class of Spring 2015.

PaceMate: Visual Pacing-Feedback System for Runners
Joshua Antonson and Aayush Agarwal

PaceMate is a visual pacing-feedback system for runners which dynamically updates a runner’s pace in response to their run. PaceMate is a portable, non-invasive, easy-to-install system which uses LED lights to guide runners as they run around a track or route, and runners just have to follow the moving LED lights to keep their run at their desired pace. With PaceMate, runners no longer have to juggle timing calculations in their head and get distracted during their runs, and they can focus on getting the pace of their run just right.

Beacon-Enabled Population-Flow Tracking for Stadiums
Kedar Amladi and Chengxiong Ruan

Crowds and lines can create frustrations for sports fans at stadiums and ballparks, especially with high attendance games and events. This project uses Bluetooth Low Energy beacons to help spectators at events navigate around crowds and away from lines by letting users see at a glance where the longest lines and biggest crowds are, helping users plan which concession stands to go to, and helping event organizers and venue owners provide spectators with a more pleasant experience.

Oculus-Enabled Virtual Reality System for Immersive Fan Experience
Gillian Tay and Ajmal Thanikkal

Virtual reality is a cost-effective way for sports fans to experience their favorite games from a new and compelling immersive perspective. This project builds a cost-effective system using the Oculus and Kinect platforms in order to provide sports fans with a unique fan experience by combining sensors and tracking technology with an immersive virtual reality system, to enable sports fans to interact with their favorite sports and sports teams.

Embedded Smart Cleats for Individual Kick-Motion Training
Jacob Nelson and Andy Choi

It is challenging for young athletes to train themselves effectively on their own in the mechanics and specialized basic motions in sports, such as proper kicking technique in soccer, without the continuous supervision of coaches. It is difficult to monitor your own body or know if your movement was correct without any past data or feedback. This project developers a cost-effective, removable insole system that integrates into existing sports cleats to monitor, record, and analyze athletes’ motions, and provide them with feedback to help them improve their own skills.

MCUp: Smart Cups for a Better Beverage Experience at Events
Peter McHale and Elena Feldman

Currently, it is challenging to effectively sell beverages such as beer to spectators at stadiums because of a lack of communication between roving hawkers and spectators. The MCUp attempts to solve this problem by embedding sensors in a cup to detect the amount of beverage left in a user’s cup. Then, by communicating with the user’s smartphone, each user’s MCUp knows when the user has finished his or her beverage and is ready to purchase more from the beverage hawkers. Additional features which can be provided include counting the number of beverages consumed, and estimating the user’s Blood Alcohol Content based on the amount of alcoholic beverage consumed.