VibroSense
Credit: Medical express

Cornell University Scientists have developed a single device that can track seventeen types of appliances using vibrations. To boost efficiency in typical households—where people forget to take wet clothes out of washing machines, turn off dripping faucets, and retrieve hot food from microwaves.

The device, known as – VibroSense, uses lasers to capture subtle vibrations in ceilings, walls, and floors, as well as a deep learning network that models the vibrometer’s data to create different signatures for each appliance—bringing scientists closer to a more efficient & integrated smart house.

“Recognizing home activities can help computers better understand human behaviors and needs, with the hope of developing a better human-machine interface,” said Cheng Zhang, assistant professor of information science and senior author of “VibroSense: Recognizing Home Activities by Deep Learning Subtle Vibrations on an Interior Surface of a House from a Single Point Using Laser Doppler Vibrometry.” The paper was published in Proceedings of the Association for Computing Machinery on Interactive, Mobile, Wearable and Ubiquitous Technologies and will be presented at the ACM International Joint Conference on Pervasive and Ubiquitous Computing, which will be held virtually Sept. 12-17.

“In order to have a smart home at this point, you’d need each device to be smart, which is not realistic; or you’d need to install separate sensors on each device or in each area,” said Zhang, who directs Cornell’s SciFi Lab. “Our system is the first that can monitor devices across different floors, in different rooms, using one single device.”

In order to detect usage across an entire house, the researchers’ task was twofold: detect tiny vibrations using a laser Doppler vibrometer, and differentiate similar vibrations created by multiple devices by identifying the paths traveled by the vibrations from room to room.

The deep learning network was trained to distinguish different activities, partly by learning path signatures—the distinctive path vibrations followed through the house—as well as their distinct noises.

The device showed nearly 96 percent accuracy in identifying seventeen different activities across five houses—including dripping faucets, an exhaust fan, an electric kettle, a refrigerator, and a range hood—in five houses over two days, according to the paper. VibroSense could also distinguish five different stages of appliance usage with an average accuracy of more than 97 percent.

In single-story houses, the laser was pointed at an interior wall at the center of the home. It was pointed at the ceiling in two-story homes.

The device is primarily useful in single-family houses, Zhang said, because in buildings it could pick up activities in neighboring apartments, presenting a potential privacy risk.

Zhang said, “It would definitely require collaboration between researches, industry practitioners, and government to make sure this was used for the right purposes.”

Among other uses, the system could help homes monitor energy usage & potentially help reduce consumption.

“Since our system can detect both the occurrence of an indoor event, as well as the time of an event, it could be used to estimate electricity and water-usage rates, and provide energy-saving advice for homeowners”.

“It could also prevent water and electrical waste, as well as electrical failures such as short circuits in home appliances.”