You can afford to store your surveillance data: Storage management basics

The Boston Marathon bombing turned the klieg lights on surveillance video. More people were taking photos and videos of the race finish line than of possibly any other location in the world at that moment, so the Boston Police Department (BPD) requested...

By expanding storage to include tape along with disk, however, storing video surveillance data becomes increasingly affordable. An effective VLM implementation is one in which newer video data and videos that may be accessed more frequently are stored on faster and more expensive storage media, while remaining video data is stored on a less expensive storage medium, moved offline, or deleted. This reduces the size and cost of storing images over time, and reduces risk without sacrificing retention time, discarding data, or creating video gaps.

With a good VLM software application and the use of some disk and tape, it’s possible and affordable to implement a storage strategy for video surveillance data, reducing the size and cost of storing images without sacrificing resolution, duration, or creating video gaps. Alternately, some choose cloud storage for their surveillance data.

Cloud storage, video surveillance as a service

Organizations are increasingly interested in cloud-based services and Video Surveillance as a Service (VSaaS), says Ricco, adding that it isn’t easy to identify any single technology in what he calls the “nebulous” idea of the cloud. IT managers are gaining confidence in security and data integrity as they move data to and from public or private clouds.

The cloud has drawbacks, however, including the bandwidth required to stream video data, and the costs involved in moving or accessing data that has been stored in a public cloud. For large amounts of video and other data, “the current monthly cost to rent storage [from cloud-based storage providers] is roughly equivalent to the amount that would be needed to purchase it outright,” says John Villasenor, Senior Fellow, Governance Studies, Center for Technology Innovation at Brookings, and professor ofelectrical engineeringandpublic policyat University of California, Los Angeles, in a 2011 paper about digital storage growth.

In-house is possible: the plummeting costs of data storage

The ubiquity of the term “Big Data” is already desensitizing us to its staggering implications. Data production is projected to be 1.8 zettabytes this year. Not many people can envision a zettabyte, which is 1,000 exabytes, or 1,000,000 petabytes, or one billion terabytes, if that helps. The quantity of data generated in 2020 is projected to be 44 times greater than it was in 2009, and the growth curve is exponential. A highly scalable storage solution is the only way to prepare for the inevitable.

Once, organizations like the NYPD—and incidentally, many totalitarian governments, according to Villasenor—had to choose the data they wanted to retain because the cost of keeping all their files was too high. Villasenor says this is no longer true.

“Over the past three decades,” says Villasenor, “storage costs have declined by a factor of 10 approximately every four years, reducing the per-gigabyte cost [of storing data on disk] from approximately $85,000 (in 2011 dollars) in mid-1984 to about five cents today. In other words, storage costs have dropped by a factor of well over one million since 1984. Not surprisingly, that fundamentally changes the scale of what can be stored.” Villasenor adds. “Memory costs do not become a major obstacle to video surveillance unless the system is truly massive. Even then, the obstacle will only be temporary.”

Controlling costs with compression

The convergence on the H.264 compression algorithm standard supports data storage at user-specified levels of compression, which allows organizations to better manage storage costs by managing the sizes of stored files. H.264 has also enabled technology developers to support and integrate one another’s software and hardware innovations, allowing for more rapid advances across the video management systems industry. Compression is one of the keys to reducing video storage overhead using a VLM system.

In-house storage options: don’t forget tape

Ricco told me about a memorable, if humbling, meeting with a “very smart” senior IT manager at a global organization a few years ago. He had gone to that meeting armed with data about using solid-state disk (SSD) for the organization’s backup and archive needs, but “he schooled me on tape versus spindles,” Ricco said, adding that he learned a lot that day about the advantages of adding digital tape to the mix of storage technologies because of its high density, energy efficiency, and low total cost of ownership.