Agencies Find New Ways to Manage the Video Data ‘Tsunami’

Asset management increasingly important as media assets multiply
Author:
Publish date:
Image placeholder title

Government agencies and business both use new cost-effective ways to store and efficiently manage video data files, even as camera networks grow exponentially, producing higher-resolution images that require ever more computer storage space.

Call it a video “tsunami” or a data bulge, IT departments are seeking more clever ways to handle huge quantities of data that needs to be stored for longer periods of time than ever before, according to Meredith Bagnulo, of the Active Archive Alliance. The organization is a 15-member industry group that promotes vendor-neutral, open-platform archiving solutions.

To complicate things, data storage demands are expected to mushroom. From the 4.4 zettabytes of stored data that is kept on computers today, devices will be asked to retain 44 zettabytes created within the decade, according to the latest Digital Universe report from EMC/IDC. Those 44 trillion gigabytes would fill 6.6 stacks of iPad Air Tablets reaching from the earth to the moon.

VOA

The Voice of America, the U.S. government’s official international broadcaster, has been hit by the increased demand, archiving more than 20 terabytes each month. Storage demands at VoA can be attributed to the vast amount of video VOA shoots and acquires on a 24/7 basis, an official said.

VoA uses the Media Asset Management platform, from Paris-based Dalet Digital Media Systems, to manage its radio and TV playout. The system transparently tracks multimedia content and metadata throughout the creation chain, preserving essential information and making it easier to distribute content across platforms, said Michael Hunt, director of VoA’s digital management division.

“The media asset management system streamlines adding metadata to video, which provides an extremely efficient and rich environment for searching both current and past events,” said Pam Commerford, VoA media asset management branch chief. “The metadata gives the assets a much longer life in the system, as we re-purpose the video to cover a wide range of news, international events and feature reporting.”

Image placeholder title

Fujifilm LTO-6 cartridge

VoA has a two-stage archiving system, first retaining the images for a week on relatively high-cost spinning disk hard drives, then moving the files to magnetic LTO (Linear Tape-Open) archive tape. After seven days, the files are purged from the disk to make room and retained on the tape until the archive gets a request for the video file, Hunt said. Before being considered for the archive, certain descriptive metadata must be added to the file or the system will create an alert to notify the producer that data needs to be added.

MAGNETIC TAPE

The VoA/Dalet system employs familiar technology: magnetic tape data storage. Magnetic tape data storage has been around since the advent of computers, and modern data magnetic tape cartridges were first introduced in the 1970s. /span>

However, LTO is not your granddaddy’s magnetic tape. Paired with the Linear Tape File System, it reduces the access latency that plagued early tape storage systems. LTO is turning heads because it’s an option that is a fraction of the cost of disk storage, easily accessible, with the ability to retain data for more than 30 years.

The data bulge is being fueled by an overarching desire to keep data not just for 30 years, but indefinitely, said David Cerf, executive vice president for strategy and business development at Crossroads Systems.

“We don’t throw anything away,” Cerf said. “We’ve become a society of images. Think of every photo that you’ve taken with your cell phone.”

And, anytime we improve imaging resolution, from 1080p to 4K for example, it means that we will need about four times more storage space to retain the higher-resolution images, Cerf said.

HOT AND COLD DATA

Image placeholder title

User interface for MTMP Solutions' Camera Master File Solution, which includes a Network Attached Storage appliance that uses LTFS technology.

Austin, Texas-based Crossroads produces StrongBox, a Network Attached Storage appliance that uses LTFS technology to better manage data archives. Typically, video data is written to a tape archive, indexed and forgotten. StrongBox allows users to keep better track of the data and move it more quickly to low-cost storage platforms.

Users can set transfer priorities when to move “hot” data, fresh files that may need frequent access, to the “cold” data portion of the archive that will not be accessed as frequently, Cerf said.

IT archiving managers are often in reactive mode for good reason: Deleted video files can be difficult to recover. Also, storage budgets don’t keep up with retention demands, which are driven by the new video technology.

“A lot of the new challenges are camera focused,” said Ryan Moriarty, market development manager for Boulder, Colo.-based Spectra Logic. “People are taking advantage of the new features on the cameras, like facial recognition and analytics.

“That, with the new retention requirements by police departments and other end users, is putting a big strain on departments and becoming a bit of a nightmare to retain that video content for a feasible amount of money,” Moriarty said.

Image placeholder title

SpectraLogic T-Series digital tape libraries use LTO 6 cartridges.

Spectra’s T-Series digital tape libraries use LTO technology to provide reasonably priced long-term storage. The T-Series stands out because it is scalable, allowing customers to add digital tape capacity for virtually unlimited, indefinite video storage.

Tape storage is not the only choice for archiving. There are a wide variety of solutions out there, like cloud storage, hard disk drives or flash memory. Although cloud storage sounds like something from the night’s television weather forecast, it actually refers to saving data to an off-site storage system maintained by a third party.

An innovative use of a cloud archive was developed by CitizenGlobal, which launched the Large Emergency Event Digital Information Repository, a free platform for law enforcement to collect, sort, analyze and distribute photos and videos during a catastrophic incident.

HYBRID ARCHIVES

Genetec, headquartered in Montreal, offers a hybrid system that writes data both to the cloud and a local archive, said Erick Ceresato, company product marketing manager. Hybrid cloud solutions provide users the ability to leverage cloud functionality, while maintaining the use of their on-premise system, Ceresato said.

“The Genetec Security Center provides customers the ability to deploy cameras that record directly to the cloud,” Ceresato said, “or even transfer video recorded on local storage devices to the cloud for long term retention.”

By making better use of the cloud, customers can easily expand their storage capacity whenever it is required, without investing capital funds in additional on-site storage hardware, Ceresato said. Security Center provides the ability to record multiple concurrent streams from single cameras, each with its own video quality settings.

Customers can record one high-quality stream for short-term retention, using high-end storage hardware, while recording a lower-quality stream for long-term retention using less-expensive hardware.

Media Technology Market Partners, based in Los Angeles, provides integrated solutions to the media and entertainment industry. At the core are the firm’s LTFS-enabled storage solutions, said company principal and COO George Anderson. The company’s solutions allow direct access to files on LTO tape, including a disk-like drag-and-drop functionality.

“MTMP’s solutions are designed to specifically help users from being overwhelmed with huge amounts of data and the related consequences,” Anderson said. “An LTO/LTFS NAS can [cost] as much as 80 percent less than an equivalent all-disk systems.”

MORE INFO Active Archive Alliance: www.activearchive.com

Crossroads/Strongbox: www.crossroads.com

Dalet: www.dalet.com

Dell: www.dell.com

Fujifilm: www.dternity.net

Genetec: www.genetec.com

Media Technology: www.mediatechmarketpartners.com

Rushworks: www.rushworks.com

Spectra Logic: www.spectralogic.com

Toshiba: www.toshibasecurity.com

Voice of America: www.bbg.com

SMART STORAGE

The Dell Fluid File System is a hybrid system that assigns data to the most appropriate archive, said Jason Mills, Dell’s director of storage solutions for the U.S. federal market. It automatically moves data around to the least expensive archive, and then pulls it back to a hard drive or flash drive if the data needs immediate access.

“It will look at the pages themselves to see if they have been touched in so many days,” Mills said. “It gives users access to the data where it needs to live.”

The Dell Compellent SC280 dense enclosure is a rack storage solution that stores up to 336 terabytes in a 5 RU footprint, which reduces data center space requirements. “Customers don’t want one flavor of storage,” Mills said. “Everyone has their own personality and attitude, and so does data.”

Fujifilm has made a major investment in tape technology, using barium ferrite magnetic particles and Fujifilm’s core NanoCubic technology for a thin and uniformly coated magnetic layer on its new LTO Ultrium 6 cartridges, according to Rich Gadomski, vice president of marketing at Fujifilm Recording Media USA.

The company has also released Dternity, a deep storage, holistic hybrid approach, said Daniel Greenberg, Fijifilm’s director of new products. Dternity uses both traditional, on-site storage and cloud vaulting (available online all the time) to provide data protection, accessibility and mobility, Greenberg said.

“It’s a unique approach in racking all these things together,” Greenberg said. “The idea is to create multiple copies and have a copy in the media cloud, which protects you if something goes down.”

NETWORK VIDEO SERVERS

Toshiba Surveillance & IP Video Products Group, based in Irvine, Calif., has just launched NVSPro, a high-performance network video server. NVSPro servers allow users to scale IP video surveillance systems to meet changing application requirements, whether it is the need to archive larger volumes of video or to support as many as 64 HD IP cameras at the same time.

Front-accessible hard drive bays can store a hefty 48 terabytes of video and audio, a critical feature for organizations that are required to archive video for longer periods of time, such as law enforcement agencies or schools. The company’s new G6 recording servers can store up to four gigabytes of data-intensive megapixel video applications, while users have the choice of either HDMI, Display Port, DVI or VGA outputs for their large-format monitors and 4K displays.

Image placeholder title

User interface for Rushworks A-List automation and asset management system

For smaller TV broadcast-style users, such as PEG channels and government access video, Rushworks’ A-List automation and asset management system provides a single point of focus for ingesting, playing out and archiving video files.

“The A-List broadcast automation system from Rushworks is a hybrid server and bulletin board system in one box,” said Rush Beesley, founder and president of Flower Mount, Texas-based Rushworks. “It’s used extensively in PEG/access channels, as well as full-power and low-power television stations.”

A-List can feed up to four channels in HD resolution, and the system has a built-in bulletin board capability for on-screen announcements. Storage can be handled on both RAID 5 and NAS systems.

Whether it’s for security, public access TV or international news broadcasting, the declining costs and increasing capacity of storage systems is good for government agencies that need to save ever-growing amounts of critical video information.

Related