IT Storage vs. Video Surveillance Storage: Protect Your Video with The Right Storage Solution
Many system integrators and network specialists still recommend IT storage products to store video surveillance footage, but this is quickly becoming an outdated practice that puts retention of your valuable video at risk. In fact, many IT storage features are not necessary for storing video surveillance and actually may do more harm than good.
While it might seem that the collection and storage of footage from video surveillance cameras would mimic that of other data used across an IT computer network, it’s not the case. Video surveillance has specific requirements that are very different from IT. This is because video surveillance information mainly deals with video and not data. To address the precise needs of video surveillance recording and storage, Rasilient began developing and manufacturing Storage Area Network (SAN) products designed specifically for video surveillance.
WHAT DOES VIDEO STORAGE NEED?
SAN, NOT NAS.
An IT computer network typically relies on Network-Attached Storage (NAS). In IT applications, the traffic patterns are diverse from small to large files, more read versus write, and various sequential and random patterns.
Storage products from manufacturers like Dell that are used in video surveillance solutions are designed for enterprise DB applications; however, these “traditional” NAS products ignore many important video surveillance-centric characteristics, such as write focus. Data from video surveillance cameras is primarily write traffic, more than 95% of the time! This is in direct contrast to general IT storage which is usually 70% read versus 30% write.
Raslient’s SAN products provide large blocks for video streams. Each video stream is sequential in nature and written to files that are generally large and the disks are formatted into large blocks, typically 64KB. General IT storage differs in that – because the files are small – the hard disks are formatted into smaller blocks, like 4 KB.
In today’s video camera-rich environment, the total number of camera video streams can literally grow into the thousands. As each file is written to a different physical location on the disks, the disk heads jump around to write to all the files concurrently. As a result, the combined pattern becomes random even though each camera’s video stream is sequential in nature. The typical, general-purpose storage system performs poorly in this multi-camera video stream environment which also becomes difficult to scale with more cameras, higher resolution cameras and longer retention periods. This demand results in high-cost-per-camera video streams.
Several factors can change, making camera traffic very dynamic. One is motion-activated traffic and another is lighting condition. The motion impact is easily seen, and the installer normally pays attention. The lighting condition, however, can be overlooked. This impacts recording bandwidth and can break the system provisioning.
For video traffic, the key question is whether the video frames are saved into the storage system. In an IT environment, if the storage system takes a long time to write something to a disk, the storage system can signal a “please wait” all the way back to the desktop computer that issued the write, and the person simply waits a few seconds longer for the data to be written to the storage product. In the video-surveillance environment, data is written to disk from a camera. The cameras will output video frames at up to 30 frames per second. If the storage cannot write the video to disk fast enough, video frames will be dropped. Dropped video frames are common with general-purpose NAS products.
The growth in surveillance data is caused by more cameras, higher resolutions and longer retention times. The architecture needs to scale storage capacity efficiently to accommodate so-called North-South traffic. This is different from datacenters Hyper-Converged and Scale-Out Cluster architecture, which scales more in computing. Each node requires powerful CPU and memory. The internode communication produces so-called East-West traffic, which demands sophisticated network infrastructure like 10GE and/or Infiniband switches.
IT storage products assume that skilled IT staff is available to install, monitor and diagnose problems. These tasks are performed by in-house staff in most cases. The video surveillance system, however, is normally installed and managed by outside service providers who might not have remote access to critical deployments due to security concerns. Any failure might not be fixed – or even noticed – for a very long time.
WHAT DOESN’T VIDEO STORAGE NEED?
Traditional IT NAS storage products often contain extravagant features that video surveillance storage just simply does not need. In fact, many of these features do more harm than good for video surveillance storage.
IT storage features such as snapshot, replication, continuous data availability, thin provisioning, 8k fixed deduplication for block and file and single instance deduplication for file, block and file compress are just a few of the NAS storage features that are actually counterintuitive to good video surveillance storage.
Let’s take a look at snapshot, for example, which is the most common storage management feature. Snapshot freezes the data in time to allow backup. Each snapshot taken becomes a recovery point. By taking snapshots continuously, you have continuous data availability.
Snapshot would only be needed when data needs to be overwritten constantly (such as in a database). You don’t want to back up data while it is changing, so you take a snapshot first. For surveillance, we only archive the data recorded hours, days or weeks ago. The data does not change. It is static. No snapshot is needed! To support snapshot, the system normally reserves 20% of the total storage capacity. In addition, there is snapshot metadata to manage. The metadata takes away the controller cache space that could be used for recording.
Replication is actually intended only for database. The replication could be “synchronous” or “asynchronous” for each IO command. This granularity ensures the correctness for each IO command across geographical locations. This is not easy to do. If the system loses synchronization between two sites due to network problems, it will need to re-sync before moving forward. Do we need per command data coherency for video surveillance? Not really. We can easily archive recorded data to a remote site without this level of synchronization.
Anything regarding duplication and compression is actually not required for video surveillance storage. Your IP cameras have already compressed video using H.264. Computing and metadata management just wastes storage resources. Anything about 8K block is for database: video surveillance only uses the large blocks we discussed earlier such as 64KB or higher. Thin provisioning is also only for IT: there is no thin provisioning in surveillance. You already know exactly how many days your video retention will be.
The bottom line is that video storage has very specific needs. Your video surveillance storage solution should be equipped with a write focus, large blocks for video streams, the ability to ensure that video frames are never dropped and much more. Traditional IT storage solutions might have an abundance of impressive features like replication and duplication, but they are not needed for video storage.
You need storage built for video surveillance! Rasilient has developed the most innovative Storage Area Network solutions on the market architected specifically for video surveillance.