RainStor Reduces the Cost of Retaining (Structured) Big Data
Big Data is getting a lot of attention, much of it related to storing massive amounts of unstructured data (e.g. images, video, email, documents). Scale-out NAS and clustered, global file systems have come out to handle the long term storage of these very large file archives. But structured data - databases, log files, protocol records, etc. - comprises a significant amount of the Big Data category as well. And, the cost of its long term storage and on-demand accessibility may be an even bigger problem. Storing large structured data sets typically involves buying more of the higher end infrastructure it was created on, since these data can’t simply be migrated over to a less expensive file storage system, and still be available for query.
Friday, July 15, 2011
RainStor is a software solution that retains structured data online, enabling read-only access to data extracted from relational databases, XML and log data. It uses a patented field and patten deduplication technology to reduce typical structured data sets by as much as 40:1 and stores them in ‘container’ files which can be searched in their compressed format using standard SQL and BI tools.
The RainStor software runs on Linux, Unix and Windows platforms and can ingest data from all major relational database management systems (RDMS), as well as logs, security events, CDR, IPDR, XML, BCP and delimited files. Read-only data to be retained is copied to a staging area and imported by the RainStor system. This process creates a ‘pattern database’, which stores only unique data, allowing subsequent iterations of patterns to be deduplicated, with 40:1 reductions in physical data footprint not uncommon. Once ingested, metadata is created, making data available for standard SQL searches, without ‘re-hydrating’ data to its original format. RainStor also integrates with content addressable storage (CAS) devices, tracking every action and event from ingestion through deletion and keeping MD5 fingerprints to enable tamper detection. It is most often deployed on commodity server and storage hardware to reduce costs.
Partnering with Dell, RainStor ported their software to the Dell DX CAS system, originally developed by Caringo. This object-based storage platform has a clustered, node architecture and runs on low-cost server and storage hardware, providing a scale-out file storage solution. With RainStor, the DX solution also supports the retention of structured data to make it a comprehensive, economical long-term storage platform.
RainStor allows companies to retain portions of their large structured data sets, removing them from the high priced infrastructure they typically occupy. Relocating and deduplicating this inactive data can result in cost savings from hardware and software as database growth is kept in check and large data sets are economically retained. But a more compelling aspect may be in operational savings as database administrators (DBAs) are among the highest paid IT personnel. By removing large sections of historical data from an RDMS, a company can similarly reduce the workload on current DBAs, since RainStor can be run by general systems administrators.
Storage Swiss Take
RainStor is a very unique technology which seems to provide an answer for the ‘back-end’ of the Big Data question. While much of the attention is being paid to analytical (OLAP) and transactional (OLTP) databases, RainStor is providing a specialized data retention (OLDR) database. This is a true ‘nearline’ storage solution for structured data, one that provides significant data reduction in the process. It’s giving organizations a way to keep the storage and administrative costs of growing databases in check, so they can potentially see big benefits from Big Data.
Eric Slack, Senior Analyst
Briefing Note
RainStor is not a client of Storage Switzerland
