神刀安全网

7 reasons to choose a file system over a database for managing your data

First| 1| 2| 3| 4| Last|  All

Introduction

Database Management Systems (DBMS) are popular as a uniform data storage, and they have become increasingly popular with proliferation of mobile and web development, where file systems are either not available at all, or have restricted access.

Such proliferation has caused certain bias to using DBMS (and specifically SQLite as a built-in DBMS on mobile platforms) as a universal solution to data storage needs.

However file systems are not the thing of the past. The advancement of mobile software brings back necessity of design templates similar to those on desktop systems. In particular, the need arises to use files and file-based approaches to management of heterogeneous information. So files are still in demand and they are more suitable for many classes of data management tasks.

Here I would like to discuss the use cases where the file system is more suitable solution than DBMS.

Encryption

Data encryption is a complicated goal which is to some extent contradictory to the nature of DBMS. The database needs to know size and in many cases content of the data fields for operating efficiently. This limits flexibility in data management. Modern DBMS (mostly client-server ones) provide means to encrypt table fields, but on mobile devices these capabilities are absent. Also, field encryption doesn’t hide metadata and doesn’t disguise the structure of data and the number of entries. So each developer needs to invent his own encryption schemes, and proper implementation of encryption is a hard to achieve goal.

At the same time encrypting files and the file system in whole is trivial. SolFS and some other file systems implement on-the-fly encryption and decryption of files in streaming mode, so you (the developer) don’t need to care about this. Moreover, certain file systems, such as SolFS, support both whole-storage and per-file encryption and you can use different keys and encryption mechanisms for different files and the storage itself. And if you need to plug your own encryption (eg. certificate-based), this is not hard to do as well via callback mechanisms offered by SolFS.

Compression

Compression is very important on mobile devices, where the user pays for traffic and where the storage space is severely limited. Databases are space-hungry and have no mechanisms for data compression, which leads to excessive complication of the application that needs to deliver data in compressed form and then import them to the database (and even then uncompressed data takes precious space in device memory).

Modern file systems have built-in data compression implemented on file level. I.e. you can compress all or some files as needed.

Versioning

Quite often you have to keep several variants of the same named entity (be it a calendar record, a contact, a text note or anything more complicated). Is it a simple Undo mechanism or logging where you need to keep one or more previous versions of the data, you need a place to store those versions. The file system with Alternate Data Streams support lets you store all variants in one file, so file operations like copying and moving perform operations on the same data.

In DBMS to have versioning you need to maintain a separate table or two as there are no built-in mechanisms for versioning. With file systems version management can be almost transparent – to create a version you open not the file itself, but its alternate stream. Just add a suffix to the filename and you have a version!

Large and heterogeneous files

Relational databases were designed for storing uniform data with predefined number and type of fields for each record. Such design is effective when processing large amounts of such uniform data, but doesn’t work well with objects that have different set of properties. Necessity to manage such object sets lead to creation of OODBMS (object-oriented DBMS) and later no-SQL DBMS. Still, SQL-based DBMS are the mainstream, especially on mobile devices.

SQL-based relational DBMS don’t handle large fields of undefined length (BLOBs): such fields break a sleek structure of the DB table. Various DBMS offer different ways to manage BLOBs. Frequently BLOBs are kept in separate file on the disk (file system comes into play again) with references from the DB table. Other DBMS have a mini-file system built into their engine to manage BLOBs and keep them in one file (here comes a file system again).

The described way DBMS work with BLOBs is an extra level of code which slows down operations with large data and does not add stability and reliability.

With a file system you can have files as large as you want, and some file systems let you add custom properties (SolFS calls them tags) to files. Such properties let you organize object storage easily.

Remote and in-memory storage

The database in DBMS is almost always a set of files in local persistent storage (file system or memory storage). Some DBMS allow creation of in-memory tables for temporary operations, but migrating data to and from such memory storage is cumbersome.

Virtual file system, such as SolFS, allows you to keep the storage anywhere – in memory, in the database (if needed), on the network disk or in the cloud. Using callback mechanism SolFS offers unprecedented flexibility in maintaining storage. Moreover, you can have on-the-fly mirroring of your storage, i.e. have a backup copy on the remote storage as soon as the storage is changed locally.

Easy backup

DBMS are not designed for easy back up. You rarely can just grab files of the DBMS and copy them – restoration of these files becomes a tricky operation. More often you need to export the data to [large and not easily maintainable] SQL file which is then packed and stored somewhere. If you need to recover the data, you need to execute this SQL file, potentially editing it.

In case of file systems backup and restore operations are trivial – you just copy the files or the container itself. In case of SolFS, which stores all files in single container, you can copy the container, then put it back and it will have anything that’s needed to continue work.

And if you use mirroring as mentioned above, you have backups created on-the-fly.

Exposure to other applications

The last but not the least is that you often need to expose the data contained in the DBMS, as files for processing by other applications. With DBMS there’s no way to accomplish the task without external assistance. And while on desktop systems you can use third-party solutions (e.g. Callback File System on Windows or FUSE on Linux) to expose custom data as files, on mobile platforms this is not possible at all.

With “regular” (OS-provided) file systems such task does not appear. SolFS also lets you expose SolFS storages as virtual disks. SolFS OS edition lets you create a virtual disk from SolFS storage on Windows, MacOS X, Linux and FreeBSD systems (unfortunately the task cannot be accomplished on mobile devices due to limitations of the system architecture on those devices).

Conclusion

While DBMS does provide a handy and inexpensive way to store certain data, the expenses and shortcomings of DBMS are numerous and alternatives must be seriously considered before the application is designed. Virtual file system can be a way better alternative to DBMS in many cases.

转载本站任何文章请注明:转载至神刀安全网,谢谢神刀安全网 » 7 reasons to choose a file system over a database for managing your data

分享到:更多 ()

评论 抢沙发

  • 昵称 (必填)
  • 邮箱 (必填)
  • 网址