Sunday, August 21, 2022

File vs Object vs Block storage


Which data storage should I use?

File vs Object vs Block storage:

After emerging of social media, data platform has evolved in respect of data storage, and processing methods to handle unstructured data. We are going to see the difference between file, block, and object storage and its pros and cons in this post.

1. File storage:

In this, all the data stored are saved together in a single file with an extension type determined by the application/tool used to create the same (Ex: .jpg,.docx,.txt). File storage uses hierarchical storage where files are organized by the user in folders and subfolders, which helps search and manage files. Mostly we manage file storage through a simple file system such as File manager.

Pros:
1.      Easy to access on a small scale.
2.      Familiar to most users
3.      Users can manage their own files
4.      Allows access rights/file sharing/file locking with the password.

Cons:
1.      Challenging to manage and retrieve large numbers of files:
2.      Hard to work with unstructured data
3.      Becomes expensive at large scales: 

Use cases for file storage:
1.      Collaboration of documents across cloud storage or Local Area Network (LAN)
2.      Backup and recovery:
3.      Archiving: due to its security and simplicity of management.

2. Block storage: 
In this storage system, the data is split into fixed blocks of data and then stored separately with unique identifiers. The blocks can be stored in different environments, such as one block in Windows and the rest in Linux. When a user retrieves a block, the storage system reassembles the blocks into a single unit. Block storage is the default storage for both hard disk drives and frequently updated data. You can store blocks on Storage Area Networks (SANs) or in cloud storage environments.

Examples: Amazon Elastic Block Store (Amazon EBS), Azure Disk Storage, Persistent Disk, and Local SSD in google.

Pros:
1.    Fast 2.Reliable: 3. Easy to modify

Cons:
1.      Lack of metadata, 2.Not searchable 3. High cost

Use cases for block storage:
1.  Databases. 
2. Email servers. 
3. Virtual machine file system (VMFS)

3. Object storage:

The object storage is a system that divides data into separate, self-contained units stored at the same level. No folders or sub-directories. Objects also contain metadata, which helps with processing and usability. Users can customize their keys and values based on their needs. Instead of the path, each object has a unique number. Objects can be stored locally on computer hard drives and cloud servers. but it manages via an Application Programming Interface (API).

Pros:
1. Handles a large volume of unstructured data 
2. Affordability, 
3. Advance search capacity

Cons:
1. Can't modify a portion 
2. slow performance 
3. Can't lock
1.IOT, 
3. Video surveillance

Examples: Amazon s3, Azure Blob
The detailed information is summarized in the below pic.





No comments:

Post a Comment

Spark- Window Function

  Window functions in Spark ================================================ -> Spark Window functions operate on a group of rows like pa...