FUNDAMENTALS OF COMPUTER

Detailed explanation-1: -HDFS stores each file as blocks, and distribute it across the Hadoop cluster. The default size of a block in HDFS is 128 MB (Hadoop 2. x) and 64 MB (Hadoop 1. x) which is much larger as compared to the Linux system where the block size is 4KB.

Detailed explanation-2: -The size of the data block in HDFS is 64 MB by default, which can be configured manually. In general, the data blocks of size 128MB is used in the industry.

Detailed explanation-3: -The default size of the HDFS data block is 128 MB. If blocks are small, there will be too many blocks in Hadoop HDFS and thus too much metadata to store. Managing such a huge number of blocks and metadata will create overhead and lead to traffic in a network.

Detailed explanation-4: -Block Size The default is 128 MB but can be changed/configured easily. In our example, a 500 MB file needs to be broken into blocks of 128 MB. 500/128 = 3 blocks of 128 MB and 1 block of 116 MB. The residual block space of 12 MB is returned back to the name node for usage elsewhere, thus preventing any wastage.

Detailed explanation-5: -The default block size is 1024 bytes for file systems smaller than 1 TB, and 8192 bytes for file systems 1 TB or larger. Choose a block size based on the type of application being run. For example, if there are many small files, a 1 KB block size may save space.