Big Data is a set of massive data, a collection of huge in size and exponentially growing data, that cannot be managed, stored, and processed by traditional data management tools.

• When we talk about Big Data, we talk about Hadoop. So, this is another Big Data interview question that you will definitely face in an interview.
• Hadoop is an open-source framework for storing, processing, and analyzing complex unstructured data sets for deriving insights and intelligence.

FSCK stands for Filesystem Check. It is a command used to run a Hadoop summary report that describes the state of HDFS. It only checks for errors and does not correct them. This command can be executed on either the whole system or a subset of files.

• NameNode – Port 50070
• Task Tracker – Port 50060
• Job Tracker – Port 50030

There are three core methods of a reducer. They are

• setup() – This is used to configure different parameters like heap size, distributed cache and input data.
• reduce() – A parameter that is called once per key with the concerned reduce task
• cleanup() – Clears all temporary files and called only at the end of a reducer task.

The term fsck stands for File System Check, used by HDFS. It is used to check discrepancies and if there is any difficulty in the file. For instance, if there are any missing blocks in the file, HDFS gets reported through this command.

HDFS NameNode supports exclusive write only. Hence, only the first user will receive the grant for file access and the second user will be rejected.

Below are the common input formats in Hadoop 

• Text Input Format –The default input format defined in Hadoop is the Text Input Format.
• Sequence File Input Format –To read files in a sequence, Sequence File Input Format is used.
• Key Value Input Format –The input format used for plain text files (files broken into lines) is the Key Value Input Format.

There are two phases of MapReduce operation.

• Map phase – In this phase, the input data is split by map tasks. The map tasks run in parallel. These split data is used for analysis purpose.
• Reduce phase- In this phase, the similar split data is aggregated from the entire collection and shows the result.

• Here are the reasons for using Hadoop in Data Science:
• Engaging Data with Large Datasets
•  Simplified methods of Data Processing
•  Using its flexible schema for Data Agility
•  Providing linear scalable storage for Data Mining

Here are the 10 most useful tools used in Big Data Solutions

• Hadoop
• Apache Spark
• Apache Storm
• Cassandra
• Rapid Miner
• MongoDB
• R Programming Tool
• Neo4j
• Apache SAMOA
• HPCC