Big Data & Hadoop Interview Questions

What is Big Data?
Big data is data that exceeds the processing capacity of traditional database systems. The data is too big, moves too fast, or doesn’t fit the strictures of your database architectures. To gain value from this data, you must choose an alternative way to process it.

Name any org. who is generating Big Data?
Facebook,Google

What is NoSQL?
NoSQL is a whole new way of thinking about a database. NoSQL is not a relational database. The reality is that a relational database model may not be the best solution for all situations. The easiest way to think of NoSQL, is that of a database which does not adhering to the traditional relational database management system (RDMS) structure. Sometimes you will also see it revered to as 'not only SQL'.

We have already SQL then Why NoSQL? 
NoSQL is high performance with high availability, and offers rich query language and easy scalability.
NoSQL is gaining momentum, and is supported by Hadoop, MongoDB and others. The NoSQL Database site is a good reference for someone looking for more information.

What is Hadoop and where did Hadoop come from?
By Mike Olson: The underlying technology was invented by Google back in their earlier days so they could usefully index all the rich textural and structural information they were collecting, and then present meaningful and actionable results to users. There was nothing on the market that would let them do that, so they built their own platform. Google’s innovations were incorporated into Nutch, an open source project, and Hadoop was later spun-off from that. Yahoo has played a key role developing Hadoop for enterprise applications.

What problems can Hadoop solve?
By Mike Olson: The Hadoop platform was designed to solve problems where you have a lot of data — perhaps a mixture of complex and structured data — and it doesn’t fit nicely into tables. It’s for situations where you want to run analytics that are deep and computationally extensive, like clustering and targeting. That’s exactly what Google was doing when it was indexing the web and examining user behavior to improve performance algorithms. 

What is the Difference between Hadoop and Apache Hadoop?
There is no diff, Hadoop, formally called Apache Hadoop, is an Apache Software Foundation project.

What is the difference between SQL and NoSQL?

Is NoSQL follow relational DB model?
No

Why would NoSQL be better than using a SQL Database? And how much better is it?
It would be better when your site needs to scale so massively that the best RDBMS running on the best hardware you can afford and optimized as much as possible simply can't keep up with the load. How much better it is depends on the specific use case (lots of update activity combined with lots of joins is very hard on "traditional" RDBMSs) - could well be a factor of 1000 in extreme cases.

Name the modes in which Hadoop can run?
Hadoop can be run in one of three modes:
i. Standalone (or local) mode
ii. Pseudo-distributed mode
iii. Fully distributed mode

What do you understand by Standalone (or local) mode?
There are no daemons running and everything runs in a single JVM. Standalone mode is suitable for running MapReduce programs during development, since it is easy to test and debug them.

What is Pseudo-distributed mode?
The Hadoop daemons run on the local machine, thus simulating a cluster on a small scale.

What does /var/hadoop/pids do?
It stores the PID.

What is the full form of HDFS?
Hadoop Distributed File System

What is the idea behind HDFS?
HDFS is built around the idea that the most efficient approach to storing data for processing is to optimize it for write once, and read many approach.

Where does HDFS fail?
Cannot support large number of small files as the file system metadata increases with every new file, and hence it is not able to scale to billions of files. This file system metadata is loaded into memory and since memory is limited, so is the number of files supported.

What are the ways of backing up the filesystem metadata?
There are 2 ways of backing up the filesystem metadata which maps different filenames with their data stored as different blocks on various data nodes:
Writing the filesystem metadata persistently onto a local disk as well as on a remote NFS mount.
Running a secondary namenode.

What is Namenode in Hadoop?
Namenode is the node which stores the filesystem metadata i.e. which file maps to what block locations and which blocks are stored on which datanode.

What is DataNode in Hadoop?
Namenode is the node which stores the filesystem metadata i.e. which file maps to what block locations and which blocks are stored on which datanode.

What is Secondary NameNode?
The Secondary NameNode (SNN) is an assistant daemon for monitoring the state of the cluster HDFS, Like the NameNode, Each cluster has one SNN, and it typically resides on its own machine as well. 

What is JobTracker in Hadoop?
The JobTracker is the service within Hadoop that farms out MapReduce tasks to specific nodes in the cluster, ideally the nodes that have the data, or at least are in the same rack. 

What are the functions of JobTracker in Hadoop?
Once you submit your code to your cluster, the JobTracker determines the execution plan by determining which files to process, assigns nodes to different tasks, and monitors all tasks as they are running. 
If a task fail, the JobTracker will automatically relaunch the task, possibly on a different node, up to a predefined limit of retries. 
There is only one JobTracker daemon per Hadoop cluster. It is typically run on a server as a master node of the cluster.

What is MapReduce in Hadoop?
Hadoop MapReduce (Hadoop Map/Reduce) is a software framework for distributed processing of large data sets on compute clusters of commodity hardware. It is a sub-project of the Apache Hadoop project. The framework takes care of scheduling tasks, monitoring them and re-executing any failed tasks. 

What are the Hadoop configuration files?
1. hdfs-site.xml
2. core-site.xml
3. mapred-site.xml

ASP.NET Web.config related Interview Questions

1). What is web.config file in asp.net?
Web.config is the main settings and configuration file for an ASP.NET web application. The file is an xml document that defines configuration information regarding the web application.This file stores the information about how the web application will act. 

2). Does web.config file case-sensitive?
Yes.

3). Web.config file is stored in which form?
Web.config files are stored in XML format.

4). Can one directory contain multiple web.config files?
No. One directory can contain only one file.

5). Can you tell the location of the root web.confit file from which all web.config file inherit ?
All the Web.config files inherit the root Web.config file available at the following location systemroot\Microsoft.NET\Framework\versionNumber\CONFIG\Web.config


6). What is the root tag of web.config file ?
<configuration> tag is the root element of the Web.config file under which it has all the remaining sub elements.

7). What is the use of customErrors tag in web.config file ?
CustomErrors tag provides information about custom error messages for an ASP.NET application. The customErrors element can be defined at any level in the application file hierarchy.

Code:

<customErrors defaultRedirect ="Error.aspx" mode ="Off">
   <error statusCode ="401" redirect ="Unauthorized.aspx"/>
</customErrors>

The customErrors section consists of defaultRedirect and mode attributes which specify the default redirect page and the on/off mode respectively.
The subsection of customErrors section allows redirecting to specified page depending on the error status code.
400 Bad Request
401 Unauthorized
404 Not Found
408 Request Timeout

8). Can you describe the funcnalitity of <httpHandlers> tab in web.config?
HttpHandler is a code that executes when an http request for a specific resource is made to the server. For example, request an .aspx page the ASP.NET page handler is executed, similarly if an .asmx file is requested, the ASP.NET service handler is executed. An HTTP Handler is a component that handles the ASP.NET requests at a lower level than ASP.NET is capable of handling.

9). What is authentication tag/section in web.config?
ASP.NET implements additional authentication schemes using authentication providers, which are separate from and apply only after the IIS authentication schemes. ASP.NET supports the following authentication providers:
Windows (default)
Forms
Passport
None
To enable an authentication provider for an ASP.NET application, use the authentication element in either machine.config or Web.config as follows:

Code:

<system.web>
   <!-- mode=[Windows|Forms|Passport|None] -->
   <authentication mode="Windows" />
</system.web>

10). For which purpose you use <appSettings> tag?
appSettings tag helps us to store the application settings information like connection strings, file paths, URLs, port numbers, custom key value pairs, etc.
Ex:-

Code:

<appSettings>
    <add key="ConString" value="Data Srouce=....."/>
</appSettings>

11). What is the use of connectionStrings tag?
<connectionStrings> is the most common section of web.config file which allows you to store multiple connection strings that are used in the application.

Code:

<connectionStrings>
    <add name ="ConString" connectionString ="Initial Catalog = abc;
        Data Source =localhost; Integrated Security = true"/>
</connectionStrings>

12). Can you write down the C# code for reading connection string which is defined in web.config?

Code:

string cnn = ConfigurationManager.ConnectionStrings["ConString"].ConnectionString;