Cheap VPS & Xen Server

Residential Proxy Network - Hourly & Monthly Packages

Starting Sqoop


Sqoop is a command line tool with following structure

  1. sqoop TOOL PROPERTY_ARGS SQOOP_ARGS [– EXTRA_ARGS]
  • TOOL indicates the operation eg: “import”, “export”.
  • PROPERTY_ARGS are Java properties in the format “-Dname=value”
  • SQOOP_ARGS mention various Sqoop parameters
  • EXTRA_ARGS are for specialized connectors, separated from the SQOOP_ARGS with a “–“

Example:

  1. % sqoop import –connect jdbc:mysql://localhost/hadoopguide –table widgets -m 1

Type “sqoop help” to get all the tools available:

Commands

What happens in backend

When you have decided to move data from RDBMS to HDFS then the first product which comes into use is Apache Sqoop. When you request to bring the data to HDFS then the following things happen.

  1. Sqoop asks for metadata information from Relation DB.
  2. Relational DB returns the required request.
  3. Based on metadata information Sqoop generates java classes.
  4. Based on primary id partitioning happens in table as multiple mappers will importing data as the same time.

Comments

comments