Talent Toppers

Job Description 

Big Data Developer [BDD]

  • 4 to 5 years of total experience
  • 2 to 3 years hands on experience in Hadoop
  • Should have experience in MVC/Programming language/ETL

Scope of work

  • Configure Big Data related services on the cluster
  • Develop data ingestion & transformation components
  • Testing and deployment of the Big Data related interfaces and scripts
  • Schedule data pipelines in Oozie to enable jobs to load data in the data lake
  • Release note and training manual
  • Write scoop scripts for extracting data from R&M systems like (ISOM, Softmar, CSL)
  • Use Apache Nifi & Kafka for streaming data and Spark for in-memory data processing
  • Write shell / Perl / python scripts to run perform and run data transformations on extracted data
  • Build necessary data views  for visualizations / analytics tools like Zeppelin and  Zoomdata for information delivery
  • Data Ingestion pipelines setup and configured as a workflow (Unix + Oozie)
  • Kafka and Spark configuration and testing
  • Data ingestion use case development (Structured & Semi structured)
  • Data ingestion use case testing (Structured & Semi structured)
  • Deployment document / Training & Operational Document
  • Data Transformation Model & Interfaces
  • Generic Data ingestion routines framework
  • Data ingestion use case development and testing (Unstructured)

Top 7 New Features of Hadoop 3