Location: Chennai
Experience: 7+ Years
Position: Big Data Analyst – Lead/ SME
Primary Skill
– Spark,Scala,HDFS,Yarn,Hive, Pig, Flume, Sqoop ,Oozie
– Strong Java hands-on
– Unix Shell script
Secondary Skill
-Prefered MapR Hadoop
-Good SQL knowledge
Roles and Responsibilities:
•Participate in technical planning & requirements gathering phases including design, coding, testing, troubleshooting, and documenting big data-oriented software applications.
•Implements, troubleshoots, and optimizes distributed solutions based on modern big data technologies like Hive, Hadoop, Spark, Elastic Search, Storm, Kafka, etc. in both an on premise and cloud deployment model to solve large scale processing problems.
•Define and build large-scale near real-time streaming data processing pipelines that will enable faster, better, data-informed decision making within the business.
•Work inside the team of industry experts on the cutting edge Big Data technologiesto develop solutions for deployment at massive scale.
•Study data, identify patterns, make sense out of it and convert it to algorithms.
•Designs and plans BI, and other Visualization Tools capturing and analyzing data from multiple sources to make data-driven decisions, as well as debugs, monitors, and troubleshoots solutions.
•Keep up with industry trends and best practices, advising senior management on new and improved data engineering strategies that will drive departmental performance leading to improvement in overall improvement in data governance across the business, promoting informed decision-making, and ultimately improving overall business performance.
•Capable to manage the team and user interaction to get the requirment and ensure quality deliverables.
•Good analtical skill with end to end ownership for the deliverables.
•Interact with MapR product team to address any issue in the platform and bring it to closure
•Provide robust solution to build BI projects for data visualization.
•Good understanding in data warehousing and Data Lake concepts
Skills and Experiences:
•Programming languages: Spark Scala MUST. Good to have a knowledge of Java and Python
•Experience with Hadoop development and QA
•Experience with MapR platforma and Apache Drill
•Experience with Unix shell scripting and other MapR ecosystem porducts
•Knowledge of Hadoop ecosystem components like MapReduce, Hive, Yarn, Oozie,HBase and, Ranger and Sentry.
•Security framework experience with Kerberos, LDAP
Qualifications:
• Degree in Computer Science or Information Technology with 4 years of Education.
• Familiar with RDBMS like SQL Server, understanding the data modeling,Parquet,JSON and XML file.
• Knowledge in Apache Drill
• Experience in a financial institution IT Environment and good knowledge of financial industry and related products is advantage.
• Data Model knowledge – an understanding of the structure of the data, what’s contained in the Data Warehouse, data sources and data quality issues
• Data Extraction, Transformation and Load – an understanding of how to extract, transform and load data; this would also involve obtaining and making available new data sources
• Tool support and installation – Monitor tool usage and perform tool upgrades.
