Job Title: Data Engineer
Location: Menlo Park, CA
Duration: 1+ year W2 contract with possible extension/ conversion
Apply proven expertise and build high-performance, scalable data warehouse application
Securely source external data from numerous global partners
Intelligently design data models for optimal storage and retrieval
Deploy inclusive data quality checks to ensure high quality of data
Optimize existing pipelines and implement new ones, maintenance of all domain-related data pipelines
Ownership of the end-to-end data engineering component of the solution
Collaboration with the program's SMEs, data scientists
Data Engineer will rotate on an On-Call shift as needed to support the team.
5+ years' experience in data engineering, proven expertise of applying DWH/ETL best practices
Proficiency in LAMP and the Big Data stack environments (Hadoop, MapReduce, Hive)
Competence with relational databases (Oracle, MySQL)
Experience working with enterprise DE tools, ability to learn in-house DE tools quickly
Coding and scripting experience with Python, Java, PHP, SQL, CLI
What we're looking for:
1. Someone who understands the difference between Kimball vs Inmon data warehouse methodology.
2. Someone who knows what and how to use different tools/mechanism to get data in and out of HDFS (Hadoop Distributed File Systems).
3. Someone who has a keen understanding of the key differences between Python and other programming languages like Java/C++.
BS/MS in Computer Science or a related technical field
APACHE HADOOP MAPREDUCE
EXTRACT, TRANSFORM, AND LOAD