Senior Software Engineer - Data Platform
- Charlestown, MA
Our client is a company dedicated to harnessing nature to help farmers sustainably feed the planet. With a vision of creating a world where farming is an economically desirable and accessible profession, this organization works alongside its growers to apply natural approaches, conserve resources for future generations, and grow healthy food for all.
Utilizing beneficial plant microbes to improve crop health and productivity, our clients' portfolio is focused on cotton, wheat, barley, corn, soybeans, and rice.
After recently joining the Unicorn club and positioning themselves as the most-funded agriculture startup in the country, they are looking to grow the engineering organization that powers the tools needed to accomplish this feat.
They are currently searching for a Senior Software Engineer to focus on the development and extension of their Data Platform. Someone who is a self-directed person passionate about database design performance at scale. This person can identify and work with data across a variety of formats while making the data available to an equally varied set of customers, from data scientists to web-based applications through APIs.
You must fundamentally understand big data ingestion techniques and the engineering necessary to push those frameworks to do more. This individual will be comfortable working with, deploying, and managing big data solutions using cloud-based technologies.
- Collaborate with Data Management to provide tools and processes to validate and repair data
- Identify critical gaps and builds ETL processes
- Be the domain expert in stored data and use cases
- Implement / Design IoT Schema and Data Ingestion (w/ tests)
- Finalize v1.x design and implementation for all R&D data
- Understands sparse data sets, storage mechanics, and the data needs for analytical applications
- Understands trade-offs between multiple data formats (flat files, parquet, orc, etc.)
- Understands data lifecycles (e.g. archiving, access, and cost of data management) and can manage/communicate these concern to the Architects and stakeholders.
- Ability to work closely with the Data Science Group to understand their needs, enabling them to more effectively do their job by working with them to be able to access the data they require
- Understands the need to pay significant attention to detail in software data management (e.g. data growth and monitoring, query performance awareness, data use cases) to be able to provide insight into problems before they are realized
- Strong sense of quality and attention to detail
- Willing to interface with many areas of the organization to ensure our data is meaningful
- 7+ years of experience
- Experience designing schemas (DDL)
- Experience architecting database topologies across a wide array of formats
- Comfortable working with complex schemas and managing views into data
- Knowledge of why/when to use NoSQL, columnar, data sharding solutions
- Experience with RDBMS (e.g. Postgres, MySQL, Oracle)
- Working knowledge of at least one NoSQL database
- Knowledgeable of in-memory database solutions
- Experience in Restful API design
- Experience in Java / Python programming languages
- Experience w/ benchmarking and performance tuning data solutions
- Knowledge of AWS data solutions (Spark / EMR / RedShift / Aurora / Athena)
- Bachelor’s degree required, Master’s degree preferred
This client is an equal opportunity employer and value diversity. They do not discriminate on the basis of race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status.
Monday, February 18, 2019
$150,000 - 180,000.00