We are looking for a Senior Data Engineer who has good experience in data modeling and data integration using Spark and SQL.
- Specialize in data integration from multiple external and internal sources in batch and real-time.
- Design and implement the platform and frameworks required for optimal extraction, transformation, and loading of data from a wide variety of data sources.
- Design, implement and operationalize data pipelines, data warehouses and semantic models in data marts using Azure Data Services (ADF, Synapse, CosmosDB,
- Spark/Databricks, Analysis Services, PowerBI etc.).
- Work with stakeholders including the management, product and data scientists to assist with data-related technical issues.
- Coordinate and enhance data quality and availability across the Coca-Cola data estate.
- Automate and optimize existing analytic workloads by recognizing patterns of data and technology usage.
- Manage own learning and contribute to technical skill building of the team.
- Embrace the engineering mindset and systems thinking.
- Gain deep technical expertise in the data movement patterns, practices and tools.
Required Qualifications
- Expertise with distributed computing and big data architectures.
- 3+ years of designing and deploying data analytics pipelines in cloud using Azure Data Services.
- 5+ years using relational databases running ETL process and managing large data transformations.
- 5+ years experience in building and performance tuning Spark code (Scala preferred).
Highly proficient in SQL. - 3+ years of experience in designing data models (STAR, Vault, Tabular) for analytics.
- Polyglot development: Capable of developing in Scala and Python with good understanding of functional programming, SOLID principles, concurrency models and modularization.
- Development experience with at least one NoSQL database. (CosmosDB preferred).
Experience with workflow scheduling / orchestration such as ADF, Airflow or Oozie.
Passion for software engineering and craftsman-like coding prowess. - DevOps: Appreciates the CI and CD model and always builds to ease consumption and monitoring of the system. Experience with Maven (or Gradle or SBT) and Git preferred.
- Believe in “Build, Ship, Monitor” Philosophy.
- Personal qualities such as creativity, tenacity, curiosity, and passion for deep technical excellence.
Desired Qualifications
- Experience with Big Data migrations/transformations programs in the Data Warehousing and/or Business Intelligence areas.
- Hands-on experience with Hadoop and Kafka.
- Experience in building self-contained applications using Docker, Vagrant. Chef.
- Innovation & Technology: The ability to work with an award-winning team that is on the cutting edge of innovation.
- Exposure to World Class Leaders: Availability to global technology leaders that will expand your network and exposure you to emerging technologies and techniques.
- Agile Work Environment: We embrace agile, with management that believes in removing barriers, so you are empowered to experiment, iterate, and innovate.
Skills:
Big Data; Data Warehousing (DW); ETL Processing; SQL; Data Pipelines; Azure Data Factory; Databricks.