Senior Data Engineer
8+ years of hands-on development experience in multiple projects, with progressively increasing responsibility
5+ years of hands-on experience working with data warehousing like applications and big data.
Experience leveraging big data technologies (One or more of Hadoop, Python, Spark) is required.
Exposure to Microsoft Azure (or other cloud) platforms is preferred
Experience working with various data exchange formats (JSON, CSV, XML etc.)
Solid understanding of relational and dimensional database design and knowledge of logical and physical data models is preferred
Experience with SDLC and / or Agile methodologies for project development, and participation in all phases of project development, is required
Excellent knowledge of SQL and Linux shell scripting
Experience in deploying and managing SQL and NoSQL databases is preferred
Experience with job scheduling (TIDAL, CAWLA, Oozie) and file transfer (e.g. SFTP)
Excellent diagnostic, analytical and problem-solving skills are preferred
Experience with continuous delivery tools (Jenkins, Bamboo, Circle CI), and an understanding of the principles and pragmatics for build pipelines, artefact repositories, zero-downtime deployment, etc. is preferred
Experience building real-time data pipelines using Kafka or spark streaming is preferred