Job Summary:

We are looking for a highly skilled Senior Data Engineer to join our innovative data team. The ideal candidate will have extensive experience in data extraction, transformation, and loading (ETL) processes, as well as a strong background in cloud data tools. You will be responsible for building and optimizing data pipelines, ensuring data quality, and collaborating with cross-functional teams to deliver actionable insights.
 

Key Responsibilities:

1. Data Management: Extract, combine, transform, load, manipulate, and cleanse data from various sources to ensure high-quality datasets for analysis.

1. Collaboration: Work closely with data modelers and business analysts to understand the data model and source-to-staging data mapping.

1. Data Structures: Create and integrate data structures suitable for analysis, including developing DDLs to create various MS Cloud SQL Server tables and views in Pond (a subset of the Data Lake) DB.

1. Databricks Development: Design and develop Databricks notebooks for extracting data from various input files, joining related entities to create Shared File Formats (SFFs).

1. Ingestion Framework: Configure the Ingestion Framework (IF) tables to utilize the existing framework for loading staging and Data Lake Core (DLC) tables.

1. Algorithm Development: Develop and deploy algorithms in performance-efficient code across analytical and data platforms.

1. Data Pipelines: Build large-scale batch and real-time data pipelines using data processing frameworks in cloud platforms or on-premise environments.

1. Continuous Integration: Leverage best practices in continuous integration and delivery to enhance data workflows.

1. Data Governance: Track data consumption patterns, collaborating with data governance and information stewards, and participate in vetting and promoting content for the curated data catalog.

1. Automation: Recommend and implement automation in existing and future integration flows to improve efficiency.

1. Data Quality Improvement: Drive advancements in data acquisition, testing, and tooling to enhance data quality.

1. Agile Collaboration: Work in cross-functional agile teams to continuously experiment, iterate, and deliver on new product objectives.

1. Requirement Analysis: Play a key role in requirement analysis and project estimates to ensure alignment with business goals.

1. Performance Monitoring: Monitor performance and optimize SQL queries for maximum efficiency.

1. Documentation: Prepare detailed design documents with prototypes and screenshots based on Business Requirement Documents (BRD) and System Requirement Documents.

1. Cloud Tools Utilization: Work with various cloud data tools such as Azure SQL, Azure Data Factory, and Azure Databricks to ingest and prepare data for analysis as per business requirements.

Qualifications:

• Education: Bachelor’s degree in Computer Science, Engineering, or a related field; Master’s degree preferred.

• Experience: 5+ years of experience in data engineering or a related field, with a strong focus on ETL processes and cloud data tools.

• Certifications required:  Certified Azure Data Engineer, Databricks Certified Data Engineer

• Technical Skills:

• Proficiency in programming languages such as Python, Java, or Scala.

• Experience with MS Cloud SQL Server, Azure Data Factory, and Azure Databricks, PySpark, Spark SQL, ADLS, SQL, PowerBI, SSIS, SSRS, Agile and Scrum

• Strong knowledge of SQL and experience with relational databases.

• Familiarity with big data technologies and data processing frameworks.

• Soft Skills: Excellent problem-solving skills, strong communication abilities, and a collaborative mindset.

SNI sp. z o.o. will process personal data for the purpose of the recruitment process in accordance with Data Privacy Policy. The data may also be stored and processed for future recruitment purposes, in accordance with the given consent.