We are seeking a highly skilledΒ QA LeadΒ with a strong background inΒ data engineering and ETL testingΒ to define and execute comprehensive test strategies. This role requires deep expertise in validatingΒ data ingestion pipelines, ensuring data integrity, transformation accuracy, and completeness at every stage of the pipeline.
The ideal candidate will design and implementΒ robust testing methodologiesΒ that extend beyond basic source-data validation. They should incorporate advanced techniques such asΒ schema validation, data profiling, reconciliation strategies, and automation frameworksΒ to enhance test coverage, reliability, and efficiency.
Key Responsibilities:
- Define and executeΒ test strategiesΒ for validating data pipelines, transformations, and data quality.
- Develop and implementΒ automation frameworksΒ to improve test efficiency and reliability.
- PerformΒ data validation and reconciliationΒ between cloud storage and databases.
- Validate structured and semi-structured data formats, includingΒ JSON, AVRO, and Parquet.
- Design and implementΒ schema validation, data profiling, and integrity checksΒ across multiple stages of data processing.
- Collaborate withΒ data engineers, analysts, and stakeholdersΒ to understand data flow and ensure high-quality data delivery.
- Troubleshoot and resolveΒ data discrepancies and quality issuesΒ using a systematic approach.
- Optimize and automate test processes to supportΒ continuous integration and deployment (CI/CD)Β in data environments.
Required Skills & Qualifications:
- Strong experience in ETL TestingΒ andΒ Data Quality Assurance.
- Proficiency inΒ PythonΒ for automation and data validation.
- Hands-on experience withΒ various data formatsΒ (JSON, AVRO, Parquet) and cloud storage solutions.
- Expertise inΒ data validation techniques, including schema checks, data profiling, and reconciliation methods.
- Familiarity withΒ cloud databases, data lakes, and big data ecosystems.
- Experience in designing and implementingΒ test automation frameworksΒ for data pipelines.
- Strong analytical and problem-solving skills to identify and resolve data issues.
- Knowledge ofΒ CI/CD practicesΒ for data testing and automation.