We’re looking for a driven professional with an inclusive mindset to join our team as a Senior Data Engineer (consultant) within the Merchandising Planning & allocation technology team. The Senior Data Engineer will play a pivotal role in developing and supporting modifications and extensions to our Planning project’s data extraction, error
identification, and transformation processes. This role includes ensuring the integrity and accuracy of data loading, monitoring progress, reporting errors, and correcting data ingestion/validation issues. Responsibilities also include migrating historical data from legacy Iseries systems to Azure or Google Cloud, setting up and configuring data connectivity, and managing data mapping between systems. Additionally, you will be responsible for setting up and extracting data from the new Planning system on Azure using Database as a Service or transfer data from Azure blob to Google cloud . A day in the life as a Data Engineer.
.
? Data Extraction and Transformation:
? Develop and support modifications/extensions to the project's source data extraction processes.
? Identify and resolve source data errors and transformation issues.
? Support the loading of data and ensure accurate progress monitoring and error reporting.
? Data Ingestion and Validation:
? Correct data ingestion and validation issues to maintain data integrity.
? Move historical data from legacy Iseries systems to DBaaS on Azure or Google Cloud.
? Data Management Support:
? Provide support to data management initiatives through data models, data controls, and integration capabilities.
? Utilize knowledge of RESTful APIs and integration techniques to enhance data
integration processes.
? Understand and work with different database platforms, including Oracle, SQL Server (Azure) and DB2.
? Develop and maintain effective KPIs to manage data integrity, and SLA adherence
? Performance Optimization & Scalability planning: Tuning and optimizing data processing pipelines and systems for improved performance. Monitoring and
addressing bottlenecks in data workflows. Assisting in planning for future data growth by designing scalable solutions. Ensuring that data infrastructure can handle increasing volumes and complexities.
? Administration: Set up and configure data connectivity and manage data mappings between the host systems and the new planning tool.
? Collaboration: Work closely with cross-functional teams, including product managers, engineers, and analysts, to understand data requirements and deliver
solutions.
? Monitoring and Troubleshooting: Monitor data flows and troubleshoot any
issues related to data integration and connectivity.
? Use SDLC processes for Data and Applications including Waterfall, Agile and DevOps
Experience Required:
? Bachelor's Degree in Computer Science, Information Technology, Data Science, or a related field with 7 years' experience
? Proficiency in working with various database platforms, including Oracle, SQL Server
(On-Prem / Azure) and DB2
? Strong SQL skills and experience with database management.
? Proficiency in programming languages used for data engineering (e.g., Python,Java,
Scala). Ability to write scripts for data manipulation, transformation, and automation.
? Strong knowledge of data governance, data validation, and data quality improvement practices. Experience with Data governance tools like Dataplex is a plus.
? Experience with cloud-based data services and platforms, specifically Azure and Snowflake.
? Strong knowledge of data pipeline development, ETL processes, and data integration.
? Experience working with API developers to connect and integrate various systems. Familiarity with RESTful APIs and integration techniques.
? Experience with big data technologies such as Kafka
? Familiarity with legacy systems (e.g., AS400) and modern Retail Merchandising planning tools is a plus.
Knowledge, Skills, and Abilities:
? Aligning with core values, we are looking for someone who:
? Brings a positive, service-oriented approach to work
? Is driven by creating unique and efficient solutions for business challenges
? Is passionate about their contribution
? Strong communication skills to understand requirements and explain data-related concepts.
? Excellent problem-solving and analytical skills.
? Outstanding skills at collaboration with