Job Search
Ми допоможемо тобі побудувати вражаючу кар’єру
Senior Data Engineer IRC230006
Job: | IRC230006 |
Location: | Poland - Krakow |
Designation: | Senior Software Engineer |
Experience: | 5-10 years |
Function: | Engineering |
Skills: | Big Data, Databricks, ETL, NoSQL, Python |
Work Model: | Remote |
Description:
Cloud-based data processing system; building analytics from metrics received from devices in the field and from the data received by automated content recognition.
Requirements:
- Bachelor’s degree in Computer Science, Engineering, or equivalent experience
- Proven experience with one of BigData technologies such Pyspark, Pandas, Spark, Hadoop, Hive
- Proficient in Python as well as AWS tools
- Proficient in SQL
- Experience with creation/maintenance of data validation tools and frameworks
- Knowledge of data modeling concepts and ETL processes
- Experience with system integration testing, end-to-end testing, databases, CI/CD pipelines
- Proven experience as a Data and QA Engineer with a focus data/ETL pipeline testing, regression data testing
- Familiarity with data integration and data warehousing technologies such as Databricks/Snowflake is a bonus
- Good communication skills
Responsibilities:
We are looking for a Data Engineer with a Dev background.
This role involves writing testing frameworks and Automation, reviewing ETL code, Test ETLs. All pipelines are built in Databricks.
• Validates data and ETL pipelines to bring new data into a data warehouse
• Data mining and detailed data analysis on data warehousing systems
• Verify data accuracy, completeness, and consistency across various data sources and pipelines
• Create and maintain test data sets for regression testing
• Provide test support for any issues that require code changes or changes made directly to the ETL pipelines
• Implement and maintain automated testing framework for data validation
• Validate data transformations, aggregations, and calculations to ensure accuracy & reliability
• Create and support data validation scripts for new & existing ETL pipeline changes
• Create visualization dashboards to analyze/monitor data for ETL pipeline changes
• Design, develop, automation tools to test ETL pipelines
• Write Python scripts in PySpark for data processing and manipulation
#LI-ZK1 #LI-Remote
What We Offer
Empowering Projects: With 500+ clients spanning diverse industries and domains, we provide an exciting opportunity to contribute to groundbreaking projects that leverage cutting-edge technologies. As a team, we engineer digital products that positively impact people’s lives.
Empowering Growth: We foster a culture of continuous learning and professional development. Our dedication is to provide timely and comprehensive assistance for every consultant through our dedicated Learning & Development team, ensuring their continuous growth and success.
DE&I Matters: At GlobalLogic, we deeply value and embrace diversity. We are dedicated to providing equal opportunities for all individuals, fostering an inclusive and empowering work environment.
Career Development: Our corporate culture places a strong emphasis on career development, offering abundant opportunities for growth. Regular interactions with our teams ensure their engagement, motivation, and recognition. We empower our team members to pursue their career goals with confidence and enthusiasm.
Comprehensive Benefits: In addition to equitable compensation, we provide a comprehensive benefits package that prioritizes the overall well-being of our consultants. We genuinely care about their health and strive to create a positive work environment.
Flexible Opportunities: At GlobalLogic, we prioritize work-life balance by offering flexible opportunities tailored to your lifestyle. Explore relocation and rotation options for diverse cultural and professional experiences in different countries with our company.