Job Search
We can help you build an exceptional career
Python Data Engineer IRC236749
| Job | IRC236749 |
| Location | Poland - Krakow |
| Designation | Senior Software Engineer |
| Experience | 3-5 years |
| Function | Engineering |
| Skills | Apache Spark / PySpark, AWS, AWS Redshift, Python, S3 |
| Work Model: | Remote |
Description
GlobalLogic is going to build an analytical platform for the client. A platform which will be gathering information about some companies from various sources, will normalize it according to the predefined flow and will show it to the user as a set of analytical dashboards. The gathered and processed information will let the customers make decisions regarding the business state of the investigated companies. Analytical information will help customers to predict the future state of the appropriate companies and improve customer’s business processes.
The client provides high value analysis and support to partner companies for identification and mitigation emerging business challenges. Today the customer’s business processes are highly manual and fragmented.
The project is aimed to define and bring to life a digital platform that connects the client employees with relevant and meaningful information about their portfolio holdings, enabling insights and action.
Requirements
- Proficiency in software development in Python (in use)
- Proven ability to work independently as well as to perform effectively in a team-oriented and open-concept environment
- Deep experience working with big data, unstructured data, structured data including cleaning/transforming/cataloguing/mapping/ etc.
- Experienced in cloud technology best practices to enable the distribution and analysis of data on the cloud (formatting/partitioning/etc.)
- Experience of ETL pipelines, managing multiple datasets and providing necessary support
- Familiarity working with data fabrics and data lakes using S3/Redshift
- Exposure to big data workflows and analytics tools (Spark/EMR/Databricks/Mongo/Casandra)
- Deep proficiency in Python with experience using Spark, Pandas or PySpark
- An understanding of CI/CD pipelines and experience with DevOps
- Experience building flexible solutions that can adapt quickly to changing requirements
- Ability to work in an entrepreneurial environment and be a self-starter
- Proven attention to accuracy and detail, highly organized with the ability to prioritize and multi-task
- Ability to work in a high performing culture with time-sensitive deadlines
- Personable, easily interacts with all types of personalities and at all levels with a high degree of professionalism.
- Strong SQL programming background with the ability to troubleshoot and tune the code
- Proven understanding and demonstrable implementation experience of cloud data platform technologies (AWS)
- Knowledge of Enterprise Data Warehouse technologies including Multi-Dimensional Data Modeling, Data Architectures or other work related to the construction of enterprise data assets
- Develop data pipelines with ETL process, connecting different data sources with custom pipelines
- Experience with Big Data querying tools
- Practical experience with Kafka
- Strong problem solving, troubleshooting and analysis skills
- Good communication skills
- Intermediate+ speaking and written English
- Ability to work at EST time zone (Toronto)
Will be a plus:
- Experience in development of ETL solutions
- Previous experience as a Python data engineer
- Substantial understanding in AWS data ingestion frameworks, data serverless processing, AWS Athena, Glue, EMR, Redshift, Lambda, Batch, Step function
- Deep familiarity with Airflow and PySpark
- Good knowledge of linux/shell
Job Responsibilities
-
Design solutions aligned with long-term architecture and technology strategy using Amazon Web Services (AWS) for Cloud development
-
Work with data team and business to select and acquire non-traditional datasets and access, clean, and pre-process data as required by use cases
-
Select appropriate datasets and data representation methods
-
Collaborate with data analysts to have necessary data pipelines built
-
Build, train, validate and test models using criteria relevant to business objectives
-
Work in a fast-paced environment collaborating with data analysts, data engineers and architects
-
Prepare, transform, combine and manage structured and unstructured data for use by business users
-
Recommend ways to improve data reliability, efficiency and quality for our business teams
-
Manage application data issues, exceptions, proactively identify the root causes and propose solutions
-
Work closely with business analysts, business teams, architecture teams, leads and developers.
#LI-TY1
We Offer
Empowering Projects: With 500+ clients spanning diverse industries and domains, we provide an exciting opportunity to contribute to groundbreaking projects that leverage cutting-edge technologies. As a team, we engineer digital products that positively impact people’s lives.
Empowering Growth: We foster a culture of continuous learning and professional development. Our dedication is to provide timely and comprehensive assistance for every consultant through our dedicated Learning & Development team, ensuring their continuous growth and success.
DE&I Matters: At GlobalLogic, we deeply value and embrace diversity. We are dedicated to providing equal opportunities for all individuals, fostering an inclusive and empowering work environment.
Career Development: Our corporate culture places a strong emphasis on career development, offering abundant opportunities for growth. Regular interactions with our teams ensure their engagement, motivation, and recognition. We empower our team members to pursue their career goals with confidence and enthusiasm.
Comprehensive Benefits: In addition to equitable compensation, we provide a comprehensive benefits package that prioritizes the overall well-being of our consultants. We genuinely care about their health and strive to create a positive work environment.
Flexible Opportunities: At GlobalLogic, we prioritize work-life balance by offering flexible opportunities tailored to your lifestyle. Explore relocation and rotation options for diverse cultural and professional experiences in different countries with our company.