We are looking for a Data Engineer to join our team.
About the Role:
The Data Engineer work on the next generation of analytics and AI product, where they will spend most of their time working on ETL’s, Data Ingestion, Apache Spark, Scala and mentoring other members of the team to design and implement our Analytics and AI/ML powered features.
The products of our client are the global standard for process intelligence and automation at more than 10,000 organizations across 90 countries. You’ll help customers accelerate their digital transformation, bringing them a quicker, easier, better way to work.
- Hands-on expertise with Apache Spark (Spark structured streaming), Scala and Spark SQL
- Experience in data munging, manipulation, cleansing and blending from multiple data sources and exposure to various data types and storage paradigms
- Recent experience as a Data Engineer
- Experience with Azure technologies
- MS Databricks experience it highly advantageous
- Python experience will be a bonus
- Familiarity with the Delta Lake House paradigm is preferable
- Familiarity with streaming data sources is preferable
Your contribution will be:
- Design, code, and unit test data engineering components and features that can be released / deployed through automation
- Ensure data completeness and correctness through cross-referencing to original data source
- Design and develop ETL’s in line with best practices and techniques for data transformation and storage
- Deliver data engineering functions that ingest, sanitize, and join data from multiple sources
- Create data models that meets the needs of management stakeholders, data scientists, and analysts as required
- Analyze and evaluate data product/output to identify and recommend optimizations as required in its extraction, cleansing, and organization
- Identify and pursue opportunities to create or improve organizational data standards
- Take ownership of assigned work and liaise proactively and collaboratively with stakeholders to ensure quality features are produced on time
- Participate in technical and team interviews as directed and provide clear input into the decision-making process
- Proactively communicate your work and progress to your manager, extended technical team, and stakeholders during the development process, and immediately escalate any product quality or delivery issues
- Contribute to the support and maintenance of data engineering functions, including liaising with the support team to resolve issues that have been escalated
- Active part of the incident management process, including on-call rotation and working with the IM team to resolve incidents
- Work with data providers to document the incoming data stream’s domain boundaries, field datatypes, and term definitions in sufficient detail to enable transformation of the data for end-user consumption
- Design and implement data engineering solutions that adhere to guidelines in support of security, disaster recovery, scalability, availability, reliability, and durability.
- Professional and career growth promotion;
- Competitive salary;
- Paid vacations and sick leaves;
- Medical Insurance (partial compensation);
- Flexible working hours;
- Regular corporate social activities;
- Regular technical training at our office;
- English courses;
- Gym, etc.