We are seeking a mid-level Data Engineer with at least 5 years of practical experience with large and complex data sets to implement Big Data technologies within our platform to enable innovation through data exploration.
You will be expected to drive a positive and supportive culture towards the Client Engagement team ensuring clients are satisfied with their service delivery.
Location: South Africa (Remote / Hybrid)
About the role
The Data Engineer will report into the Chief Analytics Officer and work closely with the Development, Marketing, and Sales teams.
As Data Engineer you will be responsible for monitoring, maintaining and developing data driven processes in the environment to support our business objectives.
- Monitor data processes to ensure continuity.
- Ensure highest data quality assurance, data accuracy and data completeness through regular and in-depth review and testing of work.
- Conduct data design, database architecture, metadata and repository creation activities and tasks as required by business stakeholders.
- Design and maintain ETL pipelines.
- Provide data management services to the Client Engagement team
- Provide input to data quality deliverables, conduct root cause analysis into anomoloes and provide feedback to data governance facilitator.
- Identify and recommend data cleansing opportunities, new metrics, attributes, datasets to support business intelligence requirements.
- Drive a data-driven approach to decisions.
- Drive on a right-data at the right-time principle to intelligence.
- Provide feedback to wider team to collaborate on digital strategy of data assets.
- Gather business needs and clearly documenting expected outcomes.
Required knowledge and experience:
- Matric / Grade 12 essential.
- Technical degree or diploma (in related industry) advantageous.
- Formal business/technical analysis training advantageous.
- Relevant data certifications advantageous.
- 5+ years’ practical experience working with large and complex data sets as a SQL Developer or Data Engineer.
- A sound understanding of data management principles including data quality assurance and data governance.
- Experience with data pipelines and workflow management.
- Good knowledge of data warehousing principles.
- Practical knowledge of working within Azure cloud services. Experience with other cloud platforms is a bonus.
- Experience in business intelligence.
- Understanding of general business terminology.
- Experience engaging with stakeholders at all levels of business.
- Solid SQL (ANSI – preferably T-SQL) experience including the ability to work with complex queries.
- Solid Knowledge of dimensional modelling (Kimble)
- Solid knowledge of RDBMS architecture.
- Solid knowledge of OLTP and OLAP concepts.
- Good knowledge of data integration concepts (ETL / ELT)
- Good knowledge of database management especially pertaining to permissions, schemas and performance tuning and quality assurance.
- Good experience working with Azure Data Factory or SQL Server Integration Services.
- Experience with Python/Scala (preferred)
- Experience with Databricks (preferred)
- General Windows operating literacy.
Required personal attributes
- Detail orientated.
- Strong analytical and problem-solving skills.
- Clear and concise communication skills.
- Able to interpret and simplify information.
- Strong ability to work to deadlines, across functions and juggle multiple projects
- Ability to work independently, but also closely with team members when the need arises.
Contact: Riva van Niekerk ([email protected])