The data engineering team at Confused.com is committed to providing high-quality data/reporting services to our customers.
We’re looking for a Senior Data Engineer, a role that has evolved from the role traditional Senior SQL Developer.
Ideally, we’re looking for an experienced cloud engineer, but we would consider taking on someone who is an experienced SQL developer with the capability and drive to fill this role.
With the rise of the cloud, data has become more important than ever, and this role provides an opportunity for the right person to shine as a data expert.
Working closely with colleagues from across the business, you’ll get the chance to expand your knowledge of different data technologies and explore new and exciting capabilities—including big data and artificial intelligence.
You’ll be working with a team of data experts tasked with fully automating a new cloud data platform as well as maintaining our existing solution.
Within Confused.com, there’s a growing focus on data and data-driven decision-making. As such, we’ll rely on you to generate well-articulated data-oriented test cases and determine the positive, negative and boundary cases.
We work in a fast-paced, agile, and continuous delivery release-driven environment. The right candidate for this position will be motivated by this challenge and will help to develop our future ways of working.
- SQL Server 2016 and above, Azure DevOps, Python, Databricks, C#, PowerShell, Azure Resource Manager (ARM), Tableau
- Ability to work closely with our cross-functional product teams to identify and deliver projects that will drive value to customers of Confused.com.
- Experience of working in an agile environment.
- Experience of building robust solutions for optimal extraction, transformation, and loading of data from a wide variety of data sources, including third-party providers, API’s and our existing Azure platform.
- Identify, design, and implement internal process improvements: automating manual processes, optimising data delivery, re-designing infrastructure for greater scalability, etc.
- Foster a self-service data culture; working closely with other data engineers, architects, and technologists to share knowledge, evaluate opportunities and proactively recommend improvements in architecture, pipelines, data models, tools and processes.
- Translate data into valuable insights that inform decisions. Involves teams in analytics and synthesis to increase consensus and challenge assumptions.
- Must be passionate about DevOps and have a resilient attitude in pushing the DevOps perspective within a Kanban team.
- Maintaining and creating continuous delivery pipelines in Azure DevOps.
- Positive outlook and desire to learn and help drive and instil technical excellence within the Confused.com IT department.
- Deliver projects to time, budget and quality in order to deliver business goals.
- Experience of tools such as VSTS (Azure DevOps).
- Designs, writes and iterates code from prototype to production-ready. Understands security, accessibility and version control.
- Knowledge of data warehousing concepts, including data warehouse technical architectures, infrastructure components, ETL/ELT and reporting/analytic tools.
- Strong SQL skills and familiarity with data modelling and relational databases.
- A passion for data, cloud-based services and customer experience.
- Knowledge and experience of Kimball design.
- Ensures risks associated with deployment are adequately understood and documented.
- Keeping current with new capabilities of and improvements in the services provided by Microsoft.
- Experience with code versioning systems (e.g. git) and CI/CD tools such as MS Build.
- Hands-on experience in cloud based services, most notably Database, Compute and Storage
- Develops fit for purpose, resilient, scalable and future-proof data services to meet user needs. Has a demonstrable understanding of how to expose data from systems (for example, through APIs), link data from multiple systems and deliver streaming services.
- Experience of cloud-based testing.
- Databricks experience
- Python programming experience
We’re currently working as a hybrid model of home and office-based working. But you will be expected to attend the Cardiff office periodically to take part in meetings, social events, hackathons, user/working groups...all depending on the needs of the team...and in line with current public health advice re Covid.
Some of our benefits include:
• Discretionary cash bonus based on personal and business performance
• 28 days holiday
• Group pension scheme
• Flexible working patterns
• Employee Assistance Programme (EAP)
• New starter buddy scheme
• SimpleHealth Scheme
If you would like to know more please get in touch.