SQL Data Engineer Purpose and Background Are you a capable sql data engineer who is passionate about the power of data to solve environmental issues?
Our client is looking for an sql data engineer to shape delivery by collaborating with data architects and modellers to contribute to the acquisition of corporate data requirements, documenting them according to the required standards utilising the prescribed methods and Azure data tools, in order to build our centralised data platform.
This is a permanent role, with responsibility for developing, constructing, testing and maintaining architectures such as data pipelines and large-scale data processing warehouses.
They will leverage industry best practice while delivering changes, such as agile backlogs, code repositories, automated builds, testing and releases.
They will be responsible for ensuring data scientists can pull relevant data sets for their analyses, and implement data pipelines to connect operational systems, data for analytics and BI systems.
They will re-engineer manual data flows to enable automation, scaling and repeatable use and develop data set processes for data modelling, mining and production.
The post holder will work closely with data architects (to determine what data management systems are appropriate) and data scientists (to determine which data are needed for analysis).
They will provide clean, usable data to the business through the data platform in accordance with governance, and analyse, design, plan, execute and evaluate data requirements to support business activities and projects.
The post holder will be central in ensuring the delivery of world-class digital products and changing the delivery culture in the company.
The Data Engineering Team’s primary remit is to improve the usability of the climate change, water, forests and cities data disclosed through robust and transparent methods.
The team will create sustainable data pipelines for data quality monitoring, data cleaning, reporting and data science modelling.
Harmonised data collected from external sources will enrich the data assets’ value and enhance accessibility for stakeholders.
The team will produce value-adding insight delivering it through data products that help internal and external stakeholders to better understand the quantitative and qualitative results of their actions.
This in turn helps stakeholders to make data-led decisions and optimise for all constraints. Through every stage the data assets are governed by the industry practice standards in a robust and transparent manner.
Key responsibilities include : Managing the investigation of corporate data requirements, documenting them according to the required standards utilising the prescribed methods and tools Implementing data flows to connect operational systems, data for analytics and BI systems.
Re-engineer manual data flows to enable scaling and repeatable use Working closely with data architects (to determine what data management systems are appropriate) and data scientists (to determine what data is needed for analysis).
Tackling problems associated with database integration and unstructured data sets Ensuring that those using the data structures and associated components have a good understanding and that any queries are dealt with promptly and efficiently In liaison with the information management or IT management functions, contributing to the development and maintenance of corporate data standards.
Ability to create efficient DW or DL structures to minimise cost of orchestration / processing and ingestion of data Required skills and experience : Strong technical process understanding regardless of technology Wide range of strong technical skills (i.
e. Azure Devops, Azure Data Factory, Data Bricks, SQL, python) Core SQL Competencies SSMS, SSIS, T-SQL, Stored Procedures ADF Pipelines to build and populate SQL databases Background in migrating traditional MS products to Azure Very high attention to detail Strong communication skills Efficient in building ETL and ELT processes for enterprise solutions Strong software delivery methods and knowledge Strong performance-tuning skills Desired skills and experience Digital delivery has a track record of working on DevOps delivery Exposure in Climate Change data legislation, practices and stakeholders Experience in Environmental related industries ie Water, Energy, Forestry related Presentation skills Understanding of architecting solutions taking into account wider considerations Structured problem-solving techniques This is a permanent full-time role, reporting to the Head of Data Engineering, which will be delivered remotely to start with.
The post holder will be required to travel to the London office from time to time when it reopens. Salary and benefits : Between £45,000 - £48,500 per annum dependent on experience, 30 days holiday excluding bank holidays, flexible working opportunities and others benefits.
Interested applicants must be eligible to work legally in the United Kingdom.