Organisation : Office for national statistics (ONS)
Outcome : Provide Data Science skills to support the Census Processing delivery goals using the ONS Data Access Platform (DAP)
Background : A team is being formed within the ONS Digital, Services and Technology (DST) Directorate to support our Census colleagues to :
1. Manage migrations to DAP (CDME / SRE etc.)
2. Manage Census Processing Specific Deliverables (Canceis, SAS, integrations)
3. Support Census Processing team deliver an automated processing system
Service Deliverables :
Provide subject matter expertise on data processing technology and application of it (i.e. Cloudera Spark / Hive / HDFS etc.) within the office.
Provide a frontline problem solving capability for issues that front end users are experiencing
Provide a guidance and advice service to teams; for example spend a day / week with a team to support them.
Promote and enable good practice within the Census Processing system
Provide an on demand data science service for Census to ensure key deliverables are expedited
Work with Software engineers and other DST team members to support the end to end processing solution for the 2021 Census
Expert in Data Science techniques using R and / or Python.
Expert in utilising Spark for distributed computing tasks.
Familiarity with Cloudera toolset desirable (HUE, Hive, Impala, Data Science Workbench, HDFS, Avro, Parquet).