Hero Image

Data Engineer

About the role

SamKnows is the global leader in internet performance measurement. We’ve been collecting data since 2008 on all manner of metrics, from how good Netflix performance is to figuring out why Fortnite isn’t working for gamers.

We're growing fast and, particularly over the past couple of years, have already undergone a period of exponential growth. This is especially true when it comes to the sheer volume of internet performance data we’re generating around the world. In anticipation of this sudden data increase, our engineers rebuilt our cloud-based data platform from scratch, now called SamKnows One. SamKnows One confidently handles and visualises the data gathered by over 30 million measurement agents every day, with this number continuing to rise. We're looking for a full-time Data Engineer to come and join our team in our London office, to help us make the most of all our data.

Responsibilities:

  • Work closely with engineers and key business stakeholders to ensure we are getting the best out of our data.
  • Monitor and ensure data integrity across our ‘big data’ platform.
  • Identify and implement cluster tuning to improve big data cluster performance.
  • Investigate and provide innovative solutions for our data ingestion pipelines.
  • Forecast and plan for data scaling to perform in line with business growth.
  • Maintain and integrate new data structures for our wide range of measurements.
  • Ensure the continued accuracy of reports and analytics, that we provide to our clients, on a daily basis.

Technologies:

  • Hadoop, Hive and Presto - Our big data platform
  • Java - Data collection infrastructure
  • PHP - Application API layer and current analytics layer
  • Mysql - Business metadata stores
  • RabbitMQ - Messaging platform

Requirements:

  • Improving the internet excites you.
  • You are a proactive, problem solver.
  • You have a strong background in Data Engineering and experience of working with very large data sets.
  • You’re keen to learn more about new technologies.
  • You have experience managing Hadoop and Hive plus ideally some experience with Presto, PHP and Java.

Benefits

We offer the following benefits, as well as awesome colleagues, engaging work, and the opportunity to help us make the internet better for everyone!

Holiday

25 days holiday each year plus bank holidays. You can sell back any unused holiday, but we strongly encourage you to take them.

Great office location

Whether you’re working at our London office, nestled just behind the Tate Modern and a stone’s throw from Borough Market, or in Washington at Tyson’s Corner, there’s plenty of culture, eateries, drinking holes, and interesting things to do.

Flexible working

Whether your child's ill, you're waiting for a package, or you have some deep-focused work to do, we all work from home from time to time.

Maternity and paternity leave

Paid maternity, paternity, and adoption leave, which you’re eligible for if you join SamKnows more than 13 weeks before your expected date of childbirth or adoption.

Pension

SamKnows have a private pension scheme and will contribute 5% to your pension if you contribute a minimum of 4%.

Private health & dental care

Private health insurance with AXA PP Healthcare, includes dentist and optician cover. Paid sick leave (for both your physical and mental health).

Hardware of your choice

Choose your own equipment and we’ll do our best to get them for you before you start.

Learning and training

We’ll contribute towards books, training courses, and conferences that help you learn and grow in your career here.

Interest free season ticket loan

Not for your favourite football team – for you to get into work for less! Pay back the loan in your monthly pay check before tax.