What Is a Machine Learning Engineer?

What Does a Machine Learning Engineer Do?

Machine learning algorithms are developed by data science teams. Those algorithms might do things like recommend you products on Amazon or movies on Netflix, or predict the end of a sentence in your email. But for users to actually receive these product benefits, there is a lot that happens in between. Consider things like:

• Where does the algorithm run?
• How can it run for 10,000 users at once?
• How does it interact with existing databases or filesystems to store data?
• How can it do all of these things securely?
• How will we keep track of whether a model needs to be updated?

These are the kind of questions that an ML engineer needs to answer, and consider when they design and implement the system. At the end of the day, users can get value from machine learning models only because ML engineers create a way to secure, serve, and monitor these models.

A machine learning system can consist of many different parts. And the required components of such a system can vary wildly depending on the use-case. Over time, we might see some of these become handled by specialised roles which emerge (such as monitoring experts). Below are some of the things that ML engineers might work on.

• Designing whole systems from end to end
• Developing solutions to meet SLAs (Service Level Agreements, e.g. less than 1-second latency for 99.9% of prediction requests)
• Building APIs and other ways to connect different parts of a system
• Containerising APIs and ML models
• Optimising model training efficiency
• Scaling up large numbers of containerised applications 
• Running clusters for services like Spark or Kubernetes
• Automatically triggering pipelines for things like continuous model retraining or model rollout
• Monitor model performance (model drift, data drift, concept drift) and system traffic (number of requests, types of requests, errors)
• Write high-quality code that will scale
• Interact with stakeholders including management, data engineers, and DevOps engineers
• Automating software updates using CI/CD
• Gracefully handling errors in production, and preventing them through testing
• Learn and adopt new tools that are being released to help with the ML workflow

Overall, the responsibility is about creating software systems that get machine learning models into real users’ hands.

An Example of Machine Learning Engineering

The prime example of machine learning engineering that you probably interact with every day is social media recommendation feeds. The first step is to build a model that predict what you are most likely to interact with given your engagement history.

Now imagine that you are Instagram and you have built this model. You need infrastructure that can serve this model millions of times a second to the hundreds of millions of users that come on Instagram every day.

On top of that, users' interests change over time so the models need to be continually updated with this new data to keep making good recommendations.

Only ML engineers know how to build the infrastructure to support this capability.

When Is a Machine Learning Engineer Needed?

In the early stages of an Ai & data team, a machine learning engineer might not be necessary.

The models you are building are simple and at a small enough scale that a software engineer can deploy them using standard software engineering tools and methods.

As the function grows, with hundreds of models deployed, millions of data points recorded and millions of users every day. The tools needed to continuously train models on new data and make that accessible at scale are not in a software engineer's toolbox so you need a specialist like a machine learning engineer.

Skills and Tools Used by Machine Learning Engineers

ML engineering requires being able to write not just code that works, but code that scales efficiently and is fault-tolerant. It also requires the ability to remain flexible and learn quickly, as there are so many new tools being released that are enabling ML engineering teams to move even faster.

Here are some of the key tools and skills which we've seen most widely used by ML engineers in industry:

• Kubernetes (K8s)
• Kubeflow (Core, Pipelines, Serving)
• Docker
• Git & GitHub
• Python
• Apache Spark
• Apache Airflow
  
• SQL
• Cloud computing
• Cloud lake and warehouse storage
• Command line

How Much Does a Machine Learning Engineer Make?

Junior machine learning engineers in the UK can expect a salary of £35,000 and above. With a few years of experience, mid level machine learning engineer salaries are around £65,000. If you stay in the industry, keep building on your skills and gain experience managing teams, senior positions pay upwards of £89,000.

What Is the Demand for Machine Learning Engineers?

The demand for machine learning engineers is grown steadily. The number of open machine learning engineer roles grew by 51% between 2019 and 2021.

Given that this is a relatively new role, most of the demand is for senior engineers who can design systems and build out teams. However, as the field matures, more junior roles are expected to open up.

Machine Learning Engineer vs. Data Scientist vs Software Engineer - What’s the Difference?

There is certainly a lot of confusion here. The field is still maturing so companies often mistake the role of data scientists and ML engineers. Imagine how hard it is for someone working in HR to define the job spec and vet candidates without having much of an idea of what Ai really looks like in the real world. To give a distinction between the two roles: data scientists analyse data, run statistical tests, and build proof of concept models, and ML engineers enable those models to get into the hands of users at scale.

You are probably thinking that ML engineering sounds a lot like software engineering, but just with a focus on ML… and that’s exactly right. The roles both involve building software systems whilst considering the efficiency of the implementation and the experience for the end user. What makes the distinction is just how different the toolkit can be when you introduce machine learning. ML is fundamentally different to traditional software in the sense that it most often requires you to solve a problem for which the solution can change over time. For example, a fashion recommendation system that works today won’t work one year from now, because style preferences change. This need for continuous training is just one aspect that brings a ton of complexity to software systems using machine learning. As such there are many specialised tools being released which only apply to systems that incorporate AI.

How To Become A Machine Learning Engineer

The most important skill for ML engineers is to have a strong foundation in software engineering, understanding how to write production-grade code and deploy using cloud technologies.

There is no single path to becoming a machine learning engineer. However, the fastest way we know to launch your career in ML engineering is through the AiCore programme.

The 18 week programme delivers the most industry-informed, hands-on education in data science. You will learn from established experts how production-grade machine learning infrastructure work then get experience by building systems that are currently deployed at companies such as Uber and Pinterest.

Launch Your Machine Learning Engineering Career Today With AiCore!

Are you considering becoming a machine learning engineer, trying to figure out if it's right for you? Maybe you are certain you want to become a machine learning engineer and are trying to find the fastest route into this lucrative career. In either case, the AiCore team would be delighted to help you. Book a 15 minute call here.