Machine Learning System Design Interview

Product Information

Valerii: Did you use the data scientist profile, because I told you that I don't like “data scientist” in my job title? I find it awful and terrible. So you’re just nudging me in my pain point. ( 58:47) We’ve trained our model with the hyperparameters that led to the best evaluation metrics in our holdout data. Should we just launch this to the entire user base? Unless you’re fresh out of school, you should know the answer is NO!

Given a text and knowledge base, find all the entity mentions in the text (Recognize) and then link them to the corresponding correct entry in the knowledge base (Disambiguate).” Alexey: Basically, when you interview they do this automatically and probably at this round, they use it to assess which level to put you. ( 55:30) Alexey: [laughs] I might be wrong with using these words. I think the recruiter probably used different words. But the reason for me failing the process – the whole interview – was machine learning system design. Not the others. I was afraid about the others. But in the others, I did well, but I failed that one. And the reason there was because the interviewer expected me to talk about actual machine learning. Instead, we talked about metrics, heuristics, and then I didn't have enough time to actually cover machine learning. Yeah, so what do you think about this? Is this typical for the process? Is it expected? ( 28:28)Valerii: Yeah, yeah. Or because there is feature shift distribution. So we need to detect this – the feature shift distribution, target distribution, model performance. And we need to have a plan B to switch that. But I need to take a look into these documents before I can tell you. [laughs] Smart people were doing that for quite some time. It's not like I can pull it from my head immediately. But there are many things which might shoot you in the leg. ( 47:01) Alexey: Perhaps if you cover all these parts during your system design interview, you're already in quite a good position. Right? ( 46:02) business requirements change, and (3) data distributions constantly shift. Without an intentional design Hammer out timing SLAs (eg. we’ll incorporate user actions into recommendations within X seconds/minutes/hours) Interviewers will generally ask you to design a machine learning system for a particular task. This question is usually broad. The first thing you need to do is ask questions to narrow down the scope of the problem and ensure your system’s requirements. You should also ask questions about performance and capacity considerations of the system.

The process of developing, evaluating, deploying, and updating models for your team has been mostly manual, slow, and error-prone. You want to automate and improve this process. Looking to start a career in data science and AI and do not know how. I offer data science mentoring sessions and long-term career mentoring: Valerii: Well, I was able, to some extent. I managed this. Because look, I mean, come on. Batch norms – there is some normalization. So? Okay. ( 33:19) Preparing for ML system design interviews Alexey: So if you're a fresh graduate and you're interviewing for a Junior position, you will not have this. But if you apply for a regular, let's say, machine learning engineer role (doesn't even have to be Senior) you will have this and then they will decide what kind of level to put you in. ( 57:01) Students will learn about data management, data engineering, feature engineering, approaches to model

Table of Contents

There’s a lot of room for creativity here and you might be able to think of some options that are very application specific. Note that you can use more than one source! Infrastructure I’m a SWE, ML with 10 years of experience ( Linkedin profile). I had offers from Google, LinkedIn, Coupang, Snap and StichFix. Read my blog. The text says, “Michael Jordan is a machine learning professor at UC Berkeley.” First, NER detects and classifies the named entities Michael Jordan and UC Berkeley as person and organization. Next, disambiguation takes place. Assume that there are two ‘Michael Jordan’ entities in the given knowledge base, the UC Berkeley professor and the athlete. Michael Jordan in the text is linked to UC Berkeley professor entity in the knowledge base. Similarly, UC Berkeley in the text is linked to the University of California entity in the knowledge base. You’re worried that there might be biases in your ML systems and you want to make your systems responsible! Once the model is launched, what other ops work will there be? How can we monitor the model to make sure it’s healthy and what operations will we have to do to keep the model performing well. What happens if we want to update our features? Leveling