Call —
Transcript
0:00 Caller: Please, this, Jen?
0:00 You: Hello? Yes, Erin. I'm well and yourself?
0:04 Caller: Yeah, hey, how are you?
0:07 Caller: I'm doing well myself, thanks.
0:10 Caller: It's now still a good time to chat a little bit about that Lola role.
0:12 You: Yes. Yes.
0:15 Caller: Okay, excellent.
0:17 You: So the biggest one is how apparently LLMs are being used as a filter or a fine-tune on what would otherwise be a traditional inventory.
0:18 Caller: So I just kind of want to run through a little bit more of your background,
0:21 Caller: what you may be looking for in your next role,
0:24 Caller: and then I can go a bit deeper into this role with Lola,
0:27 Caller: and then I may have some others that could possibly interest you.
0:29 Caller: could possibly interest you as well.
0:31 Caller: But first off, I just kind of wanted to ask you
0:33 Caller: what caught your eye about this role since I was the one
0:35 Caller: that reached out to you.
0:37 Caller: Okay.
0:47 You: So it's definitely a step removed from building another chatbot.
0:55 Caller: Okay.
0:59 Caller: Yeah, so I just kind of run through where you're at currently, I see that you've been with Rowe for a few years now.
1:10 Caller: Could you kind of explain to me a bit more of what you're doing?
1:13 Caller: It looks like you kind of transition to add engineer into more of the research engineer side.
1:17 You: I think part of that transition was predominantly motivated by tax efficiency, but that's speculation on my part.
1:27 You: The title shifted. My responsibilities functionally didn't. So I run about 250-odd production services, some customer-facing, most internally facing, primarily on Kubernetes. I was originally brought on to help the underwriting team. At the time, they were doing things very manually, copying,
1:29 Caller: I don't know.
1:31 Caller: Thank you.
1:33 Caller: Thank you.
1:35 Caller: Thank you.
1:37 Caller: Thank you.
1:39 Caller: Thank you.
1:41 Caller: Thank you.
1:43 Caller: Thank you.
1:45 Caller: Thank you.
1:46 Caller: Thank you
1:47 You: pasting back and forth across Excel spreadsheets. And so I did away with that, increasing underwriting capacity 400X with all of the automata that I put in place.
1:48 Caller: Thank you.
1:52 Caller: Mm.
1:56 Caller: Okay.
1:57 Caller: Mm.
1:59 You: And after achieving that, leadership asks, what else can you do? And now, any gaps that exist in each leader's space that can benefit from engineering attention, but is too small to dedicate a full engineering squad, comes to me.
2:02 Caller: Mm-hmm.
2:04 Caller: Mm-hmm.
2:05 Caller: Okay.
2:17 You: And that's pretty much, and now I'm at the point where every leader consumes something from me directly at Rowe.
2:18 Caller: Mm-hmm.
2:21 Caller: Okay.
2:23 Caller: Okay.
2:24 Caller: Excellent.
2:25 Caller: Excellent.
2:25 You: With whom?
2:26 Caller: So how heavily are you working like hands-on with
2:32 Caller: with LLMs there.
2:35 Caller: with LLMs with large language models. Okay.
2:35 You: Every day. So my consumption's on the order of 5 billion tokens a week, blended.
2:42 Caller: Okay.
2:44 You: And then 20 to 20.
2:47 You: 25 billion tokens a month. And my velocity, coding velocity, is at around two full squads.
2:55 You: Probably 15 to 20 people I talk to within a week regularly. And then beyond 50 to 20 people I talk to within a week regularly. And then beyond that, maybe 50 to 70. And then beyond that, maybe 50 to 70. And then,
2:57 Caller: Okay.
2:58 Caller: Excellent.
3:00 Caller: So how many people are you working directly with at Rowe?
3:07 Caller: Okay.
3:17 You: that I talk to at least once a month.
3:20 Caller: Excellent.
3:21 Caller: And then how many people are on your direct team that you work with?
3:24 You: It's just me, myself, and I, and then the dozens upon dozens of agents that perform the grunt work on my behalf, or I guess, for whom I am responsible for their output.
3:26 Caller: Okay.
3:28 Caller: Okay.
3:29 Caller: Okay.
3:34 Caller: Okay.
3:36 Caller: Okay, excellent.
3:40 Caller: So have you otherwise been looking to leave row or is it more so that you were just interested in learning
3:45 Caller: more about this position with Lola?
3:47 You: Well, as a rule, if someone sends me an in-mail, one, I know how much they cost, and I also know the gamble in spending the in-mail.
3:48 Caller: Okay.
3:54 You: So at least I'd take the call to understand why it was worth taking that gamble.
4:06 Caller: bit more about your time at Meta and your time at Spotify as well.
4:10 Caller: It looks like you were data engineer and data scientists at each role, respectively.
4:15 Caller: Could you kind of explain to me a bit more of what you were doing at Meta?
4:17 You: So meta, I was on Instagram's notification systems team.
4:24 You: So within Instagram, if you go to the notifications page, every click, impression, long click, every single event is captured by the telemetry infrastructure that I set up.
4:36 Caller: Mm-hmm.
4:36 You: So that's on the order of 1.7 million events per day.
4:39 You: And I think 1.5 petabytes of raw telemetry data that gets ingested, aggregated, and processed, and processed and transformed.
4:47 You: for consumption by the likes of data scientists, machine learning engineers, product managers, product growth analysts, and so on.
4:57 You: At Spotify, I was brought on to handle something called ATL marketing measurement.
5:02 You: So ATL is above the line, things like TV commercials, billboards, stuff you can't really measure as precisely as you can on Facebook's ad platform.
5:06 Caller: Okay.
5:11 You: And I introduced something called Bayesian structural time series. It's a mathematically defensible way to be able.
5:17 You: to measure what if. Packaged that up as something that was more accessible to the more junior
5:19 Caller: Mm-hmm.
5:21 Caller: So what were your reasons for leaving?
5:22 You: data scientists and released it to data science teams at Spotify worldwide.
5:27 You: So Spotify, worldwide. So Spotify, it was the middle of COVID. And at the time, they were
5:35 Caller: What are your reasons for leaving both Meta and Spotify?
5:39 Caller: Okay.
5:40 You: offering a material payout for those who raised their hands. So I raised my hand. So I raised my hand.
5:45 You: And then meta, this is around the time Zuck was going on about the Metaverse and how he was willing to burn $10 billion a year to make the Metaverse a thing.
5:57 You: Wall Street punished Meta stock for it.
6:00 You: And so stock was down 60% in trailing 12 months.
6:04 You: Rumors of layoffs were swirling, so I figured jump before I have to compete against my soon-to-be former Metamates.
6:05 Caller: Okay.
6:11 You: And I landed at a good spot.
6:15 You: The range you listed works for me.
6:18 Caller: Excellent.
6:19 Caller: So I just kind of want to run through as well what you'd possibly be looking for in your next role if you
6:24 Caller: were to move on from Rowe.
6:26 Caller: As far as compensation goes, what would you be targeting salary-wise in your next position?
6:32 You: The range you listed works for me.
6:34 Caller: Okay.
6:35 Caller: Excellent.
6:36 Caller: Perfect.
6:37 Caller: So let me just see as well.
6:40 Caller: You are based in New York, correct?
6:43 Caller: Okay.
6:44 Caller: Yeah.
6:45 Caller: So with this, this would be more of a hybrid role on site.
6:45 You: Thank you.
6:48 Caller: I'm not certain exactly where in New York, the company is based right now.
6:53 Caller: I think that they're looking to more of like build out in office.
6:56 Caller: They do have a pretty big presence in the Boston area at this point and then are kind of looking to expand
7:02 Caller: into New York.
7:04 Caller: Let me just pull up some of the information that I have on the role, and we can kind of determine
7:09 Caller: from there if it would kind of make sense to move forward, submitting you over their way.
7:14 Caller: Let me just see what I have on this.
7:15 You: Thank you.
7:17 Caller: Perfect. Yeah, so with this role, they kind of gave us more so like a list of different companies
7:25 Caller: that they wanted to see people coming out of and also different titles.
7:29 Caller: So with you being a research engineer, that's kind of one of the titles that they gave us.
7:33 Caller: and then they like to see people coming out of certain teams at Meta, coming out of places like Spotify.
7:39 Caller: So with this, this is more of like a retrieval and personalization, machine learning engineer position.
7:45 You: Thank you.
7:46 Caller: For the role, it's looking to own the decision layer.
7:49 Caller: So the system that turns something like, I want a romantic boutique place,
7:53 Caller: they're good food in Lisbon, plus memory, plus inventory into the right, into the right three hotels.
7:59 Caller: So this job is to essentially bottle LLM taste.
8:02 Caller: and make it reliable, scalable, and personalized.
8:06 Caller: So the LLMC and gray scale and rank by taste better than any traditional ranker.
8:11 Caller: So the bottleneck is the context they receive.
8:15 Caller: The quality of the Hindaset Set and the quality of personalization signal would be kind of the feeling of recommendation quality.
8:15 You: Thank you.
8:22 Caller: So you'd essentially be owning the whole chain.
8:25 Caller: So you'd be owning hybrid retrieval.
8:28 Caller: You'd be working on context engineering.
8:31 Caller: on context engineering for the decision step. So what subjects of inventory, memory, and
8:37 Caller: constraints lands in front of the LLM, as well as personalization plumbing, so productionizing
8:43 Caller: the three-pass memory pipeline, and then also owning evaluation infrastructure,
8:45 You: Thank you.
8:49 Caller: as well as re-ranking and creation logic that takes the LLM output and ensures consistency across
8:56 Caller: users and sessions. With this, they're looking for a minimum of like about four
9:01 Caller: years of experience, working with LLMs, a background in life retrieval, recommendation systems,
9:07 Caller: or search, and then any kind of like being comfortable across embedding, hybrid search,
9:13 Caller: re-ranking and prompting as part of one stack. So I kind of wanted to see if you had like hands-on
9:15 You: I have to use I have to use the
9:19 Caller: experience doing those kind of things like at your current role as a senior research engineer,
9:25 Caller: like if you've used LLMs like that in the past.
9:30 You: I'll have to use broad strokes because this feature hasn't been released yet.
9:31 Caller: So.
9:32 Caller: Mm-hmm.
9:33 Caller: So,
9:34 Caller: uh,
9:34 You: But a quick example would be a
9:35 Caller: .
9:36 Caller: Okay.
9:37 You: a recommendation engine for accounting classification.
9:41 You: When you go, when you have an expense on the
9:43 Caller: Mm-hmm.
9:43 You: when you have an expense on the platform,
9:44 Caller: Mm-hmm.
9:45 You: as a fintech.
9:46 You: That expense usually needs to be passed on to an accounting program so that the relevant
9:52 You: financial statements can be generated.
9:55 You: And usually that expense needs to be tagged.
9:57 You: So the recommendation engine's job is to come up with the most probable tag that the user needs
10:05 You: to set on that transaction.
10:08 You: Now I own this pipeline, I built out the full prototype and set up the spec for the rest of the team
10:13 You: to execute and implement.
10:14 Caller: Mm-hmm.
10:14 You: It's under the hood. It's basically an unsupervised ML pipeline. It has internal feedback loop to confirm that what was recommended is what was accepted, and if not, to flag that for re-evaluation.
10:24 Caller: Mm-hmm.
10:34 You: The underlying matching mechanism is predominantly a short list using traditional search and similarity type lookups.
10:44 Caller: Okay.
10:44 You: with an LLM to attempt to understand the opinions and biases of that specific account or business.
10:56 You: And it's, I don't know if I can share the precise number, but it's accurate enough that they're ready to sunset its predecessor.
10:57 Caller: Okay.
10:58 Caller: Okay.
10:59 Caller: And this is currently at row that you're doing.
11:13 Caller: currently at row that you're doing this. Okay. Okay. Excellent. Excellent.
11:14 You: Correct.
11:14 You: So I have already built out the prototype and validated it to the product team.
11:19 You: Different engineering teams are taking that prototype following the scale recommendations that I have to process in excess of a million transactions a day.
11:30 You: And then very soon, I imagine, the marketing team will be announcing this.
11:43 Caller: Perfect. And then did you any, did you do any kind of like similar work when you were at Meta or Spotify or were those like traditional like data positions?
11:44 You: So Spotify was almost exclusively analytics, not necessarily search or personalization.
11:58 You: And then meta, it was feeding upstream of the search and personalization.
12:02 You: The most relevant there would be something called want rate.
12:06 You: So being able to understand what model or what mix of the notifications that are being sent to users increase
12:13 Caller: Okay. Okay. Excellent. Perfect. And just a couple other things, too, about Meta and Spotify.
12:14 You: is retention to the platform or return to the platform.
12:18 You: So I didn't own the full process, but I definitely owned the feedback loop or the part that returned the feedback to the models.
12:43 Caller: both companies are huge. But do you know around how many people you were working with on each team there at MedOn Spotify?
12:44 You: Easily dozens, probably more at Spotify than at Meta, only because most of the Instagram team was already in New York, or at least for this particular surface that I was working in.
12:59 You: And Spotify, I work with data science teams worldwide, making sure the ATL measurement tooling was easy enough for them to use and most importantly interpret.
13:13 Caller: Okay.
13:14 Caller: Okay, excellent.
13:14 You: So.
13:20 Caller: And just a couple other things just about what you could be looking for next.
13:25 Caller: Obviously, like you've kind of had a few different titles over your career.
13:27 You: The title is tricky because that implies a singular bucket.
13:28 Caller: So I just kind of wanted to ask you what other kind of titles or kind of companies would specifically interest you if something else comes up that you may be a good fit for.
13:40 You: In practice, I can fit multiple buckets comfortably.
13:43 Caller: Mm-hmm.
13:43 You: So I...
13:44 You: less on the title and more on the problem. My skills, knowledge, and experience, the unifying theme across all of them is automation, building tools that free, liberate people from monotonous tasks. And if a company can benefit from that kind of solution, or at least has those kinds of processes that are still predominantly manual, there's a strong chance I can have outsize impact there.
14:13 Caller: Okay.
14:14 You: I think with the advent of LLMs, I'm comfortably in the 1% in terms of LLM users by volume and defensively in the top 50 worldwide among the roughly half a million profiles I track on GitHub.
14:27 You: So the sophistication of problem that I'm able to automate away has grown substantially.
14:43 Caller: Otherwise, too, I just wanted to see if you were to hypothetically, like move forward here and like receive an offer, how much notice would you need before actually getting started in this or a new position elsewhere?
14:44 You: In practice, zero, in reality zero, the contractors at will, in practice, I imagine four to six weeks only because there's a lot to unwind.
14:59 Caller: Okay. Okay. Excellent. So I just kind of want to give you a quick overview of Lola itself before we wrap up.
15:12 Caller: They are still in stealth mode right now. There's not really much information out there online.
15:14 You: Thank you.
15:16 You: Thank you.
15:18 You: Thank you.
15:19 Caller: I think that I did just actually find my first article on them earlier today, but as far as I know, they still don't have an active LinkedIn page.
15:27 Caller: So they are an early stage AI-native startup. They were founded by the co-founders of Kayak. So Paul English is a pretty big name in the industry.
15:36 Caller: And the company itself is incubated within booking holdings. And this company is within the AI
15:42 Caller: travel and consumer application space. The company itself is operating with startup autonomy,
15:48 Caller: but enterprise level expectations as far as scale security and compliance go. And then there is
15:48 You: Thank you.
15:54 Caller: expected to be immediate real world usage and a high traffic environment. So again, with this,
16:01 Caller: part of it is kind of being able to work more at the startup level, but also having some of the benefits
16:05 Caller: of a larger company working at the enterprise level. And then I believe that I had mentioned in my message to you,
16:11 Caller: They do offer booking holdings are issues. And then there is the potential to actually
16:16 Caller: have startup equity through Lola itself on top of the pretty generous base. So it's something
16:18 You: Thank you.
16:23 Caller: that I'd be happy to share your information and your resume with them to see about getting
16:28 Caller: an initial conversation started if they think it's the right fit.
16:31 You: Uncle.
16:32 You: Uncle of my resume represents my agentic capabilities, mainly
16:33 Caller: Okay.
16:33 You: of my resume represents my agentic capabilities, mainly because they, it leads more toward business outcomes.
16:41 You: But my GitHub that's more towards business outcomes, but my GitHub that's linked within the resume may be more interesting to
16:41 Caller: Okay. I mean, I do see it looks like you have your GitHub on your resume, so they can kind of check it out from there. But I mean, I did highlight some of the projects that you said they've worked on. So I'll include a brief right up along with your resume for them.
16:47 You: who's evaluating LLM expertise.
17:08 Caller: Okay. Great. So hopefully I'll have some updates for you.
17:11 Caller: Within the next few days, Lola has been honestly very quick about getting back to us with
17:16 Caller: feedback and moving people forward. So hopefully I'll have something within the next few days.
17:17 You: Okay. I think the only other relevant information for you is I'm also in process with a number of other recruiters. These are all early stages.
17:21 Caller: And then if you need anything from me in the meantime, feel free to reach out.
17:33 You: So as things move along, I'll try to make sure everyone is roughly kept up to date on what the furthest process is.
17:41 Caller: Okay, okay. That's pretty helpful to know. What other kind of roles are you applying to at this point or interviewing with?
17:47 You: Oh, it's not applying to. It's recruiters reaching out to me, but they span FinTech, they span markets. So a handful of hedge funds, a couple of startups. A pharma tech company thinks I'd be a good fit for them.
17:54 Caller: Okay.
18:02 You: And what else? A couple of PE firms are also are also looking to see if they can centralize the AI, AI function across their portfolio companies.
18:11 Caller: Okay. Excellent. No, that's definitely helpful to know. So as I said, I'll submit you over their way for now.
18:17 You: processes and trying to see what would be the best fit.
18:29 Caller: Hopefully, some updates within the next few days.
18:33 Caller: Okay. Of course, have a good one, Jed. Bye-bye.