<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Lyft Engineering - Medium]]></title>
        <description><![CDATA[How we build, scale, and secure the systems that keep Lyft moving. - Medium]]></description>
        <link>https://eng.lyft.com?source=rss----25cd379abb8---4</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>Lyft Engineering - Medium</title>
            <link>https://eng.lyft.com?source=rss----25cd379abb8---4</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 02:26:40 GMT</lastBuildDate>
        <atom:link href="https://eng.lyft.com/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com" rel="hub"/>
        <item>
            <title><![CDATA[Beyond Test Cards: A Fake Payment Provider for Testing at Scale]]></title>
            <link>https://eng.lyft.com/beyond-test-cards-a-fake-payment-provider-for-testing-at-scale-c69dddadb115?source=rss----25cd379abb8---4</link>
            <guid isPermaLink="false">https://medium.com/p/c69dddadb115</guid>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[payments]]></category>
            <category><![CDATA[fintech]]></category>
            <category><![CDATA[load-testing]]></category>
            <category><![CDATA[infrastructure]]></category>
            <dc:creator><![CDATA[Jay Patel]]></dc:creator>
            <pubDate>Tue, 10 Nov 2026 20:55:40 GMT</pubDate>
            <atom:updated>2026-11-10T20:55:38.930Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EV9F9SVDWLCKZJ_hghiIRg.jpeg" /><figcaption>Image generated with Google Gemini, 2026</figcaption></figure><p><em>Written by </em><a href="https://www.linkedin.com/in/saveliy-kolesnikov/"><strong>Savelii Kolesnikov</strong></a><strong>, </strong><a href="https://www.linkedin.com/in/jay-p-73461749/"><strong>Jay Patel</strong></a>, and <a href="https://ua.linkedin.com/in/ivanna-sivak/uk"><strong>Ivanna Sivak</strong></a></p><h3>Background</h3><p>Lyft processes billions in payments annually, and that volume continues to grow. Agentic coding has drastically accelerated feature development, but it raises the stakes for the one phase that cannot be rushed: ensuring the highest standards of reliability and correctness in critical payment systems.</p><p>Every payment platform eventually hits the same wall. Unit tests cover business rules, integration tests validate boundaries, and load tests push upstream limits, but standard testing stops the moment a request reaches a third-party processor.</p><p>Processor test cards are built for simple flows: a card either passes every payment action, from authorization to capture to refund, or fails them all. Real payment flows often have one step fail while the others succeed.</p><p>In this post, we’ll explain why test cards break down and how we built our own fake payment provider to simulate any processor behavior on demand. Unlike a typical mock, it runs safely in production, so we can load test our payment stack and trial new workflows on simulated riders before real ones.</p><h3>Test cards don’t get you there with rideshare</h3><p>Unlike paying for a product on an e-commerce website or your favorite streaming platform, Lyft, as a rideshare company, has a unique set of challenges with payments. While we can put a hold on the card on a ride request, the final charge does not always match the original hold amount on the card. We can use test cards provided by processors; however, they either succeed with all charge actions (authorization, increment, capture, direct charge, refund, etc.) or fail them all. We hit a wall if we want to test the following scenarios:</p><ul><li>A card that succeeds auth, fails capture, and then succeeds on the direct charge fallback</li><li>A card that succeeds every charge flow but fails on refund</li><li>A card that succeeds auth and increment, fails capture at the incremented amount, and succeeds when we fall back to capturing the original auth amount</li><li>The processor itself is down</li><li>The processor is still processing the charge and returns a non-terminal (pending) response</li><li>The processor returns a response, but our services have timed out</li></ul><p>None of the above scenarios are exotic. These are the exact paths our retry and fallback code was written for, and they were hard to exercise before production.</p><h3>What the limitations of test cards cost us</h3><p>These limitations cost us across the company. Without realistic failure paths, validating our own retry and fallback logic was hard. Anything touching intermediate payment failures was difficult to validate. Our end-to-end acceptance tests could only cover all success or all failure, never the middle. And because test cards are shackled to processor sandboxes, our <a href="https://eng.lyft.com/simulatedrides-how-lyft-uses-load-testing-to-ensure-reliable-service-during-peak-events-644dcb654454">simulated rides load testing framework</a> had to bypass the payment flow entirely. We’ll come back to this.</p><p>Every one of those costs points at the same root cause: the thing standing between us and confidence lives behind a processor’s API, on a release cycle we don’t control. The first instinct is to ask the processors themselves to fix it — better sandboxes, more test cards, more scenarios. That doesn’t scale. We work with several processors, each with different sandbox coverage, and sandboxes aren’t built to model our retry and fallback logic. We’d be maintaining a compatibility matrix instead of a test suite, one column per processor.</p><p>Instead of continuing to work around the sandboxes provided to us, we decided to take full control of our testing environment by building our own solution — a Fake Payment Provider designed specifically to meet the complex needs of rideshare testing.</p><h3>Architecture</h3><h3>Why build our own Fake Payment Provider</h3><p>The constraint that had been making our lives hard turned out to be the thing that saved us. Working with several processors forces you to hide all of them behind one interface, and an interface is something we could implement ourselves.</p><p>They are different companies with different APIs and their own quirks. Strip all of that away and they do the same handful of things. Tokenize. Authorize. Increment. Capture. Charge. Refund. Void.</p><p>So adding a Fake Provider is easy to describe. Implement those methods. Route some users to it. Done.</p><p>And that really is most of the design. The router resolves a provider and calls it. Nothing above the provider layer knows that this one never opens a socket. Because every processor’s response is already translated into one internal representation, the fake doesn’t have to mimic any processor’s API. It only has to produce outcomes our platform already understands. That boundary cuts both ways. Because the fake sits behind the provider interface, it never exercises the code that talks to each processor: building requests, parsing responses, and mapping their errors into ours. Those paths still rely on processor sandboxes, and closing that gap with record and replay is next on our list.</p><p>Easy to describe is not the same as easy to build. Translating this concept into practice required addressing three key questions:</p><p>◾ Access Control: Who is allowed to reach the fake provider?</p><p>◾ State Persistence: Where does the fake keep its scenario behavior across calls?</p><p>◾ Scenario Modeling: How is that test behavior actually defined and configured?</p><h3>Access Control: Only simulated users get to touch it</h3><p>Everything to do with the fake payment provider stays inside a fenced-off group of test users. In our system, they are called simulatedusers. The fake Low Value Token (LVT) is generated by us and accepted only for simulated test accounts.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*j29EwLgEIgKPP5a5JHaE-A.png" /><figcaption>Routing rules: the fake provider is only ever a candidate for simulated users</figcaption></figure><p>Notice what is not in that picture. The router has no test mode and no special case. It has rules, and for a real rider the fake never becomes a candidate in the first place. Multiple independent checks across our services ensure fake payment methods can only ever be used by the internal simulated test accounts.</p><h3>State Persistence: The scenario rides inside the token</h3><p>A processor that is not real still has to remember how it intends to behave. Which scenario are we reproducing right now? What should the next action do? What comes back for each one?</p><p>That is state, and state has to live somewhere. There were three places to put it.</p><p>◾ A new service. More infrastructure than the milestone justified. Worse than that, it would have been a data path production does not have. We would have been testing a system that does not exist.</p><p>◾ The request. A magic amount. A header. A field called test_behavior. This is the obvious one, and it breaks on the second call. Captures and refunds are started by other services. Retries arrive hours later and read their state from the ledger. Nobody is carrying your header by then. Making it work would mean every service on the path forwarding a test-only field, which is exactly the parallel test-only path we were trying not to build.</p><p>◾ The token. We already had something that travels the whole path and gets persisted on the way. So the scenario went into the token.</p><blockquote>A short detour, because the rest of this section leans on it. Payment systems try hard never to hold a card number, so every card gets swapped for a <a href="https://en.wikipedia.org/wiki/Tokenization_(data_security)">token</a>, a stand-in string. Think of a hotel. Your booking reference can’t open a single door, but the front desk swaps it for a key card that opens yours for the rest of the stay and says nothing about who you are. Card tokens work the same way. When you add a card, the app receives the booking reference: an LVT, which on its own cannot charge anything. The tokenization service plays the front desk and swaps it for the key card: a high value token, or HVT, which gets stored and used for every charge after that.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1Ks0JRysJbr97tASoJDyJg.jpeg" /><figcaption>Decoding a fake LVT: card details plus a scenario name</figcaption></figure><p>You hand that to the wallet service the same way you would hand over a real card LVT. The tokenization service decodes it, mints a high value token, and stores it exactly as it stores a token from a real processor.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7-jjF_B3UpsKZRBn_9E3bA.png" /><figcaption>Adding a card: from card to token</figcaption></figure><p>Two nice properties came out of this. The outcome became a property of the instrument rather than the transaction. That is how a real card behaves. And follow on operations read the scenario from the parent authorization rather than from the token, so the scenario cannot change halfway through a charge lifecycle.</p><h3>Scenario Modeling: A scenario is just a dictionary</h3><p>At charge time the provider decodes the scenario name and looks it up. A scenario is a map from action to an ActionSpec. The spec says whether the action succeeds, how slow it is, and what error comes back when it fails.</p><p>Failures also carry a network decline code, a two-digit number like 51 for insufficient funds or 05 for do not honor. The platform reads it to decide what to do next, including whether a charge is worth retrying on a different processor.</p><pre>Charge.Action.AUTH: PayinActionSpec(<br>    success_predicate=ACTION_FAILURE,<br>    error_code=&quot;insufficient_funds&quot;,<br>    error_message=&quot;Auth declined&quot;,<br>    network_decline_code=&quot;51&quot;,<br>    delay_ms=100,<br>    retry_specs=...<br>),</pre><p>Success is a predicate and not a flag.</p><pre>ACTION_SUCCESS: Callable[[&#39;PayinActionContext&#39;], bool] = lambda ctx: True<br>ACTION_FAILURE: Callable[[&#39;PayinActionContext&#39;], bool] = lambda ctx: False<br>ACTION_CHAOS: Callable[[&#39;PayinActionContext&#39;], bool] = lambda ctx: random.random() &lt; 0.5</pre><p>Two of those are boring on purpose. The third one is not. ACTION_CHAOS flips a coin on every call. A cleanly broken processor is easy to write code for. The one that fails half the time is what actually hurts you at three in the morning, and now we can summon it whenever we like.</p><p>High latency is a scenario, too. Set <strong>delay_ms</strong> past the caller’s timeout and you get the scariest case of all: the processor succeeded, but we gave up waiting. Now we can test what happens next. The fake doesn’t try to be clever about it. It sleeps, succeeds, and forgets. Everything after that, the caller retrying and the platform reconciling what actually happened, runs through our real payment code, which is exactly the code we wanted to test.</p><p>The part we like most is <strong>retries</strong>. An ActionSpec can carry a map of ActionSpecs keyed by attempt number. The model refers to itself, and that recursion is the whole retry mechanism.</p><p>Below is a complete card. It authorizes cleanly. It fails the capture once. It succeeds when the platform tries again.</p><pre>SOFT_DECLINE_RETRY_SUCCESS_SCENARIO = PayinFakeScenario(<br>    name=&quot;card:fake_provider:soft_decline_retry_success&quot;,<br>    actions={<br>        # Step 1: AUTH succeeds<br>        Charge.Action.AUTH: PayinActionSpec(<br>            success_predicate=ACTION_SUCCESS,<br>            delay_ms=100<br>        ),</pre><pre>        # Step 2 &amp; 3: CAPTURE fails on first attempt, succeeds on retry<br>        Charge.Action.AUTH_CAPTURE: PayinActionSpec(<br>            success_predicate=ACTION_FAILURE,<br>            error_code=&quot;transient_error&quot;,<br>            error_message=&quot;Temporary processing error&quot;,<br>            network_decline_code=&quot;91&quot;,<br>            delay_ms=80,<br>            retry_specs={<br>                2: PayinActionSpec(<br>                    success_predicate=ACTION_SUCCESS,<br>                    delay_ms=100<br>                )<br>            }<br>        )<br>    }<br>)</pre><p>Read it top to bottom, and you have the entire card. Reproducing that on demand in a processor sandbox is not something sandboxes are built for. In our case, it is a few dictionary entries.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RITNGu3feLCUnxInqiDyLw.png" /><figcaption>Each charge: decode the token, look up the scenario, pick the spec for this attempt</figcaption></figure><p>The provider decides nothing. It decodes the token, looks up the scenario, picks the spec for this action, and checks which attempt this is. Note where that attempt number comes from. It is read off the stored charge and not passed in by whoever is calling.</p><p>So that is the architecture. A provider anyone can plug in. A fence only simulatedusers cross. A dictionary that decides what happens next.</p><p>The interesting part is what people started doing with it.</p><h3>What does Fake Provider unlock for us?</h3><h3>Staging: Pick your fake scenario from a menu</h3><p>Here is the part that changed how we work day to day. You open the rider CLI. You make yourself a simulated rider. You run one command and a list comes up.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/892/1*91stRQDmMRVbsMV51PXJEw.png" /><figcaption>Choosing a fake scenario from the rider CLI</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HgtugChBeQLkFX9t7nWkjQ.png" /><figcaption>Selecting a card that declines every authorization</figcaption></figure><p>Three checkmarks and you are done. That rider now has a card that declines every authorization, and you did not have to ask anybody for it. No sandbox credentials. No waiting on a processor. No pinging someone on our team to find out which test card number does the thing you want.</p><p>So what do you do when the one you need is not among them?</p><p>◾ You are an engineer working on the retry path. You need a capture that fails once and then succeeds. That one is on the list. You need one that fails twice and then succeeds. That one is not.</p><p>◾ You are a quality engineer testing some edge case where the authorization is declined and the direct charge goes through anyway. Very specific. Almost certainly not on the list.</p><p>◾ You are a product manager and you want to see your own flow the way a rider sees it when the card is refused. Or a marketer who wants to test an offer tied to a specific card issuer.</p><p>Before AI, the answer was to go and write the scenario yourself. Define the scenario. Register it. Add a card payload with a fresh number. Generate the token. Add it to the menu. Five steps in a repository you have probably never opened.</p><p>Now the answer is a skill. Open the repository that holds the scenarios. Run the skill that adds one. Describe the card you want in a sentence.</p><pre>&gt; /fake-provider-add-scenario</pre><pre>  I need a card where the authorization succeeds but the capture fails twice<br>  with a soft decline and then goes through on the third attempt.</pre><p>It writes all of it! The AI generated scenarios still go through a human code review.</p><p>Your part of that takes a minute. One sentence is the whole contribution. Everything after it is the agent walking the repositories that Fake Provider is spread across and writing the same five things you would have written by hand.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aHYyE7BThQB3QQq5SfvkxQ.png" /><figcaption>One sentence to the /fake-provider-add-scenario skill creates a new scenario</figcaption></figure><p>That is a good place to be, but it is not where we want to stop. All of that work only exists because scenarios ship as library code. Turn Fake Provider into a service and a new card becomes data instead of a config change in multiple repositories.</p><h3>Production: Load test your services</h3><p>How do you stress-test a payment system when you can’t charge real credit cards?</p><p>Payment services were previously outside the scope of our load tests. Simulated rides short-circuited before they ever reached a real charge. No authorizations fired. No charges processed.</p><p>Fake Provider changed that entirely. By provisioning simulated users with a fake card, simulated rides now flow through the entire stack. Fake Provider transactions are tagged and kept separate from real payment activity.</p><p>This approach can answer questions we simply couldn’t before:</p><p>◾ Does the charge process slow down or start timing out once real load hits it?</p><p>◾ Is there a sane, deliberate limit on how long the system waits before giving up on a slow request?</p><p>◾ Does something that works in a developer’s local setup behave the same way once real production traffic routes through it?</p><p>◾ Where is the charge-lifecycle latency ceiling?</p><p>◾ Does auth/capture keep pace, or do retries start stacking under load?</p><p>◾ Does a fraud/risk check become the bottleneck, or does a rate limiter just mask the real capacity ceiling?</p><p>But how do you set up hundreds or thousands of users in advance for load testing? Well, every test needs the same starting point: a user who already has a payment method that works. Create a brand new one for every single run and you’re testing account setup, not payments. So there’s a pool instead — a set of test users, reused run after run.</p><p>Pools drift, though. A card expires. A test leaves a user mid-scenario and never puts them back. Something changes upstream and a card that used to work stops working. Left alone, a pool slowly rots, and the first symptom is usually a test failing for a reason that has nothing to do with the code anyone just shipped.</p><p>Here comes the Resource Manager service that we mentioned in the previous <a href="https://eng.lyft.com/simulatedrides-how-lyft-uses-load-testing-to-ensure-reliable-service-during-peak-events-644dcb654454#:~:text=An%20additional%20Resource%20Manager%20service%20exposes%20an%20interface%20for%20using%20test%20assets%20in%20simulations">article</a>. It constantly runs health checks and automatically fixes issues. Since we want load tests and other scenarios to actually succeed, we configured the fixer to make a fake-success card the default payment method for these users. Still, we take a selective approach: we can leave this check off for pools meant to test failed transactions or other corner cases. The health check’s configuration works less like one global rule and more like a per-pool setting — different pools, different defaults. It lets us get rid of old cards gradually and replace them with fake ones for simulated test users only.</p><p>Adding payments to simulated rides meaningfully increased load on our critical payment services in production. This let us verify that both our application tier and our datastores scale smoothly under the surge: autoscaling responded to demand without throttling for DynamoDB, and connection pools stayed healthy for Aurora Postgres. For the flows we pushed the hardest (authorization, capture, and direct charges), scaling kept pace end to end.</p><h3>Production: Test new workflows</h3><p>A new payment workflow — a new retry strategy, a new fallback path, a new integration with a processor — can pass every test in staging and still behave differently in production, where the surrounding infrastructure isn’t the same. Because simulatedusers can be routed through Fake Provider even in production, we can point a new workflow at simulated riders first and let them exercise the exact new code path end to end — same services, same config, same scaling behavior real riders will hit, all without a real card or a real dollar involved. If the new capture-retry logic has a bug, our simulation platform finds it, not a rider.</p><p>This gives us a few things we didn’t have before:</p><p>◾ A canary population to test our new workflow on in production before we’re confident enough to expose it to real traffic.</p><p>◾ The ability to pick the scenario the simulateduser rides through — a soft decline, a slow processor, a chaos card — so the canary isn’t just “does it work”; it’s “does it work under the specific failure mode this workflow was built to handle.”</p><p>◾ A gate before graduation: a new workflow only gets promoted to real riders once it’s been proven out against simulated ones running the same code in the same environment.</p><p>In practice, this changes how we roll out significant payment changes. We exercise the new path with simulated riders and load test it at scale first, so by the time real riders ramp on, the workflow has already handled production-level load.</p><h3>Looking ahead</h3><p>Fake Provider has already changed how we test payments: broader failure coverage in staging (manual and automated), payment services included in load tests, and a safe place for new workflows to prove themselves before real riders see them. But every scenario today lives as a dictionary entry in library code, spread across a handful of repositories. That’s been fast to build and easy to reason about, and it’s also the ceiling. As the list of scenarios grows and more teams want to define their own, “a few dictionary entries in a repo you have to know about” turns from a feature into a bottleneck. The next step is turning Fake Provider from a library into a service — a new scenario becomes a piece of data submitted through an API, not a change reviewed and deployed across multiple repositories. That shift is what opens up the rest of this list:</p><p>◾ <strong>Record and replay: </strong>Capture real request/response pairs from the processor’s sandbox environment and replay them as scenarios, so Fake Provider’s behavior tracks what processors actually do, including the quirks a hand-written approximation tends to miss.</p><p>◾<strong> Async completion notifications: </strong>Fire the delayed webhook that carries a processor’s real outcome on a schedule we control (on time, late, out of order, or never), so we can exercise the code that has to handle all of it.</p><p>◾<strong> New payment methods: </strong>Simulate payment methods beyond cards.</p><p>◾ <strong>Disputes: </strong>Simulate the full chargeback lifecycle (chargeback, evidence submission, resolution), the one flow we still can’t fully test outside a real processor sandbox.</p><p>◾ <strong>Card updates: </strong>Simulate a card-updater feed landing mid-lifecycle, the way real processors tell us about an expired or reissued card.</p><p>◾ <strong>Rate limits and outages: </strong>Simulate a processor rate limiting us or going down entirely, so fallback routing and circuit breakers get validated under real pressure.</p><p>Stay tuned as we build out the next iteration of payments resilience!</p><h3>Acknowledgements</h3><p>We would like to thank the DevEnvs team (Maura Killeen, Levi Meng, Roy Tang, Chelsea Xu, and Alex You) for their huge support in turning simulated rides into real load on the payment stack.</p><p><em>Lyft is hiring! If you love the idea of solving ambiguous problems in technology and infrastructure, visit</em><a href="https://www.lyft.com/careers"><em> Lyft Careers</em></a><em> to see our openings.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c69dddadb115" width="1" height="1" alt=""><hr><p><a href="https://eng.lyft.com/beyond-test-cards-a-fake-payment-provider-for-testing-at-scale-c69dddadb115">Beyond Test Cards: A Fake Payment Provider for Testing at Scale</a> was originally published in <a href="https://eng.lyft.com">Lyft Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Hiring Junior Engineers]]></title>
            <link>https://eng.lyft.com/hiring-junior-engineers-1edc53e0fd5f?source=rss----25cd379abb8---4</link>
            <guid isPermaLink="false">https://medium.com/p/1edc53e0fd5f</guid>
            <category><![CDATA[junior-software-engineer]]></category>
            <category><![CDATA[company-culture]]></category>
            <category><![CDATA[hiring-engineers]]></category>
            <category><![CDATA[coding-interviews]]></category>
            <category><![CDATA[ai-coding]]></category>
            <dc:creator><![CDATA[Ashley Lorden]]></dc:creator>
            <pubDate>Tue, 10 Nov 2026 17:08:06 GMT</pubDate>
            <atom:updated>2026-11-10T17:08:04.806Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*i9VQTg4Bhru37QSCKqffeQ.png" /></figure><p>Several people have asked me what the qualifications are for junior software engineers to be hired these days. This level is often referred to as “new grad,” but people with lots of different backgrounds, not just recent university graduates, can use the entry-level software engineering role as their first job in big tech.</p><p>For example, in my early career I worked as a generalist in nonprofit after studying environmental economics. Then, using experience managing a website as part of my job, coding personal projects, and going through a software bootcamp, I moved into a junior SWE role as the foundation of my career in engineering.</p><p>Once you’re in the SWE field, your background is not an overwhelming predictor of success in your job. So what is?</p><p>The #1 qualifier I look for in hiring junior software engineers is <strong>mindset and soft skills</strong>. Curiosity, humility, and eagerness to learn is one critical area for successful engineers, and strong ownership, collaboration, and drive to create impact is another. The differentiator is self-management to find the right balance between the two, so that you’re continuously both growing your skills and shipping real work.</p><p>Transparently, although this is the top thing I’d like to see, it’s really hard to demonstrate in interviews. If you have stories that show how you employ these skills, share them with your interviewers.</p><p>For <strong>hard skills</strong>, in years past candidates have studied search algorithms, practiced using data structures, and considered optimal database patterns to pass coding interviews. In a world where AI agents write lots of our code, do prospective engineers still need to learn those core building and efficiency skills? Yes.</p><p>Although you may not be writing a lot of syntax by hand, you must thoroughly know the right approach so that you can prompt and review for the correct tradeoffs. There is rarely a universal right answer; a non-optimal but easy-to-understand algorithm is sometimes the best choice, whereas in other circumstances spells disaster. Knowing which is which and when to use them is what an engineer brings to the table.</p><p>In the AI era, the contribution human SWEs make has shifted even more toward <strong>judgement</strong>. At Lyft, the software we build shapes how people get around and earn a living every day — the stakes are real. Showing the motivation to inquire and the wisdom to make reasoned decisions is required to earn the team’s trust as a steward of technical architecture where your choices impact people in their daily lives.</p><p>To be successful as a new hire at Lyft, you need <strong>recent experience producing significant amounts of production-quality code</strong>. If you’d like to apply but aren’t in this position yet, find a way to ramp up your coding so you can be a strong applicant — for example, through personal side projects, driving technical solutions as part of a non-engineering job, or pursuing a structured learning program that gets you building.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KtY2oXT21uSxH63XioHcjg.png" /></figure><p>Why is this worth it? This role gives you mentorship and support, tons of tools and resources, and the time and opportunity to develop all of your skills, whether you want to become a senior engineer, lead a team as a manager, or build novel solutions as a technical founder.</p><p>If this resonates with you, I hope you’ll consider joining a team at Lyft. Currently hiring on my team:</p><ul><li><a href="https://app.careerpuck.com/job-board/lyft/job/8900188002?gh_jid=8900188002">SWE, Android</a> (Mexico)</li><li><a href="https://app.careerpuck.com/job-board/lyft/job/8767697002?gh_jid=8767697002">Intern, Data Science</a> (Toronto)</li><li><a href="https://app.careerpuck.com/job-board/lyft/job/8767715002?gh_jid=8767715002 FE: https://app.careerpuck.com/job-board/lyft/job/8797819002?gh_jid=8797819002">Intern, Backend</a> (Mexico)</li><li><a href="https://app.careerpuck.com/job-board/lyft/job/8797819002?gh_jid=8797819002">Intern, Frontend</a> (Mexico)</li></ul><p><em>Visit </em><a href="https://www.lyft.com/careers"><em>Lyft Careers</em></a><em> to see all our openings.</em></p><p><em>Ashley Lorden is a Senior Engineering Manager leading Global Growth at Lyft, working on autonomous vehicle integration, luxury experiences including our new Alcove brand, and international expansion. She joined Lyft in 2013 as a Junior Software Engineer.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1edc53e0fd5f" width="1" height="1" alt=""><hr><p><a href="https://eng.lyft.com/hiring-junior-engineers-1edc53e0fd5f">Hiring Junior Engineers</a> was originally published in <a href="https://eng.lyft.com">Lyft Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Refreshing the Travel-Time Map Behind Lyft’s Marketplace: Rebuilding Neighborhood Reachability…]]></title>
            <link>https://eng.lyft.com/refreshing-the-travel-time-map-behind-lyfts-marketplace-rebuilding-neighborhood-reachability-5be3efbc82ea?source=rss----25cd379abb8---4</link>
            <guid isPermaLink="false">https://medium.com/p/5be3efbc82ea</guid>
            <category><![CDATA[tea]]></category>
            <category><![CDATA[data-processing]]></category>
            <category><![CDATA[airflow]]></category>
            <category><![CDATA[feature-engineering]]></category>
            <category><![CDATA[marketplaces]]></category>
            <dc:creator><![CDATA[Manjunath Shettar]]></dc:creator>
            <pubDate>Thu, 10 Sep 2026 16:12:28 GMT</pubDate>
            <atom:updated>2026-09-10T16:12:26.781Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>Refreshing the Travel-Time Map Behind Lyft’s Marketplace: Rebuilding Neighborhood Reachability Signals</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*f5bDERtosR0e_DlgBWek-Q.jpeg" /></figure><p>Every time Lyft calculates pricing to balance a market, nudges a driver toward an under-served pocket of a city, or paints a heatmap of where demand is building, there is a quiet lookup table doing work in the background. It answers a deceptively simple question: <em>how long does it take to get from here to there?</em>, for millions of pairs of places, across hundreds of regions.</p><p>That lookup table is the <strong>Neighborhood Reachability Signal</strong>, and for years large parts of it were frozen in a snapshot of the world from 2018–2019. This is the story of how we rebuilt it, why a refresh substantial enough to be worth adopting was what finally moved Pricing to switch, the cleanly positive results that came out of that switch, and where we’re taking it next, from one static file per region to time-aware travel times that change with the rhythm of the day.</p><h4>What is a Neighborhood Reachability Signal?</h4><p>A <a href="https://en.wikipedia.org/wiki/Geohash">geohash</a> is a compact way of carving the world into a grid of cells. At <strong>geohash-6</strong> resolution, each cell is roughly the size of a few city blocks. Slice a region into geohash-6 cells and you get a clean, discrete coordinate system for “neighborhoods” that downstream systems can reason about.</p><p>The Forecasting &amp; Real-Time Optimization (FORTOP) team produces the Neighborhood Reachability Signals dataset, which consists of two companion files for each region:</p><ul><li><strong>Neighborhood Reachability Matrix</strong>: the estimated travel time, in minutes, between the centers of pairs of geohash-6 cells. Think of it as a sparse origin-to-destination travel-time matrix for a region.</li><li><strong>Neighborhood Centers</strong>: the list of all geohashes that appear in the ETA files for that region, i.e. the “vocabulary” of cells that the marketplace is allowed to talk about.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*B-1oDUryuSbtKTq74pIEbw.png" /></figure><p>Both files are generated offline on a schedule by an <a href="https://airflow.apache.org/">Airflow</a> DAG. They are <em>static</em> in the sense that they are precomputed and shipped, rather than queried live (which is exactly what makes them cheap to read at high frequency in latency-sensitive systems).</p><h3>Who relies on it</h3><p>The Geohash ETA dataset is one of those pieces of infrastructure whose customer list is longer than you’d expect, because much of the dependency is indirect.</p><p>The most direct consumers are:</p><ul><li><strong>Dynamic Pricing &amp; Offer Selection:</strong> Lyft’s real-time pricing engine, internally “Graph” uses Neighborhood Reachability Signals to decide <em>which</em> demand–supply geohash pairs are even worth modeling and <em>how important</em> each potential match is. A lower ETA between a rider’s geohash and a driver’s geohash means a better matching opportunity, and therefore a stronger pull on the price. In other words, the ETA dataset literally defines the structure of the graph that pricing optimizes over.</li><li><strong>Real-Time Supply / Driver Bonus Heatmaps:</strong> The real-time supply team uses the same files to reason about where supply can reach demand, and to render the heatmaps that tell drivers where it’s worth going.</li><li><strong>Neighborhood features:</strong> The FORTOP team uses the dataset to publish a <em>neighborhood</em> version of our real-time features, rolling up each geohash together with its reachable neighbors,which a much wider set of teams then build on. If a model anywhere in the marketplace reasons about a geohash and its reachable neighbors, there’s a good chance a Neighborhood Reachability Signal is somewhere upstream.</li></ul><blockquote>Curious how FORTOP produces these real-time features in the first place? Take a look at our companion posts, <a href="https://eng.lyft.com/real-time-spatial-temporal-forecasting-lyft-fa90b3f3ec24">Real-Time Spatial-Temporal Forecasting @ Lyft</a> and <a href="https://eng.lyft.com/evolution-of-streaming-pipelines-in-lyfts-marketplace-74295eaf1eba">Evolution of Streaming Pipelines in Lyft’s Marketplace</a>.</blockquote><h3>A dataset frozen in 2019</h3><p>The version of the dataset running in production had its core ETAs last meaningfully refreshed around <strong>2018–2019</strong>. That sounds alarming, but the reason it stayed in place is more mundane than it looks: the refreshes that happened in between were <em>incremental</em>.</p><p>Early on, we introduced <strong>versioning</strong> for the ETA files, which gave us a clean way to iterate on the dataset and to run experiments against whatever version was in production. Over the years the files went through several such iterations — coverage was extended and formats changed. None of these, though, represented a large enough change to the underlying ETAs to create a forcing function for adoption by a consumer like Pricing. Swapping the dataset that a system like Graph optimizes over is not a free action: it means designing and running experiments, validating that the marketplace behaves better, and absorbing the risk of a regression. When a new version is only marginally different from the one already in production, that cost is hard to justify and the expected upside is small. Plans to move forward did exist and had real momentum, but without a clearly substantial change to point to, they never quite materialized into a launch.</p><p>There was also a hidden tax in the meantime. Because the source dataset had gaps — missing geohash pairs that were genuinely matchable, and cells that shouldn’t have been included at all — consumers had to compensate downstream. Teams maintained <strong>their own denylists</strong> to strip out bad or non-drivable geohashes, and <strong>their own allowlists</strong> to add back geohashes the source had dropped. In effect, every consumer was patching the same dataset in parallel to work around the same gaps.</p><p>Two deficiencies in the legacy data were behind much of this:</p><ol><li><strong>It under-estimated travel times.</strong> Its ETAs were systematically lower than reality, which made Graph believe drivers were closer than they actually were.</li><li><strong>It was missing pairs that were genuinely matchable.</strong> Because of how the geohash universe was built, many demand–supply pairs <em>inside</em> the matching radius simply weren’t in the dataset. If a pair wasn’t present, the system behaved as if no driver could be there at all — even when one was physically close enough to match.</li></ol><p>In the second half of 2025, FORTOP shipped a redesigned version that produced more accurate and more complete ETAs and added <strong>new fields</strong> so teams could shape the data without maintaining their own patches. Paired with validation of how it improved on the legacy data, this was the refresh that Pricing adopted.</p><h3>How the files are built</h3><p>Before diving into what changed, it helps to understand the pipeline’s shape. We’ll keep this at the altitude of a flight map rather than a wiring diagram.</p><p>At a high level, the Airflow DAG produces each region’s files in a handful of conceptual stages:</p><ol><li><strong>Decide the geohash universe.</strong> Start from a maintained list of candidate geohashes for the region, then narrow it down using real demand and supply history — primarily where riders have actually had sessions and where drivers have actually been.</li><li><strong>Form the pairs.</strong> Take that set of geohashes and build the candidate origin–destination pairs that need a travel-time estimate.</li><li><strong>Estimate the travel times two ways.</strong> For pairs with enough observed history, the ETA is derived from historical trip data. For pairs that are too sparse to trust (fewer than a handful of observations), we fall back to the Routing Simulation System (RSS), which produces a simulated ETA that also factors in traffic conditions to keep the estimate realistic. Pairs with partial history get a weighted blend of the two.</li><li><strong>Emit the two files.</strong> Write out the Neighborhood Reachability Matrix file and the Neighborhood Centers file for downstream teams to consume.</li></ol><p>The elegance of this design is that it degrades gracefully: well-traveled corridors lean on real history, and the long tail of rarely-traveled pairs still gets a reasonable estimate from simulation rather than a hole in the matrix.</p><h3>What changed in the redesign</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pAR4pIv4kAzi8p1SBKAKLw.png" /></figure><p>The redesign was anchored by a very visible gap. Coincidentally, around this time the issue was also flagged publicly — the real-time map was surfacing demand in <strong>places no car could go</strong>, such as cells sitting over open water in the SFO region — and it drew a reply from Lyft CEO <a href="https://x.com/davidrisher/status/1919035247548645416?s=20">David Risher</a>. The root cause traced back to the geohash universe itself. The pipeline never checked whether a geohash was actually <em>drivable</em> before including it and expanding to its neighbors. It was a concrete instance of exactly the kind of gap consumers had been patching by hand.</p><h3>1. Drivable geohashes as a source of truth.</h3><p>Working with the Mapping team, we brought in a set of <strong>drivable geohashes</strong> derived from the road network itself (Lyft’s directional-segment map data). Rather than just sampling road segments at their endpoints, we interpolate many points <em>along</em> each segment, which dramatically improves coverage near region boundaries and on long road segments and bridges — exactly the places naive approaches tend to drop. These drivable geohashes are used in two ways: as a positive signal when building the geohash universe, and as a final denylist that strips out any cell sitting over water or other non-navigable terrain. The map-over-water bug simply disappears.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*793LM70VcPCozhUUWtpBUA.png" /><figcaption><em>Geohash coverage overlaid on the SFO region from the legacy production Neighborhood Centers file. Cells blanket the region, including stretches over the bay where no car can actually drive.</em></figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5GrdaBG8uXMV2ESfiUgWHw.png" /><figcaption><em>The same SFO region under updated workflow. Cells removed by the drivable-geohash denylist are outlined in </em><strong><em>red</em></strong><em>, concentrated over water and other non-navigable areas.</em></figcaption></figure><p>Removal is only half the story. In dense regions the net effect is trimming, but in sparser markets, many tier-2 regions and parts of the Midwest, for example, the refreshed workflow often <em>adds</em> geohashes, filling in coverage the old pruning had dropped.</p><h3>2. A gentler pruning strategy.</h3><p>The legacy pipeline aggressively trimmed geohashes to keep the dataset small, which was a big contributor to those “missing but matchable” pairs. The redesign reworks this:</p><ul><li><strong>Established regions</strong> (more than a year of history) now retain <em>every</em> geohash that has seen at least one rider session in the past year, instead of dropping the bottom slice by session count.</li><li><strong>New and expanding regions</strong> (less than a year of history) skip session-based pruning entirely and start from the full set of drivable geohashes, so a young market isn’t penalized for not yet having a long history. These regions are automatically promoted to the “established” path once they cross the one-year mark, and the whole distinction is made dynamically inside the DAG. There’s a trade-off here, though: with no session history to attach signals to, consumers can’t use that metadata to pare down what they read for these regions. We’re evaluating exposing additional metadata derived from whatever history a region <em>has</em> accumulated so far, so that even not-so-new regions get a lever to limit the data they load.</li></ul><p>This was a deliberate trade: more coverage in exchange for more pairs to process. It directly attacks the “phantom missing supply” problem that had been quietly distorting pricing.</p><h3>3. New fields for consumers.</h3><p>The redesign surfaces additional fields so that downstream teams can shape the data without us having to fork the pipeline for each of them. For example, a field that aggregates rider-session density around each destination geohash (within an 8-minute radius) lets a consumer bound how much of the file it needs to read, and the drivable-geohash set itself is written alongside the output. Together, these additional attributes let consumers do their own pruning and filtering on top of the dataset — exactly the self-service that removes the need for hand-maintained denylists and allowlists.</p><p>This brief clip demonstrates how consumer systems utilize rider-session density to prune their specific geohash environments. Note how coverage builds dynamically, radiating outward from the highest-density hubs to the quietest corners of the region.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/1*6hcEJD_CSzSq7yKlM857ZA.gif" /></figure><h3>How Pricing adopted it — and what they saw</h3><p>Because the ETA dataset defines the very structure Graph optimizes over, swapping it out is not a quiet change — it reshapes the marketplace’s entire view of feasible supply. So Pricing treated it like a first-class experiment, running the refreshed dataset against the legacy one as a two-week time-split test across a set of top pricing regions.</p><p>The most illuminating result is <em>structural</em>. Looking at each geohash’s feasible neighbors (the cells reachable within a 20-minute ETA), the shape of “nearby supply” shifted substantially:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-PRi3GDK9FOtAkChmrLbeA.png" /></figure><p>Two forces drive this. First, <strong>over-optimism gets corrected</strong>: pairs of the old data labeled as “≤ 5 minutes away” were really 5–15 minutes away, so neighbors migrate outward into honest buckets. Second, <strong>coverage fills in</strong>: genuine medium-range pairs that were missing before now appear. The net effect is fewer phantom close-by drivers and more genuine medium-range options.</p><p>The refresh also visibly reshaped the size of the dataset, region by region.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RfnaHZTtimMnHagla1GmlQ.png" /></figure><p>With the change validated, the work now is to roll the refreshed dataset out to the remaining regions, settle each into a new equilibrium, and <strong>automate the refresh on a six-month cycle</strong> so the dataset never drifts a half-decade out of date again.</p><h3>Where we’re going next: from static to time-aware ETAs</h3><p>There’s one assumption baked into everything above that is obviously wrong if you’ve ever driven in a city: that there’s a <em>single</em> travel time between two neighborhoods. In reality, the trip that takes 8 minutes at 3 a.m. can take 25 at evening rush. The current version still produces <strong>one ETA file per region</strong>, a single all-hours average. Our next step makes the dataset <strong>time-aware</strong>. Pricing’s Graph already reasons in terms of <strong>nine time categories</strong> based on the hour of the week (morning commute, weekday day, evening commute, weekend daytime, and so on). The next version generates a <strong>separate Geohash ETA file per time category</strong>, nine files per region, so that each one reflects the traffic dynamics of its slice of the week. The historical-ETA computation is bucketed by time category, and where we fall back to simulation, the sampled requests are drawn from within the same time category as the estimate they’re filling in.</p><p>A few deliberate design choices make this practical:</p><ul><li><strong>Nine files, not one nine-times-bigger file.</strong> Splitting by time category means a consumer only loads the bucket it currently needs at inference time, instead of paying to read the whole week’s data on every model run. Both Pricing and the real-time supply team preferred this, since each can simply key its file path on region plus current time bucket.</li><li><strong>A shared library for the mapping.</strong> Converting a real-world moment (and its timezone) into the right time category is the kind of logic you only want to write once, so it lives in a shared marketplace library that any consumer can call. It handles the UTC-to-local conversion and daylight-saving edge cases so individual teams don’t have to.</li><li><strong>Latency stays flat.</strong> The trade-off is memory, roughly a couple of extra gigabytes to hold a bucket, but with simple prefetching (warm the next time category’s file before the clock rolls into it), read latency stays about the same as the single-file world.</li></ul><p>Early validation of the time-aware files is encouraging, and it points at the longer-term direction for this dataset. We’re also exploring more advanced ways to compute the estimates themselves, for example, <strong>weighted ETAs</strong>, where more recent observations carry more weight than older ones, so the dataset reflects how a city is moving <em>now</em> rather than averaging flatly over a year of history. The arc is consistent: from a single frozen snapshot, to a periodically refreshed and geographically honest map, to one that moves with the time of day, and, eventually, toward more real-time and adaptive ETAs.</p><h3>Conclusion</h3><p>Neighborhood Reachability Signals are a small piece of infrastructure with an outsized blast radius. Letting them drift to a 2019 snapshot had a real cost in mispriced markets, missed matches, and even maps that drew demand over open water. The redesign fixed the foundations: drivable geohashes from the source of truth, far better coverage, and new fields that let consumers shape the data themselves instead of maintaining their own patches. Just as important, the dataset now refreshes on a real cadence instead of by one-off fixes. Making the files time-aware is the natural next step, letting travel time vary with the day, the way it always has in the real world.</p><h3>Acknowledgements</h3><p>We would like to thank all our existing and past real-time and forecasting team members (<a href="https://www.linkedin.com/in/chunningchang/">Brian</a>, <a href="https://www.linkedin.com/in/jimchiang/">Jim</a>, <a href="https://www.linkedin.com/in/tzuchi-lin/">Jeff</a>, <a href="https://www.linkedin.com/in/joshxiaominxi/">Josh</a>, <a href="https://www.linkedin.com/in/hongru-liu/">Hongru</a>, <a href="https://www.linkedin.com/in/casey-niu/">Casey</a>) and also our partner teams like Pricing (<a href="https://www.linkedin.com/in/xiangnan-zhou-08g/">Xiangnan</a>, <a href="https://www.linkedin.com/in/shadongliu/">Simon</a>) and Driver Earnings for their valuable feedback</p><p><em>Want to build systems that balance a real-time marketplace at scale? </em><a href="https://www.lyft.com/careers"><em>Join us at Lyft.</em></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5be3efbc82ea" width="1" height="1" alt=""><hr><p><a href="https://eng.lyft.com/refreshing-the-travel-time-map-behind-lyfts-marketplace-rebuilding-neighborhood-reachability-5be3efbc82ea">Refreshing the Travel-Time Map Behind Lyft’s Marketplace: Rebuilding Neighborhood Reachability…</a> was originally published in <a href="https://eng.lyft.com">Lyft Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Rerouting the Stream: How Lyft Moved to the Apache Flink Kubernetes Operator]]></title>
            <link>https://eng.lyft.com/rerouting-the-stream-how-lyft-moved-to-the-apache-flink-operator-36f20246d250?source=rss----25cd379abb8---4</link>
            <guid isPermaLink="false">https://medium.com/p/36f20246d250</guid>
            <category><![CDATA[data]]></category>
            <category><![CDATA[data-platforms]]></category>
            <category><![CDATA[streaming-data-processing]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[apache-flink]]></category>
            <dc:creator><![CDATA[Maheep Myneni]]></dc:creator>
            <pubDate>Mon, 31 Aug 2026 19:08:16 GMT</pubDate>
            <atom:updated>2026-10-23T00:12:54.701Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fDPW6FgRQeqQXLJifjbxQg.jpeg" /></figure><p><em>Written by </em><a href="mailto:maheepm@lyft.com"><em>Maheep Myneni</em></a><em>, </em><a href="mailto:akuyumcu@lyft.com"><em>Arda Kuyumcu</em></a><em>, and </em><a href="mailto:premsantoshu@lyft.com"><em>Prem Santosh Udaya Shankar</em></a><em> at Lyft.</em></p><h3><strong>Why We Migrated</strong>: Technical Debt Meets Modern Streaming Demands</h3><p>Over the past several quarters, Lyft’s Streaming Compute team retired our internally developed Flink Kubernetes operator and moved our entire streaming fleet onto the open-source Apache Flink Kubernetes operator. This post is about why we made the switch, how we pulled it off incrementally without disrupting users, and the follow-on work it took to actually get the benefits we were after.</p><p>Back in 2020, when we first architected the Lyft Flink Kubernetes Operator, it was exactly what we needed. At that time, the open-source community hadn’t yet built a dedicated control plane, so we built our own to manage all streaming applications on Kubernetes. It worked well for our initial workloads, but as our real-time data needs increased and our engineers’ scope of ownership grew in both breadth and complexity, the cracks started to show.</p><p>First came the maintenance burden. Our operator had become a relic of Lyft’s early Kubernetes days, kept alive by a growing pile of custom code. Every Flink version upgrade meant carefully picking through layers of accumulated technical debt and hoping nothing broke on the way through.</p><p>Second came the feature gap. Streaming tooling kept moving, and our engineers kept asking for capabilities that had become table stakes elsewhere, such as autoscaling to right-size jobs, automatic rollbacks on failed deploys, and an end to hand-tuning CPU and memory. Each request left us with two options, neither of which was ideal. We could explain why we couldn’t support it yet, or spend weeks rebuilding something the open-source community had already shipped.</p><p>Third was the dependency problem. We were pinned to outdated libraries. That doesn’t break anything today, but it almost always creates new issues down the line. Security patches lagged, modern Kubernetes features stayed out of reach, and every quarter we waited made the eventual escape messier.</p><p>Meanwhile, the <a href="https://github.com/apache/flink-kubernetes-operator">Apache Flink Kubernetes Operator</a> had matured into production-grade infrastructure, and its feature set read like our wishlist: intelligent job-vertex autoscaling to kill backpressure, automatic memory tuning, safe upgrade mechanisms, and a robust reconciliation loop. Additionally, it was battle-tested by companies at massive scale. The decision in front of us became obvious. We needed to migrate Lyft’s fleet, hundreds of critical streaming jobs, from our internal operator to the Apache one. However, we had to do this migration without disrupting existing deployment patterns, altering our team’s workflows, or causing massive outages in production.</p><p>This is the story of how we pulled it off, and why sometimes the best code you can write is the code you delete.</p><h3>Our Legacy: The Lyft Flink Kubernetes Operator</h3><p>Before we dive into the migration itself, it’s worth understanding what we were migrating from. The Lyft Flink Kubernetes Operator served us well for years, and being familiar with its architecture helps illustrate both why the migration was necessary and why it was nontrivial.</p><p>At its core, Lyft’s operator follows the standard Kubernetes pattern by continuously watching custom resources and reconciling their state. We defined Flink applications using a <strong>FlinkApplication</strong> custom resource that looked something like this:</p><pre>apiVersion: flink.k8s.io/v1beta1<br>kind: FlinkApplication<br>metadata:<br>  name: my-streaming-job<br>  namespace: production<br>spec:<br>  image: lyft/my-flink-job:v1.2.3<br>  flinkVersion: &quot;1.16&quot;<br>  jarName: &quot;my-streaming-job.jar&quot;<br>  entryClass: &quot;com.lyft.streaming.MyJob&quot;<br>  parallelism: 8<br>  <br>  jobManagerConfig:<br>    replicas: 1<br>    resources:<br>      memory: &quot;2048Mi&quot;<br>      cpu: &quot;1.0&quot;<br>      <br>  taskManagerConfig:<br>    taskSlots: 4<br>    resources:<br>      memory: &quot;4096Mi&quot;<br>      cpu: &quot;2.0&quot;<br>      <br>  flinkConfig:<br>    state.backend.fs.checkpointdir: s3://lyft-flink/checkpoints<br>    state.savepoints.dir: s3://lyft-flink/savepoints<br>    execution.checkpointing.interval: 60s</pre><p>Lyft’s operator takes this spec and creates the necessary Kubernetes resources: a JobManager (JM) deployment, a TaskManager (TM) deployment, any external runtime configurations, and optionally an ingress for the Flink UI. Simple enough, in theory.</p><h3>The State Machine: Where Things Got Interesting</h3><p>The Lyft operator’s real complexity lived in the state machine. Deploying a stateful streaming application isn’t like deploying a stateless web service; you can’t simply kill the old pods and start new ones. If you did, all in-flight data and state would be lost. Our operator walked through a careful choreography of states to ensure zero data loss.</p><p><strong>Dual Deployment Mode</strong> (our default) had minimal, but non-zero downtime. When a user updated a FlinkApplication, the operator would spin up a completely new Flink cluster alongside the running one. Once the new cluster was healthy, it would trigger a savepoint on the old job, wait for it to complete, cancel the old job, tear down the old cluster, and finally submit the job to the new cluster from the savepoint.</p><p><strong>BlueGreen Mode</strong> took it a step further for truly zero-downtime deployments. Both clusters would run simultaneously (“dual running”), and you would manually switch traffic by setting a tearDownVersionHash field after verifying the new version was working correctly. This gave users a manual kill switch, which worked well for critical jobs where one would want to inspect the new version before committing.</p><p>The state machine tracked states like ClusterStarting, Savepointing, SubmittingJob, Running, DeployFailed, and our favorite debugging companion, Recovering (for when savepoints failed and we needed to fall back to externalized checkpoints). Each transition was carefully orchestrated with retry logic, timeout handling, and event logging.</p><h3>Problems: The Cracks in the Foundation</h3><p>Beyond the maintenance burden and the feature gap, two specific pain points made this decision unavoidable. Both stemmed from early design choices that suited a smaller fleet, but broke down as our workloads grew to be more varied over time.</p><h4><strong>Savepoint Fragility</strong></h4><p>Savepoints were the only first-class mechanism for preserving application state during upgrades. If a savepoint failed, the operator fell back to looking for an externalized checkpoint. However, the checkpoint fallback was a last-resort recovery path rather than a strategy users could deliberately choose. The savepoint trigger had no retry logic and no idempotency guarantees. If no recent checkpoint was available, the deploy either failed or restarted the job with no state at all. For large-state jobs where savepoints were slow or occasionally timed out, a transient failure escalated into a full recovery cycle. The Apache operator introduced last-state as an explicit upgrade mode, which uses high-availability metadata or the latest checkpoint by design. This was exactly the robustness we needed, particularly for jobs where an unhealthy JobManager couldn’t cooperate with a savepoint request.</p><h4>Rigid Resource Model</h4><p>Our operator exposed a single systemMemoryFraction knob. This represented a fraction of container memory reserved for non-JVM use. For pure Flink Java applications, this overhead was often unnecessary. For Apache Beam workloads using the Python SDK harness, it was vital. Unfortunately, how much memory those Python processes actually needed was essentially guesswork, since usage happened inside the Flink memory model. There was no observability into the utilization and no way to tune it systematically. If we set it too low, TMs would get OOM-killed. If we set it too high, we wasted resources.</p><h3>Migration: Our Blueprint for a Seamless Switch</h3><p>Setting up a new system is the easy part. The real nightmare of any migration is unwinding everything that still points to the old one. At Lyft, all Flink deployment configs start as Jsonnet files. We use a shared module to set up operational defaults, and individual teams layer their own application configs on top:</p><pre>common.sp.FlinkApplication(&quot;myflinkapplication&quot;, &quot;staging&quot;, &quot;myproject&quot;) {<br>  application+: {<br>    spec+: {<br>      parallelism: 8,<br>      flinkConfig+: {<br>        &quot;state.backend&quot;: &quot;filesystem&quot;,<br>        &quot;restart-strategy.fixed-delay.delay&quot;: &quot;10s&quot;<br>      }<br>    }<br>  },<br>}</pre><p>In the above example, an individual app extends the base factory with its own parallelism and Flink configuration. The entire system was built around the legacy operator’s FlinkApplication custom resource definition (CRD). Rewriting every library and application overlay to natively produce the Apache operator’s <strong>FlinkDeployment</strong> CRD would have been a massive and risky undertaking.</p><p>Instead, we chose to translate at the boundary, so teams could continue defining their applications exactly as they always had. The translation from FlinkApplication to FlinkDeployment happens within our deploy API. When the deploy API encounters a legacy resource, it converts the resource into something the Apache operator is familiar with. This handles the structural mapping:</p><ul><li><strong>Job Spec:</strong> We assemble flat fields like jarName and parallelism into the Apache operator’s nested spec. jarName becomes a jarURI.</li><li><strong>Upgrade Mode: </strong>We default to last-state to restore state from the last checkpoint if possible, and map legacy configs like savepointDisabled to stateless.</li><li><strong>Pod Templates:</strong> Inline container configs on the legacy spec like node selectors, tolerations, and sidecar annotations get promoted into proper PodTemplateSpec objects.</li><li><strong>Backwards Compatibility:</strong> We inject legacy environment variables like APP_NAME and HOST_NAME so application code doesn’t break.</li></ul><p>Crucially, the migration was incremental. Dynamic configuration allowed us to migrate one application at a time, starting with staging. If an app hit issues, we could instantly roll it back to the legacy operator. To the developer, the change was largely invisible.</p><h3>From the Trenches: Additional Work to Integrate the Operator</h3><p>While the translation layer got us onto the Apache operator, the migration was not yet done. We still had to address a few follow-on problems, and do additional work to make sure we saw all the benefits the Apache operator had to offer. Two of those benefits are autoscaling, which dynamically adjusts a job’s parallelism, and autotuning, which adjusts the memory resources each TaskManager gets.</p><h4>BlueGreen Deployments</h4><p>Once we migrated apps onto the Apache operator, every deploy cost us an average of 3–6 minutes of downtime, and closer to 20 minutes for our largest jobs. Lyft’s operator had managed JobManager and TaskManager lifecycles together through a complex graph of operations, which kept downtime to a minimum. The Apache operator took a different approach; it spun up only the JobManagers, configured them with the relevant savepoint and resource specs, and let them coordinate the TaskManagers. Each deploy walked through the full sequence of taking a savepoint, triggering the JM pods to tear down TM pods, tearing down the JM pods, and then starting the new FlinkDeployment’s JMs using the recently taken savepoint, which then brought up the new TM pods. For a real-time business like Lyft, this was unacceptable for several of our most important use cases.</p><p>The fix for this was <strong>FlinkBlueGreenDeployments</strong>, a newer CRD that maintains underlying FlinkDeployment objects so a new version can come up alongside the old one before any cutover. This feature was still under development when we started testing it internally; we hit a bug in how the operator renamed Flink configuration fields under BlueGreen, and contributed a minor <a href="https://github.com/apache/flink-kubernetes-operator/commit/98d041d90dfbf06573bce407cc0fc492d828c6e3">upstream fix</a>. Benefiting from the open-source community’s efforts and being able to contribute back further reinforced our decision to move to an open-source solution.</p><h3>Autoscaling</h3><p>The Apache operator’s autoscaler uses metrics like operator busyness, backpressure, and record backlog in Kafka and Kinesis to adjust each job’s parallelism. Traffic across most of Lyft’s use cases is highly variable and stream processing is expensive, so autoscaling was, in theory, an obvious win. In practice, our Flink version (v1.17) required apps to restart in order to rescale, and the apps that would benefit the most from autoscaling were precisely the ones that couldn’t afford restart-induced downtime.</p><p>Upgrading to Flink v1.19 unlocked two important changes. First, it unlocked in-place autoscaling. Jobs could change parallelism without restarting, which both preserved data freshness and avoided the spiky catch-up traffic we used to see when jobs resumed from offsets that were a few minutes old. Second, it enabled an upgrade to the new <a href="https://aws.amazon.com/blogs/big-data/introducing-the-new-amazon-kinesis-source-connector-for-apache-flink/">Amazon Kinesis Source Connector</a>, which emits record backlog metrics for Kinesis streams the same way the Kafka source does. This gave the autoscaler the signal it needed to make scaling decisions on Kinesis-bound jobs, which had previously been a blind spot.</p><h3>Autotuning</h3><p>Autotuning appealed to us for two main reasons. It promised dynamic right-sizing to cut cloud costs, and it spared our devs from tuning resource configurations by hand. However, we ran into an unexpected problem after we rolled this feature out.</p><p>Our Apache Beam workloads run user code in a Python SDK harness, which lives outside the Flink memory model. The Apache operator didn’t register that we were reserving memory for those Python processes inside the JVM overhead allocation. We had been tucking it there for years because most of our jobs barely touched their JVM overhead anyway. This hack worked until we enabled autotuning. The operator saw that these memory pools were not really used by Flink, and reclaimed them, OOM-killing TaskManagers across our Beam fleet. We ultimately moved the Python harness out of the Flink container entirely, running it as a sidecar in the same pod with its own resource limits.</p><p>Another awkward property of autotuning is that it doesn’t play well with in-place autoscaling. To tune a job’s resources, the operator has to restart it. We weren’t willing to give up either feature, so we settled on a two-tier approach where business-critical applications that power pricing, routing, and similar real-time decisions disable autotuning and keep in-place autoscaling. Less critical applications that could tolerate some downtime use both autotuning and autoscaling with restarts.</p><h3>Karpenter</h3><p>These features also made our node infrastructure obsolete. Before autotuning, we bin-packed pods onto AWS nodes efficiently by enforcing 1:4 or 1:8 CPU:memory ratios with power-of-two values. Autotuning broke this. The same application could have very different ratios from one tuning cycle to the next, and a static node pool could not keep up. We adopted Karpenter, which provisions EC2 nodes on demand based on what each pod actually needs, optimizing for cost or utilization as appropriate.</p><p>This two-tier autotuning model produced an additional unexpected efficiency win related to unused spend (the share of compute resources we are paying for that ends up unallocated). If we are paying for an 8-core node with 32GB of memory, but only allocate 4 cores and 16GB, half of the cost of the node is unused. We had already deployed a custom Kubernetes pod scheduler on our EKS cluster that uses a “most-allocated” strategy, packing pods onto busy nodes first. When autoscaling restarted apps to rescale them, the pods had a chance to move from under-allocated nodes to densely packed ones, letting us shed underutilized nodes.</p><h3>Conclusion</h3><p>This migration was a multi-quarter journey that often felt like changing the engines on a plane mid-flight. It took longer than expected to completely sunset our in-house operator, as there was a mess of state-machine quirks, memory model mismatches, and CRDs that didn’t translate cleanly. However, the difference in the platform after completing this migration is hard to overstate; the operator maintenance that used to eat engineering time on every Flink upgrade has been minimized, and the autoscaler is doing what we had hoped, which is right-sizing a fleet that had been overprovisioned by a few million dollars per year.</p><p>The important, but hard-to-quantify benefit is that we stopped being the only contributors to our operator and infrastructure. When we noticed the BlueGreen bug, we filed an issue and a fix landed upstream within days. Our team’s posture changed from “we maintain this thing” to “we use this thing, and sometimes improve it for everyone,” which is a much better place to be. What’s next is the work we actually wanted to focus on from the start. We’re building faster feedback loops for marketplace pricing, handling autonomous vehicle telemetry, and scaling streaming primitives for Lyft’s next chapter. The Apache operator finally gave us the breathing room to work on these challenging problems.</p><h3>Appreciation</h3><p>Special thanks to <a href="https://www.linkedin.com/in/aniruddhadkar/">Anirudh Adkar</a> and <a href="https://www.linkedin.com/in/ravimagham/">Ravi Magham</a>, who provided guidance to us throughout the project, and <a href="https://www.linkedin.com/in/janiceyrlee/">Janice Lee</a>, <a href="https://www.linkedin.com/in/rohanvarshney/">Rohan Varshney</a>, and <a href="https://www.linkedin.com/in/yigal-kassel/">Yigal Kassel</a> for their contributions to the team’s operations. The migration would not have been possible without their efforts.</p><p>Finally, thanks to our many internal customers of Lyft’s Streaming Compute platform, who bore with us and provided ample helpful feedback throughout the migration, including but not limited to <a href="https://www.linkedin.com/in/kyle-bilton/">Kyle Bilton</a> and <a href="https://www.linkedin.com/in/taylor-wilson-a55415b2/">Taylor Wilson</a>.</p><p>Lyft is hiring! If you love the idea of solving ambiguous problems in technology and infrastructure, visit <a href="https://www.lyft.com/careers">Lyft Careers</a> to see our openings.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=36f20246d250" width="1" height="1" alt=""><hr><p><a href="https://eng.lyft.com/rerouting-the-stream-how-lyft-moved-to-the-apache-flink-operator-36f20246d250">Rerouting the Stream: How Lyft Moved to the Apache Flink Kubernetes Operator</a> was originally published in <a href="https://eng.lyft.com">Lyft Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[From Day 1 to Production: Building Lyft’s Analytics & Rides Intelligence Assistant as Onboarding…]]></title>
            <link>https://eng.lyft.com/from-day-1-to-production-building-lyfts-analytics-rides-intelligence-assistant-as-onboarding-5c5643c192d9?source=rss----25cd379abb8---4</link>
            <guid isPermaLink="false">https://medium.com/p/5c5643c192d9</guid>
            <category><![CDATA[ai-and-machine-learning]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[front-end-development]]></category>
            <category><![CDATA[onboarding]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Sagarbaronia]]></dc:creator>
            <pubDate>Thu, 09 Jul 2026 22:01:15 GMT</pubDate>
            <atom:updated>2026-07-09T22:08:20.407Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>From Day 1 to Production: Building Lyft’s Analytics &amp; Rides Intelligence Assistant as Onboarding Project</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eJIjiV1qsbDNRc_uYHopWw.jpeg" /></figure><p><em>Written by Sagar Baronia at Lyft.</em></p><h3>A Different Kind of Day One</h3><p>Most onboarding journeys follow a familiar arc: orientation sessions, benefits enrollment, setting up your laptop, and gradually finding your footing over the first few weeks. Mine followed that arc too, but with an additional thread running alongside it from the very start.</p><p>I joined Lyft in March 2026 as a Senior Data Scientist — Algorithm on the Marketing, Business &amp; Ads team, bringing close to a decade of experience in data science, first at Oracle and then four years at Amazon. From the very first conversation with my manager, <strong>Ravi Chandra Babu Ginjipalli</strong>, it was clear that my onboarding would have an extra dimension. Alongside the standard new hire track, I would take on a real production project: one designed to teach me Lyft’s systems by shipping something through them.</p><p>The project: build the production frontend, from scratch, for Lyft’s<strong> Analytics &amp; Rides Intelligence Assistant</strong> — <strong>ARIA</strong> for short. This is the story of going from Day 1 to a live production service in three weeks.</p><h3>Aria: From Prototype to Production</h3><p>Aria is an AI-powered tool that lets Lyft employees explore ride analytics data through natural language. Instead of writing SQL queries or navigating dashboards, approved employees at Lyft can ask questions like, “How many rides were there in SF last month?” and get answers instantly.</p><p>The backend intelligence already existed. My team had built the service that powers Aria’s AI agent: the LangGraph orchestration, the SQL generation pipeline, the semantic layer, all of it. But the user-facing layer was running on Streamlit — functional for prototyping, but not production-grade. It was fine for the handful of us in the team testing it, but it couldn’t scale to the whole company, didn’t plug into Lyft auth, and lived outside the standard service framework. That meant Aria couldn’t graduate from prototype to something the rest of Lyft could actually use.</p><p>The goal was to build a proper web client using our internal Node.js framework and Next.js, connected to that existing backend, with real authentication, real deployment infrastructure, and a real path to scale. Today, Aria is actively used by approved employees across Lyft, with adoption growing week over week.</p><h3>The Onboarding Philosophy: Learn by Shipping</h3><p>Ravi’s reasoning for this project-centric approach was deliberate. At a senior level, the challenge of joining a new company isn’t learning a language or a framework — it’s understanding how the organization’s specific production ecosystem fits together. How does authentication work here? What does the deployment pipeline look like end to end? How are services discovered, routed, and monitored? A frontend project that needs to go to production forces you through all of these questions in a matter of weeks, because you simply cannot ship without answering them.</p><p>This was not about skipping the standard onboarding. Every Friday, I had dedicated hours for the regular new hire track: the compliance courses, the orientation sessions, the checklist items that every new Lyfter completes. The project ran in parallel, not as a replacement, and the two tracks reinforced each other. The onboarding material made more sense because I had real context for it, and the project moved faster because the onboarding filled in gaps I would have otherwise had to ask about.</p><p>My onboarding buddy, <strong>Alex Gao</strong>, a fellow Full Stack Applied Scientist on the team, was instrumental in those early days — helping me orient to the codebase, the team’s processes, and the company’s rhythm.</p><h3>The Support System That Made It Possible</h3><p>An ambitious onboarding project only works if the culture supports it. From my first week, Alex introduced me to the Slack channels that would become my daily references: the networking channel for all Envoy and DNS questions (critical when you’re configuring a production frontend), and the frontend engineering channels where software engineers, product designers, and frontend specialists shared patterns and solutions.</p><p>I was also introduced to staff software engineers across teams who were remarkably responsive when I hit walls that documentation alone couldn’t resolve. <strong>Glen Cheney</strong>, a Staff Software Engineer on the frontend platform team, was particularly helpful when I was working on getting the streaming connection between the frontend and backend to work properly. Rather than having me debug from scratch, Glen pointed me to an existing internal service that had already solved a similar streaming pattern. I dug into that service’s repo, traced how it handled the streaming handshake and kept the connection alive, and adapted the pieces that mapped to our setup. What could have been days of trial and error turned into an afternoon of focused reading and a working prototype by the end of the day.</p><p>That kind of guidance, where an experienced engineer helps you find the right reference instead of reinventing the wheel, is what makes the difference between a good onboarding and a great one.</p><p>This cross-team generosity wasn’t an exception. It reflected a culture where helping someone outside your immediate team is a natural part of how engineering operates at Lyft.</p><h3>Week 1: Standing Up the Foundation</h3><p>The first task was creating the Aria frontend repository and generating the Lyft Node Service scaffold. I configured all the necessary plugins — authentication, logging, metrics, service client — and built the initial layout shell for the chat interface.</p><p>Then came the infrastructure work: Envoy static configuration for routing, DNS setup with CNAME records and CloudFront aliases, and staging deployment. The first real lesson came quickly. An early deployment issue led me to diagnose an authentication plugin incompatibility, which I resolved by switching to a different plugin that was better suited to our setup. The deployment stabilized, and by the end of the week, Aria’s staging environment was live. It wasn’t doing much yet, but the scaffold was standing, and I had already learned more about Lyft’s deployment infrastructure than weeks of reading could have taught me.</p><p>Alongside the project, my first week at Lyft had its own rhythm. I went through NYC orientation, met other new hires, and started settling into the company’s culture. There were the practical logistics that come with any new job: getting payroll and benefits in order, completing verification paperwork, setting up equity accounts and commuter benefits. I also spent time configuring my development environment, getting familiar with AI-powered coding tools, and raising my first test PR just to learn how the review and merge flow works at Lyft. It was a full week in every sense.</p><h3>Week 2: Making Systems Talk</h3><p>With the frontend deployed to staging, the focus shifted to connecting it to the backend. I built the connectivity layer to the Aria backend service and began iterating on manifest file changes and deployment configurations to get the two services communicating properly.</p><p>This turned out to be less straightforward than expected. The connection kept running into issues, and the usual debugging approach of reading logs locally wasn’t enough. This is where Grafana became an invaluable tool. By tracing requests through Grafana’s log aggregation, I was able to identify that the root cause was in the Envoy setup — the session configuration wasn’t wired correctly, which was silently breaking the connection between services. Fixing it was relatively quick once the problem was visible; the real learning was building the instinct to reach for Grafana first when something feels off — tracing a request across services, scanning logs by time window, narrowing down which hop in the chain is actually broken. It’s a habit I rely on almost daily now.</p><p>This week’s other significant piece was building the XState state machine for chat state management. Chat interactions involve a surprising amount of state: session IDs, message sequences, streaming progress, error handling, connection lifecycle. XState gave us a structured way to model all the possible transitions and keep the UI predictable even when the network wasn’t.</p><p>On the onboarding side, I was still working through the remaining items from the new hire checklist and picking up the tools and processes that make day-to-day work at Lyft smoother. I also got my first taste of Lyft Pass — a small but fun perk that comes with being here.</p><h3>Week 3: The Last Mile</h3><p>With the backend connectivity solid, the final week’s focus was on getting Server-Sent Events (SSE) streaming working end to end. This is the piece that would make Aria’s chat interface feel responsive and modern, with answers streaming in token by token rather than arriving as a single block of text.</p><p>There was a minor hiccup in how the streaming needed to be configured on the frontend side. The way the SSE connection was being piped through the Next.js API route required a specific setup that wasn’t immediately obvious from the documentation. With help from the frontend platform team, we got the streaming pipeline working smoothly. In the process, we also discovered that the token chunk size coming from the backend was too large, which made the streaming feel choppy rather than fluid. This required a change on the backend side to send smaller, more frequent chunks: a good example of how frontend work often surfaces issues that live elsewhere in the stack.</p><p>The week also included production hardening: DNS management changes for CNAME and CloudFront alias in production, Envoy static config updates for production virtual hosting, and adding test cases to increase coverage across the repository. By the end of the week, Aria was live in production.</p><p>By this point, I had wrapped up the mandatory training that should be done by week three, keeping myself on track with onboarding courses and checklist items. I also spent time exploring what future iterations of the frontend could look like, having early discussions with Product Design and gathering feedback from the team on what they’d want to see next.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fhOQ0zuKil3VeRF-8Mbxpw.png" /></figure><p><em>The Aria frontend — live</em></p><h3>What This Taught Me About Lyft</h3><p>There is a line often attributed to physicist Richard Feynman: “What I cannot create, I do not understand.” The principle applies not just to physics but to any complex system. You can read every architecture document, study every infrastructure guide, and attend every onboarding session, but still not truly understand how a production system breathes until you have pushed something through it yourself. The auth plugin that wouldn’t cooperate, the Envoy session config quietly breaking the connection, the streaming chunks that were just a bit too big — none of that was in any onboarding deck. You only learn it by hitting it.</p><p>Three things stood out to me about Lyft’s engineering culture during this process.</p><p><strong>The internal tooling is mature.</strong> Lyft Node Service, Envoy, metaservice, InfraDocs — these are not aspirational frameworks. They are well-maintained, well-documented systems that a new engineer can genuinely build on top of from week one. The “getting started” guides actually get you started.</p><p><strong>Cross-team support is authentic.</strong> Staff engineers from other teams answered my questions, reviewed context, and helped me debug issues that were outside their immediate scope. It reflected a culture where helping someone outside your immediate team is a natural part of how engineering operates at Lyft.</p><p><strong>The documentation culture is real.</strong> The High-Level Design document that guided this project, the frontend platform docs, the infrastructure references — they were genuine working documents that I used daily, not artifacts gathering dust in a wiki.</p><h3>What’s Next for Aria</h3><p>The frontend is just the beginning. The roadmap ahead includes data visualization with charts for richer analytical responses, enhanced chat history management, archived chat retrieval, and custom rendering components for specialized data displays. Each of these builds on the infrastructure that is now in place: the deployment pipeline, the state management patterns, the streaming architecture.</p><p>Aria is already helping Lyft employees get answers to data questions faster than ever. The next phase is about making those answers richer and more actionable.</p><h3>Closing Thought</h3><p>The best way to understand a production system is to ship something through it. Three weeks, one new service, and a much deeper understanding of Lyft’s engineering ecosystem than any course alone could have provided. I’m grateful to Ravi for designing an onboarding that trusted me with real impact from day one, to Alex for being an exceptional onboarding buddy, to Glen for pointing me to the right references when I was stuck, and to every engineer across Lyft who answered a Slack message from the new person who had yet another question about Envoy configuration.</p><p>When onboarding gives someone a real challenge, points them at the right channels, and trusts both the technical and human systems to support them, the result isn’t just a shipped project — it’s a new hire who actually understands how the company works.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5c5643c192d9" width="1" height="1" alt=""><hr><p><a href="https://eng.lyft.com/from-day-1-to-production-building-lyfts-analytics-rides-intelligence-assistant-as-onboarding-5c5643c192d9">From Day 1 to Production: Building Lyft’s Analytics &amp; Rides Intelligence Assistant as Onboarding…</a> was originally published in <a href="https://eng.lyft.com">Lyft Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Metric Semantic Layer: How Lyft Governs and Scales Key Data Definitions]]></title>
            <link>https://eng.lyft.com/metric-semantic-layer-how-lyft-governs-and-scales-key-data-definitions-56bee3643c29?source=rss----25cd379abb8---4</link>
            <guid isPermaLink="false">https://medium.com/p/56bee3643c29</guid>
            <category><![CDATA[metric-standardization]]></category>
            <category><![CDATA[data-governance]]></category>
            <category><![CDATA[knowledge-base]]></category>
            <category><![CDATA[semantic-layer]]></category>
            <category><![CDATA[data]]></category>
            <dc:creator><![CDATA[Iraklikhorguani]]></dc:creator>
            <pubDate>Wed, 10 Jun 2026 18:42:08 GMT</pubDate>
            <atom:updated>2026-06-10T18:42:06.796Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9r1bGT0StHEZaLhFgNhI7A.png" /></figure><p><em>Written by </em><a href="https://www.linkedin.com/in/rohit-channe-5368b469/"><em>Rohit Channe</em></a><strong> </strong><em>and </em><a href="https://www.linkedin.com/in/simranmirchandani/"><em>Simran Mirchandani</em></a> <em>at Lyft.</em></p><h3>Motivation</h3><p>At Lyft, data isn’t just a resource — it’s woven into everything we do. Metrics drive key forecasts, steer operational decisions, and put our boldest hypotheses to the test. But as Lyft scaled, products launched and evolved, and team members came and went, we found ourselves at risk of different teams using different definitions for a given metric. What did “Metric ABC” actually mean? The answer often depended on the context and application of the team you asked.</p><p>The consequences were predictable. Without centralized version control or a shared standard, outdated metric definitions crept into decision-making.</p><p>Our solution was to build an internal <strong>Metric Semantic Layer (MSL)</strong>: a centralized repository that serves as a single, authoritative home for every metric’s definition — providing both a clear, plain-English description and the definitive SQL code. No more hunting across codebases or tribal knowledge — just one place to store and access a standardized, agreed-upon definition. With MSL, we have <strong>a single source of truth</strong> — consistent terminology and assumptions across every team, so everyone is genuinely speaking the same language. We achieve this through three key principles:</p><ol><li><strong>Simplified onboarding and change management</strong> — update a metric definition once, and the change automatically and frictionlessly flows through every downstream application that depends on it</li><li><strong>Intentional governance </strong>— clarified ownership, defined scope, clear accountability for data quality, and a structure resilient enough to survive org changes, team rotations, and attrition</li><li><strong>Transparency and accessibility</strong> — definitions are easy for both technical and non-technical users (and downstream applications) to find and integrate into day-to-day workflows</li></ol><h3>Solution</h3><p>Taking the above principles into account, we <strong>implemented the Metrics Semantic Layer as a Python package</strong>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*UewSVL3b5Tnn8vJMyo9tyA.png" /></figure><h4><strong>1 — Simplified onboarding and change management through flexible metric configurations</strong></h4><p>To codify the business logic and metric definitions along with supporting metadata attributes, we used YAML configurations. YAML representation provides both flexibility in — and readability of — the metric definition. The core definition is stored in the SQL jinja templates, which gives us optionality to tailor the final SQL based on time granularity and dimensions. Jinja templates were specifically chosen to make sure the SQL blocks are not redundantly declared across multiple definitions (<a href="https://en.wikipedia.org/wiki/Don%27t_repeat_yourself">DRY</a> methodology) and also have a lower learning curve compared to some of the other templating frameworks.</p><pre>YAML example<br>operational_owner: rides_table_owners<br>data_sources: <br>  - rides_table<br>view: |<br>  SELECT DATE_TRUNC(&#39;{{time_granularity}}&#39;,{{time_attribute}}) AS {{time_granularity}},<br>  {{select_dimensions(dimensions)}},<br>  {{select_metrics(metrics, include_ratio_parts)}}<br>    FROM rides_table<br>   WHERE {{time_attribute}} BETWEEN DATE(&#39;{{start_date}}&#39;) AND DATE(&#39;{{end_date}}&#39;)<br>  {% macro groupby_dimensions %}<br>time_attribute: ds<br>time_granularity:<br> - day<br> - week<br> - month<br>dimensions:<br>  - dimension_name: region<br>    dimension_expression: ride_region<br>    dimension_definition: Ride region<br><br>metric:<br>  - metric_name: rides_completed<br>    metric_description: Rides with drop off completed<br>    metric_type: Additive<br>    business_owner: rides_owner<br>    aggregated_expression: 1.0*count_if(is_dropped_off)</pre><h4><strong>2 — Intentional governance to enable scale</strong></h4><p>In order for the package to continue to be effective in the long term, we had to define structure around ongoing governance. We have done this via two primary mechanisms:</p><ul><li><strong>Selection criteria</strong>: What metrics (and when relevant, dimensions) warrant being included in the package?</li><li><strong>Ownership model</strong>: How do we make sure metrics remain up to date? Who bears the responsibility of maintaining the definitions?</li></ul><p><strong>Selection Criteria</strong></p><p>Not all metrics are created equal. There are some that are unique to one team, whereas others are of interest to people across the company. This latter group is considered “Golden Metrics”. These are metrics that have at least two distinct use cases or applications. These are the metrics that are allowed to be onboarded onto the package; anything too niche creates more overhead than it’s worth. For example, Rides Completed is actively monitored via key dashboards and also leveraged in operational ML models.</p><p><strong>Ownership Model</strong></p><p>All Golden Metrics are required to have a Business Owner <strong>and</strong> an Operational Owner. An owner should never be an individual — but rather a team — in order to be as resilient as possible.</p><p><strong>Business Owner</strong>: They are responsible for <em>metric</em> health.</p><ul><li><strong>Who</strong>? <br>- A Data Analyst or Data Scientist with deep understanding of the business, market conditions, and normal levels of volatility.</li><li><strong>What</strong>? <br>- Provide (both written and technical SQL) definitions of the metric.<br>- Maintain the definition in the package by verifying the existing definition or updating it on a regular, pre-defined cadence.<br>- Define the limitations and scalability of the metrics (e.g. allowed dimensions and time granularities). Gather approval from stakeholders.</li></ul><p><strong>Operational Owner</strong>: They are responsible for <em>data</em> health.</p><ul><li><strong>Who</strong>? <br>- A Data Engineer or Data Scientist with experience creating/supporting ETL pipelines.</li><li><strong>What</strong>?<br>- Ensure quality of the underlying data tables used by providing on-call support, creating and maintaining data quality (DQ) checks, and running backfills to fix issues if needed.<br>- Communicate delays or issues with the production of the metric, changes to the underlying assumptions/logic, and data changes due to backfills.</li></ul><p>All changes to a metric’s definition must be approved by both the Business and Operational Owners. This requirement serves two key purposes:</p><ol><li><strong>Informed and Coordinated Action</strong>: It ensures all relevant parties are aware of the change and can make necessary, reactive adjustments. For example, the Operational Owner might need to establish new DQ checks following a business definition update to maintain trust in a metric.</li><li><strong>Accountability and Quality Control</strong>: This shared responsibility utilizes the distinct knowledge of each party. One owner may be aware of critical nuances that the other is not, thereby preventing the adoption of a suboptimal change.</li></ol><h4><strong>3 — Achieving transparency through access methods and APIs</strong></h4><p>The configurations are now exposed through Python-based methods and APIs. This module helps users explore what metrics are defined, fetch their metadata and, most importantly, get or execute the SQL definition. This also serves as an interface for integration with analytics dashboards, data quality tools, data orchestration (Airflow), ML models/workflows, and AI agents/tools. The Python package is versioned and updates are deployed periodically to all dependent applications through automated code refactors so that they receive the latest updates.</p><pre>### Input<br>from metrics_source import get_sql<br>print(get_sql(metric_group_name=&#39;rides&#39;, <br>              start_date=&#39;2025-04-01&#39;,<br>              end_date=&#39;2025-05-01&#39;,<br>              dimension_names=[&#39;region&#39;],<br>              metric_names=[&#39;rides_completed&#39;],<br>              time_granularity=&#39;week&#39;,))<br><br>### Output<br>SELECT DATE_TRUNC(&#39;week&#39;,ds) as week,<br>ride_region as region,<br>1.0*count_if(is_dropped_off) as rides_completed<br>  FROM rides_table<br> WHERE ds BETWEEN DATE(&#39;2025-04-01&#39;) and DATE(&#39;2025-05-01&#39;)<br> GROUP BY 1,2</pre><h3>Accessing Standardized Metrics</h3><p>Here are some ways we’ve enabled users to access MSL without directly using the Python access functions.</p><h4><strong>Amundsen: Discoverability</strong></h4><p>For users who want to discover, explore and access metric definitions, we integrated with our internal data catalog tool <a href="https://eng.lyft.com/amundsen-lyfts-data-discovery-metadata-engine-62d27254fbb9">Amundsen</a>. This helps them search, navigate to, and access metric definitions, while also surfacing ownership and data quality information.</p><h4><strong>Self-Service Tooling</strong></h4><p>For users who wish to generate and execute a custom metric SQL without writing any code, we have introduced a self-service Metric UI. This tool allows users to select the necessary dimensions and time grains to generate SQL tailored to their specific use case, while preserving the core metric definition constructs and predicates.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UIGeeLyAAvjHUwewXOgR3w.png" /></figure><h4><strong>AI MCP/Skills</strong></h4><p>One of the biggest, and perhaps most unexpected, benefits of structuring our metrics definitions in clean, accessible YAML files — making them entirely code-friendly — is that this configuration serves as a comprehensive knowledge base for AI Agents and Skills.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PIgtzOEVbIG_mjzjOHL93g.png" /></figure><p>We created an MCP (Model Context Protocol) with tools to access the MSL library and lets users ask natural-language questions about metrics, with greater accuracy and fewer hallucinations. Users can integrate the MCP with Skills and Custom AI Agents in their local or remote workflows in Claude and Cursor. It also integrates well with popular BI tools like Hex, making the AI generated outputs from these BI tools more reliable.</p><p>Lastly, there are also guardrails baked in for evaluation of the results from the MCP using evaluation techniques like ground truth and <a href="https://arxiv.org/abs/2411.15594">LLM as a judge</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/982/1*WadDvEdTQER1lavRRN1Lbw.png" /></figure><h3>Next Steps</h3><p>The homegrown solution MSL has served as a great foundation for making sure key business applications are aligned to the latest metric definitions. Beyond that, it brought rigor in metric change management. However, as we expand our scope and seek to integrate with other third-party applications (BI tools and Databases), we are exploring options for a vendor managed solution. This will naturally extend the Metric Semantic Layer to AI-driven analysis. The deterministic structure and construct of the metric configuration and accessibility tools serve as baseline context and knowledge base for generating AI-assisted business insights we are building at Lyft — more to come on this in a follow-up blog post!</p><h3>Acknowledgements</h3><p>This work would not have been possible without contribution from many teams across the company. We want to especially shout out the following people (in no particular order) for their foundational thinking and architecture contributions: <a href="mailto:cwormsley@lyft.com">Caitlin Wormsley</a> <a href="mailto:achan@lyft.com">Andrew Chan</a> <a href="mailto:santiagom@lyft.com">Santiago Martinez Vara Ortiz</a> <a href="mailto:vzherebkin@lyft.com">Vladyslav Zherebkin</a> <a href="mailto:yxie@lyft.com">Yunyi Xie</a> <a href="mailto:karmes@lyft.com">Kristen Armes</a> <a href="mailto:dmittow@lyft.com">Devon Mittow</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=56bee3643c29" width="1" height="1" alt=""><hr><p><a href="https://eng.lyft.com/metric-semantic-layer-how-lyft-governs-and-scales-key-data-definitions-56bee3643c29">Metric Semantic Layer: How Lyft Governs and Scales Key Data Definitions</a> was originally published in <a href="https://eng.lyft.com">Lyft Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[From Chaos to Clarity: How We Built a Unified, Self-Routing Support Ops Ticketing System at Lyft]]></title>
            <link>https://eng.lyft.com/from-chaos-to-clarity-2ab50856d0c8?source=rss----25cd379abb8---4</link>
            <guid isPermaLink="false">https://medium.com/p/2ab50856d0c8</guid>
            <category><![CDATA[infrastructure]]></category>
            <category><![CDATA[automation]]></category>
            <category><![CDATA[ticketing]]></category>
            <category><![CDATA[analytics]]></category>
            <category><![CDATA[operations-management]]></category>
            <dc:creator><![CDATA[Atulgupta]]></dc:creator>
            <pubDate>Tue, 09 Jun 2026 17:30:51 GMT</pubDate>
            <atom:updated>2026-06-10T14:00:53.216Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iaQQvhuHELlD_RApdsQ45Q.png" /></figure><p><em>Written by </em><a href="http://www.linkedin.com/in/atul-gupta-ga"><em>Atul Gupta</em></a><em>, Analytics Manager — LUS Support Ops, </em><a href="https://www.lyft.com/"><em>Lyft</em></a></p><p>At Lyft, getting operators and riders connected quickly and reliably depends on more than technology — it depends on the teams working behind the scenes to keep that technology running smoothly. For the operators managing Lyft’s fleet across markets, having fast, reliable access to support is what keeps bikes on the road, stations stocked, and issues resolved before they affect riders. Building the infrastructure that makes that support possible is what our team does; this is the story of how we built it.</p><p>When I first joined Lyft Urban Solutions’ (LUS) Support Ops team in 2020, ticketing processes for our operators were still being established. There was no reliable way to raise issues, track progress, or get routed to the right person. We had a Jira Help Center, but it had become increasingly difficult to navigate.</p><p>What followed was a five-year journey of transforming that chaos into a streamlined, automated, self-routing system that now handles thousands of tickets per year, with one third of those routed automatically — saving hours of manual triage work annually.</p><h3>The Problem: Organic Growth Gone Wrong</h3><p>On the surface, a Jira Help Center sounds like a reasonable solution. In practice, ours had become a maze. Here’s what we were dealing with:</p><ul><li>Duplicate intake forms doing the same job under different names</li><li>Redundant categories with no clear ownership</li><li>Forms that didn’t capture the right information upfront, forcing follow-up back-and-forth</li><li>No auto-labeling, so tickets couldn’t be searched or reported on</li><li>No routing logic — every ticket needed a human to read it and manually assign it</li><li>No dashboards — leadership had zero visibility into ticket volume or trends</li></ul><p>With this system, tickets would’ve taken the team 10–15 minutes to manually triage. It was time to redesign the system.</p><figure><img alt="Figure 1: Before and after: four fragmented portals consolidated into one unified Help Center with multiple categories across multiple markets" src="https://cdn-images-1.medium.com/max/1024/1*9xKQyN1kaey7J2ELlI7pwg.png" /><figcaption><em>Figure 1: Before and after: four fragmented portals consolidated into one unified Help Center with multiple categories across multiple markets</em></figcaption></figure><h3>Phase 1: Restructuring</h3><p>Before building anything new, we had to tear down what wasn’t working. I started by auditing every intake form in the portal. I found forms that hadn’t been used in years sitting alongside forms that overlapped almost entirely in purpose.</p><p>Cleanup started in one of our Help Centers, we consolidated redundant intake forms into smarter, dynamic forms. But this was only the beginning. Over time, the Help Center grew far beyond operation support. Today it serves multiple categories spanning all Lyft Urban Solutions operated market teams, which includes operations, field support, data, marketing, engineering and more. The key tool that made this scalable was Jira’s Proforma, a form builder that lets you create conditional branching logic within a single intake form, capturing the right information for each category without overwhelming the submitter.</p><p>Instead of having separate forms for each issue category, we created a single dynamic form with an “Issue Type” dropdown. Depending on what the operator selected, different fields would appear and only ask for information relevant to that issue type. This same pattern was then applied across all categories as the Help Center expanded.</p><p>This eliminated confusion immediately. Operators no longer had to decide which of three similar-looking forms to use. There was one form and it asked the right questions.</p><h3>Phase 2: Automating</h3><p>With cleaner forms in place, we could now build the intelligence layer on top. Jira automations became the backbone of this system.</p><p>Automatic ticket routing was the first win. Using the data captured in our Proforma fields, we set up automation rules that would:</p><ul><li>Auto-assign tickets to the right team member based on issue type and sub-category</li><li>Add the correct label (from our 30+ label taxonomy) the moment a ticket was created</li><li>Send automated acknowledgment replies to reporters so they knew their ticket was received</li><li>Trigger PagerDuty alerts for P0 incidents, routing to the correct on-call engineer based on the specific incident type selected in the form</li></ul><p>The PagerDuty integration deserves a special mention. Previously, when a P0 blocker occurred — for example, the app going down for 5+ operators — the reporter would submit a ticket, a human would read it, identify which engineer was on call for that issue type, and then page them. That chain of manual steps added critical minutes during an active incident.</p><p>Now, when an operator selects “P0 Operating Software Incident” and specifies the incident type, the system automatically pages the right on-call engineer. There are no humans needed for an initial escalation.</p><p>Additionally, we implemented an automation that comments on and automatically closes a ticket if the user creates a P0 ticket utilizing a standard form or subsequently attempts to elevate that existing form to a blocker priority. This was essential because a regular ticket with blocker priority does not trigger an alert to an on-call engineer.</p><p>A significant portion of tickets per year now route automatically. At 10–15 minutes saved per ticket, multiple hours of manual triage work are eliminated annually.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*I6JXhPD_xMQ5Cksf7xYt6Q.png" /><figcaption><em>Figure 2: An example automation rule showing the trigger, conditional logic, and auto-assign action — the building blocks of our self-routing system</em></figcaption></figure><h3>Phase 3: Consolidating</h3><p>As my role at Lyft evolved — from Support Contributor to Platform Support Lead to Analytics Manager — so did the system’s scope. We were asked to absorb ticketing workflows from other teams, each of whom maintained their own separate portals. From an operator’s perspective, this was a nightmare as they had to navigate multiple portals.</p><p>My goal was to make our Help Center the single entry point for all LUS operated market support requests, regardless of which backend team actually owned the work.</p><p>The technical challenge: each team’s work lived in a different Jira project. We couldn’t just redirect tickets because the work still needed to live in the right project for each team’s reporting and workflows.</p><p>The solution was a combination of ticket cloning and auto-linking automation:</p><ul><li>Operator submits a ticket through our unified portal</li><li>An automation detects the issue type and determines which project it belongs to</li><li>A clone of the ticket is automatically created in the correct downstream project</li><li>Both tickets are automatically linked to each other</li><li>The original ticket is updated with an automated comment pointing the reporter to the downstream ticket for future updates</li></ul><p>This meant operators only needed to know one place to go. The routing to the right team happened invisibly and automatically.</p><p>We also deprecated the three other Help Centers, reducing operator confusion and eliminating the maintenance burden of keeping four separate portals up to date.</p><h3>Building Visibility with Jira Structures</h3><p>The final piece was dashboards. Visibility into ticket data was limited — ticket volumes, team workload, and trends were difficult to track across the system. Our team couldn’t easily track which label categories were spiking or which markets were generating the most issues.</p><p>Standard Jira dashboard widgets are limited. They handle basic counts and simple filters, but they can’t do things like show a hierarchical breakdown of tickets by label category, or display cross-project rollups in a single view.</p><p>Jira Structures gave us the ability to build those advanced views. We created:</p><ul><li>An operational overview dashboard showing real-time ticket distribution across teams and categories</li><li>A P0 incident tracker with SLA monitoring</li><li>A cross-project rollup showing all tickets regardless of which downstream project they lived in</li><li>A quality tracker showing tickets missing required labels, which helped enforce labeling discipline across the team</li></ul><p>But we didn’t stop there. For deeper analytics — the kind needed to power Monthly Business Reviews and spot longer-term trends — we built an additional reporting layer by connecting our Help Center ticket data to Mode via a Jira ETL pipeline. This unlocked richer querying and custom visualizations that weren’t possible with native widgets, including breakdowns by market, category-level volume trends over time, and SLA performance across teams. This all lived in one place and was accessible to leadership without them needing to log into Jira at all.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OGvpdxCt3w7-yofQWJu20A.png" /><figcaption><em>Figure 3: Month-over-month percentage change in ticket volumes across projects, showing system stability and the impact of the Help Center consolidation over time</em></figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fcOuY3y3kMz1pK4N8aDYew.png" /><figcaption><em>Figure 4: Ticket volume breakdown by request and issue type, showing distribution across categories (top) and month-over-month trends from Jan 2024 to Nov 2025 (bottom)</em></figcaption></figure><h3>What We Learned</h3><p>Four lessons from this build that might be useful if you’re tackling something similar:</p><ol><li><strong>Design for the submitter, not the triage team. </strong>The instinct when building intake forms is to ask for everything that would be useful for an investigation. Resist this. Operators filling out forms are often in the middle of an operational situation — they want to submit quickly and get help. Ask only what’s truly necessary to route and resolve the ticket.</li><li><strong>Automate the boring triage, not the nuanced triage. </strong>Jira automations are excellent at routing “if issue type = X, assign to person Y” style decisions. However, they’re not a replacement for human judgment on complex, multi-symptom issues. The goal is to free up your team for the work that actually requires expertise.</li><li><strong>Labels are a data model. </strong>When we built our label taxonomy with 30+ labels across business areas, we were really building a reporting schema. Treat labeling as seriously as you’d treat database schema design, because the quality of your reporting entirely depends on it.</li><li><strong>Consolidation is a forcing function for alignment. </strong>Merging four Help Centers into one required us to have conversations with four different teams about ownership, priorities, and workflows. Those conversations would have been valuable regardless — the consolidation project just gave us a reason to have them.</li></ol><h3>The Results</h3><p>Over five years, the team consolidated four Help Centers into one, built a 30+ label classification system, and created a system that now serves multiple operated LUS markets. A significant portion of all incoming tickets route automatically with no manual triage needed.</p><p>Beyond metrics, this system changed how our team operates. We spend less time as ticket routers and more time as analysts. The operators we serve get faster responses and clearer ownership. We improved leadership visibility into ticket data so they also understand where support demand is growing.</p><p>If you’re building or inheriting a support ops system and dealing with the same kind of organic sprawl we had, I hope this gives you a playbook. All the tools: Jira Proforma, automations, Structures are accessible. The hard part is the design thinking that has to come before you touch any of it.</p><h3>What’s Next: Migrating to Atlassian Cloud</h3><p>This story doesn’t end here. Later this year, Lyft is migrating from Jira Server to Jira Cloud — and that migration brings a new chapter of challenges and opportunities for everything we’ve built.</p><p>What we’re currently navigating:</p><p><strong>Jira Structures is being deprecated.</strong> The advanced dashboarding layer we built — the cross-project rollups, the SLA monitors, the label health trackers — was powered by it, and we’re retiring it internally as part of the Cloud migration. Every dashboard we’ve built needs to be rebuilt from scratch.</p><p><strong>The Mode ETL pipeline is going away.</strong> Beyond Jira Structures, we also built a set of deeper analytical dashboards by piping Jira ticket data into Mode via an ETL pipeline. This allowed us to run richer queries, build custom visualizations, and power our Monthly Business Reviews with live ticket data — things that weren’t possible with native Jira widgets alone. The Jira-to-Mode ETL pipeline will be discontinued post-migration. We’re exploring cloud-native analytics tools. We’ll be rebuilding our reporting suite there — which is both a challenge and an opportunity to consolidate everything into one tool rather than maintaining two separate dashboarding layers.</p><p><strong>PagerDuty integration needs reconfiguration. </strong>The PagerDuty services we set up to auto-page on-call engineers for P0 incidents don’t migrate automatically. We’ll need to manually reconfigure every integration in the new Cloud environment.</p><p><strong>API and bot updates are required. </strong>Any Jira bots and integrations — including our Boomi integration — will need to be updated to use Jira Cloud’s new REST API v3. This is a meaningful engineering effort that needs to be coordinated carefully to minimize downtime.</p><p><strong>Proforma intake forms need validation. </strong>The dynamic intake forms at the heart of our routing system are built on Proforma, which does exist in Jira Cloud — but all forms will need to be reviewed and tested to confirm they migrate cleanly and all conditional logic survives the transition.</p><p>Atlassian Analytics is more powerful than Jira Structures for reporting purposes. The forced rebuild is a chance to design dashboards that are better than what we had.</p><p>The migration is scheduled for later this year. The same principles that guided the original build — design intentionally, automate the boring work, consolidate rather than sprawl — will guide the rebuild.</p><p>After five years, the system looks nothing like what it started as and that’s the point. A good team keeps finding ways to make things better.</p><p><em>Lyft is hiring! If you’re passionate about building scalable support systems, automating operational workflows, and analytics that drive real decisions — visit </em><a href="https://www.lyft.com/careers"><em>Lyft Careers</em></a><em> to see our open roles.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2ab50856d0c8" width="1" height="1" alt=""><hr><p><a href="https://eng.lyft.com/from-chaos-to-clarity-2ab50856d0c8">From Chaos to Clarity: How We Built a Unified, Self-Routing Support Ops Ticketing System at Lyft</a> was originally published in <a href="https://eng.lyft.com">Lyft Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How We Built a Smarter Pickup Experience for Gated Communities]]></title>
            <link>https://eng.lyft.com/how-we-built-a-smarter-pickup-experience-for-gated-communities-47416e9df029?source=rss----25cd379abb8---4</link>
            <guid isPermaLink="false">https://medium.com/p/47416e9df029</guid>
            <category><![CDATA[mapping]]></category>
            <category><![CDATA[lyft]]></category>
            <category><![CDATA[marketplaces]]></category>
            <category><![CDATA[rideshare]]></category>
            <dc:creator><![CDATA[winnieyan]]></dc:creator>
            <pubDate>Thu, 23 Apr 2026 19:16:44 GMT</pubDate>
            <atom:updated>2026-04-23T19:17:14.179Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cxVOjt0GnB5tRXbNktweOA.jpeg" /></figure><p>If you live in a gated community, you’ve been there: You request a ride from your apartment complex, expect your driver to come to you as usual, and then — your driver’s car icon just stops right at the front gate. You watch helplessly as the ETA ticks up. A chat message comes in: <em>“Hey, how do I get in?”</em> You scramble to remember the gate code. They try it. It doesn’t work. You end up meeting them awkwardly on the sidewalk outside while your coffee gets cold — <strong>a pickup journey frustrating for both you and your driver.</strong></p><figure><img alt="An example gated community in real life, Photo by Bingqian Li on Pexels" src="https://cdn-images-1.medium.com/max/1024/1*Pp8iEDacju32V13VKgsOHg.jpeg" /><figcaption>An example gated community in real life, Photo by Bingqian Li on Pexels</figcaption></figure><p>It turns out you’re not alone: Gated community pickups can make up 25–30% of Lyft rides in selected markets. For a long time, our app offered no special guidance in these situations. Riders would drop their pin inside the gates (fair enough — that’s where they <em>are</em>), while drivers would pull up to a locked entrance with no way in, leaving both parties to sort things out over chat. The result was predictable: <strong>more cancellations, longer waits, and a lot of unnecessary stress for our customers</strong>.</p><p>The Lyft Mapping team decided it was time to fix this properly — not with a band-aid, but a new end-to-end experience. Here’s how we did it.</p><h3>What Was Actually Going Wrong?</h3><p>We looked through gated ride examples, zoomed into our metrics data, and found two root causes behind most of the friction.</p><p>The first was an <strong>inflexible selection of pickup spots</strong>. Our app would suggest pickup spots near a rider’s location — which, for riders inside a gated community, often means <em>inside the gate</em>. But our data told a different story: many riders actually preferred meeting their driver right outside the gate, knowing their driver couldn’t access the property. The app wasn’t giving them that option clearly.</p><p>The second was a <strong>communication black hole</strong>. Even riders who <em>knew</em> how to get their driver through the gate had no good way to pass along access instructions in advance. Instead, they’d wait until the driver was already idling at the entrance before firing off a text with the gate code, setting off a frantic last-minute back-and-forth.</p><p>Two problems, two different moments in the ride flow. This signaled that we needed to fix things both <em>before</em> and <em>after</em> a ride was requested.</p><h3>Only on Lyft Maps</h3><p><strong>Lyft’s Mapping team sits in a unique position to solve this problem</strong>, with workstreams across map data, pickup spot recommendations, routing, and the rider and driver app experience.</p><p>This one needed all of the pieces to complete the map (pun intended).</p><h3>Piece 1: Drawing Gated Communities on the Map</h3><p>Before anything else, we needed a map that actually <em>knew</em> about gated communities — where they are, where their entrances are, and how many gates they have.</p><p><strong>Our Map Data team built a gate area shape generation algorithm</strong> to do exactly this. Gated communities come in all shapes and sizes: some are small apartment complexes with one entrance, and some are large developments with multiple gates and even their own internal road networks. The algorithm had to handle all of it reliably and provided the foundation accurate enough to build our new pickup experience for riders within these gated communities.</p><p><em>How does the app even know I’m inside a gated community?</em> When you open the app, we cross-reference your GPS location against the gate area shapes we generated. If you’re within a known gated community boundary, the app quietly switches into “gates mode” and adjusts what you see as your pickup spot selection process. No extra steps on your end.</p><p><em>Is my community in the system?</em> We’ve built our gate coverage from sources like OpenStreetMap and driver feedback, and we’re continuously monitoring for gaps and adding new communities during every map update — so coverage grows over time without you having to do a thing. That said, no map is perfect, and local knowledge is invaluable. If you don’t see the new experience or encounter issues at your gate, reaching out to Lyft support helps us prioritize the communities that need attention most.</p><h3>Piece 2: Giving Riders a Smarter Selection of Choices</h3><p>Now, when you open the Lyft app from inside a gated community, you’ll see pickup spots at your complex entrances (labeled “I’ll walk outside the gate”), right alongside the usual spots near your current location (labeled “Pick me up inside gate”). You can decide freely whether you prefer to wait locally (maybe it’s raining outside) or want to meet your driver outside the gate (get some steps in, right?).</p><figure><img alt="The new pickup spot selection process for riders in gated community" src="https://cdn-images-1.medium.com/max/800/1*ijs537jdaX1t6EJSAVybtA.gif" /><figcaption>The new pickup spot selection process for riders in gated community</figcaption></figure><p><strong>The team looked at historical ride patterns to surface spots that real riders have actually used near gates</strong> — so we’re not just pointing you toward the gate in theory, we’re pointing you to where riders in your community <em>actually</em> go.</p><figure><img alt="Pickup location heatmap of an example gated community" src="https://cdn-images-1.medium.com/max/467/1*bu616vMnvIaTPlze3Vs8pg.png" /><figcaption>Pickup location heatmap of an example gated community</figcaption></figure><p>We also designed the new UI to feel familiar, borrowing patterns from our existing Venues pickup flow so our riders wouldn’t have to learn something from scratch.</p><p>Before shipping, we ran a controlled experiment to make sure we weren’t accidentally making things worse — specifically, that the new flow wouldn’t add enough friction to make riders bail on requesting a ride altogether. Good news: no meaningful drop-off. Even better news: we saw actual pickups happening at the spots that riders chose initially, meaning that they no longer have to walk around to find their drivers.</p><h3>Piece 3: Routing Drivers to the Right Gate</h3><p>Recommending gate-aware pickup spots to a rider is only half the job. The driver also needs to be <em>routed there</em> correctly.</p><p>Normally, a pickup route is simple — get the driver from point A (their location) to point B (the rider’s pickup spot). <strong>For gated communities, our Routing team introduced a new “detour”: the gate itself becomes an intermediate stop</strong>, an arbitrary, “invisible” stop that the driver passes through on the way to the rider. We use our map data to direct drivers to the <em>right</em> gate, accounting for road direction and the most logical approach path.</p><p>This seemingly small routing tweak creates something valuable: a precise moment in the driver’s journey where we can surface gate instructions at exactly the right time.</p><h3>Pieces 4: Actually Getting the Driver Through the Gate</h3><p>Now that we get drivers <em>to</em> the gate, we still need to get them <em>through</em> the gate to meet those who asked to be picked up inside, and we really can’t let our customers resort to back-and-forth communications.</p><p><strong>Our Map Experience team brought this to life</strong>: After a rider is matched with their driver, they’re prompted to share access information. Rather than presenting a blank text field — which puts all the pressure on the rider to know exactly what to say, right after they’ve just requested a ride — <strong>our Design team drew inspiration from the intercom panels riders use every day</strong>: a familiar numpad for gate codes, and a short list of plain-language options for everything else. The goal was to make instruction sharing effortless.</p><p>We leveraged historical data from gate instructions that riders had previously shared to identify the most common access scenarios. <strong>Our Content team worked to make each option feel conversational and specific enough to be useful</strong>, without being so wordy that riders would skip past them. The free-text fallback is still there for the edge cases, but in practice, most riders find what they need in the list.</p><figure><img alt="The new gate instructions sharing feature" src="https://cdn-images-1.medium.com/max/236/1*1l-RXL-PfkkYaPsoNeNctw.png" /><figcaption>The new gate instructions sharing feature</figcaption></figure><figure><img alt="The new gate instructions sharing feature — Sharing gate codes" src="https://cdn-images-1.medium.com/max/236/1*tjUIMB7QaP0er7qP91XwlA.png" /><figcaption>Sharing gate codes</figcaption></figure><figure><img alt="The new gate instructions sharing feature — Sharing special gate instructions" src="https://cdn-images-1.medium.com/max/236/1*ZSxPFZ7mLxoOT-vbgmCD5w.png" /><figcaption>Sharing special gate instructions</figcaption></figure><p>You might be thinking: <em>I usually text my driver the gate code — what’s actually different here?</em> <strong>The difference is timing and reliability.</strong> When you text, you’re reacting: the driver might be busy driving, or is already at the gate, already stopped, already waiting.</p><p>We also paid special attention to how we should show the shared instructions to drivers. They are likely navigating with eyes on the road — so a paragraph of gate instructions on their screen at the wrong moment is unhelpful and more importantly, unsafe. <strong>We designed the gate instructions to surface as a small, but obvious banner in the navigation screen at precisely the moment the driver approaches the gates</strong>, timed to the routing waypoint we added in Piece 3. The information is brief and scannable — just enough to get through the gate. No extra steps or unnecessary calls or texts.</p><figure><img alt="Gate instructions shared to drivers during navigation" src="https://cdn-images-1.medium.com/max/233/1*DqImu0uPvhvpAA4yjRxgAg.png" /><figcaption>Gate instructions shared to drivers during navigation</figcaption></figure><p>We also know that sharing a gate code feels more sensitive than sharing a pickup pin — it’s access to where you live, <strong>and your privacy is important to us. </strong>So we built in privacy controls that let you view or delete your instructions at any time after sharing. Gate codes are never stored between trips, and only your matched driver ever sees them on their navigation screen on their way to pick you up. To further ensure your gate codes stay protected, we implemented safeguards to prevent screenshots.</p><figure><img alt="Privacy protection for riders to remove shared gate details" src="https://cdn-images-1.medium.com/max/223/1*WAyZHz_ED1a-kXJEeeYZTw.png" /><figcaption>Privacy protection for riders to remove shared gate details</figcaption></figure><h3>Does It Actually Work?</h3><p>We asked riders directly through our optional feedback survey. <strong>About 95% responded positively</strong> — and their comments said it better than any metric could:</p><p><em>“Driver successfully picked me up OUTSIDE the gate, as requested.”<br></em> <em>“Picked me up where I asked.”</em></p><p>We also see data reflecting our goals: riders and drivers are not giving up on a ride just because there’s a gate in the way. Drivers are not looping around a community entrance trying to figure out how to get in. Riders are not wandering the sidewalk looking for a car that’s stuck on the other side of a barrier. Phones are not fired off with chats and calls because our riders and drivers can’t find each other. <strong>Those are the moments we were trying to eliminate — and the numbers reflect it.</strong></p><p>We observe lower cancellation rates from both riders and drivers, when riders share gate instructions.</p><figure><img alt="Rider cancellation rate (after driver match) for gated community rides" src="https://cdn-images-1.medium.com/max/1024/1*5doZWavnvAujBNb0ht8H1A.png" /><figcaption>Rider cancellation rate (after driver match) for gated community rides</figcaption></figure><figure><img alt="Driver cancellation rate (after driver match) for gated community rides" src="https://cdn-images-1.medium.com/max/1024/1*_Z4XJy-qi_YtrvZfIZu6aA.png" /><figcaption>Driver cancellation rate (after driver match) for gated community rides</figcaption></figure><p>In addition, riders are walking less to their final pickup locations from their requested pins, waiting less, and don’t need to reach out to their drivers as much. We also see drivers needing to change course in the final stretch less. Each of these represents a ride that went smoothly instead of sideways.</p><figure><img alt="Graph on distance between pin and actual pickup location for gated community rides" src="https://cdn-images-1.medium.com/max/1024/1*Us4IyuqjdztCFvoply0sDw.png" /><figcaption>Distance between pin and actual pickup location for gated community rides</figcaption></figure><figure><img alt="% of gated community rides with wait longer than 5 minutes" src="https://cdn-images-1.medium.com/max/1024/1*XMH8nvbqX9c7Rtwm_rJYaw.png" /><figcaption>% of gated community rides with wait longer than 5 minutes</figcaption></figure><p>Now, that same ride from your gated community looks a little different: your driver’s icon moves smoothly past the gate with the instruction you share with a simple tap, and you get into your ride stress-free, with your coffee warm. <strong>The results gave us confidence that the approach works — and got us thinking about where else we could take it.</strong></p><h3>More Gates, and The Bigger Picture: Bringing the Real World to the Map</h3><p>For gated communities specifically, we’re just getting started. We want to <strong>provide riders with the option to save non-sensitive gate instructions</strong> <strong>for future trips, </strong>so riders who regularly get picked up from the same community don’t have to re-enter the same information every time. Your gate intercom instruction isn’t going to change — we shouldn’t make you tell us every time. We will also work on <strong>smarter gate selection</strong> for communities with multiple entrances, choosing the best gate based on where both you and your driver are at that moment, not just the nearest one on the map.</p><p>But the more we worked on this project, the more we realized gates are just one example of a much bigger problem. <strong>The real world is full of physical constraints that affect where a driver can go and where a rider can safely and conveniently be picked up — and we see a big opportunity of leveraging our Mapping stack to improve customers’ pickup experiences across many of these situations.</strong></p><p>Consider road closures. A street gets blocked for construction, a parade, or a marathon, and suddenly the route our app plotted doesn’t work anymore. The driver gets rerouted at the last second, the pickup pin is now on the wrong side of a barrier, and rider and driver end up circling each other trying to figure out where to actually meet.</p><p>Or consider unsafe road segments — a pickup spot that looks perfectly fine on a map but sits on a high-speed tunnel exit way with an accident-prone history. The rider waits on the curb, and the driver pulls over in a spot, attempting to avoid moving traffic. Neither of them had any idea there was a safer option just half a block away.</p><p>What the gated community project gave us is a repeatable playbook for all of these situations: <strong>encode the real-world constraint into the map, surface that context in pickup spot recommendations, thread it through routing, and deliver the right information at the right moment and place in the app</strong>. Whether the obstacle is a gate, a closed road, or an unsafe stretch of curb, the same approach applies.</p><p>Gated communities were a specific problem. But the approach behind solving them isn’t specific at all. <strong>Every time the real world gets in the way of a safe and convenient pickup, there’s an opportunity to make the map smarter — and the ride a little less stressful for everyone involved.</strong></p><p><em>Lyft is hiring! If you’re passionate about maps, visit </em><a href="https://www.lyft.com/careers"><em>Lyft Careers</em></a><em> to see our openings.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=47416e9df029" width="1" height="1" alt=""><hr><p><a href="https://eng.lyft.com/how-we-built-a-smarter-pickup-experience-for-gated-communities-47416e9df029">How We Built a Smarter Pickup Experience for Gated Communities</a> was originally published in <a href="https://eng.lyft.com">Lyft Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Predicting Rider Conversion in Sparse Data Environments with Bayesian Trees]]></title>
            <link>https://eng.lyft.com/predicting-rider-conversion-in-sparse-data-environments-with-bayesian-trees-07227ff92789?source=rss----25cd379abb8---4</link>
            <guid isPermaLink="false">https://medium.com/p/07227ff92789</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[ridesharing]]></category>
            <category><![CDATA[marketplaces]]></category>
            <category><![CDATA[statistics]]></category>
            <dc:creator><![CDATA[Zammit Alban]]></dc:creator>
            <pubDate>Mon, 30 Mar 2026 14:43:41 GMT</pubDate>
            <atom:updated>2026-03-30T14:43:39.644Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OMZl9ZxHl02QWvOMcaUaCQ.jpeg" /></figure><p>At Lyft, understanding how riders go through our user experience is fundamental to operating a healthy marketplace. Specifically, it is important to have a robust model determining if a rider will actually request a ride after entering a destination and viewing a price and ETA. Accurately predicting this decision, that we call <em>conversion</em>, informs countless decisions across our platform. Whether it is to better balance supply and demand, improve user experiences, optimize recommendations and advertisement, understand long-term engagement, decide how to distribute coupons… rider conversion prediction is a central challenge for the Lyft business.</p><p>However, predicting human behavior at scale is incredibly complex, and the exact same person might well open the app just to check current availability or actually to request a ride after viewing our prices. The contexts under which riders make their conversion decisions are extremely diverse and almost unique to each session. A user’s intent changes based on where they are and where they want to go, what time it is, their previous interactions with the platform, current supply-demand market conditions, to cite a few.</p><p>When we try to model this using standard machine learning approaches, we run into a significant challenge: <strong>data sparsity</strong>.</p><h3>The Challenge of High Cardinality and Sparsity</h3><p>To accurately predict conversion, we need to slice our data very thinly across many categorical features. Imagine trying to predict the conversion probability for a business traveler leaving the suburbs of Detroit at 4:00 AM on a Tuesday to catch their flight at the airport 30 minutes away. While Lyft has vast amounts of data overall, the amount of data available for that specific intersection of contexts often reveals to be very tiny. Maybe we only have ten examples in history.</p><p>If we use standard techniques like <a href="https://en.wikipedia.org/wiki/Gradient_boosting">Gradient Boosted Trees</a> (e.g., LGBM, XGBoost), we encounter severe overfitting. A standard model looking at 10 examples in the training data where 8 converted might confidently predict an 80% conversion probability. But that’s likely statistical noise. The next 10 examples might show only 2 conversions.</p><p>One could argue that much more complex models, such as Deep Neural Networks or even Large Language Models in a few-shot fashion, could be able to handle this sparsity if properly tuned and calibrated; but this comes at the cost of greatly increased inference time. Indeed, predicting session conversion is very often needed live, while the rider interacts with our systems, and we want to be able to confidently predict their conversion as soon as we know their destination. And of course, no one wants to see a loading wheel rotating for hours, so when you think about the user experience, we need a model that can handle an extremely high-cardinality set of different situations, provide robust and accurate predictions even when data is scarce, and serve those predictions with ultra-low latency.</p><h3>Overcome Sparsity with Bayesian Trees</h3><p>To solve this, our team developed a modeling framework to predict sessions’ conversions in real-time. At its core, it is a highly optimized, hierarchical lookup structure designed to handle sparse categorical data by leveraging Bayesian theory. It incorporates priors (knowledge gained from broader contexts) to smooth predictions in specific, data-rare contexts.</p><h4>The Hierarchical Tree Structure</h4><p>We organize the training data into a tree structure based on a hierarchy of partition keys. These keys define the context. While the exact keys we use depend on the specific application, imagine a hierarchy like this:</p><ul><li>Root: All the sessions</li><li>Level 1 Split (e.g., Spatial Context): City Region</li><li>Level 2 Split (e.g., Temporal Context): Time of Day / Day of Week</li><li>Level 3 Split (e.g., Congestion Context): Current local supply-demand balance</li><li>…</li></ul><p>As we move down the tree, the context becomes more and more specific, and the data available at each node becomes sparser and sparser.</p><figure><img alt="Figure 1 — Hierarchical decomposition of rider sessions. The tree structure illustrates how the model splits global data into progressively granular segments, from City Region down to specific Temporal and Congestion contexts, to address data sparsity. (Image generated with Gemini’s Nano Banana model)" src="https://cdn-images-1.medium.com/max/1024/1*wx3nbQJF2UJDE3roAyTAKA.png" /><figcaption><em>Figure 1 — Hierarchical decomposition of rider sessions. The tree structure illustrates how the model splits global data into progressively granular segments, from City Region down to specific Temporal and Congestion contexts, to address data sparsity. (Image generated with Gemini’s Nano Banana model)</em></figcaption></figure><h4>The Secret: Bayesian Smoothing with Gaussian Priors</h4><p>How do we deal with a leaf node that only has five data points? We definitely cannot trust only the 5 data points and train an independent model, but also we cannot have a global model that trains on the whole data, as it would miss the specifics of this leaf node that guide rider conversion decision-making.</p><p>Our conversion model uses the statistical properties of the parent node to inform the prediction at the child node. This is achieved through <strong>Bayesian smoothing</strong>, specifically utilizing gaussian priors on model parameters. It allows data-sparse segments (e.g. a leaf node in the tree) to default to the robust average of their parent group.</p><p>Here is the intuition:</p><ol><li><strong>Model architecture: </strong>Each node in the tree is a copy of the same parametric model. It can be a logistic regression, a support vector machine, or even a shallow neural network. To make a prediction y from an input x, you rely on a parametric model f that depends on trainable parameters Θ such that y = f(x, Θ). Each node has its own version of Θ that we want to be as specific as possible to the segment the node represents.</li><li><strong>Training</strong>: The tree is trained top-down, starting from the root of the tree, level by level, until the leaves, with less and less data as we dive deeper and deeper. For each individual node, its parent node has been trained on more data. Hence Θ_parent provides a strong belief about what Θ_child should look like when training the child node. As such, when fitting the child’s model on the child’s specific data, we add in the training a L2 penalization for diverging too far from the parent’s, that is ||Θ_parent — Θ_child||². This is known as a Gaussian Bayesian prior, centered at the parent’s mean.</li></ol><figure><img alt="Equation 1 — Example of the training loss used to train a child node’s parameters Θ_child using its parent parameter Θ_parent via a L2 regularization of strength λ" src="https://cdn-images-1.medium.com/max/1024/1*lpezD3yNpON6G6FIUKb4AQ.png" /><figcaption><em>Equation 1 — Example of the training loss used to train a child node’s parameters Θ_child using its parent parameter Θ_parent via a L2 regularization of strength </em>λ</figcaption></figure><p>The choice of the regularization strength λ is critical and should depend on the data size available between the parent’s node and its children, such that the model automatically balances parent’s trust with specification to child’s data. If a leaf node has very little data, its prediction will be heavily pulled towards the parent’s stable mean. As the leaf node acquires more data, the model gradually trusts the local data more, moving the prediction away from the parent’s prior and towards the local raw average.</p><figure><img alt="Figure 2 — How Bayesian smoothing prevents overfitting. If a parent model shows conversions peaking at 5-mile trips, but a sparse child node indicates a 2–3-mile peak, relying solely on the child’s data causes irrational overfitting. Bayesian smoothing creates a compromise by blending the local signal with the parent’s prior, ensuring robust predictions even when local data is scarce." src="https://cdn-images-1.medium.com/max/990/1*DrMVDdCFSpCXH6ZnXwGz2A.png" /><figcaption><em>Figure 2 — Bayesian Smoothing in Action. Imagine we are at a specific node in the tree where the parent model (black line) tells us that, for the parent’s segment of sessions, rider conversion peaks around 5-mile trips. Then, for a sparser segment one level below in the tree, i.e. for one of the parent’s child nodes, we observe in the child’s specific training data that shorter 2–3-mile trips actually convert more. Standard Overfitting (purple dashed line): If we relied solely on this child’s data, the child model would overfit, irrationally predicting near-zero conversion for 5-mile trips. Bayesian Smoothing (pink line): The smoothed model finds a compromise. It respects the local signal by shifting the peak a bit to the right, but also relies on the parent’s prior to maintain a good probability mass around 5 miles, ensuring robust predictions even where local data is missing.</em></figcaption></figure><h3>Ensuring Behavioral Consistency</h3><p>Beyond handling sparsity, we often have domain knowledge about how conversion should behave relative to certain continuous variables. For instance, consider the rider’s historical conversion rate: intuitively, as this increases, the predicted conversion probability for the current session should also increase. All else being equal, we should logically predict the conversion of a rider with a 90% historical rate higher than the one for another rider with a 40% historical rate. The model should enforce a monotonic relationship between historical conversion and current session’s conversion.</p><p>However, standard ML models can sometimes learn erratic shapes that violate this intuitive logic due to noise in the training data, ending up with non monotonic relationships between some input and the output that shouldn’t be. With Bayesian trees, we can keep very simple parametric models at each node because the use case is already very specific (that is the whole point of Bayesian trees, right?). Using simpler models<em>— </em>e.g. a logistic regression cvr = σ(Θ . cvr_hist) <em>—</em> can give much more control on the model monotonicity (e.g. enforce Θ&gt;0) and ensure appropriate behavior of the model, and as such better explainability. This guarantees that the model’s outputs are not only statistically robust but also directionally intuitive, reliable and interpretable.</p><figure><img alt="Figure 3 — Bayesian priors to enforce logical constraints, like monotonicity. For instance, a rider with a higher historical conversion rate should logically have a higher current conversion probability. While standard models overfit to noisy data, creating jagged, illogical curves; the Bayesian model enforces a monotonic constraint. This ignores noise and produces a smooth, logically consistent prediction that aligns with business logic." src="https://cdn-images-1.medium.com/max/989/1*r5apu5JDsMmvtttHoUAMkg.png" /><figcaption><em>Figure 3 — Enforcing Logical Constraints with Bayesian Priors. Imagine we are analyzing a specific feature used to predict a session’s conversion, such as the “Rider’s Historical Conversion Rate.” Business Logic dictates that this relationship should be monotonic: a rider who has converted frequently in the past should generally be more likely to convert now, not less. Standard Overfitting (purple dashed line): A standard model blindly chases the sparse noisy data (blue dots), creating a jagged curve that irrationally becomes non-monotonic with the Rider’s Historical Conversion Rate. Monotonically-constrained Model (pink line): By enforcing a monotonic constraint on the relationship between input and output, the Bayesian model produces a smooth, logically consistent prediction. It ignores the noise that suggests a dip, ensuring that a higher historical rate always translates to a higher predicted probability, aligning with our domain knowledge.</em></figcaption></figure><h3>Conclusion</h3><p>Bayesian Conversion models represent a significant step forward in our ability to model rider conversion decision-making in highly dynamic, sparse environments. By combining a highly structured hierarchical approach with the robust statistical grounding of Bayesian smoothing, we can generate accurate, stable predictions where traditional models fail. This architecture allows us to be hyper-local and specific when the data supports it, while gracefully falling back to broader, stable trends when faced with the unknown. It’s a key piece of infrastructure that helps Lyft make smarter decisions in real-time.</p><p><em>Lyft is hiring! If you’re passionate about developing state of the art machine learning/optimization models or building the infrastructure that powers them, read more about them on our blog and </em><a href="https://www.lyft.com/careers"><em>join our team</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=07227ff92789" width="1" height="1" alt=""><hr><p><a href="https://eng.lyft.com/predicting-rider-conversion-in-sparse-data-environments-with-bayesian-trees-07227ff92789">Predicting Rider Conversion in Sparse Data Environments with Bayesian Trees</a> was originally published in <a href="https://eng.lyft.com">Lyft Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Beyond A/B Testing: Using Surrogacy and Region-Splits to Measure Long-Term Effects in Marketplaces]]></title>
            <link>https://eng.lyft.com/beyond-a-b-testing-using-surrogacy-and-region-splits-to-measure-long-term-effects-in-marketplaces-9cb06d628f2d?source=rss----25cd379abb8---4</link>
            <guid isPermaLink="false">https://medium.com/p/9cb06d628f2d</guid>
            <category><![CDATA[marketplaces]]></category>
            <dc:creator><![CDATA[Iraklikhorguani]]></dc:creator>
            <pubDate>Wed, 25 Mar 2026 13:56:39 GMT</pubDate>
            <atom:updated>2026-07-21T14:37:10.699Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iVC-FQekAWZXMelyqruN5g.png" /><figcaption>Image generated with Gemini 3 Pro (Google), 2026.</figcaption></figure><p><em>Written by </em><a href="https://www.linkedin.com/in/amber-yuehui-wang-b283b0aa/"><em>Amber Wang</em></a> <em>and </em><a href="https://www.linkedin.com/in/yoonji-kim-48731b239/"><em>Y</em>oonji Kim</a> <em>at Lyft.</em></p><h3>Background</h3><p>Whenever you use the Lyft app, there is a complex balancing act happening behind the scenes. Various levers are used to keep the marketplace running smoothly; Base prices and coupons for riders affect demand, while driver pay and bonuses impact the level of available supply. Since every change to prices and payments impacts Lyft’s costs and revenue, they lead to key optimization problems, such as:</p><ul><li><em>How should we allocate budget between driver incentives and rider incentives?</em></li><li><em>How do we invest resources to achieve x% rides growth, and how much does it cost in terms of short term profit?</em></li></ul><p>These are the questions the Foundational Models team at Lyft tries to answer in a systematic way. A key ingredient is understanding the effects of different types of investments — for instance, <em>what will happen if we increase the total budget for driver incentives by x%? What will happen if we increase the rider price of all rides by y%?</em> It’s worth noting that the long term effects of such decisions tend to dominate the short term effects: we may earn more short term profit from a ride if we charge riders more and pay drivers less, but lose riders and drivers in the long run.</p><p>Estimating the long term effects of resource allocation decisions is challenging in a multi-sided marketplace such as Lyft. Because these decisions tend to be consequential, their effects go beyond first order effects on directly affected users. For example, if we increase driver incentive spending by x% in week 1, drivers will drive more in week 1 (short term effect), and may return to drive a bit more in the following weeks (direct long term effects). But this is not the full picture: in week 1, when there is a positive increase in driver hours as the result of more incentives, riders will enjoy better experiences (e.g. less surge pricing, shorter wait times) and may want to return to Lyft in the future. However, more driver hours in the market also mean the average driver is less busy, and that idleness may discourage them from driving in the future. An analog holds for any decision that boosts short term demand — riders are more likely to encounter bad experiences such as higher surge prices and longer wait times, whereas drivers may become busier and earn a bit more. At Lyft, we refer to such indirect effects as the “market-mediated effects.” Compared to the direct long term effects, these market-mediated long term effects are much harder to estimate.</p><p>Below is an illustration of the causal relationships we consider for the policy change “increase driver incentive spend.”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PREa8hWRI4ABqpCCxg8aig.png" /></figure><h3>Summary of our solution</h3><p>This article introduces the methodology framework we use to estimate the “market-mediated long term effects,” the more challenging component of the long-term effects. In short, we use a two-step approach that can be thought of as a surrogacy approach in its broad sense. Under the assumption that the “market-mediated” outcomes are fully mediated through negative user experiences, (1) we first estimate how our policy changes affect the distribution of core negative user experiences, then (2) estimate how negative user experiences affect users’ future behavior.</p><p>Both steps are done with observational causal inference, allowing fast and inexpensive updating. Verification is another major challenge. There is no single form of experiment that can provide a perfect verification; therefore, we have to combine multiple imperfect signals. Specifically, the first step methodology can be verified using a switch-back experiment, and the second step of estimation can in principle be verified using user-split experiments.</p><p>As the last step, we combine the direct long term effects (estimated separately) and the market-mediate long term effects and use region-split experiments to verify the overall long term effects. In general, region-split experiments suffer from poor pre-intervention fit and low power; we developed a forward selection algorithm to optimize experiment design by picking the treated and control regions.</p><p>Below, we describe our methodology in detail.</p><h3>Step 1: from decisions to negative user experiences</h3><p>We first start with a question: <em>when we move a policy decision today (e.g., raise rider price water-level or change driver incentive budgets), how does the short-term user experience shift?</em></p><p><strong>What we model</strong>: <em>How does a policy change affect short-term user experiences?</em> We focus on a set of negative user experiences that matter most downstream — such as long waits, high surge, and cancellations for riders, or lower hourly earnings, idleness and incentive earnings for drivers — and maintain the critical assumption that these are the only channels through which today’s decisions affect the future (by affecting today’s market). The goal of this step is to estimate the effect of our policy decisions on the distribution of these negative user experiences.</p><p>The challenge is that negative user experiences are highly cyclical and seasonal, varying with time‑of‑week, holidays, weather, and shifting supply/demand. A naive regression would blend these rhythms with the true effect of our decisions. Therefore, our approach:</p><ul><li>Residualizes predictable time-of-week patterns so we compare like-with-like, which mimics a “random shock” by capturing the deviation from this period’s normal status.</li><li>Controls for remaining market information such as supply and demand so the coefficient of our policy decision reflects its incremental impact on negative user experience.</li></ul><p><strong>How we estimate </strong>(observational, residualized regressions)<strong>:</strong> Conceptually, for any negative user experience metric (e.g., wait time), we fit a residualized model on deviations from the market’s own baseline:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dDlbSBg2EK8v-cbJ6meK0w.png" /></figure><p>Below is a visualization of a simulated example where we have one contextual variable. With residualized terms, we can capture how policy change would marginally change negative experience (green solid line).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Rmmh0X8przGlbV0um4wZjA.png" /></figure><p>The coefficient beta policy tells us how much negative user experience moves when a policy decision (like rider price water‑level or incentive spend) deviates from its usual level, after accounting for typical time‑of‑week rhythms and market conditions. Because we measure everything as “deviation from normal,” the effects read like elasticities around everyday operating conditions. For example, a higher‑than‑usual price suppresses demand and hence is associated with a predictable decrease in negative rider experience (e.g. high surge, long wait time), holding everything else constant. The output is a calibrated response function that turns a policy change into a forecasted shift in the distribution of negative user experience, not just its average, with uncertainty to reflect real‑world variability.</p><p><strong>How we validate: </strong>To validate this mapping, we use switch‑back experiments that alternate policy settings across comparable time slots and compare the model’s predicted changes in negative user experience to the experimental lifts we observe. The experiment either verifies our model or informs its iteration (e.g., changing controls or introducing additional ones).</p><h3>Step 2: from negative user experiences to future outcomes</h3><p>The next question is: <em>given a change in a certain negative user experience today, how do future outcomes move (e.g., future rides, retention, driver hours)?</em> In the broader surrogacy framing, Step 1 captures short‑term shifts in experience, and Step 2 translates those shifts into long‑term behavior under the assumption that long‑term market-mediated effects are completely mediated by short‑term negative user experiences.</p><p><strong>What we model</strong>: The long term effects of negative user experiences. We treat these negative experiences as exposures that vary naturally across users, times, and places, and estimate their impact on future outcomes — such as future rides — while controlling for confounders (time, location, and rider history).</p><p><strong>How we estimate</strong> (observational, double‑robust): We use Augmented Inverse Probability Weighting (AIPW, <a href="https://academic.oup.com/ectj/article/21/1/C1/5056401">Chernozhukov et.al, 2021)</a>, a doubly robust causal estimator combining (i) a propensity model for exposure (the likelihood of facing a given level of negative user experience, given context) and (ii) outcome models for future metrics, conditional on confounders. This yields average treatment effects for negative user experience. We summarize the mapping via a “surrogacy index” that quantifies how much short-term negative experiences will affect long‑term outcomes; this is the scaling we use to move from short‑term exposure to negative experiences to long‑term impact.</p><p><strong>How we validate</strong>: We run user‑split experiments that perturb negative experiences and compare the model’s predicted changes in future outcomes to the experimental lifts, checking calibration (predicted vs. observed) for validation.</p><p><strong>How it works together with Step 1</strong>: Step 1 converts a policy change into a shift in the distribution of negative user experiences; Step 2 converts that exposure shift into forecasted changes in future outcomes (e.g., future rides). Together, via the surrogacy index, they provide a causal link from today’s decision to long‑term business impact.</p><h3>Step 3: verify overall LTE by region-split experiments</h3><p>With the market‑mediated effect from Steps 1–2, we combine the direct long term effect (estimated separately) using a transparent formula grounded in market mechanics. This yields a single policy‑level forecast for long‑run rides and financials that reflects both mediated and direct channels.</p><p>To validate end‑to‑end predictions, we run region‑split experiments. We developed a forward‑selection algorithm, inspired by the forward difference‑in‑differences (FDiD, <a href="https://pubsonline.informs.org/doi/10.1287/mksc.2022.0212">Li, 2024</a>) approach, to choose treated and control regions: starting from a single treated region, we iteratively add treated regions that best improve pre‑period fit and expected power. The region-split experiments allow us to observe the overall long-term effects of a policy intervention to an entire market.</p><p>Below is a simulated example (using simulated data) of a region-split experiment. We find a set of treated regions where we can find another set of control regions that mimic the average behavior of treated regions. Once we inject an intervention shock on the treated regions (e.g., increase of incentives), we track the discrepancy between the average rides of treated and control regions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xsMU4a9HfE_7aVrq39skSw.png" /></figure><h3>Conclusion</h3><p>We presented a framework to connect today’s resource decisions to long‑run marketplace impact. First, we translate policy changes into shifts in negative user experience using residualized modeling and verify those short‑term responses with switch‑back experiments. Second, we map negative user experience to future outcomes with doubly‑robust observational inference (AIPW) and a surrogacy index, then validate with user‑split experiments. Finally, we combine the market‑mediated and direct long-term effects and validate end‑to‑end predictions via region‑split experiments, using a forward‑selection design to choose treated and control regions.</p><p><strong>What this enables</strong>:</p><ul><li>Scenario planning and policy evaluation.</li><li>Budget allocation across levers (pricing, incentives) informed by the short-term profit v.s. long-term rides Pareto Frontier.</li><li>Continuous calibration as markets evolve, grounded in experimentally-verified observational causal inference.</li></ul><p>The result is a model‑based, experiment-verified causal engine: decisions move user experience, user experience moves future behavior, and the composition yields long‑term business impact. This enables fully-informed decisions on resource allocation.</p><p><em>Lyft is hiring! If you’re passionate about Data Science, visit </em><a href="https://www.lyft.com/careers"><em>Lyft Careers</em></a><em> to see our openings.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9cb06d628f2d" width="1" height="1" alt=""><hr><p><a href="https://eng.lyft.com/beyond-a-b-testing-using-surrogacy-and-region-splits-to-measure-long-term-effects-in-marketplaces-9cb06d628f2d">Beyond A/B Testing: Using Surrogacy and Region-Splits to Measure Long-Term Effects in Marketplaces</a> was originally published in <a href="https://eng.lyft.com">Lyft Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>