RTR Struts the Data Runway with Prefect

The Stakes

Rent the Runway is a fashion company. It’s a rental company. It’s an e-commerce company and a subscription service. It may be America’s largest dry-cleaner. But at its heart, it’s a data-driven technology company.

Every click, every delivery, every return, every review delivers rich, actionable data, and the data science team puts it to good use, delivering insights to stakeholders across the company who use it in decision-making. Serving over 2.5 million customers and partnering with 750+ designer brands, RTR has disrupted the need to manufacture as many as 1.3 million new garments since 2010. But behind the scenes, they’ve built a unique and interesting technology-driven business, even building their own customer software to drive the reverse logistics of shipping out clothing, then receiving it back, cleaning it, repairing it, and shipping it right back out to a new customer, sometimes as early as the same day.

The fashion industry as a whole has tapped into data science in a big way, collecting and analyzing data about customer preferences and behaviors as well as details about garments, from fabric to size to style, to match people with clothing they love. RTR collects and processes data around fit, cut, fabric, style, color, and more for each garment, as well as specific customer details and preferences.

RTR’s machine learning based fitting technology is one tool they offer users, backed by data science. “Member feedback and machine learning technology give us a leg up—providing next-level insights on what runs large, small, and everything in between. Advanced sizing data help us generate the most accurate size prediction for you, making planning your next shipment a breeze.” By combining millions of data points with customer reviews and the client’s own input regarding fit and style, RTR promises “you’ll have data-backed fit-friendly superpowers on your side.”

Given the critical importance of data to its business model, Rent the Runway constantly tests new technologies “in pursuit of a high-performing tech stack that is fun to work with and serves our customers well.” One of these new technologies is Prefect.

The Tech

“The local UI was a huge benefit during the POC because it allowed me to get immediate feedback on how a flow’s tasks mapped to the DAG that appeared in the UI. I appreciated having that laboratory to help experiment with piecing together tasks to make my flow's DAG look exactly as I expected it to—it really helped me get more comfortable and fluent with Prefect.” - Sean Talia, Senior Engineer

The migration process from their previous orchestration solution to Prefect has largely been a smooth process. “Setting up some of the initial patterns took time and careful thought, but since we’ve become comfortable with those patterns, things have been seamless. Moving some of our more standardized workflows from our old orchestration tool to Prefect now takes barely five minutes!”

“Because our team is small,” says Sr. Director, Machine Learning Emily Bailey, “we have been executing the migration bit by bit over time. We focused first on critical flows where we wanted more stability. Along the way we also halted any new flow development with other tools, to minimize future migrations.” Now, “the team has fully migrated to Prefect, using it to ingest data into the data warehouse and dbt to transform data.”

While the data team at Rent the Runway is “still exploring” the full range of available features in Prefect Cloud, a few highlights stood out:

“The ease of setting up cloud hooks was appealing. We were largely excited to have a fully hosted cloud solution, as opposed to having to stand up and manage an instance of the tool ourselves, which is incredibly valuable for a small team such as ours.” - Sean Talia, Senior Engineer

They also list notifications and state handlers as “huge” impact features. “Having it configured in code lets us use the notifications to assign ownership to the appropriate team.”

Problems Solved

Rent the Runway turned to Prefect because the orchestration tool they were using was problematic in a few ways. In particular, they were disappointed in its inactive open source community, and felt complex dependency management was not workable. Prefect’s strengths in these areas were what first attracted them.

“Reading [Prefect CTO] Chris White’s ‘Why Not Airflow?’ Medium post was influential in our decision to POC Prefect first,”

- Sean Talia, Senior Engineer at Rent the Runway.

“The initial exploration was a collaboration between myself and a colleague who had worked with Airflow (albeit in a limited fashion). We considered one of our workflows with a complicated DAG and tried to get some (simplified version) of the nodes of that DAG up and running in isolation. From there, we could see that Prefect would make it very easy to stitch those nodes together via its orchestration pattern.” -Sean Talia, Senior Engineer

Flexibility

"The flexibility of Prefect made things a little difficult at the beginning. Picking up a new tool is hard when there aren't forums or guides out there for the implementation you're envisioning. Fortunately, the Slack community has been an excellent resource since Day 1.”

-Sean Talia, Senior Engineer

“We really value Prefect's flexibility,” says Bailey, “but it also required greater investment from us to standardize good patterns and avoid potential bad ones. This took time, so analysts and scientists began to think of Prefect as ‘hard to use,’ and that perception became an obstacle in its own right. If I were to do this again I would focus on supporting batches of specific use cases so that people could see immediate value in a big way.”

To battle the flexibility paradox—flexibility is good because it lets you customize your tools to your specific needs, but it can also introduce complications and confusion in the beginning—the Rent the Runway data team turned to Prefect’s robust and active Slack community for help and guidance. They also appreciated the general transparency of Prefect as an open source project with a strong community:

“Insight into the product roadmap helped us understand what patterns we might set up that would be well supported.”

Business Impact

The team’s overall objective was “to implement a more robust workflow orchestration tool that would be able to easily handle the DAGs for some of our more complex workflows, and to speed up developer workflow by adding stability to important scheduled jobs and making more complex workflows easier to implement.”

“A huge win is that we've been able to get a lot of our workflows easily up and running on AWS ECS thanks to Prefect. We also have just been able to dramatically reduce the workload on some of the legacy EC2 instances we were using for our workflow execution, with the ultimate goal of completely doing away with those instances. Prefect will make that goal attainable much more quickly.”

- Sean Talia, Senior Engineer

Already, the team is seeing impact:

“Our team members who have already been fully spun up on Prefect are really excited about it. We’ve created a lot of custom Prefect tasks that have made our workflows much more streamlined, and there’s no doubt that those have reduced implementation time.”

They have bigger plans for the future, hoping to use it for ML and AI to power their fashion recommendation platform and track shipments; today, though, they’re using Prefect to transform and modernize their entire data practice to enable those use cases. “We are still on the ramp-up for more complex workflows training ML models, etc.” says Bailey. But already, “on the data engineering team and for simple data science flows, yes, absolutely” Prefect has provided a reduction in engineering/implementation time.

It’s also helpful in recruiting new talent. “I've been using it as a recruiting tool when interviewing candidates for our team. When I describe to them how we’re using Prefect and what we plan to do with it, they get excited!”

Finally, the team at Rent the Runway says they would recommend Prefect to another data team. “It’s well-suited for mid-size companies with large enough data teams to develop good patterns out of the many possibilities.”