The 5-Year Data Payoff: Why Clearcover Keeps Building on Prefect's Pythonic Foundation

Industry: Auto Insurtech

Use Case: Data Pipeline Orchestration, ELT, and Business Process Automation

Key Outcomes:

Sustained self-service for five years: A Python-first platform allows a small team to manage all critical ETL/ELT pipelines within secure, governed environments.
Engineered a 5-minute rollback: Prefect and dbt enable a resilient warehouse swap architecture, ensuring fast data recovery.
Automated the front office: Authorized CX agents trigger Prefect flows via Slack for instant self-service document generation within Clearcover's internal access and governmental controls.

Meet Clearcover

Clearcover is a modern auto insurtech company founded in 2016 with a mission to make insurance simpler and more technology-centric than traditional carriers. The company operates in nearly 20 states, focusing on private passenger auto insurance. Serving certain segments of both the standard and non-standard auto markets, they have been expanding to offer more coverage options for drivers who often have a harder time securing coverage.

For a technology-driven company, data is everything. To run everything from pricing models to customer-facing services, their data architecture has to be robust, agile, secure, compliant, and accessible.

Richard Wilson, Manager of Data Engineering, Analytics Engineering, and Data Analytics, of Clearcover, oversees the data platform. Wilson leads a lean, single squad of engineers who handle all of Clearcover's core data infrastructure, giving them the agility of a small startup without requiring a large engineering footprint.

The Five-Year Test: Why Accessibility Endures

When Clearcover first adopted Prefect over five years ago, they were fleeing a common orchestration headache. The team had started with AWS Step Functions and ECS Fargate, but found that its reliance on AWS-specific knowledge created a frustrating, high-friction environment for analysts.

"Initially, the engineering team at Clearcover found success orchestrating their data pipelines with a combination of AWS Step Functions and AWS ECS Fargate, but quickly encountered the high barrier to entry for non-AWS professionals," according to the 2020 case study.

Looking back at the initial evaluation, the team recognized they had to prioritize simplicity, while maintaining compliance and governance controls, above all else.

As the 2020 case study noted, "It was clear that accessibility was a critical success metric for any tool under consideration. We found Prefect to be way more accessible than Dagster."

Five years later, that focus on simplicity still sets the standard for how the team works. Wilson's team needed a platform that wouldn't slow down new hires. He confirms the ease of adoption:

"I have enjoyed Prefect. I think the learning curve is a little less steep on it, especially if you have somebody coming in that knows Python," says Wilson.

Prefect as the Core Data Engine

Clearcover embraced Prefect's flexibility from the start, integrating it deeply into their existing AWS environment. They even contributed specialized open-source integrations, like the popular DbtShellTask, to create a custom, self-service CI/CD process for all data work.

Today, Wilson's team maintains this resilient system using a classic ELT model:

Ingestion: They use approximately 50-75 independent Prefect flows to pull raw data from approved and vetted external vendor sources (databases, SFTP servers, APIs) into their Snowflake data lake.

Transformation: One large flow is responsible for cleaning, standardizing, and transforming that raw data into their analytics layer.

"We use Prefect for the kind of raw ETL jobs where we grab it from a source. We don't really transform it, and we just place it into our data lake the way it is... Then, we allow dbt (also orchestrated by Prefect) to do all of that transformation for us in that one large flow."

Innovative Use Cases

The longevity of the partnership is best demonstrated by how Clearcover leverages Prefect for two of its most critical and most creative use cases.

Engineering the 5-Minute Rollback

New data is staged and verified in an isolated environment first. If the new load fails, the system can instantly revert to the last known healthy state.

"What this does for us is support," explains Wilson. "If the production environment behaves unexpectedly, we can trigger a Prefect flow that reverts the schema pointers back to the verified state. We can restore service in approximately five minutes, rather than scrambling for hours to fix a broken pipeline."

This resilient architecture is a huge win for business continuity and also keeps infrastructure costs low.

"The benefit of having Prefect handle our orchestration system is that it allows us to do some of this custom functionality... but it can also keep us from being more inclined to need to purchase dbt cloud. It allows us to run off of their free versions."

Automating the Front Office via Slack

What was once a manual request for a simple "letter of experience" is now an instantaneous, self-service action. Authorized agents can trigger actions right in their Slack channel, bypassing engineering friction entirely while operating within Clearcover's internal security and compliance standards.

"Being able to trigger from Slack was a huge unlock for us. That's been kind of our big unlock in the last year," says Wilson.

The team relies on the same webhook capability for production monitoring:

"We use Prefect's custom webhooks for a lot," Wilson explains. "They allow us to push notifications into Slack and to integrate with a paging service, which we use for our production alerts." These integrations operate within Clearcover's internal security and compliance standards, ensuring that automated workflows maintain the same governance controls and oversight as manually executed processes.

Key Takeaways

Clearcover's decision to anchor its data platform to a flexible, Python-based orchestrator has delivered value for half a decade. That initial investment in accessibility and customization now allows a small team to maintain complex, resilient infrastructure, ensuring data reliability (with a five-minute rollback) while extending powerful automation to non-technical business units.

For any team considering an orchestration tool, Wilson's advice is: "It's easy to find people who can program in Python and using Prefect is just another extension of Python."

Disclaimer:

The information presented in this document is provided for general informational purposes only and reflects Clearcover's use of Prefect and related technologies as of the date of publication. Statements describing Clearcover's systems, processes, or technology stack are illustrative and may not represent the full scope of the company's operations or future plans. Clearcover makes no warranties, express or implied, regarding the accuracy, completeness, or continued applicability of the information contained herein. Any references to third-party products or services (including Prefect, dbt, Slack, or others) are for descriptive purposes only and do not constitute endorsement or partnership beyond their documented integration or usage. All trademarks and product names are the property of their respective owners.