Customer: Foundry
Industry: Cryptocurrency infrastructure
Use Case: Data pipeline orchestration
Key Outcomes:
Foundry supports large-scale institutional Bitcoin mining. Its flagship product, the Foundry USA Pool™, helps clients consolidate hash rate and increase revenue from mined blocks. Foundry also builds software products that support hardware fleet management and mining firmware, as well as provides site operations services to institutional miners.
Behind the scenes, a lean but critical data team powers internal reporting and internal customer dashboards. Siva Nadesan, Director of Data, leads the platform and analytics efforts. Josh Caskie, a data engineer on the team, helped lead the evaluation and implementation of Prefect.
“We’re focused on institutional clients only. These are major operations, not hobbyists, with huge fleets of miners and serious operational needs.” — Siva Nadesan, Director of Data
For years, Foundry's data team relied on GitLab CI for scheduling and orchestration. While effective at allowing the team to get basic pipelines running with low development time, it lacked reliable scheduling and success/failure monitoring. This left the team firefighting issues manually in a disconnected system where each pipeline lived in its own repo, further reducing visibility into jobs.
“At the time, we lacked basic retry logic. If a job failed, you had to go run it manually. It was ad hoc every time.” — Josh Caskie, Data Engineer
Schedules defined in GitLab CI weren’t reliable due to shared runner contention and job start times drifted. More importantly, the system wasn’t built to handle parameterized runs or recover from failures with context.
“The data quality issues were real. And because we didn’t have the right tooling, the team couldn’t be accountable. We couldn’t say confidently what failed and when.” — Siva
As usage grew and expectations rose, the cost of broken workflows and invisible failures became too high to ignore. They needed orchestration that could keep up, not just with their pipelines, but with the business they were supporting.
Siva had experience with orchestration tools going back to the mainframe era. He’d previously used CA7, AutoSys, Control-M, Airflow, and Astronomer, but didn’t want to automatically default to old tools. He wanted a modern solution that fit the team’s Python-based workflows and wouldn’t force a rebuild.
Josh ran a structured evaluation of Prefect against other leading orchestrator platforms. He ran POCs, reviewed architecture, and compared costs.
“We needed something easy for engineers to pick up, where we wouldn’t be locked into a whole new paradigm. We didn’t have time to train the whole team for six months just to get workflows running, which ruled out many options.” — Josh
It was soon discovered that other tools wouldn’t be a fit. Their opinionated modeling approach would have required the team to rewrite their existing pipelines, which wasn’t feasible. Pricing models based on cloud credits made it difficult to forecast costs reliably, and, most critically, the team didn’t have time for a long ramp-up period. They needed something their engineers could use productively right away, not a tool that would take months to adopt.
“The more we dug into other tools, the more it became clear that they weren’t the right fit. For our team specifically, we were looking at around six months at least to get every repo migrated, which didn’t work for us, even if we ended up loving the end result.” — Josh
In contrast, Prefect offered a better fit across the board.
Foundry runs three Prefect workspaces: development, stage, and production. Everything is managed through Terraform, including workspaces, work pools, and blocks. Most flows are batch jobs moving data between internal systems, S3, and Snowflake. They also use Prefect to trigger dbt models and generate reports.
One of the most critical pipelines Prefect supports is the Operational Dashboard, a key operational tool used by Foundry’s business development and product teams. The dashboard consolidates performance metrics, helping to streamline information the team relies on for daily conversations and decision-making.
“We ingest and transform data for our dashboards using Prefect and dbt. If someone has a call coming up, they check the Operational Dashboard for the most accurate look at the data.” — Siva
Before Prefect, failures in this pipeline were often hard to detect. Now, alerts are immediate, and the data team can address issues before they affect the business.
Before Prefect, the team spent too much time firefighting silent issues. Now, issues are surfaced immediately, teams are alerted through PagerDuty, and incidents are resolved quickly.
“Now it’s part of our on-call process. Our daily standups include checking Prefect. Everything’s visible.” — Josh
Prefect didn’t just improve observability, it changed the team’s posture. Engineers now have the tools and accountability they need. That shift restored internal trust in data and enabled better business decisions.
“We didn’t have the right tools to make ourselves accountable. Prefect brought that in and helped us continue to build trust with stakeholders.” — Siva
That trust is critical. Foundry’s dashboards inform customer conversations and product strategy. If a stakeholder is preparing for a call, they need to know the data is right. Prefect helps make that possible.
Prefect didn’t just modernize Foundry’s orchestration. It delivered clear, measurable improvements through reliability, speed, and team efficiency:
These changes gave the team back hours of manual effort, improved engineering velocity, and helped restore trust in the data powering internal reports and customer conversations.
Foundry didn’t choose Prefect just to modernize their pipelines, they chose it to build trust, eliminate manual effort, and ensure their platform could scale with the business. Better orchestration gave the data team time back and gave the business data it could count on.
“With Prefect in place, we don’t have to think about orchestration as a problem every day. The platform just works.” — Siva
“We could get our jobs running in production quickly. Prefect wasn’t trying to reinvent how we work. It just fit.” — Josh
