About Fanatics
Fanatics is a global digital sports platform and the official e-commerce partner of all major U.S. sports leagues. The company manages over 900 online stores, serving more than 100 million customers worldwide and offering a product catalog of 14 million products.
Most queries hit Iceberg base tables directly, with data cache keeping hot data blocks local. Async materialized views are added only when base-table performance isn't sufficient.
Key takeaway · Materialized views only where they are needed
Behind the scenes, Fanatics processes around 1 billion events daily from 800+ event types, spread across 120,000+ datasets. Data fuels everything from on-site personalization to internal decision-making.
The challenge
The previous architecture
Fanatics' previous architecture spanned multiple engines and storage layers:
- Data warehouses: Amazon Redshift and Snowflake
- Data lake: S3 queried via Athena (often with >30s latency)
- Real-time layer: Druid, used for real-time dashboards
Where it fell short
This setup resulted in performance bottlenecks, rising costs, and increased operational complexity:
- Interactive queries on Athena were too slow for real-time use cases (>30s median latency).
- Cross-warehouse joins were impractical, forcing teams to maintain redundant data copies.
- Druid couldn't support complex requirements like fact-to-fact joins, primary-key updates, or point lookups.
- Warehouse and data transfer costs were high because of overlapping workloads and fragmented data ownership.
- Governance was inconsistent, and ownership across tools and teams was unclear.
- Forced denormalization: Every piece of data had to be denormalized because of the join limitations of the underlying engines, which created duplicated data across systems and costly preprocessing pipelines.
The solution
Alternatives considered
Fanatics evaluated other OLAP engines, including ClickHouse, but found similar limitations around join capabilities and operational integration. They selected StarRocks for its fast, full-featured query engine, native support for Glue Catalog, and asynchronous materialized views with automatic query rewrite.
The new architecture
Fanatics adopted a lakehouse architecture built on:
- Apache Iceberg as the open table format, with S3 as the source of truth
- StarRocks as the high-performance query and serving layer
- AWS Glue Catalog as the central metadata store
- dbt Core for transformations, with Superset and a Bedrock-based text-to-SQL agent for visualization and access
Ingestion and table design
Fanatics supports several ingestion patterns into StarRocks:
| Source | Path | Notes |
|---|---|---|
| Kafka → Iceberg | Spark batch jobs with external configs | Table maintenance is embedded in ingestion to avoid delete-file overhead on hot partitions |
| Kafka → StarRocks | Routine Load with Kafka Streams pre-processor | Stable at ~100K msgs/sec; ~800 routine loads run on the production cluster with low overhead |
| S3 → StarRocks | External Tables and Broker Load | External tables for Parquet/ORC; Broker Load for CSV and other formats |
Tables are designed with the workload in mind:
- Primary-key tables for fast reads and real-time update support
- Duplicate-key tables for high-throughput append-only data
- Tablet sizes are optimized (~1 GB), and tenant-aware partitions are designed to avoid skew and small files
Performance optimization
To improve query speed and concurrency, Fanatics applies several techniques:
- Async materialized views (AMVs) are used selectively for the hottest workloads, with fine-tuned refresh windows and partition alignment. Automatic query rewrite removes the need for manual SQL rewrites.
- EXPLAIN/ANALYZE is used to validate plan efficiency, pruning, and pushdown.
- Colocate and shuffle join strategies are monitored and tuned to reduce inter-node I/O.
- Multi-stage aggregation is configured where needed to support high-cardinality metrics.
Caching and storage
Fanatics takes a layered approach to performance:
- Most queries hit Iceberg base tables directly, with data cache enabled to keep hot data blocks local.
- Async materialized views are introduced only when base-table performance isn't sufficient.
- StarRocks runs on EC2 with EBS for shared-nothing deployments today, with a roadmap toward shared-data clusters for elastic scaling.
The results
- Apache Druid was eliminated for real-time dashboards, yielding significant savings while enabling joins, PK updates, and point lookups on the same platform.
- Snowflake footprint and cost dropped substantially. As reported, usage was reduced by up to 95% and cost savings were ~90% after consolidation of use cases.
- Ad-hoc queries on Iceberg data now execute 10x faster than in Athena.
- Fanatics now hosts thousands of tables, views, and AMVs on a single StarRocks cluster, with multiple petabytes of Iceberg data accessible through one query layer.
- Dashboards that previously took hours, days, or weeks can now be assembled in minutes, as reported.
What's next
- Broaden AI usage, for example conversational analytics across multiple datasets.
- Wider adoption of Streamlit for lightweight apps on top of the platform.
- More automation, toward "touchless" onboarding of new databases and tables.
- Cluster metadata backup and restore to strengthen disaster recovery.
- Unify BI tooling under StarRocks as the standard serving platform.