How JOIN Changed How We Approach Data Infra At NAVER

About NAVER

As the data platform team at NAVER, we power the analytics backbone behind South Korea's leading web portal. NAVER supports an ecosystem of over 200 interconnected services, including search, e-commerce, media, and AI-driven applications. With most Koreans relying on our platform and 20+ PB of data accumulated in our Apache Iceberg lakehouse, we handle some of the highest data traffic volumes in the country.

This article was originally published on the StarRocks Engineering blog on Medium by Youngjin Kim, Team Leader, and Moweon Lee, Data Engineer, at NAVER.

Our analytics system has to provide real-time insights, process complex metrics, and scale with growing traffic. It serves two key missions:

1. Service performance monitoring: Making sure our services operate efficiently and consistently meet user expectations.
2. User behavior analysis: Understanding how users interact with our services to drive data-informed decisions.

The system supports internal stakeholders, from engineers optimizing service performance to executives shaping strategy. To do this, we process and analyze large volumes of log data, including user agents, service URLs, and clickstreams, and turn the raw data into metrics people can act on.

The challenge

When we built the first version of our analytics platform, we wanted to move quickly and chose ClickHouse. Its fast aggregated query performance let us deliver results early on. As the platform evolved, however, we ran into significant limitations.

Fixed dimensions

ClickHouse's lack of JOIN support forced us to rely on denormalized tables, which limited users to fixed dimensions and hindered real-time analysis. With numerous data sources and tables, scaling this approach was impractical, so we could serve only a fraction of our data.

Scalability issues

Scaling ClickHouse was another major challenge. Balancing data across nodes required manual intervention, a time-consuming process without automation. As data volumes grew, maintaining that balance became more complex and resource-intensive.

Limited data upserts and deletes

ClickHouse uses merge-on-read to handle real-time mutable data. This can provide data freshness, but it severely degrades performance, which was unacceptable for us. It made many scenarios difficult, if not impossible, to support, and restricted analytics workflows that require mutable data and complex schemas.

As our needs expanded to include dynamic dimensions, raw data queries, and horizontal scaling, these limitations became more apparent, and we needed a more versatile foundation for NAVER's analytics platform.

The solution

Options considered: Trino, Pinot, Druid, StarRocks

We evaluated and benchmarked Trino, Pinot, Druid, and StarRocks against the following criteria:

• Multi-table JOINs: Handling complex queries across multiple tables without relying on denormalization.
• On-the-fly aggregated query performance: Fast, efficient query execution for dynamic, real-time analysis.
• Scaling capabilities: Scaling horizontally with growing data volumes and minimal operational overhead.
• Data upserts: Supporting real-time data upserts without impacting query performance.

After extensive testing, we selected StarRocks for these reasons:

• Multi-table queries out of the box: StarRocks natively supports complex multi-table JOINs, which removes the need for denormalized tables.
• Federated analytics: Integration with Apache Iceberg and other open formats lets us analyze internal and external datasets through one query interface.
• Aggregated query performance: StarRocks matched or exceeded ClickHouse's performance for aggregated queries, even with dynamic workloads.
• Cloud-native scalability: Its decoupled storage-compute architecture simplifies scaling, with data shared across nodes and efficient resource management.

Testing performance and resource allocation

Before migrating, we ran comprehensive tests with our real-world queries and datasets to find the right resource allocation and verify StarRocks' performance. The tests covered multi-column aggregated queries, multi-table JOIN queries, and horizontal scalability, comparing StarRocks with ClickHouse on our 1-hour, 6-hour, 12-hour, 18-hour, and 24-hour data.

Multi-column aggregated queries

The first benchmark focused on multi-column GROUP BY queries. We evaluated StarRocks and ClickHouse in two configurations, small and large clusters on Kubernetes. StarRocks outperformed ClickHouse in both.

JOIN queries

Next, we tested how StarRocks and ClickHouse handle multi-table JOIN operations on the same cluster configurations.

StarRocks completed all JOIN queries with low latency. ClickHouse failed to finish four of the five queries.

Horizontal scalability

Our infrastructure is fully containerized and runs on Kubernetes, so horizontal scalability is crucial for consistent performance and cost control. The chart below compares how StarRocks and ClickHouse behave when scaling in and out:

StarRocks shows linear growth as resources are added, which lets us handle increasing workloads while keeping performance predictable.

Resource allocation

Based on the test results, our StarRocks deployment includes:

• 5 frontend (FE) pods: Query parsing, planning, and coordination.
• 80 backend (BE) pods: Data storage, each with 48 CPUs, 50Gi of RAM, and 10TB of SSD storage.
• 70 stateless compute node (CN) pods: Compute that scales as needed, each with 48 CPUs and 50Gi of RAM, for high query volumes and complex analytics workloads.

Setting up monitoring

Monitoring is critical to the reliability and performance of the platform. We set it up with the pre-configured Grafana templates provided in the StarRocks documentation.

The template includes an installation guide and pre-defined dashboards for metrics such as cluster status, compaction status, and BE and FE status. These let us watch system health and performance in real time, plan maintenance ahead, and resolve issues quickly.

Accelerating queries with materialized views

To improve query performance further, we use StarRocks materialized views. They act as an intermediate cache for the base tables and speed up queries without manual maintenance. Key features include:

• Query rewrite: Queries are automatically rewritten to use materialized views when applicable, which reduces the need to rewrite SQL by hand.
• Auto/scheduled refresh: Views stay up to date with minimal manual intervention.

For the query above, we created a denormalized materialized view to bypass the JOIN, which gave a 6x performance improvement. MVs can be added on demand, and because of StarRocks' query rewrite, the original SQL does not need to change. This lets us tune query performance at any time while keeping our workflows simple.

The results

The move to StarRocks changed our analytics platform in four ways:

• Flexible interactive queries: Engineers can now query raw data directly in SQL, instead of the rigid, pre-defined metrics we had with ClickHouse.
• Faster queries on all data: Multi-table JOINs and multi-column GROUP BY queries run significantly faster, even on datasets with real-time data upserts and deletes.
• Unified query platform: StarRocks provides one query interface across the StarRocks internal catalog for high-performance queries, the Apache Iceberg external catalog for governed external data, and the Apache Hive external catalog for legacy siloed datasets.
• Scalability: Horizontal scaling on Kubernetes gives linear growth in processing power, so the platform handles dynamic and heterogeneous workloads cost-effectively.

What's next

We plan to extend our StarRocks deployment:

1. Performance optimization: Refining partitioning strategies to improve query efficiency, particularly for timestamp-based queries.
2. Community contributions: Contributing our internal improvements back to the open-source StarRocks project.
3. Broader integration: Expanding StarRocks' role to handle more external datasets, using Apache Iceberg for dynamic and governed data queries.