RedNoteAnalytics and content search

Scaling Shared-Data Architecture and Hybrid Search at RedNote

RedNote's OLAP team consolidated several StarRocks clusters into one shared-data cluster with six isolated warehouses at about half the resource cost. A second project moved full-text and vector search into StarRocks, so search results can be filtered and analyzed in the same engine.

200,000+

CPU cores across 150+ StarRocks instances

RedNote's internal OLAP platform

~60PB

of data in StarRocks

Largest shared-data cluster: about 7 PB

55,000

vector queries per second over 1.3 billion rows

Semantic deduplication with vector search

CompanyRedNote
Use caseAnalytics and content search
Previous solutionShared-nothing StarRocks clusters, Elasticsearch and Milvus
ReplacedElasticsearch and Milvus

This story is based on a NextQuery 2026 session by Qi Ma, Software Engineer at RedNote. NextQuery is PhoenixAI's Data + AI Infrastructure Summit. Watch the session

About RedNote

RedNote (Xiaohongshu) is a lifestyle community where people share real experiences. Users post notes, photos, and videos, and others search and discuss them to inform their own decisions. RedNote has more than 400 million monthly active users, over 70 million comments per day, and more than 800 million searches per day.

RedNote's OLAP team runs the internal platform for interactive analytics behind advertising, experiment platforms, user behavior analysis, and ad hoc analytics. Users expect answers in seconds on queries over hundreds of billions of rows, and many questions arrive with SQL whose complexity no one can predict in advance.

RedNote's OLAP team chose StarRocks because its MPP architecture, vectorized execution, and query optimizer serve multi-table joins and aggregations interactively, because new data can be queried soon after it lands, and because standard SQL fits both daily reports and ad hoc work. Today RedNote runs more than 150 StarRocks instances with over 200,000 CPU cores and about 60 PB of data. RedNote's earlier use of StarRocks for advertising analytics is described in RedNote Delivers Integrated Advertising Analytics with StarRocks.

The challenge

Shared-nothing clusters sized by storage

At RedNote's scale, more than 150 StarRocks instances with over 200,000 CPU cores, the OLAP team faced two separate problems: clusters that scaled with storage instead of demand, and search tools that could not keep up with new questions.

Most of RedNote's StarRocks clusters were shared-nothing, with compute and storage on the same nodes. When disks filled up first, the OLAP team had to add whole nodes even when CPU sat idle. Each new node triggered a redistribution of tablets, which slowed scaling and added operational work.

Queries, ingestion, and compaction also shared the same backend nodes, so a spike in one workload raised latency for the others. To keep workloads apart, RedNote ran multiple clusters, and the same data could end up stored more than once.

Search that could not keep up with new questions

Two of RedNote's internal platforms ran into the limits of their search tools. Platform A gives internal teams brand and industry insights on trends, discussions, products, and reputation. Analyzing a new industry or topic on Platform A started with defining it, collecting keywords and samples, and running model-based processing and batch labeling. For a large industry, that could take months. Coverage was a second gap: users describe things in colloquial ways no keyword list anticipates, and information inside an image or video may never appear in a text field.

Platform B serves multimodal pretraining data to RedNote's algorithm teams, at a scale of tens of billions of samples. The same data went into Elasticsearch for attributes and full-text search and into Milvus for vector search, and the application merged the results.

Both engines stored the data and had to stay aligned. Vector database cost limited how much of Platform B's dataset could be searchable, and queries on the Elasticsearch cold tier could easily time out.

The solution

Shared storage, separate compute

RedNote solved its cluster and search problems as two separate projects, both on StarRocks. The first moved RedNote's clusters from shared-nothing to shared-data. In shared-data mode, persistent data lives in S3 object storage. A frontend (FE) cluster handles metadata, query planning, and routing, and compute runs in warehouses of compute nodes (CNs) that cache hot data locally. Every warehouse reads the same data, so isolating a workload no longer means copying it.

At RedNote, latency-sensitive interactive queries, compute-heavy complex analytics, and throughput-first ingestion and compaction each get their own warehouse, which can grow or shrink on its own.

Reference architecture: business workloads such as advertising, experiment platforms, user behavior analysis and ad hoc analytics, plus data loads, connect to an FE cluster that handles metadata, query planning and routing. The FE routes work to three warehouses of compute nodes with local cache, one for latency-sensitive interactive queries, one for compute-intensive complex analytics and one for throughput-first ingestion and compaction. All three read one shared copy of the data in S3 object storage.
RedNote's shared-data StarRocks cluster with separate warehouses per workload

Shared-data StarRocks now runs in more than five large RedNote clusters and several smaller ones, and it is the default for new clusters.

One shared-data cluster with 30 million tablets

RedNote's largest StarRocks deployment replaced several clusters, which kept repeated copies of the same data, with one shared-data cluster: six warehouses of 80 CNs with 64 CPU cores each, 30,720 cores in total, on one copy of more than 350 tables, about 7 PB of data, and about 30 million tablets.

Before and after for RedNote's largest cluster. Before: multiple clusters with repeated data copies. Now: one shared-data cluster with six warehouses of 80 compute nodes and 64 cores each, 30,720 CPU cores in total, on one shared data copy, at about 50% lower resource cost. The cluster holds 350+ tables, 7 PB of data and 30M tablets.
RedNote's largest production cluster. Source: RedNote, NextQuery 2026

At 30 million tablets, RedNote's OLAP team hit a new bottleneck in this cluster. Every node change, whether planned scaling, a node failure, or a replica change, starts a background shard checker that verifies and rebalances the mapping between shards and nodes. With this much metadata, the checker ran for hours while holding a control-plane lock, and other operations waited behind it.

RedNote's engineers cut the work the shard checker does under the lock. The checker now caches shard hash results and reuses intermediate balancing data instead of rescanning metadata, and it submits scheduling actions in batches instead of one shard at a time.

Search and analytics in one StarRocks engine

The second project moved search for Platform A, RedNote's brand and industry insights platform, and Platform B, its multimodal training-data platform, into StarRocks. Full-text search, vector search, and structured filters now run on the same data RedNote analyzes. Upstream models still handle image understanding and vector generation.

RedNote built on StarRocks' community inverted index and added what its business needed. Phrase matching respects word order, which helps with product names and specific scenarios. BM25 scoring ranks results by relevance, so text results can be combined with vector results. On the vector side, RedNote introduced HGraph in place of HNSW, added RaBitQ quantization, and trimmed overhead from the query path.

A query with clear keywords can use full-text search alone, and one about meaning or image-text similarity can use vector search alone. For broader coverage, Reciprocal Rank Fusion (RRF) merges both rankings, while filters on brand, product, time, and RBAC permissions limit the scope. The ranked rows then go straight into joins and aggregations in StarRocks, without moving to another system. For a closer look at this pattern, see Retrieval Is a Query: Search Where Your Business Data Lives.

RedNote's hybrid search pipeline. Full-text search with phrase matching and BM25, vector search for semantic and image-text recall, and scalar filtering by brand, product and time feed Reciprocal Rank Fusion, which fuses the two result sets while scalar predicates constrain candidates. The fused results go to an analysis step for joins, aggregates, trends and AI.
Hybrid search pipeline. Source: RedNote, NextQuery 2026
StarRocksS3 object storage

The results

Compute that follows demand

RedNote's move to shared-data StarRocks changed how compute is sized, isolated, and scaled.

  • Right-sized clusters. A petabyte-scale security log cluster with relatively low query volume used to be sized by its storage. After migration, it runs on 20 CNs instead of 60.
  • Isolated workloads. In one migrated workload, query latency alerts went from about once a week to almost none.
  • Faster scale-out. New compute nodes are ready in seconds instead of hours, because adding them no longer moves data.
  • Lower cost. The largest cluster costs about 50% less in resources than the multi-cluster setup it replaced, and capacity can be set per workload.
  • Predictable scaling and recovery. In a stress test with production-scale metadata, a full run of the background shard checker fell from more than 12 hours to about 1 hour. Other control-plane operations no longer wait hours behind it, so scaling and failure recovery are more predictable.

Faster text queries and billion-row vector jobs

Some of RedNote's text queries had used regular expressions over large amounts of raw data. With the StarRocks full-text index narrowing candidates first, cold-read latency on representative queries dropped from 66.2 s to 10.2 s, and data read fell from 707 GB to 1.46 GB. Other queries used about 84% less CPU, and peak memory per query fell from about 1.137 GB to 3.309 MB.

Vector search in StarRocks also runs at scale in RedNote's batch work. A semantic deduplication job over about 1.3 billion rows finishes in about 18.7 hours, with its nearest-neighbor phase running for about 6 hours at 55,000 queries per second.

Platform A and Platform B on one engine

Platform A, RedNote's brand and industry insights platform, adds about 10 million valid comments per day, and users can now search all of them, including comments no label covers. Queries combine full-text and vector recall with filters on industry, brand, product, and time, and return aggregated results in seconds.

When a new colloquial expression appears, analysts can find relevant comments right away and decide later whether it deserves a long-term label.

Platform B, which serves multimodal pretraining data to RedNote's algorithm teams, cut its monthly resource cost by about 69% after the move from Elasticsearch plus Milvus to StarRocks. Full-text and vector search share one dataset, so there is no duplicate storage to keep aligned, and searches that used to time out on the cold tier now return in milliseconds. The platform covers more than 100 datasets and tens of billions of samples.

Platform B after moving to StarRocks. Monthly cost fell from 100% to about 31%, a reduction of about 69%. Data storage went from duplicated copies in Elasticsearch and Milvus to one StarRocks dataset for full-text and vector search. Query latency went from cold-tier timeouts to millisecond search served by one engine. The platform covers 100+ datasets and tens of billions of samples.
Platform B, three benefits of a unified engine. Source: RedNote, NextQuery 2026

For another example of vector search in StarRocks, see How Tencent Runs Vector Search for RAG and AIGC Image Management on StarRocks.

What's next

Running shared-data StarRocks at RedNote's scale taught the OLAP team that fast compute scaling is only one part of elasticity. Background scheduling and cache warm-up have to keep pace.

The next driver is AI agents. Daily queries, ingestion, and analytics are relatively predictable, but agents are starting to send more queries to OLAP, with timing, concurrency, and SQL complexity that are hard to forecast. RedNote wants to move toward a more fully serverless model that allocates compute on demand, protects daily workloads, and absorbs bursts from agents, along with more multimodal retrieval and analysis in the same engine.