TossService analytics and AI agents

Managing Workloads and Permissions for StarRocks–AI Agent Integration

Toss replaced batch pipelines that served data through MySQL with StarRocks, then used resource groups, per-team accounts and an MCP gateway to run live services, lake queries and employees' AI agents side by side. Each agent query runs as the employee who asked it, under that person's existing Ranger permissions.

5.7×

faster than Impala in an internal TPC-H test

22 TPC-H queries on lake data

90%+

of queries finish in under 100 ms

From SQL logs across Toss's clusters

40B+

rows of raw data aggregated in real time

Pre-aggregated on load with Aggregate Key tables

CompanyToss
Use caseService analytics and AI agents
Previous solutionSpark and Impala on Hadoop, served through MySQL
ReplacedHadoop-to-MySQL batch pipelines

This story is based on a NextQuery 2026 session by Jeongho Choi and Yujin Lee, data engineers at Toss. NextQuery is PhoenixAI's Data + AI Infrastructure Summit. Watch the session

About Toss

Toss is a financial super-app in South Korea with more than 30 million registered users, over half of the country's population. Securities, banking, insurance, shopping and advertising all run inside one app.

Each of those businesses brings its own data needs, so the Toss data platform team serves product backends, operations dashboards, analysts, batch jobs and, more recently, AI agents. Toss has run StarRocks in production since the second quarter of 2023. Today five StarRocks clusters carry product backends, dashboards, batch jobs and AI agents, and lake data stays in Hive and Iceberg tables on HDFS that every cluster reads in place.

The challenge

Until 2023, every data request at Toss followed one path. Engineers wrote a Spark job or Impala SQL, the pipeline computed the result on Hadoop, and the result was loaded into MySQL and served from there.

That held up until the requests changed. Report services and admin consoles needed dynamic filters on whatever a user picked in the interface, but the pipelines had already fixed the numbers, so every new filter meant a new pipeline. Product services wanted subsecond answers over billions of rows, and the existing engines were too slow and could not handle enough concurrent queries. Because everything ran in batches, data was always 5 to 10 minutes behind.

Tuning MySQL sped up existing queries but not new ones. Querying Hadoop directly was too slow for many users at once. More pipelines meant more jobs and more copies of the data to keep in sync.

The solution

Why Toss chose StarRocks

Toss compared several OLAP databases before choosing StarRocks. Toss's data platform team wanted speed for both loading and queries, plus resource isolation, automatic materialized view rewrite, flexible deployment and easy scale-out. Most of Toss's transactional databases run MySQL, so StarRocks's MySQL protocol and syntax let Toss reuse most existing queries unchanged.

In Toss's internal TPC-H test of 22 queries on lake data, StarRocks reading through a Hive catalog finished in 105.7 seconds in total, against 393.9 seconds for Spark and 601.9 seconds for Impala. StarRocks was also the fastest on its own tables. ClickHouse was fast too, but StarRocks did better on complex joins and large datasets, and ClickHouse was harder to run and scale out.

Slide showing a bar chart of per-query run times for 22 TPC-H queries, with totals of 601.9 seconds for Impala, 393.9 seconds for Spark and 105.7 seconds for StarRocks through a Hive catalog, beside notes that StarRocks was fastest on its own tables and on Hive and Iceberg tables, and that ClickHouse was harder to run and scale out
Toss's internal TPC-H comparison of Impala, Spark and StarRocks. Source: Toss, NextQuery 2026

Where StarRocks sits in the Toss data platform

At Toss, change data and logs land in Kafka. Flink and Kafka Connect stream them continuously into both Hadoop and StarRocks, and Spark and dbt load data in scheduled batches. Impala and Spark still run batch jobs and heavy queries on the lake. StarRocks handles real-time aggregation and serves batch data at low latency, which removed the pipelines that copied results from Hadoop into MySQL.

Reference architecture: app events and MySQL and MongoDB change data flow into Kafka, then through Flink and Kafka Connect into five StarRocks clusters and the HDFS data lake. Three clusters (active service, standby service, commerce) run shared-nothing on local SSD, and two (monitoring and batch, lakehouse) run shared-data on HDFS. StarRocks serves service backends, BI dashboards and AI agents through MCP, while Impala and Spark serve analysts on the lake
Where StarRocks sits in the Toss data platform

Toss runs five StarRocks clusters and picks the deployment model per workload. Latency-critical services run shared-nothing on local SSD, where point queries on the prefix index return in under 10 ms. Even with cache warm-up, an unpredictable query on shared-data can set off a cache storm, and live services cannot absorb that. The monitoring and batch cluster and the lakehouse cluster run shared-data on HDFS for flexibility and scale.

Every cluster reads the same Hive and Iceberg catalogs, so lake data is never copied. Native tables are mirrored between clusters with StarRocks cross-cluster replication and checked for lag and consistency.

Logs go into Duplicate Key tables and change data into Primary Key tables. High-volume ads data goes into Aggregate Key tables that sum on load, which made real-time aggregation over 40 billion rows of raw data practical.

Keeping live services fast on a shared cluster

On Toss's StarRocks clusters, product services run thousands of point queries that read a few megabytes, batch jobs read gigabytes, and dashboards query constantly. AI agents add repetitive yet unpredictable patterns. Without resource groups, one heavy job had slowed down an entire cluster and even taken down a node.

Scatter plot of data read per query against queries per hour, on log scales. Services such as merchant lookups and the shopping service run tens of thousands of small queries per hour, batch jobs read up to tens of gigabytes per query, Grafana dashboards sit high on both axes, and the AI agent (MCP) workload sits in the middle
Query shapes by workload. Source: Toss, NextQuery 2026

Toss's SQL logs show the two extremes: over 90% of queries finish in under 100 ms, while a few batch jobs take 10 seconds or longer and read most of the data.

On the commerce cluster, a high-QPS shopping service ran at almost the same priority as heavy batch queries. When a large batch query started, backend CPU jumped from nearly 0 to 80 or even 100%, and service latency rose with it.

Slide showing the shopping service and commerce batch queries sharing CPU on the commerce cluster, with a Grafana chart of backend CPU jumping from near 0% to 80 to 100% when a heavy job starts
A batch job slowing down a live service. Source: Toss, NextQuery 2026

Toss fixed it in two steps. A higher CPU weight for serving helped, but weights only apply under contention, and spikes returned when batch jobs overlapped. Reserving exclusive CPU cores for the serving group made the spikes disappear. With CPU borrowing on, other groups can use those cores while they sit idle.

Toss now starts with weights and moves to dedicated cores if latency still suffers. Commerce, search and ads get dedicated cores or high weights. Loading gets dedicated cores and a memory cap. Dashboards get low weights and concurrency limits. AI agents get their own group.

Moving lake scans off the serving nodes

On Toss's StarRocks clusters, only 6% of queries touch Hive or Iceberg, but they read 36% of the data. In April 2026, backend CPU on the service cluster hit 85 to 92% every 30 minutes, when a scheduled insert scanned 91 days of a Hive table on the same nodes that served live traffic. Lake queries saw p95 latency 6 to 30 times worse. The compute nodes added for lake work sat at 8 to 11% CPU, because compute node routing was turned off globally.

Each Toss team now has two StarRocks accounts. The regular account runs internal-table queries on the backends with a high CPU weight. A second account for lake queries prefers compute nodes and is scheduled on compute nodes only, with a low CPU weight and a memory cap. The account decides where a query runs, and the resource group decides how much it can use. Toss's data platform team verifies placement with EXPLAIN and query profiles.

Letting AI agents in through one door

At Toss, AI agents never touch the StarRocks service cluster. They run on the shared-data lakehouse cluster. The biggest use case is self-service MCP: once access is approved, any employee can query StarRocks through their own AI assistant. Operations agents also analyze slow queries and open DDL pull requests.

Agents connect only through the company MCP gateway, which forwards the caller's LDAP ID to an MCP server. The server exposes two read-only tools, query and metadata inspection, and masks personal data.

On the lakehouse cluster, unreviewed agent SQL runs in its own StarRocks resource group. Agents loop over the same tables, so repeat reads can be served from the compute nodes' local disk cache instead of HDFS, and Toss can add compute nodes as agent traffic grows without moving data. For the aggregations agents repeat, materialized view rewrite answers from a view without changing the SQL. Agent access through MCP follows a similar pattern to Conductor's agentic analytics.

Agents query as the employee who asked

Most of what Toss employees ask their AI agents about lives in Hadoop, where Apache Ranger already manages table permissions. A second set of grants in StarRocks could drift from Ranger, and a shared agent identity could expose more data than intended. So Toss added a separate Hive catalog on the same metastore with Ranger enabled. Internal StarRocks tables keep native role-based access control.

When an employee such as Jane asks a question, Toss's MCP server, logged in to StarRocks with its service account, runs EXECUTE AS 'jane.doe' WITH NO REVERT. Ranger then checks Jane's own permissions, and the audit log records Jane. The session cannot switch back, so an impersonated connection is never reused for another user. Agents get no extra privileges. They act as the person asking.

StarRocksApache KafkaApache FlinkKafka ConnectApache SparkdbtApache ImpalaApache HiveApache IcebergHDFSApache RangerMySQL

The results

StarRocks replaced the batch pipelines into MySQL and now carries Toss's services, dashboards, lake queries and AI agents.

  • Fast serving: over 90% of queries finish in under 100 ms, and point queries on the prefix index return in under 10 ms.
  • Real-time aggregation: Aggregate Key tables aggregate over 40 billion rows of raw data in real time.
  • Protected live traffic: dedicated cores removed batch-driven latency spikes on the commerce cluster, and lake scans moved to compute nodes.
  • Query acceleration without code changes: materialized view rewrite made one service query about 10 times faster with the SQL unchanged.
  • Governed AI access: self-service MCP has been live since July 2026 on the lakehouse cluster, which runs v4.1 with 41 compute nodes. Every agent query runs under the employee's own identity and Ranger policies.

For Toss, AI agents are one more workload on StarRocks, managed by the same rules as services, batch jobs and lake queries. Toss lets agents in through one gateway, runs their queries as the person asking and keeps them in their own resource group.

Next, Toss is looking forward to multi-warehouse support and plans to move more workloads to shared-data. It is building a disaster recovery cluster on shared-data now. Toss's data platform team also plans to contribute its connector and replication improvements back to the StarRocks community.