CH3: A Mixed Workload Benchmark for Scalable NoSQL

Database management systems that support hybrid workloads (i.e., HTAP or HOAP) first arose in the relational world. Such hybrid data management support in the document database (NoSQL) world is also gaining popularity in both commercial and research arenas. The CH2 benchmark was proposed in 2021 to evaluate such hybrid NoSQL platforms. In addition to operational and analytical services, full-text search is a key component of NoSQL platforms that provides a search engine-like query processing capability on JSON documents. In this paper, we present CH3, a mixed workload benchmark for evaluating scalable NoSQL platforms with OLTP, OLAP, and full-text search (FTS) workloads. Like CH2, the CH3 benchmark borrows from and extends both TPC-C and TPC-H. However, CH3 generates meaningful text content and includes FTS indexes and FTS queries on these indexes to model an FTS workload. This paper presents the required extensions from CH2 to address FTS workloads, the detailed design of CH3, and performance results obtained by running the CH3 benchmark against Couchbase Server (which offers Query, Analytics, and Search Services). The results provide insight into the performance of the Search Service, the performance isolation among OLTP, OLAP and FTS workloads, and the horizontal scalability of Couchbase Server as well as the effectiveness of CH3 for evaluating the mixed workload performance of such NoSQL platforms.