SingleStore
SingleStore is a distributed, relational SQL database management system (RDBMS) that combines transactional and analytical processing in a single platform, enabling real-time data ingestion, querying, and AI-driven applications at scale.[1][2] Founded in 2011 as MemSQL by co-founders including Nikita Shamgunov and Eric Frenkiel through Y Combinator, the company initially focused on in-memory rowstore databases for high-speed online transaction processing (OLTP).[3][4] In 2020, it rebranded to SingleStore to better reflect its evolution into a converged data platform supporting both OLTP and online analytical processing (OLAP) workloads across structured, semi-structured, and unstructured data.[5] Headquartered in San Francisco, California, and led by CEO Raj Verma since 2019, SingleStore has raised significant funding, including a $116 million round in 2022 led by Goldman Sachs, and in October 2025 completed a growth buyout led by Vector Capital to accelerate its enterprise AI and real-time data solutions.[6][7][8] The platform's core architecture features Universal Storage, which integrates an in-memory rowstore for low-latency transactions with an on-disk columnstore for efficient analytics, allowing sub-second query responses on massive datasets.[9] It supports multi-model data handling, including JSON documents, time-series, geospatial, full-text search, and vector embeddings for machine learning and AI applications, all queried via standard ANSI SQL.[1][2] SingleStore optimizes performance through techniques like just-in-time code generation for queries and distributed computing across clusters, making it suitable for highly concurrent workloads in industries such as finance, healthcare, and retail.[10] Deployable as SingleStore Helios—a fully managed SaaS offering on AWS, Azure, and Google Cloud—or as self-managed on-premises/hybrid setups, the platform simplifies data architectures by eliminating the need for separate ETL pipelines or data warehouses, potentially reducing total cost of ownership by up to 75%.[11][5] Notable customers include Comcast, Dell Technologies, Kroger, Samsung, Siemens, and ZoomInfo, leveraging it for real-time insights and AI innovation.[1] As of November 2025, SingleStore continues to emphasize AI-native capabilities, with expansions into markets like Japan to support global enterprise adoption.[12]Overview
Core Functionality
SingleStore is a distributed, relational SQL database management system (RDBMS) that provides full ANSI SQL compliance, enabling real-time data ingestion, transactional processing, and querying for data-intensive applications.[13][14] It supports standard SQL syntax, including joins, filters, and analytical functions, while maintaining compatibility with existing SQL drivers and tools.[14] The database is optimized for mixed online transaction processing (OLTP) and online analytical processing (OLAP) workloads, delivering low-latency query responses, often in milliseconds, even under high concurrency and across petabyte-scale datasets.[15][16] This performance stems from its ability to handle real-time analytics and operational workloads in a unified platform, reducing the need for separate systems.[17] Originally launched as MemSQL, the company rebranded to SingleStore in October 2020 to emphasize its vision of a single database that consolidates transactional and analytical processing, thereby simplifying data architectures and eliminating silos between OLTP and OLAP databases.[18] SingleStore achieves this through basic operational modes that leverage in-memory processing for rapid query execution and persistent on-disk storage for data durability and recovery.[2]Supported Data Models
SingleStore supports a range of data models, enabling it to handle diverse workloads within a unified SQL-based system. Originally focused on relational data, the database evolved into a multi-model platform with enhancements including the general availability of SingleStore Kai in 2024 and the native vector data type in version 8.5 (January 2024), allowing a single instance to manage structured, semi-structured, and specialized data without requiring separate databases.[19][20][21] At its core, SingleStore provides full support for relational tables using standard SQL for structured data storage and querying, including ACID transactions and joins across large datasets. It also natively handles JSON documents through a dedicated JSON data type that accommodates maps, arrays, and nested structures, with built-in functions for extraction, modification, and aggregation. For geospatial data, SingleStore offers spatial functions and types like GeographyPoint, enabling operations on points, lines, and polygons; while lacking native GeoJSON parsing, it integrates GeoJSON via JSON support and computed columns for flexible ingestion and querying.[22][23][24] Additionally, SingleStore can function as a key-value store by leveraging rowstore tables with primary keys and varbinary or text columns for values, supporting high-throughput inserts and lookups akin to NoSQL systems. It includes a vector data type for storing embeddings, with similarity search functions like cosine and Euclidean distance to facilitate AI applications. In November 2025, SingleStore introduced AI Functions, enabling developers to call generative AI models and generate embeddings directly within SQL queries for enhanced AI application development.[25][26][27] For time series data, the platform provides specialized functions such as TIME_BUCKET for aggregation and SERIES_TIMESTAMP for efficient ingestion, optimized for rowstore or columnstore tables with timestamp columns.[28] Semi-structured data, including JSON and vectors, integrates seamlessly with SQL queries, allowing relational joins, filtering, and analytics on mixed formats. Full-text search is available on JSON attributes and vector embeddings, enhancing discoverability in document-heavy or AI-driven datasets. This multi-model approach supports use cases like real-time analytics on IoT-generated time series data for monitoring sensor streams or AI-driven vector similarity searches for recommendation engines and semantic retrieval.[23][29][28][26]Company History
Founding and Early Milestones
SingleStore was founded in January 2011 by Eric Frenkiel, Nikita Shamgunov, and Adam Prout as MemSQL, with an initial focus on building an in-memory database for operational transaction processing (OLTP) to enable high-performance, real-time data handling.[30][31] The founders, drawing from their experience at companies like Facebook, aimed to combine the speed of in-memory computing with SQL compatibility to address limitations in traditional databases for modern applications.[32] Headquartered in San Francisco, California, the company began operations as a Y Combinator graduate, establishing its core technology around lock-free data structures and distributed architecture for scalable performance.[31][33] On April 23, 2013, MemSQL launched its first generally available version, introducing a distributed relational database that supported wire-compatibility with MySQL and enabled horizontal scaling for OLTP workloads across clusters.[34] This release marked the company's entry into the market as a high-speed, in-memory solution capable of handling real-time analytics and transactions, positioning it as an alternative to specialized NoSQL systems while retaining relational features.[35] A pivotal early milestone came in 2015 with the addition of columnstore functionality, which extended the platform to support analytical (OLAP) queries efficiently by storing data in columnar format for compression and faster aggregation, bridging OLTP and OLAP in a single system.[36] This enhancement enabled high-speed data ingest rates, up to 1 TB per hour in benchmarks, facilitating real-time pipelines for big data applications.[3] By 2017, MemSQL had forged key partnerships, including with Hewlett Packard Enterprise, to integrate its database with enterprise infrastructure for optimized deployment in production environments.[37] The company also expanded its workforce to around 100 employees by 2018, supporting growth in engineering and customer adoption.[38]Rebranding and Expansion
On October 27, 2020, the company formerly known as MemSQL rebranded to SingleStore to better align its name with its expanded capabilities as a unified platform for both transactional (OLTP) and analytical (OLAP) workloads, moving beyond its original focus on in-memory databases.[39][18] This change emphasized the platform's universal storage architecture, enabling efficient handling of diverse data types across multi-cloud and on-premises environments without data replication, thereby broadening its appeal in the enterprise data management market.[18] Following the rebrand, SingleStore pursued aggressive global expansion, opening offices in London in 2021, Hyderabad in 2022, and Lisbon in 2023 to support growing international demand.[40] By April 2022, the company's employee count had reached nearly 400, reflecting rapid scaling in engineering, sales, and operations teams.[41] These moves were bolstered by strategic hires, including key executives for revenue, innovation, and marketing, as well as enhanced presence in EMEA and Latin America.[42] In April 2023, SingleStore launched real-time AI features, including initial vector search capabilities, to enable faster processing for AI-driven applications.[43] This was followed in May 2023 by the introduction of generative AI (GenAI) tools, such as enhanced analytics integrations to support AI model development and deployment.[43] Key partnerships accelerated this momentum: in July 2023, SingleStore announced a collaboration with AWS to advance real-time data analytics and AI workloads on cloud infrastructure.[44] In August 2023, IBM integrated SingleStore with its watsonx.ai platform to facilitate generative AI application building using real-time data.[45] Further advancements included the general availability of MongoDB API support via SingleStore Kai in January 2024, allowing over 100x faster analytics on JSON data without application changes.[20] In September 2024, SingleStore partnered with Snowflake for bi-directional integration with Apache Iceberg, enabling seamless data sharing and real-time AI processing within Snowflake's ecosystem while maintaining governance.[46] The year culminated in the October 2024 acquisition of BryteFlow, an Australian data integration platform, to strengthen change data capture from enterprise sources like SAP and Salesforce, thereby accelerating real-time analytics and GenAI adoption.[47] In August 2025, SingleStore expanded its global presence by opening an office in Japan, marking its eighth office worldwide and targeting growing demand in the Asia-Pacific region for enterprise AI solutions.[12] These developments marked a strategic pivot toward cloud-native, AI-optimized databases, with SingleStore positioning itself as a frontrunner in real-time analytics as of late 2025 through enhanced ingestion, vector processing, and ecosystem integrations.[48]Funding Rounds
SingleStore has raised approximately $464 million in equity funding across 11 rounds through October 2022, with key investments supporting product development, engineering expansion, and market growth.[49] In September 2025, the company underwent a growth buyout led by Vector Capital Management, which acquired a majority stake in a transaction speculated at around $500 million; this move provided capital for accelerated growth in AI and real-time data solutions while retaining investments from long-term shareholders such as Google Ventures, Dell Technologies Capital, IBM, and REV Venture Partners. The buyout, expected to close in Q4 2025, represented a strategic alternative to traditional equity rounds or an IPO.[50][51] Prominent investors include Insight Partners, Accel, Google Ventures (GV), Goldman Sachs, and Prosperity7 Ventures, alongside others such as Khosla Ventures, Dell Technologies Capital, and Hewlett Packard Enterprise.[52][6] The company's funding history began with a Series A round in January 2013, raising $5 million led by Data Collective (DCVC) and IA Ventures to fund initial platform development and engineering team building.[53] In January 2014, SingleStore secured $35 million in a Series B round led by Accel, with participation from Khosla Ventures, aimed at accelerating distributed in-memory database technology for real-time analytics.[54] Subsequent rounds built on this foundation. The Series C in April 2016 raised $36 million, led by Caffeinated Capital and REV Venture Partners, to drive growth in real-time analytics databases.[55] The Series D in May 2018 brought in $30 million, led by GV and Glynn Capital Management, to enhance scalability for insight-driven enterprises.[56] Later-stage funding accelerated amid cloud and AI focus. In December 2020, a Series E round of $80 million was led by Insight Partners, with participation from Dell Technologies Capital and others, supporting global expansion and integration of disparate data silos following the company's rebranding from MemSQL.[57] The Series F in September 2021 raised another $80 million, again led by Insight Partners and Hewlett Packard Pathfinder, to fuel triple-digit cloud growth and strategic initiatives.[52] In 2022, funding reached its peak with two significant raises. A $116 million round in July, led by Goldman Sachs Asset Management and including Sanabil Investments, advanced real-time data platform capabilities.[6] This was followed in October by a Series F-2 extension of $30 million from new investor Prosperity7 Ventures (the venture arm of Saudi Aramco), bringing the total for that extended round to $146 million and supporting product enhancements, sales investments, and geographic expansion into Europe and Asia.[58][30] These investments have primarily been directed toward research and development in distributed systems, cloud-native expansions like SingleStore Helios, and emerging AI features to power data-intensive applications.[52][58]Products and Services
SingleStore Database Engine
SingleStore DB serves as the flagship product of SingleStore, functioning as a distributed relational database management system (RDBMS) that provides high-performance data processing for both transactional and analytical workloads.[14] It achieves ANSI SQL compliance, enabling seamless use of standard SQL syntax for operations such as joins, aggregations, and window functions, while maintaining compatibility with the MySQL wire protocol to integrate with existing tools and applications.[14] This design supports real-time ingestion and querying of large-scale datasets, making it suitable for applications requiring low-latency analytics on operational data. At its core, SingleStore DB operates through a distributed architecture comprising aggregator nodes and leaf nodes. Aggregator nodes handle query coordination, parsing, optimization, and result aggregation across the cluster, ensuring efficient distribution of workloads.[59] Leaf nodes, in contrast, manage data storage, partitioning, and local query execution, allowing for horizontal scaling by adding more nodes to increase capacity and performance.[59] This separation enables the system to process complex queries in parallel while maintaining data consistency. SingleStore DB offers flexible licensing options to accommodate various use cases. The free edition is an unlimited-time trial of the Standard edition, restricted to a total capacity of 8 vCPU and 32 GB RAM across the cluster as of July 2025, providing core functionality for development and small-scale testing.[29][60] The Enterprise edition extends this for production environments, incorporating advanced security features like audit logging and compliance certifications (e.g., SOC 2, GDPR), along with premium support and unlimited scaling capabilities.[29] Key integration tools enhance SingleStore DB's usability for data workflows. SingleStore Pipelines facilitate extract, transform, and load (ETL) processes by enabling continuous data ingestion from sources such as Apache Kafka for streaming data and Amazon S3 for batch files, with built-in transformation capabilities to prepare data for analysis.[61] Additionally, it supports semantic search for natural language querying of vectorized data and offers native compatibility with business intelligence tools like Tableau, allowing direct connections via JDBC or ODBC drivers for visualization and reporting.[62][63] For cloud deployments, the engine can be hosted via SingleStore Helios, the managed service offering.[11]SingleStore Helios
SingleStore Helios is a fully managed database-as-a-service (DBaaS) offering launched in September 2019, providing elastic scalability and high availability for modern applications.[64] It is available on major public clouds including AWS, Azure, and Google Cloud Platform, enabling users to deploy clusters without managing underlying infrastructure.[65] Key features include autoscaling, which automatically adjusts compute resources based on workload demand to optimize performance and cost, and zero-downtime upgrades that ensure continuous availability during maintenance.[66][67] In 2024, SingleStore introduced a "Bring Your Own Cloud" (BYOC) option, allowing private deployments within a customer's own cloud account for enhanced control and compliance.[68] Additionally, serverless compute capabilities, such as Cloud Functions, support variable workloads by executing code without provisioning servers.[69] The service builds on the core SingleStore database engine, delivering distributed SQL processing in a cloud-native environment.[70] Pricing follows a pay-per-use model, charged based on compute units (measured in credits per hour) and storage (per GB-month), with granular metering by the second to align costs with actual usage.[71] A free shared tier is available for development and testing, offering limited resources without upfront commitment.[71] Helios reduces operational overhead by handling provisioning, patching, backups, and monitoring automatically, allowing teams to focus on application development.[65] Integrated monitoring provides real-time insights into performance metrics, while tools like SingleStore Flow enable seamless data migration from on-premises or other systems with minimal disruption.[72] This managed approach supports scalable, low-latency workloads for real-time analytics and AI applications.[70]Management Tools
SingleStore Studio is a web-based graphical user interface designed for interacting with SingleStore clusters in self-managed deployments. It serves as a visual SQL client, enabling users to execute queries, view results in tabular and graphical formats, and perform schema exploration through SQL commands. The tool supports performance visualization via workload monitoring features that analyze query execution details, resource utilization, and cluster health metrics.[73][74][75] In SingleStore Helios, the cloud service, management extends to Data Studio, which includes support for Jupyter notebooks allowing development with both SQL and Python code directly connected to workspaces. This facilitates schema management, ad-hoc querying, and data analysis in a notebook environment.[76] SingleStore Tools provide a command-line interface (CLI) suite for administering self-managed clusters, encompassing deployment, configuration, backups, and diagnostics. Key components include thesinglestoredb-toolbox package, which offers commands such as singlestore-admin for operational tasks like starting, stopping, and reconfiguring nodes, as well as memsqlctl for lower-level management. Historical operations, such as generating diagnostic reports for troubleshooting, are handled through tools like sdb-report, which captures cluster state and performance data for analysis. These utilities are packaged for Debian, RPM, and tarball distributions, supporting scripted automation in distributed environments.[77][78][79]
SingleStore's built-in monitoring capabilities capture cluster events and metrics via a dedicated metrics database and pipelines, integrated with Grafana for visualization. It tracks query latency through historical workload analysis, including execution times and run counts for parameterized queries, and monitors resource usage such as CPU, memory, and disk I/O per node. Alerting is supported through configurable thresholds in the monitoring setup, with compatibility for exporting metrics in Prometheus format starting from version 7.3, enabling integration with external observability tools for proactive notifications on issues like high latency or resource saturation.[80][74]
In 2024, SingleStore introduced enhanced autoscaling tools for Helios users, allowing programmatic adjustment of compute resources based on workload demands to maintain performance without manual intervention; autoscaling is now generally available as of 2025.[66][68][81]
Architecture
Data Storage Formats
SingleStore employs a dual storage architecture that combines rowstore and columnstore formats to optimize for diverse workloads, enabling efficient handling of both transactional and analytical queries within the same database. The rowstore format stores data in a row-oriented layout entirely in memory, facilitating rapid inserts, updates, and point lookups ideal for online transaction processing (OLTP) scenarios.[82] In contrast, the columnstore format, also known as Universal Storage, persists data on disk in a columnar orientation, supporting high compression and fast aggregations for online analytical processing (OLAP) tasks.[83] This hybrid approach allows tables to be defined as either rowstore or columnstore based on workload needs, with columnstore serving as the default since version 7.3.[84] The rowstore uses a row-wise storage model where each row's fields are kept together in RAM, minimizing latency for transactional operations and enabling lock-free concurrency through skiplist and hash indexes for efficient range scans and exact matches.[82] This format excels in high-concurrency environments with frequent writes and deletes, as data remains fully in memory without routine disk I/O during queries.[83] For persistence, rowstore data is synced to disk via periodic checkpoints that capture the table's state, combined with a write-ahead transaction log for recovery, ensuring durability without compromising in-memory performance.[85] Columnstore organizes data into row segments—groups of sorted rows—and column segments that store values for each column within those rows, allowing selective access to only relevant columns during scans and aggregations.[86] Compression techniques, such as dictionary encoding for repeating values, reduce storage footprint significantly, while metadata like min/max bounds per segment enables elimination of irrelevant data blocks to accelerate OLAP queries.[86] Tables in columnstore are automatically sorted by one or more sort key columns defined in the table creation statement, optimizing data layout for common access patterns and further enhancing scan efficiency.[86] A hidden in-memory rowstore buffer handles small or incremental inserts before merging into the on-disk columnstore, bridging operational and analytical workloads seamlessly.[87] This dual-format system supports hybrid usage where a single table type—typically columnstore—can manage both OLTP and OLAP demands through appropriate indexing, while rowstore is reserved for pure transactional tables requiring sub-millisecond response times.[83] Persistence in columnstore occurs through checkpoints that write compressed blobs to disk, maintaining data integrity across restarts.[86] Overall, the architecture scales to petabyte-level datasets without performance degradation, leveraging distributed nodes for storage and processing.[17]Indexing Strategies
SingleStore employs a variety of indexing strategies to optimize query performance across its rowstore and columnstore storage formats, enabling efficient data access for both transactional and analytical workloads. These indexes are designed to support fast lookups, range scans, and specialized searches while maintaining low overhead during data ingestion. Primary indexes form the core structure for each table type, with secondary and specialized indexes providing additional acceleration for targeted query patterns.[88] In rowstores, primary indexing relies on skip lists, which are probabilistic, lock-free linked lists that replace traditional B-tree structures for in-memory operations. Skip lists achieve O(log n) average-case performance for insertions, deletions, and lookups by maintaining multiple levels of linked lists, allowing nodes to "skip" ahead during traversals. They support ascending or descending order and are particularly efficient for sorted forward scans, making them suitable for high-velocity transactional data. This implementation ensures fast insert performance without the locking overhead of B-trees.[89][4] For columnstores, the primary index is defined by the SORT KEY clause, which orders rows within logical segments to facilitate segment elimination during queries. Each segment stores metadata with min/max values per column, enabling the query optimizer to skip irrelevant segments based on predicates, such as range filters. This sorted organization also improves compression ratios by grouping similar values and supports efficient column-wise scans without requiring full table sorts. Empty SORT KEY declarations result in unsorted segments, which may suffice for certain append-only workloads but reduce elimination opportunities.[90][88] Secondary indexes in SingleStore primarily use hash indexes to accelerate equality-based lookups on non-primary keys. These indexes employ a hash function to map keys to buckets in a sparse array, providing constant-time O(1) access for exact matches while supporting multi-column configurations. Only one unique hash index per table is allowed, and they exclude floating-point types like FLOAT or DOUBLE due to precision issues. Queries must reference all indexed columns to leverage the hash structure fully, making them ideal for point lookups in both rowstore and columnstore tables.[89][88] Full-text indexes utilize inverted structures to enable efficient text searches on CHAR, VARCHAR, TEXT, or LONGTEXT columns in columnstores. Created via the FULLTEXT clause in CREATE TABLE, these indexes map terms to their document locations, supporting MATCH...AGAINST queries across multiple columns. Version 2 of the index offers enhanced functionality for broader search expressions, including table-level matching. Geospatial indexes, on the other hand, apply R-tree structures to GEOGRAPHY and GEOGRAPHYPOINT columns in rowstores, organizing spatial data hierarchically for fast intersection and containment queries. An optional RESOLUTION parameter (ranging from 6 to 32) controls the granularity of polygon or linestring decomposition, balancing query speed against memory and insert costs—lower values favor ingestion speed, while higher ones improve accuracy.[91][24][88] Index maintenance in SingleStore is fully automated, with updates occurring synchronously during data ingestion or modification to ensure consistency without manual intervention. The system supports online index creation, allowing concurrent reads and writes on the table, which minimizes downtime for secondary and specialized indexes. This approach keeps overhead low for real-time workloads, as indexes are incrementally updated rather than rebuilt periodically.[92][88]Distributed Processing
SingleStore employs a distributed architecture consisting of aggregator and leaf nodes to handle large-scale data processing and query execution. Leaf nodes are responsible for storing data in partitions and performing computations on those subsets, while aggregator nodes manage query coordination. Client applications connect to an aggregator, which parses incoming SQL queries, generates execution plans, and distributes subqueries to the appropriate leaf nodes based on data locality. The leaf nodes execute these subqueries in parallel and return intermediate results to the aggregator, which then merges and aggregates them before sending the final output back to the client. This separation of concerns allows for efficient scaling by pushing the majority of processing workload to the leaves.[59] Data partitioning in SingleStore uses hash-based sharding to distribute tables across leaf nodes, ensuring even workload balance and parallelism. For each table, rows are assigned to partitions via a hash function applied to the shard key—typically the primary key or user-defined columns—which determines the partition number. By default, the number of partitions equals the number of leaf nodes multiplied by a configurable factor (default_partitions_per_leaf), with each leaf holding multiple partitions. In columnstore tables, data is further organized into segments within these partitions, enabling parallel scans and operations across distributed nodes. This sharding approach minimizes data movement during queries when filters align with the shard key. Replication of partitions across multiple leaves provides fault tolerance, as detailed in the reliability features section.[59][93] Query execution follows a massively parallel processing (MPP) model, where operations such as scans, joins, and aggregations are distributed across leaf nodes for concurrent execution. The aggregator optimizes the query plan to route operations to relevant partitions, using shard key predicates to target specific leaves and avoid full cluster scans. On the leaves, queries leverage pipelined execution to process operators in a streaming fashion, reducing intermediate materialization and network shuffling—particularly for co-located joins on shard keys. If data redistribution is needed for cross-partition operations, the system employs efficient shuffle mechanisms, but prioritizes local processing to maintain performance. This MPP paradigm supports high-throughput analytics on petabyte-scale datasets by parallelizing both storage and computation.[94][95] Horizontal scaling in SingleStore is achieved by adding leaf nodes to the cluster, which automatically redistributes partitions to balance load and achieve near-linear gains in storage capacity and query throughput. New leaves integrate online without downtime, with data rebalancing occurring in the background via controlled partition migrations. Aggregator nodes can also be scaled independently to handle increased connection volumes. The system supports orchestration via Kubernetes operators, enabling automated deployment, resizing, and management in containerized environments for elastic resource allocation.[96][97][98]Reliability Features
Durability Mechanisms
SingleStore ensures data durability through a combination of transaction logging and periodic snapshots, which provide persistence for in-memory updates and enable recovery to a consistent state following failures or restarts.[99] These mechanisms support both synchronous and asynchronous durability modes, configurable during database creation or restoration, where synchronous mode guarantees that log writes complete before transaction commits, while asynchronous mode prioritizes performance by allowing commits before disk persistence.[100] Transaction logs in SingleStore function as a write-ahead log (WAL), asynchronously recording all database updates from DDL and DML operations to disk before they are committed in memory.[101] Each log is partition-specific and captures changes in a non-human-readable binary format, ensuring that upon server restart, the system can replay these logs from the last consistent point to reconstruct the in-memory state.[99] This approach enables point-in-time recovery by allowing selective replay of logged transactions up to a specific timestamp, preventing data loss even in the event of crashes.[101] To manage disk usage, logs are truncated after being incorporated into snapshots, with retention controlled by engine variables likesnapshot_retention_period to balance recovery windows and storage efficiency.[102]
Snapshots serve as durable checkpoints of committed in-memory rowstore data, created periodically or manually to capture the state of databases across all partitions.[101] Triggered automatically when transaction logs reach a configurable size threshold (defaulting to 1 GB per partition via snapshot_trigger_size), snapshots are written to disk and include only rowstore portions, as columnstore data persists separately through segmented files.[99] During recovery, the most recent snapshot is loaded into memory, followed by replay of any subsequent transaction logs, minimizing restoration time.[101] For columnstores, background compaction merges and optimizes segments to maintain efficiency and durability without interrupting queries, ensuring long-term data integrity in analytical workloads.[103]
SingleStore achieves ACID compliance, particularly for transaction isolation, through multi-version concurrency control (MVCC) implemented in its rowstores, which maintains multiple versions of data rows to allow concurrent reads and writes without blocking.[103] This lock-free mechanism, utilizing skip lists and hash tables, ensures read committed isolation level while supporting high-throughput OLTP operations, with atomicity and durability enforced via the transaction log writes.[104][105] Consistency is upheld by validating transactions against MVCC versions during commit, preventing partial updates from surviving failures.[106]
Backup strategies in SingleStore include incremental physical backups via the BACKUP DATABASE command, which capture changes since the last full backup to minimize storage and time overhead, applicable to local storage databases but not unlimited storage ones.[107] Logical exports are supported through the MySQL-compatible mysqldump utility, allowing export of databases or tables in SQL format for portability and recovery.[108] In SingleStore Helios, continuous backups to object storage enable point-in-time recovery to any timestamp within a configurable retention period (default 7 days), combining snapshots and logs for granular restoration without downtime.[109] These approaches ensure comprehensive data protection, with recovery processes leveraging the same logging and snapshot infrastructure for verifiable integrity.[109]
Replication and High Availability
SingleStore implements high availability (HA) through a redundancy model that pairs leaf nodes in availability groups, where data partitions are synchronously replicated from primary to secondary leaves to ensure redundancy and minimal downtime. In this setup, theredundancy_level is set to 2, creating replica partitions on secondary leaves that mirror the primaries in real time, allowing the system to maintain data availability even if a primary leaf fails.[110][111]
The primary replication type for HA is synchronous, where writes to a primary partition are committed only after successful replication to the paired secondary leaf, providing strong consistency guarantees and preventing data loss in the event of a single node failure. Automatic failover occurs when the master aggregator detects a primary leaf failure, promoting the secondary's replicas to primaries immediately, with the process completing in seconds to minutes depending on cluster size and load. Asynchronous replication is available as an option for HA but is not recommended due to weaker consistency; it is more commonly used for cross-cluster replication, where a secondary cluster lags behind the primary. To balance durability and performance, SingleStore supports tunable consistency by allowing administrators to switch between sync and async modes via database-level commands like ALTER DATABASE.[112][111][110]
For read scalability in HA mode, SingleStore employs a master-master-like approach in paired leaf configurations, where each leaf can serve reads from both its primary and replica partitions, distributing query load across the cluster while maintaining synchronous updates. Strong consistency is enforced through quorum-based writes, requiring acknowledgment from the primary and its replica before commit, though this can be tuned for lower latency at the cost of potential brief inconsistencies during network partitions. Failovers are designed to avoid data loss in standard HA setups, as synchronous replication ensures all committed transactions are durably stored on at least two leaves; however, simultaneous failure of both leaves in a pair could lead to temporary unavailability until recovery. Split-brain scenarios are prevented through monitoring; in synchronous HA, there is no grace period and failover is immediate, while asynchronous replication uses a grace period mechanism of up to 300 seconds for repeated failures to allow network recovery and avoid divergent states. This integrates with durability mechanisms like the write-ahead log (WAL) for local persistence, ensuring replicated data is transactionally safe.[110][113][114]
Advanced Integrations
Apache Iceberg Support
SingleStore introduced bi-directional integration with Apache Iceberg in June 2024 (currently in preview, with public preview for ingestion and private preview for full bi-directional support), enabling seamless interoperability between its distributed SQL database and Iceberg-based data lakes. This support allows users to query Iceberg tables directly within SingleStore without data movement, while also exposing SingleStore tables as Iceberg-compatible catalogs for external access. The integration leverages catalogs such as AWS Glue or Hive Metastore to manage metadata, facilitating shared data ecosystems across tools like Snowflake or Spark.[115] Key features include ACID-compliant transactions on Iceberg data through merge pipelines that handle append, overwrite, replace, and delete operations on snapshots, ensuring transactional consistency during ingestion and querying. Schema evolution is supported with automatic detection of changes, such as column additions, renames, or type promotions, allowing pipelines to pause and resume without manual reconfiguration. Time travel queries are supported through Iceberg-native features via external table querying, enabling access to historical snapshots of Iceberg tables, while partitioning alignment optimizes data scanning by applying file-level filters to reduce unnecessary reads.[116] This integration supports use cases centered on unifying operational databases with data lakes, where SingleStore acts as a high-speed layer for real-time analytics on large-scale datasets. For instance, organizations can perform SQL-based analytics directly on S3-stored Iceberg files, combining transactional workloads with batch processing to power low-latency applications and AI-driven insights without ETL overhead.[117] Performance benefits stem from zero-copy access to external Iceberg tables, which avoids data duplication and minimizes latency by reading files in place. Metadata caching in the catalog further accelerates queries by storing schema and partition details locally, enabling sub-second response times even on petabyte-scale lakes when integrated with SingleStore's distributed processing.[115]Vector Search and AI Capabilities
SingleStore provides robust vector search capabilities, enabling efficient storage, indexing, and querying of high-dimensional vectors for AI-driven applications. The platform supports vector data types alongside traditional relational data, allowing unified processing of structured and unstructured information in a single database. This integration facilitates real-time analytics and similarity searches essential for modern AI workloads.[118] Vector indexing in SingleStore leverages algorithms such as Hierarchical Navigable Small World (HNSW) for approximate nearest neighbor (ANN) searches, which construct proximity graphs to achieve logarithmic search times with high recall rates. HNSW variants, including HNSW_FLAT for optimal accuracy and HNSW_PQ using product quantization for reduced memory usage, are built on implementations from Facebook AI Similarity Search (Faiss). Additionally, Inverted File (IVF) indexes, such as IVF_FLAT and IVF_PQFS, cluster vectors for scalable indexing on large datasets. For precise results, exact k-nearest neighbor (k-NN) searches perform full scans without indexing, trading speed for 100% accuracy in smaller-scale queries. These indexing options were introduced in SingleStore release 8.5, significantly reducing query times—for instance, from 3-4 seconds to 400 milliseconds on 160 million vectors.[119][120] Key AI features include real-time vector ingestion and similarity search, launched in advanced form during 2024 to support dynamic AI pipelines with low-latency updates. Similarity metrics like cosine similarity and Euclidean distance enable semantic matching for text, images, and embeddings. In 2024, SingleStore enhanced integration with large language models (LLMs) through the SingleStore Kai API, a MongoDB-compatible interface that allows seamless ingestion of JSON data and vector embeddings for LLM applications without ETL processes. This API delivers over 100x faster analytics on MongoDB workloads, powering real-time RAG (Retrieval-Augmented Generation) systems.[43][121][122] Further advancements in October 2025 introduced AI Functions and ML Functions, enabling direct SQL calls to ML models and LLMs for agentic AI support, where autonomous agents perform low-latency tasks like real-time decision-making and multi-step reasoning in milliseconds, alongside Aura Analyst for conversational data exploration. These functions, combined with Zero Copy Attach for instant data replication, ensure scalable, reliable execution under heavy AI loads. SingleStore's distributed architecture maintains ACID compliance while handling dynamic agent workflows.[123][124] Full-text search has been improved with hybrid semantic and keyword capabilities, blending vector-based semantic matching with traditional keyword indexing in a single SQL query via full outer joins and re-ranking using Reciprocal Rank Fusion (RRF). This approach, launched in June 2024, enhances result accuracy—for example, weighting vector scores at 70% and full-text at 30%—while executing in under 200 milliseconds on datasets exceeding 160 million vectors. Autoscaling features, added in mid-2024, dynamically adjust resources for AI workloads, optimizing compute for vector searches and ensuring cost efficiency during peak GenAI demands.[125][126][68] These capabilities support key use cases such as RAG pipelines, where vectors retrieve context for LLM generation; recommendation systems, leveraging similarity searches on user embeddings; and GenAI analytics on unified data, combining real-time ingestion with SQL-based insights for personalized applications. Integrations with frameworks like LangChain and LlamaIndex further streamline development for these scenarios.[127][124]Deployment Options
On-Premises Deployments
SingleStore supports on-premises deployments through self-managed installations on customer-owned infrastructure, enabling full control over hardware and data locality. These deployments require Linux-based operating systems, including Red Hat Enterprise Linux (RHEL) or AlmaLinux 7 and later, Debian 8 or 9 (with 9 preferred), and Ubuntu 14.04 or later, along with a minimum kernel version of 3.10 and glibc 2.17 or higher for SingleStore version 8.1 and above.[128] Installation uses RPM packages for RHEL, CentOS, or AlmaLinux distributions and DEB packages for Debian or Ubuntu, with adequate permissions required on target machines for package management via yum or apt-get.[129] For production high-availability (HA) setups, a minimum of four nodes is recommended to demonstrate distributed database capabilities, typically comprising at least one aggregator node and multiple leaf nodes.[130] Hardware specifications include an x86_64 CPU with a minimum of four cores (eight vCPUs recommended, especially for leaf nodes), optimized for SSE4.2 and AVX2 instruction sets, though functional on systems without them.[128] Aggregator nodes require at least eight GB of RAM, while leaf nodes need a minimum of 32 GB to align with licensing units and performance expectations.[128] Storage should be at least three times the main memory capacity per node, with SSDs recommended for columnstore workloads, and compatible filesystems such as ext4 or XFS; rowstore data requires approximately five times RAM, while columnstore sizing depends on raw data volume (doubled for HA).[128] The setup process begins with deploying the SingleStore engine using command-line interface (CLI) tools for cluster initialization, supporting both online (internet-connected) and offline modes for air-gapped environments.[129] Online deployments download components directly, while offline setups involve manual transfer of tarballs or packages, ensuring compatibility for regulated or isolated networks.[131] Since 2020, the SingleStore Kubernetes Operator has provided orchestration capabilities for Kubernetes environments, including on-premises clusters, facilitating automated management on platforms like Red Hat OpenShift.[98] This operator, certified for OpenShift, enables seamless cluster resizing, node replacement, and version upgrades, though prior Kubernetes experience is advised.[98] Scaling in on-premises deployments involves manual addition of nodes via CLI commands or the Kubernetes Operator, allowing horizontal expansion to handle increased workloads without automated provisioning.[132] Security features include role-based access control (RBAC) for administering user permissions and synchronizing them across the cluster, alongside row-level security (RLS) for fine-grained data access.[133] Encryption in transit is enforced via SSL/TLS for client-server and intra-cluster connections, with configurable certificates for secure communications.[134] For encryption at rest, SingleStore integrates with OS-level solutions like LUKS or third-party tools such as Thales CipherTrust Transparent Encryption and IBM Guardium Data Encryption, protecting data files, backups, and logs.[134] These measures support compliance in regulated environments by securing persistent storage and network traffic. On-premises deployments lack built-in autoscaling, requiring manual intervention for resource adjustments, which contrasts with cloud-managed options but provides advantages in air-gapped or highly regulated settings where data sovereignty is paramount.[132] Offline installation options ensure operation without external connectivity, making it suitable for environments with strict isolation requirements.[131]Cloud and Managed Services
SingleStore offers SingleStore Helios as its primary cloud database-as-a-service (DBaaS) platform, enabling users to deploy and manage distributed SQL databases without handling underlying infrastructure.[65] Helios supports both transactional (OLTP) and analytical (OLAP) workloads on a unified architecture, with features like low-latency queries (single-digit milliseconds), high concurrency, and real-time data ingestion via pipelines from sources such as Kafka and Amazon S3.[70] It operates on a shared-nothing distributed model with separation of compute and storage, allowing independent scaling of resources to handle elastic demands.[65] As a fully managed service, SingleStore Helios automates key operational tasks, including provisioning, configuration, upgrades, backups, and monitoring, under a shared responsibility model where SingleStore manages the platform and users secure their data and applications.[65] High availability is ensured through multi-AZ failover and continuous backups, with service level agreements (SLAs) of 99.9% uptime for single-AZ deployments and 99.99% for multi-AZ configurations in Standard and Enterprise editions.[11] Security features include compliance with ISO/IEC 27001, SOC 2 Type 2, GDPR, HIPAA, and CCPA, along with built-in encryption, audit logging (Enterprise edition), and customer-managed encryption keys (CMEK).[65] Disaster recovery options, such as point-in-time recovery (PITR) at microsecond granularity, are available in the Enterprise edition on AWS and GCP.[135] Helios is available across major public cloud providers: Amazon Web Services (AWS), Microsoft Azure, and Google Cloud Platform (GCP), with deployments in multiple regions to meet data residency and latency needs.[65] Users can choose from several editions tailored to different use cases:- Free Shared Edition: Designed for development, prototyping, and non-production workloads, this edition provides shared workspaces with basic SQL features, monitoring, and self-support but no SLA or production-scale performance guarantees.[135] It incurs no cost and supports limited throughput for evaluation.[11]
- Standard (Dedicated) Edition: Suited for general-purpose production applications, it offers dedicated workspaces with full MySQL compatibility, support for JSON, time-series, vector, geospatial, and full-text search functionalities, plus resource governance and multi-AZ high availability.[135] Pricing starts at $0.99 per compute unit-hour, with usage-based billing and 600 free credits for initial setup.[11]
- Enterprise Edition: Targeted at mission-critical, customer-facing workloads, this builds on Standard features with advanced recovery (online PITR, Smart Disaster Recovery), enhanced security (SCIM integration with Okta, audit logging, CMEK), and cross-region replication.[135] It starts at $1.49 per compute unit-hour and supports deployment across multiple clouds and regions.[11]