StarRocks debuts cloud-native SQL engine to speed up real-time analytics

California-based StarRock has announced a cloud-based version of its SQL engine
for enterprises to provide them with faster and more affordable analytical
capabilities.
As data continues to explode, companies are increasingly looking for ways to extract
value from the growing pool within their own backyards. Real-time data has been
leveraged to create an application that allows people to see the status of their
orders as they happen. It is the new Holy Grail, with a number of companies
processing and reacting to event stream data for use cases such as outage
detection.
Image Source: Getty Images
However, when it comes to analytics, which involves finding patterns in data, working
in real-time can be a major challenge for some companies. Analytical queries
become difficult to perform when information is constantly being added, updated,
and even deleted from databases. When dozens of people try to query data at once,
the problem becomes even worse.
Universal Engine Of StarRocks
StarRocks solves the problem of having too many different tools for analyzing data
by combining them into a single tool called StarRocks.
“This is a new kind of analytic database that addresses the critical technical
challenges in big data analytics, such as the ability to handle massive amounts of
data, the need to normalize data, and the challenge of scaling up for thousands of
concurrent users.” We created a brand new query engine using many breakthrough
technologies.
Image Source: medium.com (Quick Response Time)
The engine was purposely designed to support real-time data and a large number
of concurrent users, with multi-table joins. According to the company’s claims, it can
ingest data at 100 MB/s per node and perform more than 10,000 searches per
second. This eventually allows enterprises to combine their latest streaming
transaction records with historical records for effective recommendation and
decision-making.
More than 500 companies have already adopted the solution, including Airbnb,
TripAdvisor, and Lenovo.
New Cloud Native Version
With the new cloud-native version, available as a fully managed SaaS platform
called StarRocks, StarRocks is increasing its value for enterprises.
Basically, StarRockets Cloud allows organizations to integrate their existing data
infrastructures in the cloud and do without regular engineering and administrative
tasks needed for real-time analytics. From setting up the servers/VMS to deploying
the software.
Image Source: Getty Images
Besides this, cloud computing also brings in various cloud-specific features such as
separation of computation and storage, automatic resource management, etc, that
not only reduces costs but also gives data teams extra time to focus on query
performance and improve the time to insights for end-users.
Competition Level
There are currently several organizations looking into the real-time analytics market,
including ClickHouse, Imply (Apache Druid), Starmtree (Apache Pinot), and Rockset
(ROCKSdb). However, StarRockets claims to offer a much lower price-performance
ratio than its competitors.
“StarRockets has better performance than its competitors – we have published
benchmark tests showing we have three to five times the performance of our
competitors.
In addition, StarRockets is less expensive than its competitors – with our industryleading query engine, we are capable of achieving superior query performance
without complex data processing and heavy indexing processes.
Image Source: Getty Images
As a result, we’re able to achieve a much better price-performance ratio.” Finally,
StarRocks is the most flexible. Our unique design can easily handle frequent updates
to past transaction records while still maintaining high query speed in real-time.
This allows us to provide real-time analytics for use cases that were previously
considered unsuitable for real-time analytics.
StarRocks Cloud is expected to become generally available in Q3 2022 on Amazon
Web Services, with support for the Google cloud platform coming in later.
VentureBeat’s mission: To be a digital town square where technical decision makers
can learn about transformative enterprise technology and transact.
Wide-table Testing Using SSB
Star Schema Benchmark (SSB) is a tool used to measure the basic performance
metrics of various online analytical processing (OLAP) databases. SSB uses a Star
Schema Test Set [1] that is commonly used in academia and industry.
ClickHouse flattens out the star schema into a wider flat table and then rewrites the
SSBs into a flat table benchmark. Therefore, in flat table scenarios, we use the CREATE
TABLE statement in ClickHouse to create a PK test on StarRock, ClickHouse, and
Apahe Druid.
We perform additional performance testing on the aggregation for low cardinality
fields.
Multi-table Testing Using TPC-H
TPC-H benchmarks are used by companies to determine whether their applications
run efficiently enough for them to be able to handle large amounts of transactions.
TPC-H (Tiny Parquet Compressed Hierarchical) can be used to build data
warehouses for simulating the data warehouse of a business system.
Image Source: researchgate.net
The main performance metrics include the response time for each query. The TPC-C
benchmark evaluates a database system by testing its ability to handle queries with
various types of joins.
• Analyze large amounts of data.
• Complex queries require complex processing.
• Answer critical business questions.
ClickHouse and Apache Drill cannot complete the TPC-H benchmark. Since there is
no primary key between StarRocks and Trio, we perform a non-unique constraint test
between them.