Repository navigation
Add performance comparision with prometheus-compatible systems #299
Description
You can use VictoriaMetrics/prometheus-benchmark.
From project README:
Prometheus-benchmark provides the following features:
- It generates production-like workload for both data ingestion and querying paths:
- It generates write workload from real node_exporter metrics.
This is the most frequently used exporter for Prometheus metrics. - It generates read workload from typical alerting rules for
node_exportermetrics - see chart/files/alerts.yaml.
- It generates write workload from real node_exporter metrics.
- It allows generating time series churn rate
via scrapeConfigUpdatePercent
and scrapeConfigUpdateInterval
options. The churn rate is typical for Kubernetes monitoring. - Multiple systems can be tested simultaneously - just add multiple named entries
underremoteStoragessection at chart/values.yaml.
The following systems can be tested with prometheus-benchmark:
- Single-node version of VictoriaMetrics:
- Cluster version of VictoriaMetrics:
- Grafana Mimir:
- Cortex:
- Thanos:
So, list of compatible systems:
- Prometheus
- VictoriaMetrics
- Cortex
- Thanos
- Grafana Mimir
Activity
flyingmutant commentedon Mar 2, 2023
Thanks for the suggestion, an interesting benchmark. Maybe one day we'll find time to run it or similar one, but to be honest I don't see a lot of value.
The things we focus on (multi-level multi-tenancy with fair resource budgeting, low query latency under mixed read-heavy workloads, space-efficient long-term storage with automatic downsampling) are quite different from the things being measured here, and will be much harder to quantatively benchmark. Highlighting only the raw throughput (even though I believe should be excellent in StatsHouse) does not do StatsHouse justice.
ernado commentedon Mar 2, 2023
Thank you for your repsonse!
I think that it is possible to measure low query latency, space efficiency and down-sampling.
System performance under high multi-tenant load should demonstrate statshouse overall superiority and resiliency for overloads.
This will be a pretty strong argument for using statshouse instead of existing systems, people love benchmarks.
I understand that advanced benchmarks are hard to setup, but I think that profit worth the effort.