e2e/

README

Benchmark Suite Comparison

Overview

This document compares the performance benchmarks of two packages: qoordinator and ncorest. The qoordinator is a client for nco-qoordinator server (ncoq-api) and ncorest is a client for the native Netcool OMNIbus REST API.

The benchmarks are run on the same hardware and environment, with results measured for different scenarios.

Scenarios

• insert 100 random generated rows into alert.status table
  • single: BenchmarkQoordinatorInsert_SingleThread_100rows-8
  • multi: BenchmarkQoordinatorInsert_MultiThread_100rows-8
• 100 times select rows from alerts.status, each request returns 10 rows
  • single: BenchmarkQoordinatorSelect_SingleThread_10rows_100repeat-8
  • multi: BenchmarkQoordinatorSelect_MultiThread_10rows_100repeat-8
• 10 times select rows from alerts.status, each request returns 100 rows
  • single: BenchmarkQoordinatorSelect_SingleThread_100rows_10repeat-8
  • multi: BenchmarkQoordinatorSelect_MultiThread_100rows_10repeat-8

Single means that there is only one API-client.

Multi - there are a few clients that share work: for instance, 100 inserts by 10 clients, each one makes 10 queries.

System Information

• Operating System: Linux
• Architecture: amd64
• CPU: 12th Gen Intel(R) Core(TM) i7-1260P

Two OMNIbus Object Servers are running inside docker containers with 4 CPUs and 2GiB memory in each one:

CONTAINER ID   NAME                       CPU %     MEM USAGE / LIMIT   MEM %     NET I/O          BLOCK I/O   PIDS
9deb08e18ee8   nco_objserver-omnibus1-1   0.04%     43.05MiB / 2GiB     2.10%     225MB / 1.17GB   0B / 0B     46
f50f0b2cab1c   nco_objserver-omnibus0-1   0.05%     43.37MiB / 2GiB     2.12%     225MB / 1.17GB   0B / 0B     46

Benchmarks

Comparison

Execution time per benchmark iteration in ms, less is better.

Memory used by client per benchmark iteration in KB, less is better.

qoordinator

Benchmark	Iterations	Time (ns/op)	Memory (B/op)	Allocations (allocs/op)
BenchmarkQoordinatorInsert_SingleThread_100rows-8	14	76588191	731674	8015
BenchmarkQoordinatorInsert_MultiThread_100rows-8	55	23408627	851534	8667
BenchmarkQoordinatorSelect_SingleThread_10rows_100repeat-8	8	129790946	9283941	123349
BenchmarkQoordinatorSelect_MultiThread_10rows_100repeat-8	20	50188410	9628030	123496
BenchmarkQoordinatorSelect_SingleThread_100rows_10repeat-8	27	41080279	7032588	110159
BenchmarkQoordinatorSelect_MultiThread_100rows_10repeat-8	55	23556700	7110121	110176

ncorest

Benchmark	Iterations	Time (ns/op)	Memory (B/op)	Allocations (allocs/op)
BenchmarkNcoRestInsert_SingleThread_100rows-8	25	46429086	1379349	15929
BenchmarkNcoRestInsert_MultiThread_100rows-8	1	1284330526	2253120	18655
BenchmarkNcoRestSelect_SingleThread_10rows_100repeat-8	1	1072702005	8240744	126400
BenchmarkNcoRestSelect_MultiThread_10rows_100repeat-8	1	1229445891	8733448	129046
BenchmarkNcoRestSelect_SingleThread_100rows_10repeat-8	9	113508944	9736496	110617
BenchmarkNcoRestSelect_MultiThread_100rows_10repeat-8	1	1027271154	9904992	111432