Database Test 5 (DBT-5) Documentation

BrokerageHouse

BrokerageHouse is responsible for executing all of the database transactions. It implements the EGenTxnHarness and EGenDriverDM.

MarketExchange

MarketExchange emulates stock exchanges and sends messages to the BrokerageHouse for initial Trade Request and Mark Feed transactions. It implements the EGenDriverMEE.

Driver

Driver emulates customers and drives the workload. It implements EGenDriverCE.

Required Software

In addition to the database management system software, the following is also required:

• C++ compiler

• patch

• Database development libraries and header files

  • PostgreSQL https://www.postgresql.org

    • psql

• SQLite https://www.sqlite.org/index.html is required for processing test results

Optional software:

• sar, pidstat http://pagesperso-orange.fr/sebastien.godard/ (While the scripts assume this particular version of sar and pidstat, it is possible to run on non-Linux based operating systems with some modifications to the kit.)

• jq https://jqlang.github.io/jq/ and toml-cli https://github.com/gnprice/toml-cli are required to use TOML configuration files for executing tests.

Building TPC-E Tools

Download the TPC-E Tools directly from the TPC: https://www.tpc.org/tpc_documents_current_versions/current_specifications5.asp

The kit requires the TPC-E Tools to be built for the specific database management system to be tested. The TPC-E Tools is developed in such a way that it needs to rebuilt or another copy needs to be built if a different database management system is to be tested.

DBT-5 provides a script to apply patches and code to compile the TPC-E Tools. The patches that are applied are minor code changes and code is supplied to build sponsor supplied code.

For example, to build the TPC-E Tools for PostgreSQL (pgsql), unzip the TPC-E Tools zip file into an empty directory and run dbt5-build-egen against the resulting directory:

mkdir /tmp/egen
cd /tmp/egen
unzip /path/to/*-tpc-e-tool.zip
dbt5-build-egen /tmp/egen

PostgreSQL

cd storedproc/pgsql/c
make
make install
dbt5 pgsql-load-stored-procs -t c

dbt5 build --tpcetools=/tmp/egen pgsql

dbt5 run -d 120 pgsql /tmp/results

Database Connection Scaling

This test is performed by the test-user-scaling script.

The purpose of this test is determine how many users, or database connections, are needed to get as much throughput as possible out of the system.

This script is run a series of tests starting with 1 user up to the number of detected processors of the driver system, adding 1 user per step in the series. The script is primarily intended to be run in a 1-tier system configuration, but may be run in a 2-tier configuration and may needs some additional intelligence or options for that.

Each step in the series will spend 1 minute to warm up and establish all connections to the database, then run for an additional 5 minutes and stop before starting the next test in the series. The warm up time and test duration can be varied.

At the end of the test, the script will create a bar plot of the reported metric of each test vs. the number of users. The sar data from all the tests will be aggregated and plotted.

Here are some examples plots from a system with 4 logical processors where it spend 1 minute warming up before running another 5 minutes.

Database Parameter Effects

This test is performed by the test-db-param script.

The purpose of this test is to evaluate the effects of a changing a single database parameter.

This script must be used with a configuration file and cannot currently be controlled with just command line arguments.

The configure file must contain the following details in addition to any other required configuration parameter (see configuration instructions in User Guide section of the documentation):

1. database start command

2. database stop command

3. database parameter name

4. database parameter values to test

Here is an example for PostgreSQL that is testing 3 difference values for shared_buffers:

db_start_command = "pg_ctl -D /tmp/pgdata start"
db_stop_command = "pg_ctl -D /tmp/pgdata stop -m fast"
db_param_name = "shared_buffers"
db_param_values = ["1GB", "2GB", "3GB"]

At the end of the test, the script will create a bar plot of the reported metric of each test vs. the value of database parameter being evaluated. The sar data from all the tests will be aggregated and plotted, as shown in the Database Parameter Effects section.

Here is an example plot of the above example.

Building the Kit

CMake is build system used for this kit. A Makefile.cmake is provided to automate some of the tasks.

Building for debugging:

make -f Makefile.cmake debug

Building for release:

make -f Makefile.cmake release

Building source packages:

make -f Makefile.cmake package

See the AppImage section for details on building an AppImage. There are additional requirements for the appimage target in the Makefile.cmake. Alternatively, the kit provides scripts in the tools diretory to create a container that can create an AppImage.

AppImage

AppImages are only for Linux based systems: https://appimage.org/

The AppImageKit AppImage can be downloaded from: https://github.com/AppImage/AppImageKit/releases

It is recommended to build AppImages on older distributions: https://docs.appimage.org/introduction/concepts.html#build-on-old-systems-run-on-newer-systems

The logo used is the number "5" from the Freeware Metal On Metal Font.

See the README.rst in the tools/ directory for an example of creating an AppImage with a Podman container.

./configure --without-ldap --without-readline --without-zlib \
      --without-gssapi --with-openssl

mkdir egen
cd egen
unzip /tmp/*-tpc-e-tool.zip

EGEN=tpc-e-tool.zip tools/build-appimage-container
EGEN=tpc-e-tool.zip tools/build-appimage