diff --git a/DocBook/docs.xml b/DocBook/docs.xml index 27507c5e..0811e7db 100644 --- a/DocBook/docs.xml +++ b/DocBook/docs.xml @@ -22,7 +22,7 @@
Release Notes - The following are the release notes for HammerDB v4.8. + The following are the release notes for HammerDB v4.9.
Nomenclature Change @@ -84,6 +84,23 @@ viewing Jobs with the Web Service.
+
+ SQL Server Schema build BCP Option + + From v4.9 HammerDB has been enhanced to improve schema build + performance using the SQL Server BCP utility. This option is selected + by default with an option to also build the schema using inserts as + done in previous versions. +
+ +
+ MariaDB/MySQL No Stored Procedures option to TPROC-C + + From v4.9 HammerDB has added an option to the TPROC-C workload + for MariaDB and MySQL that uses Client SQL instead of stored + procedures for comparative purposes. +
+
Known Third-Party Driver Issues @@ -198,35 +215,51 @@ DIAG_DDE_ENABLED=FALSE
- HammerDB v4.8 New Features + HammerDB v4.9 New Features - HammerDB v4.8 New Features are all referenced to HammerDB v4.9 New Features are all referenced to GitHub Issues and Pull Requests, where more details for each new feature and related pull requests can be found here: - [TPC-Council#560] Allow using existing empty database for Db2 - TPROC-C and TPROC-H + [TPC-Council#619] Network Error When Loading Data + + [TPC-Council#616] Remove Unneeded Indices From TPROC-H + MSSQL + + [TPC-Council#614] TPROCC MSSQLS BCP Load by Warehouse + + [TPC-Council#613] Alter MSSQL TPCH Colstore Index Steps + + [TPC-Council#611] Implement BCP Option for MSSQLS TPCH Load build + database - [TPC-Council#558] Enhancement for storing and retrieval of - HammerDB results and configurations + [TPC-Council#610] Fix for TPROC-H calculates o_totalprice + incorrectly - [TPC-Council#556] MySQL/MariaDB L_ORDERKEY to BIGINT + [TPC-Council#602] Add MariaDB/MySQL No Stored Procedures option to + TPROC-C - [TPC-Council#541] Update mssqlsoltp.tcl + [TPC-Council#599] Fix MySQL/MariaDB TPROC-H build FK error - [TPC-Council#539] Auto scale factor for TPC-H (issue 23) + [TPC-Council#596] Edited Dockerfile to install a specific version + of msodbcsql18 - [TPC-Council#535] Fix doc bug for VU/warehouse count + [TPC-Council#592] Added use_bcp default as true and enabled bcp + with windows authentication - [TPC-Council#531] Repair ora_compatible OSTAT procedure + [TPC-Council#591] Skip blank lines in binds in recent Oracle trace + files - [TPC-Council#530] Add jobs getchart command + [TPC-Council#587] Added Use bcp as a user option - [TPC-Council#522] Add CLI savescript and custommonitor - commands + [TPC-Council#586] Add Optional Invisible Primary Key to History + for Replication in MySQL and MariaDB - [TPC-Council#517] Create CONTRIBUTING.md + [TPC-Council#585] Update precision of w_ytd to prevent numeric + field overflow + + [TPC-Council#569] Container file simplification
@@ -237,10 +270,12 @@ DIAG_DDE_ENABLED=FALSE tested against. The matrix is not an exclusive support or configuration matrix and HammerDB has been designed to be compatible with the supported databases running on different architectures and operating - systems. HammerDB is built for the x88-64 architecture on Linux and - Windows. Where the database is not running on either Linux or Windows on - x86-64 HammerDB can be run on Linux or Windows and connect to the target - database on another architecture over a network. + systems. The HammerDB release files are built for the x86-64 + architecture on Linux and Windows. To support other database versions + HammerDB supports building from source as described further in this + documentation. Where the database is not running on either Linux or + Windows on x86-64 HammerDB can be run on Linux or Windows and connect to + the target database on another architecture over a network. HammerDB has been built and tested on the following x86 64-bit Linux and Windows releases. @@ -317,8 +352,8 @@ DIAG_DDE_ENABLED=FALSE MariaDB - 10.2 / 10.3 / 10.4 / 10.5 / 10.6 / 10.8 / 10.10 / - 11.1 + 10.2 / 10.3 / 10.4 / 10.5 / 10.6 / 10.8 / 10.10 / 11.1 / + 11.2 @@ -352,12 +387,12 @@ DIAG_DDE_ENABLED=FALSE which can be downloaded at no cost from Microsoft and run as follows: - fciv -both HammerDB-4.8-Win-x86-64-Setup.exe + fciv -both HammerDB-4.9-Win-x86-64-Setup.exe and on Linux with md5sum and sha1sum as shown: - md5sum HammerDB-4.8-Linux.tar.gz -sha1sum HammerDB-4.8-Linux.tar.gz + md5sum HammerDB-4.9-Linux.tar.gz +sha1sum HammerDB-4.9-Linux.tar.gz
@@ -540,8 +575,9 @@ sha1sum HammerDB-4.8-Linux.tar.gz In addition to the HammerDB software deletion there is also - a number of database files by default stored in the temporary - area. These can also be safely removed if preferred. + a number of SQLite database files by default stored in the + temporary area. These can also be safely removed if + preferred.
@@ -578,10 +614,10 @@ sha1sum HammerDB-4.8-Linux.tar.gz To install from the tar.gz run the command - tar -zxvf HammerDB-4.8.tar.gz + tar -zxvf HammerDB-4.9.tar.gz This will extract HammerDB into a directory named - HammerDB-4.7. + HammerDB-4.9.
@@ -608,7 +644,7 @@ sha1sum HammerDB-4.8-Linux.tar.gz HammerDB Docker Container Build & Run Containers are lightweight prebuilt environments that are easy to - distribute and deploy. Version 4.8 includes both a pre-built Docker + distribute and deploy. Version 4.9 includes both a pre-built Docker Image and a Dockerfile that can be used to create a HammerDB Docker containers. This container is based on an Ubuntu 20.04 base image and provides a ready to use environment to run HammerDB. HammerDB has a @@ -695,7 +731,7 @@ docker pull tpcorg/hammerdb:mssqls To create an image: Go to the folder containing the Dockerfile and run the docker build command. - docker build -t hammerdb-v4.8 . + docker build -t hammerdb-v4.9 .
docker build @@ -726,7 +762,7 @@ docker pull tpcorg/hammerdb:mssqls build is successful but the client libraries for IBM are not installed. - docker run -it --name hammerdb hammerdb-v4.8 bash + docker run -it --name hammerdb hammerdb-v4.9 bash
docker run @@ -768,7 +804,7 @@ docker pull tpcorg/hammerdb:mssqls communicate with remote databases. The following is an example of adding host network to the container. - docker run --network=host -it --name hammerdb hammerdb-v4.8 bash + docker run --network=host -it --name hammerdb hammerdb-v4.9 bash
@@ -798,7 +834,7 @@ docker pull tpcorg/hammerdb:mssqls and type librarycheck. - HammerDB CLI v4.8 + HammerDB CLI v4.9 Copyright (C) 2003-2023 Steve Shaw Type "help" for a list of commands The xml is well-formed, applying configuration @@ -1101,7 +1137,7 @@ Driver=/opt/microsoft/msodbcsql/lib64/libmsodbcsql-17.0.so.1.1 UsageCount=1}}

MySQL - HammerDB version 4.8 (4.X and version 3.3) has been built and + HammerDB version 4.9 (4.X and version 3.3) has been built and tested against a MySQL 8.0 client installation, hammerDB version 3.0-3.2 has been built against MySQL 5.7. On Linux this means that HammerDB will require a MySQL client library called @@ -1115,8 +1151,8 @@ Driver=/opt/microsoft/msodbcsql/lib64/libmsodbcsql-17.0.so.1.1 UsageCount=1}}

MariaDB - HammerDB version 4.8 has been built and tested against MariaDB - 10.5.10 client installation. On Linux this means that HammerDB will + HammerDB version 4.9 has been built and tested against MariaDB + 10.6 client installation. On Linux this means that HammerDB will require a MariaDB client library called libmariadb.so.3. This client library needs to be referenced in the LD_LIBRARY_PATH as shown previously in this section. On Windows the client library is called @@ -1160,7 +1196,15 @@ Driver=/opt/microsoft/msodbcsql/lib64/libmsodbcsql-17.0.so.1.1 UsageCount=1}}

+ desired. In the listing below the Data Base File corresponds to the + XML file with the same name. Note that after the first start and + creation of the Data Base File the XML configuration file will not be + re-read again. If you wish to reset the configuration for a particular + database if you the Data Base File for a particular database it will + be recreated from the XML configuration at the next HammerDB start. + The hammer.DB file contains the data related to jobs storage. + Similarly if this file is deleted it will be recreated on the next + HammerDB start. <sqlitedb> <sqlitedb_dir>TMP</sqlitedb_dir> @@ -1205,7 +1249,7 @@ Driver=/opt/microsoft/msodbcsql/lib64/libmsodbcsql-17.0.so.1.1 UsageCount=1}}

- The existing SQLite DBs are from version v4.7. SQLite DBs will be reset to v4.8. + The existing SQLite DBs are from version v4.8. SQLite DBs will be reset to v4.9. For this reason before running another version any existing configuration that is wished to be saved should be exported using @@ -1630,7 +1674,7 @@ Success ... loaded library mariatcl for MariaDB 4. Include a copy of the original copyright notice - it is not permitted to modify the license on a build from source + It is not permitted to modify the license on a build from source distribution. @@ -3381,6 +3425,17 @@ GO the schema build can take advantage of the higher core count to run the build more quickly. + + + Use BCP Option + + This option uses the SQL Server BCP utility to bulk + load data. Temporary staging files will be created and + deleted in the location defined by the TEMP environment + variable. When unselected an insert based build is used. A + BCP build offers vastly improved performance over an + insert based build. + @@ -3677,6 +3732,17 @@ GO Partition Order Line Table for improved scalability. + + + History Table Primary Key + + In the TPC-C specification that the HammerDB TPROC-C + workload is derived from the history table does not have a + primary key. However for MySQL replication all tables must + have a primary key. This option creates the history table + with an invisible primary key at supported MySQL versions. + + @@ -3857,6 +3923,17 @@ GO Partition Order Line Table for improved scalability. + + + History Table Primary Key + + In the TPC-C specification that the HammerDB TPROC-C + workload is derived from the history table does not have a + primary key. However for MySQL replication all tables must + have a primary key. This option creates the history table + with an invisible primary key at supported MySQL versions. + + @@ -5220,6 +5297,20 @@ P99: 2.067ms P95: 1.488ms P50: 0.772ms SD: 79101.309 RATIO: 4.187% + When Jobs are enabled by default the time profile data will be + stored alongside the Jobid and can be viewed with the web service. + Charts are automatically generated to show the response times. + +
+ graphical response times + + + + + + +
+ An alternative time profiling package time called etprof can be selected by modifying the timeprofile section in generic.xml to change xtprof to etprof. A key difference between etprof and xtprof is that @@ -5232,7 +5323,8 @@ P99: 2.067ms P95: 1.488ms P50: 0.772ms SD: 79101.309 RATIO: 4.187% well as cumulative values for all of the test at the end of the test run. For etprof the time profile values are recorded in microseconds. etprof should be viewed as way to drill down into an individual - virtual user after analyzing xtprof data. + virtual user after analyzing xtprof data and is not recorded in the + jobs related data for viewing with the web service. Hammerdb Log @ Fri Jul 05 09:55:26 BST 2019 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+- @@ -6138,6 +6230,30 @@ Vuser 1:Timing test period of 5 in minutes feature. +
+ No Stored Procedures Option for MySQL and MariaDB + + From version 4.9 for MySQL and MariaDB there is an advanced + option of No Stored Procedures. When selected this option will drive + the TPROC-C workload entirely from the client, running the same SQL + statements with the application logic within HammerDB. This option is + useful when looking to run a workload with an increased network load + and in tests increases the Bytes received and Bytes sent by 6X and 8X + respectively. This option should not be expected to deliver NOPM/TPM + values as high as using stored procedures due to the additional time + spent at the network. + +
+ No Stored Procedures Option + + + + + + +
+
+
Oracle AWR Reports @@ -6973,7 +7089,8 @@ done and display configuration meaning that the agent must be installed on the SUT. This can be accomplished by installing HammerDB on the SUT as well as the server. On Linux the sysstat package must be pre-installed where the - agent is running. + agent is running. An agent on Linux can can communicate with the HammerDB + display running on Windows and vice versa. $ mpstat -V sysstat version 11.5.7 @@ -7335,6 +7452,13 @@ pgsentinel_ash.max_entries = 1000000 the status bar shows the mode that HammerDB is running in. By default this will be Local Mode. + Note that communication between Primary and Replica is enabled by + the comm module which opens a default channel called ::comm::comm. If an + exterior application attempts to connect then an warning such as follows + will be shown. + + hammerdb>Warning: Connection from 127.0.0.1/49436 received on ::comm::comm with unknown protocol "ddffdfdfd" +
Mode @@ -7652,7 +7776,7 @@ hammerdb> To learn CLI commands type "help". - HammerDB v4.8 CLI Help Index + HammerDB v4.9 CLI Help Index Type "help command" for more details on specific commands below @@ -9388,9 +9512,9 @@ HAMMERDB RESULT and on Windows. - C:\Program Files\HammerDB-4.8>set TEMP=C:\temp -C:\Program Files\HammerDB-4.8>hammerdbcli.bat -HammerDB CLI v4.8 + C:\Program Files\HammerDB-4.9>set TEMP=C:\temp +C:\Program Files\HammerDB-4.9>hammerdbcli.bat +HammerDB CLI v4.9 Copyright (C) 2003-2023 Steve Shaw Type "help" for a list of commands Initialized new Jobs on-disk database C:/temp/hammer.DB @@ -11365,6 +11489,17 @@ infinidb_import_for_batchinsert_delimiter=7 Generation Server or SUT if HammerDB is running there. + + + Use BCP Option + + This option uses the SQL Server BCP utility to bulk + load data. Temporary staging files will be created and + deleted in the location defined by the TEMP environment + variable. When unselected an insert based build is used. A + BCP build offers vastly improved performance over an insert + based build. + diff --git a/DocBook/docs/images/ch1-10.PNG b/DocBook/docs/images/ch1-10.PNG index 4e65730b..7d91d9a9 100644 Binary files a/DocBook/docs/images/ch1-10.PNG and b/DocBook/docs/images/ch1-10.PNG differ diff --git a/DocBook/docs/images/ch1-11.PNG b/DocBook/docs/images/ch1-11.PNG index 89f628bf..c3715c9b 100644 Binary files a/DocBook/docs/images/ch1-11.PNG and b/DocBook/docs/images/ch1-11.PNG differ diff --git a/DocBook/docs/images/ch1-12.PNG b/DocBook/docs/images/ch1-12.PNG index 4032ad80..f19b171a 100644 Binary files a/DocBook/docs/images/ch1-12.PNG and b/DocBook/docs/images/ch1-12.PNG differ diff --git a/DocBook/docs/images/ch1-12aa.PNG b/DocBook/docs/images/ch1-12aa.PNG index 6c8cca37..774b6509 100644 Binary files a/DocBook/docs/images/ch1-12aa.PNG and b/DocBook/docs/images/ch1-12aa.PNG differ diff --git a/DocBook/docs/images/ch1-12aaa.PNG b/DocBook/docs/images/ch1-12aaa.PNG index 70b2da90..4ef6a498 100644 Binary files a/DocBook/docs/images/ch1-12aaa.PNG and b/DocBook/docs/images/ch1-12aaa.PNG differ diff --git a/DocBook/docs/images/ch1-17.PNG b/DocBook/docs/images/ch1-17.PNG index b0bd4743..47619f44 100644 Binary files a/DocBook/docs/images/ch1-17.PNG and b/DocBook/docs/images/ch1-17.PNG differ diff --git a/DocBook/docs/images/ch1-17aa.PNG b/DocBook/docs/images/ch1-17aa.PNG index 4b773c16..7553af65 100644 Binary files a/DocBook/docs/images/ch1-17aa.PNG and b/DocBook/docs/images/ch1-17aa.PNG differ diff --git a/DocBook/docs/images/ch1-4a.PNG b/DocBook/docs/images/ch1-4a.PNG index aa677425..ce0b5072 100644 Binary files a/DocBook/docs/images/ch1-4a.PNG and b/DocBook/docs/images/ch1-4a.PNG differ diff --git a/DocBook/docs/images/ch1-6.PNG b/DocBook/docs/images/ch1-6.PNG index 85910c71..c6e9e90b 100644 Binary files a/DocBook/docs/images/ch1-6.PNG and b/DocBook/docs/images/ch1-6.PNG differ diff --git a/DocBook/docs/images/ch1-8.PNG b/DocBook/docs/images/ch1-8.PNG index d5905617..303eeebd 100644 Binary files a/DocBook/docs/images/ch1-8.PNG and b/DocBook/docs/images/ch1-8.PNG differ diff --git a/DocBook/docs/images/ch1-9.PNG b/DocBook/docs/images/ch1-9.PNG index bc53c7fb..f6d9bdbf 100644 Binary files a/DocBook/docs/images/ch1-9.PNG and b/DocBook/docs/images/ch1-9.PNG differ diff --git a/DocBook/docs/images/ch13-5.PNG b/DocBook/docs/images/ch13-5.PNG index a8a19db3..33814816 100644 Binary files a/DocBook/docs/images/ch13-5.PNG and b/DocBook/docs/images/ch13-5.PNG differ diff --git a/DocBook/docs/images/ch2-6.PNG b/DocBook/docs/images/ch2-6.PNG index 97400a2d..722cf4c7 100644 Binary files a/DocBook/docs/images/ch2-6.PNG and b/DocBook/docs/images/ch2-6.PNG differ diff --git a/DocBook/docs/images/ch4-15p.PNG b/DocBook/docs/images/ch4-15p.PNG new file mode 100644 index 00000000..a9c037f6 Binary files /dev/null and b/DocBook/docs/images/ch4-15p.PNG differ diff --git a/DocBook/docs/images/ch4-5.PNG b/DocBook/docs/images/ch4-5.PNG index 0ffd3f9c..b272aef2 100644 Binary files a/DocBook/docs/images/ch4-5.PNG and b/DocBook/docs/images/ch4-5.PNG differ diff --git a/DocBook/docs/images/ch4-7.PNG b/DocBook/docs/images/ch4-7.PNG index 476ddbf8..93493930 100644 Binary files a/DocBook/docs/images/ch4-7.PNG and b/DocBook/docs/images/ch4-7.PNG differ diff --git a/DocBook/docs/images/ch4-7a.PNG b/DocBook/docs/images/ch4-7a.PNG index 4064424e..f4726a47 100644 Binary files a/DocBook/docs/images/ch4-7a.PNG and b/DocBook/docs/images/ch4-7a.PNG differ diff --git a/DocBook/docs/images/ch7-10.png b/DocBook/docs/images/ch7-10.png index 7305e0f3..f1680a84 100644 Binary files a/DocBook/docs/images/ch7-10.png and b/DocBook/docs/images/ch7-10.png differ diff --git a/DocBook/docs/images/ch9-9.PNG b/DocBook/docs/images/ch9-9.PNG index 316ecc7e..139f9030 100644 Binary files a/DocBook/docs/images/ch9-9.PNG and b/DocBook/docs/images/ch9-9.PNG differ