Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

LID benchmarking tool #1276

Merged
merged 30 commits into from
May 19, 2023
Merged

LID benchmarking tool #1276

merged 30 commits into from
May 19, 2023

Commits on Apr 4, 2023

  1. feat: LID benchmarking tool

    @dirkmc
    dirkmc committed Apr 4, 2023
    Configuration menu
    Copy the full SHA
    8e72cce View commit details
    Browse the repository at this point in the history

  2. fix: bench thread safety

    @dirkmc
    dirkmc committed Apr 4, 2023
    Configuration menu
    Copy the full SHA
    0713d3f View commit details
    Browse the repository at this point in the history

  3. refactor: structured logging

    @dirkmc
    dirkmc committed Apr 4, 2023
    Configuration menu
    Copy the full SHA
    c3e69a0 View commit details
    Browse the repository at this point in the history

  4. refactor: postgres bulk insert

    @dirkmc
    dirkmc committed Apr 4, 2023
    Configuration menu
    Copy the full SHA
    806b8ec View commit details
    Browse the repository at this point in the history

  5. lid bench: Add foundationdb impl

    @magik6k @dirkmc
    magik6k authored and dirkmc committed Apr 4, 2023
    Configuration menu
    Copy the full SHA
    a66c57e View commit details
    Browse the repository at this point in the history

  6. lid fdb: Fix Tx sizing, parallel chunk puts

    @magik6k @dirkmc
    magik6k authored and dirkmc committed Apr 4, 2023
    Configuration menu
    Copy the full SHA
    482f842 View commit details
    Browse the repository at this point in the history

  7. lid fdb: More efficient sample generation

    @magik6k @dirkmc
    magik6k authored and dirkmc committed Apr 4, 2023
    Configuration menu
    Copy the full SHA
    f2eeb12 View commit details
    Browse the repository at this point in the history

  8. feat: array of piece count / blocks per piece (#1314)

    @dirkmc
    dirkmc committed Apr 4, 2023
    Configuration menu
    Copy the full SHA
    91fee5b View commit details
    Browse the repository at this point in the history

  9. lid bench: print add rate

    @magik6k @dirkmc
    magik6k authored and dirkmc committed Apr 4, 2023
    Configuration menu
    Copy the full SHA
    e10de81 View commit details
    Browse the repository at this point in the history

  10. lid bench: Add retry to postgres put (#1316)

    @magik6k @dirkmc
    magik6k authored and dirkmc committed Apr 4, 2023
    Configuration menu
    Copy the full SHA
    1d0dfe7 View commit details
    Browse the repository at this point in the history

  11. lid bench: Make cassandra put much more robust (#1318)

    @magik6k @dirkmc
    magik6k authored and dirkmc committed Apr 4, 2023
    Configuration menu
    Copy the full SHA
    366b58a View commit details
    Browse the repository at this point in the history

Commits on Apr 5, 2023

  1. instrumentation for bench tool (#1337) 

    * instrument postgres

* more instrumentation

* check for err getoffsetsize

* emit metrics every 10sec

* ignore errors

* add postgres-drop

* use directly tables

* fix: go mod tidy

* use INSERT INTO instead of tmp tables

* try to catch sig

* remove transaction commit

* fixup

* add postgres-init

* fixuop

* split create and init

* fixup

* remove if not exist

---------

Co-authored-by: Dirk McCormick <dirkmdev@gmail.com>
    @nonsense @dirkmc
    nonsense and dirkmc committed Apr 5, 2023
    Configuration menu
    Copy the full SHA
    f8ca07d View commit details
    Browse the repository at this point in the history

  2. feat: batch insert queries for postgres

    @dirkmc
    dirkmc committed Apr 5, 2023
    Configuration menu
    Copy the full SHA
    313421f View commit details
    Browse the repository at this point in the history

  3. feat: add flag to insert into postgres using tmp table

    @dirkmc
    dirkmc committed Apr 5, 2023
    Configuration menu
    Copy the full SHA
    49e5105 View commit details
    Browse the repository at this point in the history

Commits on Apr 6, 2023

  1. refactor: merge changes from nonsense/lid-bench

    @dirkmc
    dirkmc committed Apr 6, 2023
    Configuration menu
    Copy the full SHA
    e307ecf View commit details
    Browse the repository at this point in the history

  2. refactor: just use one database (dont create bench database)

    @dirkmc
    dirkmc committed Apr 6, 2023
    Configuration menu
    Copy the full SHA
    064f03b View commit details
    Browse the repository at this point in the history

  3. refactor: remove unused params

    @dirkmc
    dirkmc committed Apr 6, 2023
    Configuration menu
    Copy the full SHA
    f7fb109 View commit details
    Browse the repository at this point in the history

Commits on Apr 7, 2023

  1. refactor: command structure

    @dirkmc
    dirkmc committed Apr 7, 2023
    Configuration menu
    Copy the full SHA
    6102aa5 View commit details
    Browse the repository at this point in the history

Commits on Apr 13, 2023

  1. fix: cassandra - dont use batch insert for PayloadToPieces

    @dirkmc
    dirkmc committed Apr 13, 2023
    Configuration menu
    Copy the full SHA
    d799fd4 View commit details
    Browse the repository at this point in the history

  2. fix: create tables CQL

    @dirkmc
    dirkmc committed Apr 13, 2023
    Configuration menu
    Copy the full SHA
    b12415b View commit details
    Browse the repository at this point in the history

Commits on Apr 14, 2023

  1. fix: increase payload to pieces insert parallelism

    @dirkmc
    dirkmc committed Apr 14, 2023
    Configuration menu
    Copy the full SHA
    5820c69 View commit details
    Browse the repository at this point in the history

  2. fix: use simple replication strategy

    @dirkmc
    dirkmc committed Apr 14, 2023
    Configuration menu
    Copy the full SHA
    8a1ad94 View commit details
    Browse the repository at this point in the history

  3. feat: use yugabyte cassandra driver

    @dirkmc
    dirkmc committed Apr 14, 2023
    Configuration menu
    Copy the full SHA
    4963af9 View commit details
    Browse the repository at this point in the history

  4. fix: remove bench binary

    @dirkmc
    dirkmc committed Apr 14, 2023
    Configuration menu
    Copy the full SHA
    36a80a3 View commit details
    Browse the repository at this point in the history

  5. update metrics endpoint

    @nonsense
    nonsense committed Apr 14, 2023
    Configuration menu
    Copy the full SHA
    371d915 View commit details
    Browse the repository at this point in the history

  6. fix random generated piece cid

    @nonsense
    nonsense committed Apr 14, 2023
    Configuration menu
    Copy the full SHA
    c159cff View commit details
    Browse the repository at this point in the history

  7. fixup

    @nonsense
    nonsense committed Apr 14, 2023
    Configuration menu
    Copy the full SHA
    4add736 View commit details
    Browse the repository at this point in the history

  8. fix: cassandra bitswap benchmark

    @dirkmc
    dirkmc committed Apr 14, 2023
    Configuration menu
    Copy the full SHA
    136f579 View commit details
    Browse the repository at this point in the history

Commits on May 19, 2023

  1. remove foundationdb

    @nonsense
    nonsense committed May 19, 2023
    Configuration menu
    Copy the full SHA
    6023cf7 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'nonsense/piece-store' into feat/lid-bench

    @nonsense
    nonsense committed May 19, 2023
    Configuration menu
    Copy the full SHA
    d083ec9 View commit details
    Browse the repository at this point in the history