Announcing pgio (The SLOB Method for PostgreSQL) Is Released Under Apache 2.0 and Available at GitHub

This is just a quick post to advise readers that I have released pgio (The SLOB Method for PostgreSQL) under Apache 2.0. The bits are available at the following link: https://github.com/therealkevinc/pgio/releases/tag/1.0. The README is quite informative.

My last testing before the release showed “out of the box” data loading into Amazon Aurora with PostgreSQL compatibility at a rate of 1.69 TB/h. I only modified the pgio.conf file to specify the connection string and to set scale to 128 GB per schema:

After loading data I edited pgio.conf to increase the number of threads per schema to 16 and then easily drove IOPS to the current, advertised IOPS limit of 120,000 for Amazon Aurora with PostgreSQL compatibility.

Testing PostgreSQL physical I/O on any platform could not be any easier, repeatable, nor understandable.

Enjoy pgio!

 

 

5 Responses to “Announcing pgio (The SLOB Method for PostgreSQL) Is Released Under Apache 2.0 and Available at GitHub”


  1. 1 DomenSetar (@DomenSetar) October 30, 2019 at 6:09 am

    I have a problem:

    ./runit.sh
    Date: Wed Oct 30 13:57:46 CET 2019
    Database connect string: “pgbench”.
    Shared buffers: 1GB.
    Testing 4 schemas with 2 thread(s) accessing 100M (12800 blocks) of each schema.
    Running iostat, vmstat and mpstat on current host–in background.
    Launching sessions. 4 schema(s) will be accessed by 2 thread(s) each.
    pg_stat_database stats:
    datname| blks_hit| blks_read|tup_returned|tup_fetched|tup_updated
    BEFORE: pgbench | 1215174 | 108818 | 742598 | 274296 | 102
    AFTER: pgbench | 1215174 | 108818 | 742598 | 274296 | 102
    awk: cmd. line:1: (FILENAME=- FNR=1) fatal: division by zero attempted

  2. 4 karthik (@karthik365365) March 15, 2022 at 3:44 am

    Hi Mr.Kevin, I was trying SLOB for postgres and hit the same error. Could you please help here? ‘setup.sh’ ran fine and loaded approx 400gb data but ‘runit.sh’ is failing,

    ======
    postgres@st-caf-postgresdb pgio-master]$ ./runit.sh 1
    Date: Tue Mar 15 10:27:24 UTC 2022
    Database connect string: “actdb”.
    Shared buffers: 128MB.
    Testing 2 schemas with 2 thread(s) accessing 2G (262144 blocks) of each schema.
    Running iostat, vmstat and mpstat on current host–in background.
    Launching sessions. 2 schema(s) will be accessed by 2 thread(s) each.
    pg_stat_database stats:
    datname| blks_hit| blks_read|tup_returned|tup_fetched|tup_updated
    BEFORE: actdb | 164038048 | 69196715 | 283795857 | 3920392 | 123
    AFTER: actdb | 164038048 | 69196715 | 283795857 | 3920392 | 123
    awk: cmd. line:1: (FILENAME=- FNR=1) fatal: division by zero attempted
    =====

    As suggested by you in the earlier comment, ran” sh -x ./runit.sh > runit.out 2>&1″ and from the runin.out file i see below might be the problem,
    =========
    ++ psql actdb -t –quiet -c ‘select blks_hit from pg_stat_database where datname = ‘\”actdb’\”’
    + after_cache_hits=’ 164040682′
    + run_time=0
    + echo ‘0 69196715 69196715 actdb 2 2 164040682 164040682’
    + f_format_output
    + awk ‘{ printf(“DBNAME: %s. %d schemas, %d threads(each). Run time: %d seconds. RIOPS >%ld%ld<\n\n", $4,$5,$6, $1 , ($2 – $3 ) / $1, ($8 – $7) / $1 ) }'
    awk: cmd. line:1: (FILENAME=- FNR=1) fatal: division by zero attempted
    + /bin/kill -9 4718 4719 4720
    ===========


Leave a comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.




DISCLAIMER

I work for Amazon Web Services. The opinions I share in this blog are my own. I'm *not* communicating as a spokesperson for Amazon. In other words, I work at Amazon, but this is my own opinion.

Enter your email address to follow this blog and receive notifications of new posts by email.

Join 743 other subscribers
Oracle ACE Program Status

Click It

website metrics

Fond Memories

Copyright

All content is © Kevin Closson and "Kevin Closson's Blog: Platforms, Databases, and Storage", 2006-2015. Unauthorized use and/or duplication of this material without express and written permission from this blog’s author and/or owner is strictly prohibited. Excerpts and links may be used, provided that full and clear credit is given to Kevin Closson and Kevin Closson's Blog: Platforms, Databases, and Storage with appropriate and specific direction to the original content.