Modifying Regression Test Results for Benchmarking Apache AGE

Question

I'm currently working on making modifications to the source code of the Apache AGE extension for PostgreSQL, and I'm interested in performing some benchmarks to evaluate the impacts of my changes.

Apache AGE uses a series of regression tests, which I've been executing via the make installcheck command.

So I would like to know from where in the source code of Postgres the following results are printed, so I can modify it for my benchmarking purposes:


============== creating temporary instance            ==============
============== initializing database system           ==============
============== starting postmaster                    ==============
running on port 61958 with PID 3343
============== creating database "contrib_regression" ==============
CREATE DATABASE
ALTER DATABASE
============== installing age                         ==============
CREATE EXTENSION
============== running regression test queries        ==============
test scan                         ... ok          240 ms
test graphid                      ... ok           21 ms
test agtype                       ... ok          198 ms
test catalog                      ... ok          116 ms
test cypher                       ... ok           32 ms
test expr                         ... ok          627 ms
test cypher_create                ... ok          125 ms
test cypher_match                 ... ok          517 ms
test cypher_unwind                ... ok           62 ms
test cypher_set                   ... ok          148 ms
test cypher_remove                ... ok          103 ms
test cypher_delete                ... ok          120 ms
test cypher_with                  ... ok           89 ms
test cypher_vle                   ... ok         1166 ms
test cypher_union                 ... ok           42 ms
test cypher_call                  ... ok           61 ms
test cypher_merge                 ... ok          223 ms
test age_global_graph             ... ok          172 ms
test age_load                     ... ok         1830 ms
test index                        ... ok           98 ms
test analyze                      ... ok           35 ms
test graph_generation             ... ok           82 ms
test name_validation              ... ok          166 ms
test drop                         ... ok          236 ms
============== shutting down postmaster               ==============
============== removing temporary instance            ==============

======================
 All 24 tests passed. 
======================

Any advice or insights from those who have experience modifying PostgreSQL extension regression tests would be greatly appreciated. Thank you!

score 1 · Accepted Answer · answered Jul 26 '23 at 14:39

1

AGE does not have its own regression test driver. It inherits the PostgreSQL's one.

You can find it in here and make changes:

https://github.com/postgres/postgres/blob/master/src/test/regress/pg_regress.c

answered Jul 26 '23 at 14:39

Mohayu Din

433
9

score 0 · Answer 2 · answered Jul 24 '23 at 17:10

In the regress folder you will find test cases and their output. The structure is as follows:

age_load: Has data which is used
expected: Has expected output of the test cases.
sql: Has all the test cases written in .sql files.

You can modify the files in sql folder.

score 0 · Answer 3 · answered Jul 24 '23 at 17:55

0

The regression tests are found in the regress directory of Apache AGE repo.

The subdirectories ./expected and ./sql contain files that tests the extensions. Specifically expr.out and expr.sql, these files tests various functions.

answered Jul 24 '23 at 17:55

Tito

289
8

score 0 · Answer 4 · answered Jul 24 '23 at 18:47

The tests itself are done through PostgreSQL's test strategy and Apache AGE make use of it, which means Apache AGE are following the same schema/structure of the tests and seeds the scripts to be tested inside the PostgreSQL server.

To add a new testing script simply you can do the following

Locate it under the correct directory like the following age/regress/sql/$(your_script.sql)
Add that to the list of REGRESS in age/Makefile
Run the tests (it will fail) copy the results of your script to the expected directory under regress after revising and making sure of it.
Run again (it should pass now)

score 0 · Answer 5 · answered Jul 25 '23 at 01:32

you can find the regression tests at src/backend/utils/ directory.

an example is the file name validation.c which contains a lot of functions to test the name validation cases. You can create your own function to test the desired benchmarks.

but you have to change the MAKEFILE to run the created test function. For example, if you have created the test_vertexfunc

REGRESS = scan \
      graphid \
      agtype \
      catalog \
      cypher \
      expr \
      cypher_create \
      cypher_match \
      cypher_unwind \
      cypher_set \
      cypher_remove \
      cypher_delete \
      cypher_with \
      cypher_vle \
      cypher_union \
      cypher_call \
      cypher_merge \
      age_global_graph \
      age_load \
      index \
      analyze \
      graph_generation \
      name_validation \
      test_vertexfunc \ <<<<<<
      drop

score 0 · Answer 6 · answered Jul 25 '23 at 12:20

The regression tests can be found in the sql files inside /regress/sql. And their expected results can be found in /regress/expected in the form of .out files.

In case you wish to add or modify a regression test, modify the query in .sql file (if you have to) and modify the expected result in the .out file.

When the tests are run, the output of the tests is checked against the expected outputs and suitably they are either failed or passed.

Any regression test differences (in case of failed tests) can be found in /regress/regression.diffs . This file can be opened using any text editor.

score -1 · Answer 7 · answered Jul 27 '23 at 12:59

-1

As Mohayu Din mentioned here, to find these results you need to access the postgres regression test driver, since AGE inherits it.

You can find the link here.

answered Jul 27 '23 at 12:59

mtoaima

1
2

score -1 · Answer 8 · answered Jul 27 '23 at 18:02

-1

Firstly, Apache AGE is in the development phase and it does not have its own regression tester so far.
The regression tester of Postgresql is inherited by AGE. So you can access that from here and modify it according to your benchmarking purposes.

answered Jul 27 '23 at 18:02

adil shahid

125
4

score -1 · Answer 9 · answered Jul 31 '23 at 06:48

-1

Apache Age depends on the regression tester of Postgresql. You can find this link useful

answered Jul 31 '23 at 06:48

Prachi

39
3

Modifying Regression Test Results for Benchmarking Apache AGE

9 Answers9