1

I wish to write some automated integration tests for a C++ application that uses an external shared library - specifically WiringPi on a Raspberry Pi. This library is used to read/write GPIOs on the RPi. I am currently using WiringPi-Sim on an Intel/AMD Linux host, which is just a set of basic stubs that do nothing but allow the application to be built and run on the non-RPi host, which is where I'd prefer to run the tests.

The C++ application-under-test (AuT) makes a handful of TCP connections to other processes, and receives UDP datagrams from another process. It also writes some data to a serial TTY (normally an external RS485 adapter).

For the tests, I need to check a few things by treating the AuT as a black box and removing as many dependencies on the RPi, serial device, external processes, etc, as possible:

  • That the AuT starts up correctly and initialises a set of GPIOs to specific values.
  • That the AuT changes a set of GPIOs during execution to specific values.
  • That the AuT responds to incoming data on the TCP and UDP sockets.
  • That the AuT writes and reads specific data on the serial TTY.
  • That the AuT shuts down correctly on receipt of an appropriate signal (e.g. SIGINT), that specific data is seen on the TTY as a result, and that the GPIOs are set to specific final values.

My current thinking is to use Python (e.g. pytest) to implement the tests:

  • Simple TCP servers can be created within the Python test app to pretend to be the external TCP server applications that the AuT can connect to,
  • A simple UDP sender within the Python test app can send appropriate data to the UDP socket on the AuT.
  • A fake TTY can be created to act as the serial port, and the python test application can read/write from that (maybe via socat?), as can the AuT,
  • A library or shared object based on WiringPi-Sim can be written that allows the Python test app to monitor and influence the GPIO state (as far as the AuT can see).
  • Then the python app can start the main process and send appropriate signals to check behaviour.

The bit I'm not so sure about is the WiringPi "mock" shared object. I know it can be injected into the AuT process with LD_PRELOAD but I'm not sure how to communicate with the mock object from the Python test application. Would I need to utilise shared memory or some kind of IPC to allow the library instance in the AuT to communicate with the Python test app? Could the Python test app run a TCP server and the WiringPi mock connects "behind the scenes" to relay data/control to the test app? Could the mock object somehow integrate with Python in some python-friendly way?

Can the mocked shared object communicate with the test app via a UNIX socket or pipe, perhaps?

I have the full source for the C++ AuT so I could implement most or all of this in C++ rather than Python, however some aspects (e.g. the simple TCP servers) would be faster and simpler to write in Python. I also like pyenv quite a it.

I wonder if there's already a solution for this kind of test harness. It doesn't need to be Python-based, but that is what I'm used to. Any ideas?

EDIT: another idea is to extract the core of the AuT and test this using C++ language bindings, with dependency injection and C++ mocks. However an important requirement is to test the built application, rather than a source-level modification of the application, as it is important to run these tests on the same binary that is shipped, for safety and audit reasons.

davidA
  • 12,528
  • 9
  • 64
  • 96
  • Someone offline has suggested using a container (LXC, Docker, maybe even QEMU?) to hold the AuT, and then inspect/control the GPIOs from outside the container. Or maybe there's a way to redirect the Linux GPIOs to some kind of dummy driver? I'm not even sure if these approaches would work - has anyone done this before? – davidA Oct 10 '19 at 01:25
  • I'm thinking two simple shared memory buffers would work - one for input, one for output. Then I could hack wiringpi-sim to read/write from the shared memory, and the Python test app can access it from the other side. – davidA Oct 10 '19 at 02:23

1 Answers1

1

Here is a suggestion how it could be done. It is an idea only - I have not tried it out.

First, interaction between Python and C is described in the Python manuals: https://docs.python.org/3/extending/extending.html. You could write your test environment in python, then from some test case call from Python the AuT implemented in C++. And, from this C++ code you can then call some Python code again.

Thus, what might come close to what you are looking for would be the following call chain:

python-test-case -> AuT -> wiringPi-sim -> python-test-callbacks

From the python-test-case you would configure the respective python-test-callbacks to behave as needed for this specific test case. After this setup the python-test-case would call the AuT, which in turn calls one or several wiringPi-sim functions, each of which calls its corresponding python-test-callback.

The problem is, that currently wiringPi-sim implements the interface in a very simple way that does not support to inject specific behaviour. For example, the digitalRead function is stubbed like this:

int digitalRead(int pin) {
    return 1;
}
;

To make it more useful, you would have to extend its functionality such that instead of doing the return 1 you would then call some Python callback for the digitalRead function which can be configured from the Python test case. This would require to modifiy wiringPi-sim or write your own. From what I have seen, however, wiringPi-sim is only a small amount of code, so it might be feasible to do for you.

Dirk Herrmann
  • 5,550
  • 1
  • 21
  • 47