2

Is there a tool for helping to automate integration tests for Gnome Shell extensions?

In web development, it's common to use tools like Selenium to simulate a human interaction by programmatically interacting with a web browser and confirming functionality.

Is there anything similar for scripting interactions in Gnome and Gnome Shell?

I maintain an open source Gnome Shell extension, where most of the functionality is in the form of GUI elements exposed in the panel. Unfortunately, it's common for user-submitted bug fixes to contain regressions. A major pain point are new Gnome releases, which are frequent and often contain backwards incompatible changes whose effects can be subtle.

I currently have some minimal automated testing to confirm the installation process works, but it doesn't catch many feature bugs. All real testing has to be done manually, which is very tedious and time consuming, even though the nature of the tests isn't complicated. It's also difficult to test a new version of Gnome, as that requires installing it system wide, which often isn't practical.

To speed up development and reduce the workload of myself and other volunteer testers, I'd like to write some automated tests in something like Docker that install the extension in a specific version of Gnome, restart Gnome Shell, confirm the GUI elements are present, click a button, confirm a preference dialog appears, etc.

Is this possible?

Cerin
  • 60,957
  • 96
  • 316
  • 522

1 Answers1

3

The short answer is not yet.

There is a Dockerfile in GNOME Shell for running basic syntax, linting and packaging tests. There is an example of it in use for the Top Icons extension here.

Evan Welsh has a project that can generate TypeScript definitions from GObject Introspection (https://gitlab.gnome.org/rockon999/gi.ts). If GNOME Shell were documented with TypeScript annotations, it could be scraped and you could run much more comprehensive static analyses on extensions. This would probably cover 95% of breakages between versions.

As far as automated GUI tests, dogtail exists, although I would be surprised if that worked with GNOME Shell. The go to for distributions seems to be OpenQA (eg. Fedora), but I'm not aware of anyone employing this on a smaller scale.

I believe it's only recently that upstream GNOME has started providing full, vanilla VM images (See: https://gitlab.gnome.org/GNOME/gnome-build-meta/pipelines). Again, I'm not aware of anyone using these for automated testing of extensions (yet).

andy.holmes
  • 3,383
  • 17
  • 28
  • This is what I feared, but thanks for the summary. Dogtail looks interesting, even if no one yet seems to be using it for extension testing. – Cerin Mar 11 '20 at 12:57