4

I am trying to download files from a Google Firebase Test Lab device. These files are generated from running instrumented tests. The files are screenshots. I need to download the screenshots because subsequent runs of the tests will compare screens to these screenshots. Thus the screenshots are a baseline for visual change detection.

I've tried looking for a way to connect to the Firebase Test Lab device's Device File Explorer. But there does not seem to be a way to access it remotely.

Since Firebase Test Lab cleans up the device after the test, I assume the only way I can achieve what I want to do is to add code to the test to push the files someplace.

One place I'd like to push the files to is the Google Cloud storage bucket where the test results are stored. From the documentation below, it seems like that is an intended purpose of the storage bucket:

After Test Lab completes testing of your app, you can review test results in the Firebase console or in a Google Cloud storage bucket in your project. You can also add a gsutil command to the shell command shown above to copy the test results data to your local computer. To learn more, see Analyzing Firebase Test Lab Results. (Firebase documentation)

For this approach, how would my instrumented tests push the files to the bucket? How does it know what bucket to use? What is the API to perform the file transfer?

Another approach I could take is Take Screenshots from Firebase Test Lab Instrumentation Tests That would get the screenshots into the bucket. But I see two potential problems.

The first problem is the screenshot library's screenshots might differ from the screenshots taken from UiAutomation.takeScreenshot(). In other words, the baseline screenshot is taken with screenshot library but on subsequent runs I would take the screenshot with UiAutomation.takeScreenshot() and compare them. Since the libraries are different, their screenshots might differ. A possible solution would be to take both screenshots with the screenshot library. But then I would need to figure out how to read the screenshot back into memory so I can perform my comparison operations. Today I read them in as Bitmaps.

For this approach, how would my instrumented tests load screenshot library's screenshots as Bitmaps?

A second problem is figuring out how to download the screenshots from the bucket to my local computer. The user interface for the bucket does not have a download button. It only has an upload button:

enter image description here

For this approach, how would I download files from the bucket to my local computer so I can add them to the resources directory that my instrumented test can read?

Maik
  • 3,419
  • 1
  • 23
  • 34
Michael Osofsky
  • 11,429
  • 16
  • 68
  • 113

2 Answers2

5

Alright, I hope I captured all your questions below:

By default all files from /sdcard/Screenshots are copied from the device to the GCS bucket you have access to. You can write your own files during the test into the same folder on the device. Test Lab will pick them up and copy them to the bucket.

If you want to download additional folders and download them locally, I can recommend flank which has an option called "directories-to-pull" to specify multiple directories. It already provides a lot of what you try to achieve by hand and more.

In case you don't want to use flank, gcloud has this option too, but puts the file into your GCS bucket. No download to local machine.

The Cloud Storage UI seems fairly limited. It allows you to download individual files, but not folders as a zip. Use gsutil to download files recursively:

gsutil -m cp -r gs://path/to/bucket/folder path/to/local/folder

By default Test Lab provides a GCS bucket location for you where it copies the files to, but you can use your own bucket and location e.g. using the gcloud client:

gcloud firebase test android run --results-bucket=gs://my-bucket --results-dir=path/to/folder
Maik
  • 3,419
  • 1
  • 23
  • 34
  • Thank you maik. To write my own files to `/sdcard/Screenshots` from my instrumented test, I tried APIs described at https://developer.android.com/training/data-storage/files but none of them worked. For example, `getApplicationContext().getExternalFilesDir(Environment.DIRECTORY_PICTURES)` gave me `/storage/emulated/0/Android/data/com.locuslabs.android.sdk/files/Pictures` instead of `/sdcard/Screenshots`. I could also try taking screenshots via https://firebase.google.com/docs/test-lab/android/test-screenshots?authuser=0. What would you do? – Michael Osofsky Jan 28 '19 at 18:41
  • I should also add, that I tried saving directly to `/sdcard/Screenshots` but I got `java.io.FileNotFoundException` ... `(Permission denied)`. – Michael Osofsky Jan 28 '19 at 19:01
  • However, I did get it working when I simply save the screenshot to a file in a directory I opened with `File imageFolder = new File("/sdcard/screenshots/")`. My mistake may have been thinking I could run the instrumented test in Android Studio on the debugger using a physical device. – Michael Osofsky Jan 28 '19 at 19:19
  • 1
    `Environment.getExternalStorageDirectory()` should give you the sdcard directory. The path might be something like `/storage/emulated/0`, but it should actually point to the same directory as `/sdcard`. – Maik Jan 28 '19 at 20:00
1

A couple more bits of information for you. You don't need to use Flank in order to use --directories-to-pull. That flag is available in Test Lab's gcloud firebase test android run command -- see https://cloud.google.com/sdk/gcloud/reference/firebase/test/android/run for more information.

Also, you can certainly download files from the Cloud Storage UI -- just click on the filename and it will ask you where you want to save the file on your local drive (or depending on the file's type, you may have to right-click and choose "Save Link As...").

P. Davis
  • 929
  • 6
  • 10