HTML Publisher plugin failed to extract (only when Serenity tests fail)

Question

I am using Serenity BDD for my automation test framework and now I wish to publish the the results onto Jenkins using the HTML Publisher plugin.

When all my tests pass, no problem, I can publish the report with no issues, but if 1 single test fails, I get an error during the publishing stage from the HTML Publisher plugin. The error I receive is due to "truncated TAR archive". Not really sure what this means. Looking deeper at the error, I can see: Caused by: java.io.IOException: Entry 'pagesource11965942031956863624.html' closed at '0' before the '106438' bytes specified in the header were written, but I also don't know what this means.

Here is the full error (with some directory partly hidden):

java.io.IOException: Failed to extract /home/jenkins/agent/workspace/*<someJenkinsDirs>*/target/site/serenity/**/*
    at hudson.FilePath.readFromTar(FilePath.java:2982)
    at hudson.FilePath.copyRecursiveTo(FilePath.java:2742)
    at hudson.FilePath.copyRecursiveTo(FilePath.java:2706)
    at hudson.FilePath.copyRecursiveTo(FilePath.java:2694)
    at hudson.FilePath.copyRecursiveTo(FilePath.java:2677)
    at htmlpublisher.HtmlPublisher.publishReports(HtmlPublisher.java:242)
    at htmlpublisher.workflow.PublishHTMLStepExecution.run(PublishHTMLStepExecution.java:62)
    at htmlpublisher.workflow.PublishHTMLStepExecution.run(PublishHTMLStepExecution.java:44)
    at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
    Suppressed: java.util.concurrent.ExecutionException: java.io.IOException: Entry 'pagesource11965942031956863624.html' closed at '0' before the '106438' bytes specified in the header were written
        at hudson.remoting.Channel$2.adapt(Channel.java:1036)
        at hudson.remoting.Channel$2.adapt(Channel.java:1030)
        at hudson.remoting.FutureAdapter.get(FutureAdapter.java:66)
        at hudson.FilePath.copyRecursiveTo(FilePath.java:2745)
        ... 12 more
    Caused by: java.io.IOException: Entry 'pagesource11965942031956863624.html' closed at '0' before the '106438' bytes specified in the header were written
        at org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.closeArchiveEntry(TarArchiveOutputStream.java:429)
        at hudson.util.io.TarArchiver.visit(TarArchiver.java:117)
        at hudson.util.DirScanner.scanSingle(DirScanner.java:51)
        at hudson.util.DirScanner$Glob.scan(DirScanner.java:146)
        at hudson.FilePath.writeToTar(FilePath.java:2943)
        at hudson.FilePath$CopyRecursiveRemoteToLocal.invoke(FilePath.java:2908)
        at hudson.FilePath$CopyRecursiveRemoteToLocal.invoke(FilePath.java:2893)
        at hudson.FilePath$FileCallableWrapper.call(FilePath.java:3492)
        at hudson.remoting.UserRequest.perform(UserRequest.java:211)
        at hudson.remoting.UserRequest.perform(UserRequest.java:54)
        at hudson.remoting.Request$2.run(Request.java:376)
        at hudson.remoting.InterceptingExecutorService.lambda$wrap$0(InterceptingExecutorService.java:78)
        at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at hudson.remoting.Engine$1.lambda$newThread$0(Engine.java:121)
        at java.base/java.lang.Thread.run(Unknown Source)
        Suppressed: java.io.IOException: This archive contains unclosed entries.
            at org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.finish(TarArchiveOutputStream.java:291)
            at org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.close(TarArchiveOutputStream.java:309)
            at hudson.util.io.TarArchiver.close(TarArchiver.java:124)
            at hudson.FilePath.writeToTar(FilePath.java:2942)
            ... 12 more
        Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to JNLP4-connect connection from 61.0.33.251/61.0.33.251:52128
            at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1784)
            at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:356)
            at hudson.remoting.Channel$2.adapt(Channel.java:1034)
            at hudson.remoting.Channel$2.adapt(Channel.java:1030)
            at hudson.remoting.FutureAdapter.get(FutureAdapter.java:66)
            at hudson.FilePath.copyRecursiveTo(FilePath.java:2745)
            at hudson.FilePath.copyRecursiveTo(FilePath.java:2706)
            at hudson.FilePath.copyRecursiveTo(FilePath.java:2694)
            at hudson.FilePath.copyRecursiveTo(FilePath.java:2677)
            at htmlpublisher.HtmlPublisher.publishReports(HtmlPublisher.java:242)
            at htmlpublisher.workflow.PublishHTMLStepExecution.run(PublishHTMLStepExecution.java:62)
            at htmlpublisher.workflow.PublishHTMLStepExecution.run(PublishHTMLStepExecution.java:44)
            at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
            at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
            at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
            at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
            at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
            at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.io.IOException: Truncated TAR archive
    at org.apache.commons.compress.archivers.tar.TarArchiveInputStream.read(TarArchiveInputStream.java:743)
    at java.base/java.io.InputStream.read(InputStream.java:205)
    at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1309)
    at org.apache.commons.io.IOUtils.copy(IOUtils.java:978)
    at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1282)
    at org.apache.commons.io.IOUtils.copy(IOUtils.java:953)
    at hudson.util.IOUtils.copy(IOUtils.java:52)
    at hudson.FilePath.readFromTar(FilePath.java:2972)
    ... 13 more

I've also tried using archiveArtefacts but I get the exact same error (guessing they both function in similar ways).

How do I fix this error?

EDIT:

I've done some more digging into this issue and I've found that the pagesource<random20numbers>.html is generated when either a test fails or a test has an error (not sure if there are other situations in which this file in generated). The file itself seems to be the HTML and CSS code for the webpage that Serenity can "see" in the test failure/error (ie, the page being viewed by the Selenium webdriver when the test fails).

I've tried deleting the file in my Jenkins pipeline with a simple rm pagesource<numbers> command, but ran into permission issues (will speak to people on team to resolve this) and I've also tried to use the includes parameter for the HTML publisher plugin. The Glob expression I have tried is **/!(pagesource)* but this then causes me to get the error:

ERROR: Directory '<someJenkinsDirs>/target/site/serenity' exists but failed copying to '<someJenkinsDirs>/builds/29/htmlreports/Serenity_20Report'.

score 2 · Accepted Answer · answered May 18 '23 at 19:33

I wanted to update the official answer of this question as I have now found a viable solution that allows for all the Serenity Reporting functionality as well as the ability to zip/compress the report and attach it as an artefact to builds or publish the report in another way in a Jenkins pipeline. I have decided to add this as a new answer as it's been about 10 months since I posted my last answer, which is actually more of a workaround/hack that didn't really solve the problem, but still allowed you to zip the report with some missing functionality. Please be aware that this is a direct copy/paste from the issue that I opened on the serenity-core Github.

For some reason, the pagesource<LotsOfNumbers>.html files have some different permissions to all the other files generated in the report directories, no idea why this is the case. Anyway, I have just simply done a chmod -R 777 <mySerenityReportDirectory> to give full permissions to the files for all users.

After this I can very easily use a tar shell command in my pipeline and do whatever I want with the report files. In my case, I am using the archiveArtifacts plugin to add the report to the build in a way that allows me to download it.

Some caveats to this solution that people should bear in mind. I am using Jenkins with Kubernetes so I have a container with Maven installed and I am running both my test execution command (mvn clean verify [options]) and the permission change command (chmod -R 777) inside this container. Running the chmod outside the container results in operation not permitted when attempting to tar the files, again I think this has something to do with permissions of the generated files.

Here's some of my pipeline code:


            container('maven') {
                stage('Run tests') {
                    FAILED_STAGE = env.STAGE_NAME
                    catchError(buildResult: 'SUCCESS', stageResult: 'UNSTABLE') {

                        currentBuild.displayName = params.BRANCH
                        currentBuild.description = params.TAG_FILTER;

                        sh("""
                            mvn -B clean integration-test \
                                -Denvironment=${params.ENVIRONMENT.toLowerCase()} \
                                -P ${params.TAG_FILTER.toLowerCase()} \
                                serenity:aggregate
                        """)
                    }
                    sh('chmod -R 777 target/site/serenity')
                }
            }
            
            stage('Publish test report') {
                FAILED_STAGE = env.STAGE_NAME
                cucumber buildStatus: 'null', fileIncludePattern: '**/cucumber.json', jsonReportDirectory: 'target/cucumber-reports', reportTitle: 'Cucumber Test Report'

                sh('tar -czf SerenityReport.tar.gz target/site/serenity')
                archiveArtifacts(artifacts: "SerenityReport.tar.gz", fingerprint: true)
            }

score 1 · Answer 2 · answered Jul 31 '22 at 19:47

I've managed to find 1 solution to this problem, it's not quite what I wanted, but it'll until I find or am shown another way.

Serenity takes screenshots during the test and this functionality can be customised in the serenity.properties file. The options available are:

FOR_EACH_ACTION
BEFORE_AND_AFTER_EACH_STEP
AFTER_EACH_STEP
FOR_FAILURES
DISABLED

I simply disabled the screenshot functionality and the pagesource<numbers>.html is no longer generated. More about managing Serenity screenshots here

I believe the pagesource.html file is generated so that a tester can debug why the test is failing or erroring out. Disabling screenshots means that the following page isn't available in the Serenity report, meaning that there is no need to generate the pagesource.html file (the HTML source button links to the pagesource.html file).

If anyone finds a better solution to this problem that doesn't involve disabling screenshots, please post it, as I'd prefer to have screenshots enabled so that it is easier for our BA's to see the user journey.

I posted this issue on the Serenity-core github page and the creator of Serenity says that this issue should be fixed with the latest release. I haven't tried it myself quite yet but I intend to. [Link to the issue page](https://github.com/serenity-bdd/serenity-core/issues/2878). — Mo0rBy, Jan 23 '23 at 08:59

score 1 · Answer 3 · answered May 18 '23 at 13:33

The serenity report failure issue was figured out.thanks to @Mo0rBy So now you can get the screenshots as well no need to disable screenshots.

For some reason the serenity report files are having different permission and hence the archive issue.

Below is the pseudo code:

I have put the clean test aggregate command inside a catcherror block so that it continues to execute other commands. I have given full permission to the target/site/serenity folder to everyone.

stages {
  stage('Run Test Cases') {
    steps {
      container('myproject') {
          catchError(buildResult: 'SUCCESS', stageResult: 'UNSTABLE') {
              sh """
          pwd
          ls
          echo ${WORKSPACE}
          gradle --info clean test aggregate -Denvironment=${params.APP_ENV} -Dtags='envr:'${params.APP_ENV} -Dcucumber.options='--tags @envr='${params.APP_ENV}
         """
          }
          sh """
          echo 'Allowing Full permission to target File'
          chmod -R 777 target/site/serenity
          """
      }
    }
  }
post {
  always {
    publishHTML (target : [allowMissing: false,
      alwaysLinkToLastBuild: false,
      keepAll: true,
      reportDir: 'target/site/serenity',
      reportFiles: 'index.html',
      reportName: 'qa-my-project-bdd-reports',
      reportTitles: 'The Serenity Report'])
  }
}

Thanks for this, I am Mo0rBy on Github by the way, I probably should have updated my answer here as well though. — Mo0rBy, May 18 '23 at 19:23
I've updated this question with my answer from [the github issue](https://github.com/serenity-bdd/serenity-core/issues/2878) as well — Mo0rBy, May 18 '23 at 19:34

score 0 · Answer 4 · answered Jan 20 '23 at 12:12

0

For others, that might be Googling and find a similar stack trace. We had that cryptic stack trace, because Windows Defender removed file while copying. The fix was, that we had to disable "Real-time protection".

answered Jan 20 '23 at 12:12

Alojzij Blatnik

81
11

HTML Publisher plugin failed to extract (only when Serenity tests fail)

4 Answers4