Copy and retain previous output for backup of transformed json data

Question

I have a perl script that transforms json data to perl and saves output in files called teams.txt, backyard, and also a file called backup.txt, where the output of teams.txt is copied from. The following are two snippets from the script/the part of it that writes the data to the text files:

my %manager_to_directs;
my %user_to_manager;
my @users;
my $url = "https://xxxxxxxxxxxxxx.com/api/v1/reports/active/week";

my $useragent = LWP::UserAgent->new();
my $response = $useragent->get(($url));
if ($response->code !~ "200" || $response->code !~ "204" ){

while ($url && $url ne "Null") {
  my $data = fetch_json($url);
  last if !defined $data;

  $url  = $data->{next};
.
.
.

# write backyard.txt
open my $backyard_fh, ">", "backyard.txt";
foreach my $user (sort keys %user_to_management_chain) {
  my $chain = join ',', @{$user_to_management_chain{$user}};
  print $backyard_fh "$user:$chain\n";
}
close $backyard_fh;

# write teams.txt
open my $team_fh, ">", "teams.txt";
foreach my $user (sort @users) {
  my $followers = $manager_to_followers{$user};
  my $followers_joined = $followers ? join (',', sort @$followers) : "";
  print $team_fh "$user:$followers_joined\n";
}
close $team_fh;

# write backup.txt, backup for teams.txt
open my $backup_fh, ">", "backup.txt";
copy("teams.txt", "backup.txt")
    or die ("Can't copy teams.txt \n");
close $backup_fh;

This works almost exactly how I want it to, but now I've been testing with a negative scenario, where the .json url provided in the script is false/nonexistent, and I have to make sure that not another teams.txt file is created and the backup.txt file is still retained from the last execution.

I tested by replacing

my $url = "https://xxxxxxxxxxxxxx.com/api/v1/reports/active/week";

with

my $url = "https://fakeUrl.com/api/v1/reports/active/week";

And in this scenario, 404 would be passed and the program is supposed to fail. With this test, I noticed that the the contents of teams.txt and backyard.txt get wiped, but the backup.txt file gets wiped too...and that's not good.

I'm fine with teams.txt and backyard.txt being overwritten per each run of the script, but I need the backup.txt file to be retained no matter what, unless the program runs successfully and there's new content from teams.txt to be copied over to backup.txt.

Any help I can get is highly appreciated!

I don't see that the code actually exits (or skips processing) if response is bad ... ? For one, why not test with `if (not $response->is_success) { die $response->status_line }` or such? Note that as soon as a file is opened for writing its content is wiped out (truncated). — zdim, Jun 02 '20 at 17:15
I actually noticed something weird..and it may or may not be related to this question, but I've since tested with `die $response->status_line`, and `$response` is bring printed out as "HTTP::Response=HASH(0x7f8e50b3fc08)". So now I'm thinking that my perl script isn't able to read and process the http header. — Ian Balas, Jun 02 '20 at 17:58

score 1 · Answer 1 · answered Jun 02 '20 at 18:34

Following code snippets taken almost directly from documentation for modules.

May be you should try this approach.

use strict;
use warnings;
use feature 'say';

use LWP::UserAgent ();

my $url = 'https://metacpan.org/pod/HTTP::Tiny';
   $url = 'https://fakeUrl.com/api/v1/reports/active/week';
my $ua = LWP::UserAgent->new(timeout => 10);
$ua->env_proxy;

my $response = $ua->get($url);
my $data; 

if ($response->is_success) {
    $data = $response->decoded_content;
}
else {
    die $response->status_line;
}

# Process further data
say $data;

Output

500 Can't connect to fakeUrl.com:443 (Bad file descriptor) at C:\....\http_lwp.pl line 19.

use strict;
use warnings;
use feature 'say';

use HTTP::Tiny;

my $url = 'https://metacpan.org/pod/HTTP::Tiny';
   $url = 'https://fakeUrl.com/api/v1/reports/active/week';
my $data;

my $response = HTTP::Tiny->new->get($url);

if( $response->{success} ) {
    $data = $response->{content};
} else {
    say "$response->{status} $response->{reason}";
    exit 1;
}

# Process further data
say $data;

Output

403 Forbidden

Copy and retain previous output for backup of transformed json data

1 Answers1