libssh/scp - Copying the contents of a remote directory except some specified subfolders

Question

I am writing a C program to copy a directory from a remote machine using libssh and scp. This directory contains a large number of subfolders and I want to exclude several (less than 10) of them.

I tried iterating over the subfolders that I need (I obtained the directory contents using the SFTP subsystem) and copying each of them with a separate SCP session. But as there are over 20,000 subfolders, it would probably be more efficient if I could copy everything in one session.

I want to know if there is a way to open an SCP session in recursive mode and skip reading a certain directory after a SSH_SCP_REQUEST_NEWDIR request for that directory.

Here is my function for reading the directory.

int scp_read_folder(ssh_session session,char* location){
    
    /*Open scp session*/
    ssh_scp scp=ssh_scp_new(session, SSH_SCP_READ|SSH_SCP_RECURSIVE,location);
    if (scp == NULL){
        printf("Error allocationg SCP session: %s\n",ssh_get_error(session));
        return SSH_ERROR;
    }
    
    int rc= ssh_scp_init(scp);
    if (rc!=SSH_OK){
        printf("Error initializing SCP session: %s\n",ssh_get_error(session));
        ssh_scp_free(scp);
        return rc;
    }
    
    /*Read folder*/
    int mode;
    const int buffer_size = 1024;
    char* filename;
    char buffer[buffer_size];
    int fd;
    int bytes_read;
    
    while ((rc = ssh_scp_pull_request(scp))!= SSH_SCP_REQUEST_EOF){

        if (rc == SSH_SCP_REQUEST_NEWFILE){
            
            filename=strdup(ssh_scp_request_get_filename(scp));
            mode = ssh_scp_request_get_permissions(scp);
            printf("Receiving file %s: permissions %o\n",filename,mode);
            
            ssh_scp_accept_request(scp);
            
            /*Read and write to a file in chunks*/

            fd = open(filename, O_CREAT | O_WRONLY, mode);
            
            if (fd == -1){
                printf("Error opening file %s/%s\n",get_current_dir_name(),filename);
                perror(filename);
                ssh_scp_free(scp);
                free(filename);
                return SSH_ERROR;
            }
            free(filename);
            while ((bytes_read=ssh_scp_read(scp,buffer,buffer_size))!=SSH_ERROR){
                write(fd, buffer, bytes_read );
            }
            close(fd);
            
        }else if (rc == SSH_SCP_REQUEST_NEWDIR){

            /*This is where I want to check the folder name and avoid reading it*/
            
            filename = strdup(ssh_scp_request_get_filename(scp)); //folder name in this case
            mode = ssh_scp_request_get_permissions(scp);

            printf("Creating and entering %s: mode %d\n",filename,mode);
            mkdir(filename,mode);
            if (chdir(filename)==-1){
                printf("Unable to enter directory %s\n",filename);
                free(filename);
                ssh_scp_free(scp);
                return SSH_ERROR;
            }
            free(filename);
            ssh_scp_accept_request(scp);
            
        }else if (rc == SSH_SCP_REQUEST_ENDDIR){
            printf("Leaving directory\n");
            chdir("..");
            
        }else if (rc == SSH_SCP_REQUEST_WARNING){
            printf("Warning received: %s\n",ssh_scp_request_get_warning(scp));

        }else if (rc == SSH_ERROR){
            printf("%s\n",ssh_get_error(session));
            ssh_scp_close(scp);
            ssh_scp_free(scp);
            return SSH_ERROR;
        }
    }
    ssh_scp_close(scp);
    ssh_scp_free(scp);
    return SSH_OK;
}

Methods I tried:

• Calling ssh_scp_deny_request when I reach a folder that I don't need
  This doesn't work because it then replies with SSH_SCP_REQUEST_EOF and ends the requests, so I can't read the folders after this one.

• Setting a DONT_READ flag when I reach a folder that I don't need and just calling ssh_scp_accept_request without reading anything until the end of that folder.
  This will throw an error when I try to call ssh_scp_pull_request without reading a file. I can call ssh_scp_read without writing anything to my local directory to prevent this, but it would be a waste of time as there can be a lot of files inside a subfolder.

I will be okay with using something other than scp for this if it has a better solution.