ASP.NET C# search for files on ftp (better code)

Question

Hoping I explain this like I am thinking

I have a web app which has products, from a list of suppliers. e.g. www.example.com.au/Browse.aspx?SupplierID=XYZ displays products for XYZ suppliers only. This then goes to the details page www.example.com.au/Product.aspx?ProductID=123. This page displays a list of details for that product (gallery of images, product data sheets, i.e PDFs, etc).

Now, each supplier is given an ftp account, where he can upload their product information - ftp://resources.example.com.au, and this is accessible via resources.example.com.au/*. So, let's say supplier XYZ will have something like:

resources.example.com.au/XYZ/123.jpg <== main product image
resources.example.com.au/XYZ/123_a.jpg <== secondary image
resources.example.com.au/XYZ/123_b.jpg <== secondary image
resources.example.com.au/XYZ/123_c.jpg <== secondary image

(Due to application requirement, the web app sits on a different server, than the resources folder)

Now, to get the list of related product images, i have the following:

public List<string> GetFiles(string strDirectoryName, string strStartsWith)
{
    List<string> files = new List<string>();

    FtpWebRequest request = (FtpWebRequest)FtpWebRequest.Create(new Uri(strFTPUrl + strDirectoryName));
    request.Method = WebRequestMethods.Ftp.ListDirectory;
    request.Credentials = new NetworkCredential(strFtpUser, strFtpPassword);

    FtpWebResponse response = (FtpWebResponse)request.GetResponse();
    Stream responseStream = response.GetResponseStream();
    StreamReader reader = new StreamReader(responseStream);

    while (!reader.EndOfStream)
    {
        string filename = reader.ReadLine().ToString();
        if (filename.Length > 4)
        {
            if (!string.IsNullOrEmpty(strStartsWith))
            {
                if (filename.StartsWith(strStartsWith, StringComparison.OrdinalIgnoreCase))
                {
                    files.Add(filename);
                }
            }
        }
    }

    response.Close();
    responseStream.Close();
    reader.Close();

    return files;
}

Unfortunately, this function goes through all the files (which could be around 100,000's), and selects only the file i need.

My question: is there a better way to filter only the images i need, rather than going through the entire directory?

Or, perhaps another way to get these images?

score 0 · Answer 1 · answered May 10 '12 at 07:10

I didn't find a way to do a directoryList Ftp request with filters but i might have a way to improve the performance of your solution.

You could start a new thread that fetches the directoryList every x minutes and stores the result in memory.
If you are looking for a file you could serve the list from memory(greatly improving performance) at the risk of not finding the file (at which point you could perform your current code). In a worst case scenario the memory search operation will cause a little overhead but it should be neglectable compared to an ftp request with such a big result set.

ASP.NET C# search for files on ftp (better code)

1 Answers1