-2

I have a 4gigabyte text file upload onto a server. I am using this code

WebRequest request = WebRequest.Create("http://www.UrlToDownloadStringFrom.com");
WebResponse response = request.GetResponse();
Stream stream = response.GetResponseStream();
StreamReader streamReader = new StreamReader(stream);

// the result should return "firstWord~:::~secondWord" as expected.
string result = streamReader.ReadToEnd();

// split the string apart whenever the string ~:::~ appears within it.
string[] resultSplit = result.Split(new string[] { "~:::~" }, StringSplitOptions.None);

// resultSplit[0] is firstWord, resultSplit[1] is second word
string secondWord = resultSplit[1];

to read the text file line by line and splitting it between ~:::~

Is there anyway to compress the text file so that the C# code doesn't take super long to find a specific line of text?

Dgameman1
  • 378
  • 5
  • 24

2 Answers2

1

Compressing the file will only save you storage - it will take the same ammount of time to browse through the file.

The easiest way to improve performance would be by sorting the lines in the .txt file and using a binarySearch algorithm.

Daniel
  • 10,641
  • 12
  • 47
  • 85
-1

If the file is really big and takes a while to download I would process the file, looking for the required result, as it's being downloaded. That way you can stop the download once you've found the result you are looking for.

This example should start you off: https://stackoverflow.com/a/2269627/74585

Community
  • 1
  • 1
Matthew Lock
  • 13,144
  • 12
  • 92
  • 130