2

I was wondering how Select-String reads a file content?

Does it load the content into the memory first or does it read the content in chunks?

(The reason I am curious about it is because I wonder if Select-String can create too much I/O load depending on its implementation)

Thanks

marc_s
  • 732,580
  • 175
  • 1,330
  • 1,459
pencilCake
  • 51,323
  • 85
  • 226
  • 363
  • 1
    Select-String works on single lines so it doesn't need to load a lot of data in memory. If it loaded everything in memory it wouldn't be able to read large files. – Panagiotis Kanavos Mar 30 '15 at 12:04
  • You could use `Get-Content` to read the file in chunks using `-ReadCount` piping that into `Select-String`? – Matt Mar 30 '15 at 12:33
  • If you are using Select-String at the RHS of pipeline, be sure that it will process one by one basis. – zerocool18 Mar 30 '15 at 12:52
  • This article also is useful in regards to my question: https://rkeithhill.wordpress.com/2007/06/17/optimizing-performance-of-get-content-for-large-files/ – pencilCake Mar 30 '15 at 13:19

1 Answers1

0

Based on the MS article linked below, Select-String reads lines of input and performs a regular expression comparison to determine if there is a match (or nomatch) to the pattern provided - it DOES NOT load the whole input file into memory it simply iterates through the lines of the input looking for the pattern match or not match, depending on the parameters provided.

https://technet.microsoft.com/en-us/library/hh849903.aspx?f=255&MSPPError=-2147217396

Stephen Correia
  • 81
  • 4
  • This is simply a link to a possible answer; since links change this may eventually become an invalid answer. Please post relevant code or documentation here, as well. – rfornal Apr 06 '15 at 00:12
  • This is a link to the Microsoft documentation for the select-string command that the question was referencing. If it goes away so does the select-string command in which case this whole question is irrelevant. – Stephen Correia Apr 06 '15 at 00:42