I'm using a powershell script to split a daily text file that contains several headers and footers inside, the header record always start with "0" and the footer starts with "9". My script is working fine on splitting the original file to N output files every time it finds a new header record, the problem is when the original text file is larger, i'm using the Get-Content cmdlet on my script and I know that it's not very efficient with large files, but I'm new on powershell and I don't know how to do it with other functions, can you help?
This is my script:
# Define input and output paths
$inputFilePath = "D:\TEMP\Input\MyFile.TXT"
$outputFolderPath = "D:\TEMP\Output"
# Read the input file
$content = Get-Content $inputFilePath
# Initialize the output file content, filename and counter
$outputFileContent = ""
$outputFileName = ""
$fileCounter = 0
# Loop through each line in the input file
foreach ($line in $content) {
# If the line starts with "0C2B", create a new output file with a timestamp
if ($line.StartsWith("0C2B")) {
# If there is an existing output file content, write it to a file
if ($outputFileContent) {
$outputFileName = "MyFile$fileCounter$((Get-Date).ToString("HHmmssfff")).txt"
$outputFilePath = Join-Path $outputFolderPath $outputFileName
Set-Content $outputFilePath $outputFileContent.TrimEnd("`r`n")
$fileCounter++
$outputFileContent = ""
}
}
# Append the line to the current output file content
$outputFileContent += "$line`r`n"
}
# If there is remaining output file content, write it to a file
if ($outputFileContent) {
$outputFileName = "MyFile$fileCounter$((Get-Date).ToString("HHmmssfff")).txt"
$outputFilePath = Join-Path $outputFolderPath $outputFileName
Set-Content $outputFilePath $outputFileContent.TrimEnd("`r`n")
}