Virtual memory exception when dealing with large binary files

Question

I get a virtual memory exception error when transforming a (2~3 megabytes) binary file to a different format. This error only happens on low performance PCs with 100 Mbytes of virtual memory. Performance report in realtime (.net Memory profiler)

here is the code where this memory usage increase occurs:

Using reader As BinaryReader = New BinaryReader(New FileStream(strSrcFileName, FileMode.Open))
    Dim BuffOffset As Integer 'offset of reading the DAT file every time
    Dim RecSize As Integer 'no of records to process each time
    While intRecordsCount > 0 'reader.BaseStream.Position <> reader.BaseStream.Length
        If intRecordsCount > RecordsChunck Then
            RecSize = RecordsChunck
            intRecordsCount -= RecordsChunck
        Else
            RecSize = intRecordsCount
            intRecordsCount = 0
        End If
        Dim tempBuffer(RecSize * intRecordLength - 1) As Byte
        Dim outputbuffer(RecSize * (intRecordLength - 8) - 1) As Byte

        reader.BaseStream.Seek(BuffOffset, SeekOrigin.Begin)
        reader.Read(tempBuffer, 0, tempBuffer.Length)
        BuffOffset += tempBuffer.Length 'prepare the offset for the next reading

                'conversion is done here

        'append the converted buffer into the target binary file
        Using writer As BinaryWriter = New BinaryWriter(File.Open(strTargetFileName, FileMode.Append))
            writer.Write(outputbuffer)
        End Using
    End While 
End Using

The increase is happening as expected when appending the target file (writer stream)

My question is what can I to do to avoid that (much) memory usage when both source and target files were loaded, and what is the best practice in such cases?