Easy way is to use unsafe context and lock some bits. Oversimplified sample:
unsafe
{
var bitmapData = bmp.LockBits(new Rectangle(0, 0, bmp.Width, bmp.Height), ImageLockMode.ReadWrite, bmp.PixelFormat);
byte* first = (byte*)bitmapData.Scan0;
byte a = first[0];
byte r = first[1];
byte g = first[2];
byte b = first[3];
...
bmp.UnlockBits(bitmapData);
}
However, if you still need to use BinaryReader and you know how many bytes per pixel there are, It is possible to just skip the header (you can find it's length in @Bradley_Ufffner 's link) and access the bytes.