I'm new to rust & msgpack so thought i'd see if this is expected performance and if there's any significantly faster ways to use this crate. I have a file (it happens to be geolocation data from a maxmind csv database). It's approximately 180mb. I need to be able to deserialise the data as quick as possible and much quicker than a second if possible.
This is roughly what i'm doing, i've stripped all the extraneous stuff.
#[derive(Debug, PartialEq, Deserialize, Serialize)]
struct Record {
network: String,
geoname_id: String,
registered_country_geoname_id: String,
represented_country_geoname_id: String,
is_anonymous_proxy: String,
is_satellite_provider: String,
postal_code: String,
latitude: String,
longitude: String,
accuracy_radius: String
}
fn main() -> std::io::Result<()> {
let f = File::open("src/ips.csv")?; // maxmind geolite ipv4 database ~180mb
let mut r = csv::Reader::from_reader(BufReader::new(f));
let mut records = Vec::new();
for result in r.deserialize() {
let record: Record = result?;
records.push(record);
}
let buf = rmp_serde::to_vec(&records).unwrap();
let now = Instant::now();
let values: Vec<Record> = rmp_serde::from_read_ref(&buf).unwrap();
println!("time {}", now.elapsed().as_secs());
Ok(())
}
I also dumped buf into a file to interrogate it. It's ~180mb. Which i thought would be smaller with message pack?
