Why does a Future from a member struct outlive the parent struct?

Question

I want to provide an abstracted struct for writing records to a file. I want to be able to specify different format options for that file, so I use an enum to internally represent the actual writing operation on the file, while the outer struct mainly obtains records over some channel and enqueues them in a FuturesUnordered to have itself evaluated as a Stream from.

The problem I'm running into is that this enqueueing fails due to conflicting lifetimes and I can't figure out how to resolve them (playground):

use std::{pin::Pin, task::Poll};

use anyhow::Result;
//use csv_async::AsyncSerializer;
use futures::{Future, FutureExt, Stream, StreamExt};
use serde::Serialize;
use tokio::{
    fs::File as TokioFile,
    io::AsyncWriteExt,
};

#[derive(Serialize)]
struct Record {
    timestamp: String,
    data: std::collections::HashMap<String, f64>,
}

enum WriterStrategy {
    RawFile(TokioFile),
    Csv/*(AsyncSerializer<TokioFile>)*/
}

impl WriterStrategy {
    async fn write(&mut self, records: &[Record]) -> Result<()> {
        match self {
            Self::Csv/*(ser)*/ => {
                /*for rec in records {
                    ser.serialize(rec).await?;
                }*/
                Ok(())
            },
            Self::RawFile(file) => {
                let mut write = String::new();
                for rec in records {
                    for (key, val) in &rec.data {
                        write.push_str(&format!("{} - {}: {}\n", rec.timestamp, key, val));
                    }
                }
                AsyncWriteExt::write(file, &write.as_bytes()).await.map(|_| ()).map_err(|e| e.into())
            },
        }
    }
}

struct FormatWriterStream<'a> {
    writer: WriterStrategy,
    channel: std::sync::mpsc::Receiver<Record>,
    open_queries: futures::stream::FuturesUnordered<
        Pin<Box<(dyn Future<Output = Result<()>> + Send + 'a)>>,
    >,
}

impl Stream for FormatWriterStream<'_> {
    type Item = Result<()>;

    fn poll_next(self: Pin<&mut Self>, c: &mut std::task::Context) -> Poll<Option<Self::Item>> {
        let mut records = Vec::new();
        while let Ok(rec) = self.channel.try_recv() {
            records.push(rec);
        }
        self.open_queries.push(self.writer.write(&records).boxed());
    
        self.open_queries.poll_next_unpin(c)
    }
}

The compilation error on this particular version is a little misleading with the following output:

error[E0495]: cannot infer an appropriate lifetime for lifetime parameter in function call due to conflicting requirements
  --> src/lib.rs:61:32
   |
61 |         self.open_queries.push(self.writer.write(&records).boxed());
   |                                ^^^^
   |
note: first, the lifetime cannot outlive the anonymous lifetime defined on the method body at 56:28...
  --> src/lib.rs:56:28
   |
56 |     fn poll_next(self: Pin<&mut Self>, c: &mut std::task::Context) -> Poll<Option<Self::Item>> {
   |                            ^^^^^^^^^
note: ...so that the reference type `&mut Pin<&mut FormatWriterStream<'_>>` does not outlive the data it points at
  --> src/lib.rs:61:32
   |
61 |         self.open_queries.push(self.writer.write(&records).boxed());
   |                                ^^^^
note: but, the lifetime must be valid for the lifetime `'_` as defined on the impl at 53:36...
  --> src/lib.rs:53:36
   |
53 | impl Stream for FormatWriterStream<'_> {
   |                                    ^^
note: ...so that the expression is assignable
  --> src/lib.rs:61:32
   |
61 |         self.open_queries.push(self.writer.write(&records).boxed());
   |                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   = note: expected `Pin<Box<dyn futures::Future<Output = Result<(), anyhow::Error>> + std::marker::Send>>`
              found `Pin<Box<dyn futures::Future<Output = Result<(), anyhow::Error>> + std::marker::Send>>`

The problem becomes clearer when also using an explicit lifetime on the block impl<'a> Stream for FormatWriterStream<'a>, where it is pointed out that the WriterStrategy's write function does not return a Future with a matching lifetime boundary (it would be expected to be 'static if the explicit lifetime was removed from FormatWriterStream, which is certainly also not what I'd want).

I'm unable to find a way to match them somehow - I tried multiple approaches to get around the issue:

1. not stating the write function as async but instead explicitly returning a Pin<Box<(dyn Future<Output = Result<()>> + Send + 'a)>>, which requires both its arguments to have the 'a lifetime, which cannot be reconciled with the strict self: Pin<&mut Self> requirement of the poll_next function.
2. moving away from the enum approach entirely and instead use a(n async_)trait that I'd implement on the structs wrapped by the enum thus far and have FormatWriterStream's writer member be a + 'a-bounded trait object, but also to no avail.

I have a similar construction working in a different module, however in that case the analogue of FormatWriterStream contains a proper &'a reference to the struct that produces the futures that get pushed into the FuturesUnordered, so the lifetimes work out. As it makes more sense for me to have the file handle be tied to its writer abstraction, I don't find it reasonable to set it up somewhere outside the scope of the FormatWriterStream and pass it in as a reference.

Why do the lifetimes of futures that are obtained from the WriterStrategy member struct conflict with the FormatWriterStream's lifetime requirements? And is there some way I can make clearer to the compiler what I'm intending to do here?

I know that the control flow in the poll_next function is not very good. In my actual code I evaluate the channel based on an IntervalStream, but since it would complicate the already lengthy code sample even more, I decided to leave that part out.

I've come across Lifetime issues with FuturesUnordered (among others) but was not able to apply any of its answer's suggestions to my problem as hopefully demonstrated above