Dynamically setting an image's srcset and src attributes triggers one request too many

Question

I'm going for a lazy loading solution where an <img>'s src and srcset attributes are both set dynamically.

If an image has its src and srcset attributes both already present in the HTML (traditional), then a srcset-capable browser will safely ignore the src and only trigger one request, picked from the srcset list. (this guy says it works, but two requests being triggered from HTML was indeed a Chrome bug until recently.)

<img src=""
    data-lazy="fotos/000_180x120.jpg"
    data-lazy-srcset="fotos/000_180x120.jpg 180w,
    fotos/000_360x240.jpg 360w,
    fotos/000_720x480.jpg 720w,
    fotos/000_1080x720.jpg 1080w,
    fotos/000_1440x960.jpg 1440w,
    fotos/000_2160x1440.jpg 2160w"
    sizes="(min-width: 700px) 700px,
    100vw">

src is initially empty. In JavaScript both src and srcset attributes get set to the values in the data- attributes.

Now, unlike when the attributes are already set in the html, setting both attributes triggers two requests. This goes for whichever attribute is set first. It's understandable that setting src before setting srcset will already fire off one request.

What I want is to set srcset and then have a srcset-capable browser find its match, while ignoring the src attribute that is set afterwards and is meant as a fallback for other browsers.

Does anyone have experience with lazy-loading a srcset + fallback?