Jackson serializes java.time.Duration as a string in one part of my object graph and as a double in a different part

Question

How can I force Jackson to consitently serialize both durations as a string?

    record MyObject(@JsonProperty("duration") Duration duration) {
    }

    @Test
    public void testJackson() throws JsonProcessingException {
        var map = Map.of(Duration.ofHours(1), new MyObject(Duration.ofHours(1)));
        ObjectMapper om = new ObjectMapper();
        om.registerModule(new JavaTimeModule());
        var json = om.writerWithDefaultPrettyPrinter().writeValueAsString(map);
        System.out.println(json);
    }

Results:

{
  "PT1H" : {
    "duration" : 3600.000000000
  }
}

Tried extending DurationSerializer and annotating MyObject with @JsonSerialize(using = MyDurationSerializer.class, but this didn't work

public class MyDurationSerializer extends DurationSerializer {

    @Override
    protected boolean useTimestamp(SerializerProvider provider) {
        return false;
    }
}