What is the purpose of the Duplicate encoder instruction in HTTP/3 QPACK?

Question

In HTTP/3 QPACK there exists an instruction for duplicating an existing entry in the dynamic table; supposedly it is used to avoid adding reference to an older entry which may block inserting new entries.

However, I fail to see how this is helpful.

If the dynamic table does not contain enough room for a new entry without evicting other entries, and the entries that would be evicted are not evictable, the encoder MUST NOT insert that entry into the dynamic table (including duplicates of existing entries).

To ensure that the encoder is not prevented from adding new entries, the encoder can avoid referencing entries that are close to eviction. Rather than reference such an entry, the encoder can emit a Duplicate instruction (Section 4.3.4), and reference the duplicate instead.

   +--------+---------------------------------+----------+
   | Unused |          Referenceable          | Draining |
   | Space  |             Entries             | Entries  |
   +--------+---------------------------------+----------+
            ^                                 ^          ^
            |                                 |          |
      Insertion Point                 Draining Index  Dropping
                                                       Point

(Taken from IETF-QUIC-QPACK-DRAFT16)

Is the idea here that once a field passes the draining index, we want to make it a lower priority to exist, so we'll add a reference to it, make any new requests refer to that reference, so that the old field becomes unreferenced and it will be removed? And between when it becomes removed, and the new reference gets added, those intermediate requests can be populated into the dynamic table which would otherwise become blocked?

Thank you.

Answer 1

Duplication of the old entry makes a new copy of that table entry; it does not take a reference to the old entry. This allows for the old entry to age out (be evicted).

The Duplicate instruction achieves three goals:

1. it allows the old entry to be evicted (covered above).
2. it allows for the continuous use of the dynamic table rather than the inserting a new entry (copy of the old entry) again and potentially risking a blocked hearer; and
3. duplicating an old entry is cheaper -- from the perspective of number of bytes needed to send over the network -- than insertion of a new entry; this improves compression performance.

The QPACK Internet Draft touches upon when issuing Duplicate instruction may be profitable. Choosing when and whether to issue a Duplicate instruction is an important decision which is likely to affect compression performance greatly. Each encoder will select its own strategy.

For example, ls-qpack contains this code:

    candidate = NULL;
    STAILQ_FOREACH(entry, &enc->qpe_all_entries, ete_next_all)
    {
        if (!qenc_entry_is_draining(enc, entry))
            break;
        if (candidate && ETE_SIZE(entry) < ETE_SIZE(candidate))
            continue;
        for (next = STAILQ_NEXT(entry, ete_next_nameval); next;
                                    next = STAILQ_NEXT(next, ete_next_nameval))
            if (next->ete_nameval_hash == entry->ete_nameval_hash
                    && next->ete_name_len == entry->ete_name_len
                    && next->ete_val_len == entry->ete_val_len
                    && 0 == memcmp(ETE_NAME(next), ETE_NAME(entry),
                                                        next->ete_name_len)
                    && 0 == memcmp(ETE_VALUE(next), ETE_VALUE(entry),
                                                        next->ete_val_len))
                break;
        if (!next
                && qenc_hist_seen(enc, HE_NAMEVAL, entry->ete_nameval_hash)
                        && qenc_has_or_can_evict_at_least(enc, ETE_SIZE(entry)))
            candidate = entry;
    }

The code says: find a draining entry that has not already been duplicated, whose name and value have been used recently (qenc_hist_seen()), and that can be copied (qenc_has_or_can_evict_at_least()).