MUI DatePicker + date-fns localization problem

Question

The problem occours when I'm using MUI DatePicker with LocalizationProvider and AdapterDateFns with Hungarian local.

<LocalizationProvider dateAdapter={AdapterDateFns} adapterLocale={hu}>
      <DesktopDatePicker
        value={value}
        onChange={(newValue) => {
          setValue(newValue);
        }}
        renderInput={(params) => (
          <TextField
            {...params}
            inputProps={{
              ...params.inputProps,
              placeholder: "yyyy. MM. dd."
            }}
          />
        )}
      />
</LocalizationProvider>

All latest dependencies.

The problem is when I'm trying to type in the date manually the curzor jumps around wierd in the month and day part. The LocalizationProvider transforms the input mask to "yyyy. MM. dd." and that is what I want to have, but it seems like the curzor doesn't adapt to the new format and bugs out.

You can try it here:

CodeSandbox

Is there any way to fix this or work around it?

NOTE: The cause seems to be the date-fns. It works fine with dayjs, but I have to use date-fns, so I have to make that work somehow.