React hook causing slow render on mousemove

Question

I'm using React & TypeScript in a project and recently pieced together the following hook:

export const useDrag = (selectionRef: React.MutableRefObject<HTMLDivElement>) => {
  const [dragStart, setDragStart] = useState<Coordinates>(null);
  const [dragEnd, setDragEnd] = useState<Coordinates>(null);

  const onMouseDown = (e: MouseEvent) => setDragStart(getMouseOffset(e, selectionRef.current));
  const onMouseUp = () => {
    setDragStart(null);
    setDragEnd(null);
  };

  const onMouseMove = (e: any) => {
    if (dragStart) {
      setDragEnd(getMouseOffset(e, selectionRef.current));
    }
  };

  useEffect(() => {
    const current = selectionRef.current;
    if (current) {
      current.addEventListener('mousedown', onMouseDown);
      current.addEventListener('mouseup', onMouseUp);
      current.addEventListener('mousemove', onMouseMove);
      return () => {
        current.removeEventListener('mousedown', onMouseDown);
        current.removeEventListener('mouseup', onMouseUp);
        current.removeEventListener('mousemove', onMouseMove);
      };
    }
  }, [selectionRef, dragStart]);

  return {
    pendingSelection: dragStart && dragEnd && { start: dragStart, end: dragEnd },
  };
};

This pretty much works as expected. The problem is when I'm using this hook in another component like so:

...
const selectionRef = useRef<HTMLDivElement>();
const { pendingSelection } = useDrag(selectionRef);

return (
  <div className="selection" ref={selectionRef}></div>
  {pendingSelection && (
    <div
      style={{
        height: pendingSelection.end.y - pendingSelection.start.y,
        width: pendingSelection.end.x - pendingSelection.start.x,
      }}
    ></div>
);
...

It seems like the 'mousemove' event occurs so frequently that it is actually slowing down the rendering process (to the point of temporarily freezing the page).

I've seen react packages that handle drag events like this with very smooth updates. I know I could switch to one of these, but would really rather understand what I am doing wrong here and avoid a dependency for such a small use case.

Would be happy to provide more code or a working example. Any information appreciated!