Reanimated + Gesture handler setState callback needs to be on the JS thread

Question

I've recently added React Native Reanimated (v3) to my project. It already had React Native Gesture Handler usages but now one of the callbacks chained to the Gesture onStart method shows an error that it I need to either mark it as a worklet or run it on the JS thread.

Looking through the docs I can't seem to see how to allow it to run on the UI thread. I'm also not sure if it can since it's setting React State.

My Pan gesture looks like this:

const [backgroundColor, setBackgroundColor] = useState(
    theme.colors.transparent
)

const panGesture = Gesture.Pan()
    .activeOffsetX([-10, 10])
    .onChange(handleGestureEvent)
    .onStart(() => {
        setBackgroundColor(theme.backgroundColors.error)
    })
    .onEnd(animateReset)

When trying to run this, I get the following error:

[react-native-gesture-handler] Some of the callbacks in the gesture are worklets and some are not. Either make sure that all calbacks are marked as 'worklet' if you wish to run them on the UI thread or use '.runOnJS(true)' modifier on the gesture explicitly to run all callbacks on the JS thread.

I've tried "marking it as a worklet" by adding the directive like so:

const panGesture = Gesture.Pan()
    .activeOffsetX([-10, 10])
    .onChange(handleGestureEvent)
    .onStart(() => {
        "worklet";
        setBackgroundColor(theme.backgroundColors.error)
    })
    .onEnd(animateReset)

But that gives me the following error:

ReanimatedError: Tried to synchronously call a non-worklet function on the UI thread.

I can remove the error by either chaining runOnJs(true) to the Gesture or by wrapping the callback in Reanimated's runOnJS function.

import { runOnJS } from "react-native-reanimated"

...

const panGesture = Gesture.Pan()
    .activeOffsetX([-10, 10])
    .onChange(handleGestureEvent)
    .onStart(
        runOnJS(() => {
            setBackgroundColor(theme.backgroundColors.error)
        })
    )
    .onEnd(animateReset)

My question is, why does this need to be the case. Can I not run callbacks that set state on the UI Thread along with the other methods?