I had some very similar questions when I decided on going down the Redux rabbit hole. In my case I was developing relatively small apps and components using just the React internal state, which still works really well. It was only when the app and the number of components grew much larger that setState became somewhat unwieldy.
I don't think this is emphasized enough - you don't always have to use Redux if you're working on a relatively small app or a few components that are easy enough to keep track of without a robust immutable store solution. There is a lot of necessary boilerplate just to get started, which can be overly time consuming in some instances.
That being said, React + Redux is a great design paradigm to follow once you are used to it and have your own boilerplate you can call upon. Redux essentially favors props over state, which forces immutable design as you (generally) can't mutate props. This is what makes the Redux DevTools "time machine" possible, along with all of the middleware that you can throw at your store as previously mentioned by others.
Middleware is a big part of it, especially with the advent of more precise solutions for sync and async tasks alike like redux-saga (https://redux-saga.js.org/). IMO "thunks" (https://github.com/gaearon/redux-thunk) are easier to grok than sagas when you're just getting started, unless you're already an expert with generators, but sagas are much more predictable and therefore testable in the long run.
So instead of having a separate internal state for each component, you can have a set of actions / reducers that you use instead, knowing that you can't mistakenly mutate the state. I've found the combination of ImmutableJS (https://facebook.github.io/immutable-js/) and Reselect (https://github.com/reactjs/reselect), which gives you composable selectors - to be great for this. No need for Object.assigns
or tons of spread operators, it creates a new object for you.
I wouldn't be in a hurry to convert existing non-Redux projects into Redux, the workflow is different enough to cause significant confusion, but you'd be hard pressed to find boilerplates for new projects that don't already have Redux in their workflow. Such as "React Boilerplate" (https://github.com/react-boilerplate/react-boilerplate), I know this kind of blew my mind at first, but definitely worth getting a feel for. It really tests your functional programming chops.