I have owned the depicted Agave attenuata since about half a year. It was outside until early November when local temperatures became too cold. Now it resides in in a light, moderately heated room. (Unfortunately I cannot provide it with a cooler one.) I am concerned about the newly yellow leafs that have been and are emerging and especially about the by now sad state of the offshoot.
How can I better care of this plant and help it over the winter season?
I've had a few of these and I don't see anything out of the ordinary. You don't mention any specifics as to where you are and what temperature you consider to be "too cold".
It looks like a plant that has either gotten some cold weather, some hot weather, and/or not enough water. None of this is fatal.
Note the long stem. It used to have leaves all over it and they all turned yellow and fell off. This is what they do. They grow a little differently in habitat, but grown in a pot, they look pretty much like this.
They are one of the more frost-tender agaves I've encountered, so it's really best to keep it at 50F or higher.
