Actually, it is a federal crime to open another person's mail, regardless of whether you are related to them.
The relevant statute is 18 U.S. Code § 1703 - Delay or destruction of mail or newspapers, section (b), paragraph 2:
Whoever, without authority, opens, or destroys any mail or package of newspapers not directed to him, shall be fined under this title or imprisoned not more than one year, or both.
It's also illegal to take mail out of your box with intent to prevent it reaching someone else (§ 1702, emphasis mine):
Whoever takes any letter, postal card, or package out of any post office or any authorized depository for mail matter, or from any letter or mail carrier, or which has been in any post office or authorized depository, or in the custody of any letter or mail carrier, before it has been delivered to the person to whom it was directed, with design to obstruct the correspondence, or to pry into the business or secrets of another, or opens, secretes, embezzles, or destroys the same, shall be fined under this title or imprisoned not more than five years, or both.
There is no exception in the law for spouses.
That said, if it is your customary behavior to allow someone else (spouse or otherwise) to collect and open your mail, then that is a valid defense if they're ever prosecuted for opening your mail, and (barring hostile circumstances, such as going through a divorce) it's unlikely they would be punished.
Once the mail has been delivered to the person to whom it is addressed (and in the case of a spouse that could simply be bringing it into the house - no matter who does so), then opening it appears to be considered theft/destruction of personal property, rather than of mail. (source)