Defining a non-default constructor doesn't automatically mean you need a destructor etc - you may just be using that constructor as a convenience to fill in a POD type.
That said, the same applies to default constructors.
Basically, the "do you need the big three?" is triggered when the class will manage a resource. You still might not need a default constructor, but you do need a constructor to set up a valid initial state.
That newly-constructed state may not own an instance of the resource yet, but if it doesn't, then it must know that it doesn't (e.g. have a null pointer).
However, the big three are also used implicitly in a lot of cases. Temporaries, for example, are default-constructed. One reason you would need a non-default constructor is simply to block the implicit default constructor from being provided and used.
So one reason to define all three at once is to ensure that your code (including that provided implicitly by the compiler) remains sane.
Almost always, if you're managing a resource, you will have a default constructor, so that's why the rule mentions that - but so long as you define some kind of constructor, you should be OK.