The Standard addresses this directly in the only place it mentions any restrictions on the declarations of user-defined literals, §13.5.8/2:
A declaration whose declarator-id is a literal-operator-id shall be a declaration of a namespace-scope function or function template (it could be a friend function (11.3)), an explicit instantiation or specialization of a function template, or a using-declaration (7.3.3).
If the friend is also declared at namespace scope, then there is no distinction between definition in class or namespace scope. Note that there is no requirement for definition at namespace scope, which your question as currently worded asserts.
If not declared at namespace scope, since it cannot be found by ADL, the friend could be used privately inside the class where it is scope by regular unqualified name lookup. This is the only way of declaring a literal operator which is not an external interface.
If the friend is defined inside a class template, then two instantiations of the template will generate two identically-named functions at namespace scope, which collide even though they are both invisible outside the class scope.