.NET Framework 4 uses a different Runtime, so it might indeed cause problems. 3.5 uses the Runtime version 2, and to be able to use assemblies compiled for the 2.0 Runtime, you'll have to edit the app.config of your main executable (which would have to be 4.0) and add this to the configuration
tag (or edit it if the element already exists):
<startup useLegacyV2RuntimeActivationPolicy="true">
<supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.0"/>
</startup>
4.0 -> 2.0 references should work that way, but probably not the other way around.
The best solution IMO though would be to upgrade them all to 4.0 if possible. And of course users would need the .NET Framework 4.