While what you want to do IS possible, it's certainly not as easy as "all you need do is add references to your assemblies within the LS Client project and then work with any types you defined within those referenced assemblies and you are good as gold".
You can't use .NET assemblies in the client (as you noted, the client is based on Silverlight at the moment). LightSwitch doesn't have a direct way for the Silverlight client to access .NET assemblies, but they can be used in the Server project.
There's a technique, that a number of us refer to as the "command table pattern" (though it's not an official pattern as such). What this involves is having a "command" table (or an "action" table, or a "dummy" table, it doesn't matter what it's called), to which you can programatically add & save a record from the client tier, then intercept the saving of that "dummy" record in the server tier, allowing you to "tirgger" the use of any .NET code you mmight need to use.
It's certainly not as straightforward as it could be (hopefull this will be addressed in future versions), but it does allow you to get around any restrictions that Silverlight might be causing you. I'd be interested to hear where you got the 20% figure from, I've never heard of that before, & I doubt that Silvelight's functionality, compared to .NET, is that low.