Besides specifying "v4.0", you will need to add the reference "Microsoft.CSharp.dll" in order to use the "dynamic" keyword.
var codeProvider = new Microsoft.CSharp.CSharpCodeProvider(
new Dictionary<string, string> { { "CompilerVersion", "v4.0" } });
var parameters = new System.CodeDom.Compiler.CompilerParameters
{
GenerateInMemory = true,
GenerateExecutable = false,
IncludeDebugInformation = true,
TreatWarningsAsErrors = false
};
// Here add more referenced assemblies
parameters.ReferencedAssemblies.Add("Microsoft.CSharp.dll");
Don't forget to compile the solution using the Framework 4.0+.
I hope this helps.