Each time you fill in a template variable, or a combination thereof, that has not been used by the compiler, the compiler will instantiate all the code for that template. This can lead to a considerable larger executable which in some situations is not desirable at all for instance if you are working on an embedded processor. Not everybody is working on a pc where the amount of memory is huge.
The process that is in effect is called code bloat. Sometimes the amount of code that is being generated is a lot more than you anticipated for instance when you are using STL. STL containers have quite a bit of code, there is the container , but there is also the iterators and so on.
Because template classes are based on header files, every method is inline. What the compiler will do with all this depends but the end result is that the use of a lot of template classes can blow up the size of your code.
This might not be a problem for you, for instance if you are writing software that is going to run on a pc, then it is not likely that this will impose a problem. But I am writing embedded software in C++ and templates are allowed but heavily discouraged because of there effect on the code size.
There are ways to minimize the code bloat, by using OO techniques, for instance instead of creating a vector for each type you can create a vector for the base type. Push as much functionality as possible to base classes where your template classes derive from and so on. This requires some experience.
Another thing worth mentioning is that when you are working with templates compilers have a tendency to create a big error message if something is wrong with the code. This error message can be hard to interpret by people who are not experienced with templates.