The link you provided does not contain any information about the required dll file, which is the native library for JCo 3 (on linux the file is named libsapjco3.so for example).
Take a look inside the javadoc folder of the sapjco3 archive you got from SAP, there is a file named intro.html which contains instructions on how to install JCo 3 on many operating systems (click on the Installation link at the top of the page).
Quote from intro.html in the sapjco3-NTAMD64-3.0.10 archive:
To install JCo for Windows unzip the appropriate distribution package into an arbitrary directory {sapjco3-install-path}.
Note: Do not copy the sapjco3.dll neither into the {windows-dir}\system32 nor into the {windows-dir}\SysWOW64 directory. This will break the operability of other JCo versions that are already installed on the same system. Furthermore you would risk that the current installation also would not work anymore, if the sapjco3.dll gets replaced in the respective Windows system directory in the future.
Then add {sapjco3-install-path} to the PATH environment variable.
Finally, add {sapjco3-install-path}\sapjco3.jar to your CLASSPATH environment variable.
I have followed the above instructions and got it working on Windows 7 64 bit and JBoss EAP 6.1.
See my other post for a screenshot of the modified PATH environment variable.