Apparently there is no way to configure proxy in Jenkins. The proxy settings that are available through "Manage Jenkins --> Manage Plugins --> Advanced" are only for plugins. So if I try to configure it there, I still don't see this configuration on System Properties page.
The only way that I managed to configure the proxy successfully is via command line, for example:
java -Dhttp.proxyHost=<host> -Dhttp.proxyPort=<port> -Dhttps.proxyHost=<host> -Dhttps.proxyPort=<port> -DJENKINS_HOME="C:\.jenkins" -jar jenkins.war --httpPort=<port>
But I have Jenkins set up as Windows service and not sure how can I supply these command line parameters. When I look at the task managed, Jenkins process appears with a bunch of parameters but I am not sure where does it read it from.