I'm speaking only about servers that doesn't have any SSL certs (self-signed or by authority).
Is it normal for site that doesn't have anything about SSL (default config, clean installed apache/nginx) to return Unable to connect (firefox) or refused to connect (chrome) if accessed via https (e.g. https://example.com) ?
Is it really indeed or misconfiguration on my server
Yes this is normal. HTTPS works on different ports, and with many web servers the HTTPS version of a site is essentially a different site than the the HTTP version of the side. Fortunately, most administrators will configure the two sites to serve the same content.
If you have parts of the site that require authentication, they should only be accessible on the HTTPS version of the site. Often the HTTP version of the site will redirect such requests to the HTTPS version of the site.
Some web server have self-signed certificates available to enable you to test your HTTPS setup. However, these generate certificate warnings and are not appropriate for public sites. You will need to replace them with certificates recognized by a recognized certificate authority.
You can obtain a free certificate from Let's Encrypt. This is a certificate authority designed to make self-registration simple and secure. The organization has support from many organizations.
Many search engines are being to penalize sites that don't offer service using HTTPS with valid certificate. (A valid certificate is a non-expired certificate signed by a recognized certificate authority.) For my site, Google is providing HTTPS links instead of HTTP sites. This began after I switched from a self-signed certificate, to a certificate from Let's Encrypt.
Yes, you wont be able to connect if you don't have a certificate. It is needed for the key exchange. Luckily getting a self-signed one is not that much work. Nginx gives following error message,
no "ssl_certificate" is defined in server listening on SSL port while SSL handshaking
In step one in this link they show you how to create a self-signed certificate https://www.digitalocean.com/community/tutorials/how-to-create-an-ssl-certificate-on-nginx-for-ubuntu-14-04
While I concur with the other answers for "non-SSL" as per the title, there is actually an unexcluded middle in the question body; SSL/TLS does have some ciphersuites that do not use a server certificate: some entirely unauthenticated (anonymous) and some authenticated by noncertificate means (PSK or Kerberos). These are rarely used, often prohibited, and sometimes unimplemented.
HTTPS as one application (profile) of SSL/TLS should not use these because RFC2616 specifies matching the URL authority's host identity against the certificate, but it is technically possible. Browsers (AFAIK all) do not implement them, but some other clients do or can. AFAICT it is not possible to configure Apache or nginx to choose them, but other servers (or quasi-servers like an ALG, IDS/IPS/DLP, or front-end/load-balancer) might and it is fairly easy to do so in Java if you use a keystore that is not set up correctly (often due to following bad advice from some websites) and enable all supported protocols (following bad advice from many websites, even stackexchange).
In this case, if a server IS listening on 443 but does not have a certificate:
