I'm not sure why you think it's Javascript that is the problem? It looks to me like it's the initial HTML that is causing the problem.
Time to First byte for a resource is the time take from after sending the request (e.g. GET /
) until it receives the first byte back. It excludes the DNS lookup, TCP connection and SSL handshake time, so really is a measure of the time take to start receiving that resource. Note that the "first byte" time at the top of the waterfall is the full end to end time, including DNS/TCP/SSL and any redirects, but the TTFB for each resource this is split out more.
I don't know how your home page is created - I would guess it's not a static page so whatever is generating this (PHP?) is taking too long. Whether this is due to bad backend code, an under-resourced server, a slow database, or something else is impossible to say from the outside. I would suggest getting in touch with your hosting provider and/or reviewing your code and server.