I have a lot of "no reply" in tracepath youtube.com. Sometimes it is showing 200ms ping for Europe-US packets.
I am not sure either of these is much of a symptom.
You don't say where in Europe you are, but checking the available options for transatlantic cables, it seems that there is going to be at very least 50 to 100ms of latency built-in, plus the additional distance to be covered on each side of the Atlantic.
At 20mb/s, you probably have residential DSL, and that 20mb/s figure is the maximum possible rate the underlying technology supports, not the actual bandwidth you get.
The 20mb/s pipe is generally between you and your nearest cabinet/DSLAM, too - you then use a shared uplink from that to your ISP's core network, and from there onto your destination - and generally, that means an actual average throughput somewhat lower than 20mb/s.
As you suggest, peak load probably factors in. What might also factor are your local network setup (if using Wifi, there are several things that could be causing your issue in addition to the factors mentioned above), whether or not your ISP has edge devices for video traffic (such as Netflix's OpenConnect), and whether your ISP is doing traffic shaping.
Your question itself seems to indicate this mostly works OK, except when it doesn't - which unfortunately sounds like that might also be the answer to your question: it depends.
One resource you could check to see if your ISP generally does shaping is the net neutrality map.
You could also consider using some public iperf servers to check your bandwidth, and/or using Wireshark (or tcpdump) to check for substantial packet loss/re-transmits.
I would though make sure you rule out any issues on your local network before you dig into packet captures.
You would also need to run repeated tests over time to really be able to draw any substantial conclusions - traffic shaping might only be used (or might be 'enhanced') when there is congestion/high-use.
