For anyone wondering what the gap actually is:
As ThinkingStiff mentions, these are inline elements. That means that by default they will try to align themselves with the baseline of text. If you had some adjacent text, it would be easier to see what's happening.
The amount of space left below the svg is the size of a descender at the current font-size. This is why Teg's solution only works for the default font-size. The default font-size is 16px and 4px of that is dedicated to the descender. If you increase the font-size, the descender will also increase.
See the same piece of DOM using font-sizes of default (16px), 50px and 0px;
div{
border: 1px solid blue;
}
canvas{
border: 1px solid red;
}
#two{
font-size:50px;
}
#three{
font-size:0px;
}
<div id="one">
xy<canvas width="100" height="100"></canvas>
</div>
<div id="two">
xy<canvas width="100" height="100"></canvas>
</div>
<div id="three">
xy<canvas width="100" height="100"></canvas>
</div>