It depends how heavy you mean by 'heavy'...
In a lot of projects I will use the minified CSS source intact and override some styles with my own project SCSS, which I compile with a gulp build process.
This is simplest and best if you are mainly intending to change colours, border styles etc, and not changing components fundamentally.
In rare projects that require more fundamental differences, I will handcraft most of the SCSS and just rip out the few parts of Bootstrap which I want to use from the source.
This approach is much more labour-intensive and only done when the vast majority of Bootstrap code will be redundant or clashing with my own code.
For JavaScript, I tend not to use Bootstrap's, as I generally write or include Angular directives from other sources, but that just fits my particular workflow and stack choices.
It's pretty easy to rip out unused JavaScript from Bootstrap's source if you want to reduce the amount of bloat.