Executable gets larger when removing sections from linker script

Question

I'm really struggling with linker scripts, so I decided to start with a minimal test and start tweaking things. But removing lines for the default linker script for sections not even in my object file, can cause the executable to grow drastically!

---

program: wait_input.S

    .intel_syntax noprefix
#include <sys/syscall.h>

    .text
    .globl  _start
_start:
    # read(0,rsp,1)
    xor edi,edi
    mov rsi,rsp
    mov edx,1
    mov eax,SYS_read
    syscall

    # exit(0)
    xor edi,edi
    mov eax,SYS_exit
    syscall

Ultra small, but enough to see it do something. Now build it and grab the default linker script from the verbose output (which is delineated by lines of ====).

$ gcc -c wait_input.S
$ gcc -o wait_input -nostdlib wait_input.o -Wl,--verbose | awk '!/==/ && p; \
/==/ {p=1-p};' > defaultlinker.x

Taking a look we see there are only three sections in the object file (and only one has any data in it), and the resulting executable is small (its more ELF header than code at this point).

$ objdump -h wait_input.o
wait_input.o:     file format elf64-x86-64

Sections:
Idx Name          Size      VMA               LMA               File off  Algn
  0 .text         0000001a  0000000000000000  0000000000000000  00000040  2**0
                  CONTENTS, ALLOC, LOAD, READONLY, CODE
  1 .data         00000000  0000000000000000  0000000000000000  0000005a  2**0
                  CONTENTS, ALLOC, LOAD, DATA
  2 .bss          00000000  0000000000000000  0000000000000000  0000005a  2**0
                  ALLOC

$ wc --bytes wait_input
880 wait_input

The default linker script refers to all kinds of sections I don't have in my object file, so I tried to pare it down but then suddenly the executable file ballooned in size. Playing with it a bit, I eventually figured out that just removing two lines (for sections my object file didn't even have!), caused it:

$ sed -i -e "/build-id/d" defaultlinker.x
$ gcc -nostdlib -o wait_input wait_input.S -Wl,-T defaultlinker.x
$ wc --bytes wait_input
880 wait_input
$ sed -i -e "/interp/d" defaultlinker.x
$ gcc -nostdlib -o wait_input wait_input.S -Wl,-T defaultlinker.x
$ wc --bytes wait_input
2098048 wait_input

WHAT!?

Can someone please explain?

I've used linker scripts for microcontrollers before and they were quite straight forward. I'm really struggling to understand them for ELF files. Nothing seems to match the documentation, and the default linker script seems to be missing all kinds of important info.