How to remove YAML frontmatter from markdown files?

Question

I have markdown files that contain YAML frontmatter metadata, like this:

---
title: Something Somethingelse
author: Somebody Sometheson 
---

But the YAML is of varying widths. Can I use a Posix command like sed to remove that frontmatter when it's at the beginning of a file? Something that just removes everything between --- and ---, inclusive, but also ignores the rest of the file, in case there are ---s elsewhere.

could you specify the input and output? my understanding is you want to remove lines inside `---` and `---`. If so, try `sed -i -n '/---/,/---/d' [file]` — qqibrow, Jan 29 '15 at 18:25
qqibrow, your `sed` command as written is being too greedy and will remove more then what was asked. — user3439894, Jan 29 '15 at 18:42

score 11 · Accepted Answer · edited Mar 27 '15 at 03:01

I understand your question to mean that you want to remove the first ----enclosed block if it starts at the first line. In that case,

sed '1 { /^---/ { :a N; /\n---/! ba; d} }' filename

This is:

1 {              # in the first line
  /^---/ {       # if it starts with ---
    :a           # jump label for looping
    N            # fetch the next line, append to pattern space
    /\n---/! ba; # if the result does not contain \n--- (that is, if the last
                 # fetched line does not begin with ---), go back to :a
    d            # then delete the whole thing.
  }
}
                 # otherwise drop off the end here and do the default (print
                 # the line)

Depending on how you want to handle lines that begin with ---abc or so, you may have to change the patterns a little (perhaps add $ at the end to only match when the whole line is ---). I'm a bit unclear on your precise requirements there.

score 4 · Answer 2 · edited Jul 30 '19 at 13:16

4

If you want to remove only the front matter, you could simply run:

sed '1{/^---$/!q;};1,/^---$/d' infile

If the first line doesn't match ---, sed will quit; else it will delete everything from the 1st line up to (and including) the next line matching --- (i.e. the entire front matter).

edited Jul 30 '19 at 13:16

hoijui

3,615
2
33
41

answered Mar 27 '15 at 02:49

don_crissti

361
1
5
13

as front matter can also end with `...` instead of `---`, you might want to replace the second appearance of `---` with `$---\|\.\.\.$` – hoijui Jul 30 '19 at 13:18
if the input file has no front-matter, this solution reduces the file to its first line -> not good! – hoijui Aug 25 '19 at 13:34

Ben Grimm · Answer 3 · 2015-01-29T18:51:55.187

1

If you don't mind the "or something" being perl.

Simply print after two instances of "---" have been found:

perl -ne 'if ($i > 1) { print } else { /^---/ && $i++ }' yaml

or a bit shorter if you don't mind abusing ?: for flow control:

perl -ne '$i > 1 ? print : /^---/ && $i++' yaml

Be sure to include -i if you want to replace inline.

edited Jan 29 '15 at 18:51

answered Jan 29 '15 at 18:42

Ben Grimm

4,316
2
15
24

score 0 · Answer 4 · 2019-04-10T16:06:22.007

you use a bash file, create script.sh and make it executable using chmod +x script.sh and run it ./script.sh.

#!/bin/bash

#folder articles contains a lot of markdown files
files=./articles/*.md

for f in $files;
do
    #filename
    echo "${f##*/}"
    #replace frontmatter title attribute to "title"
    sed -i -r 's/^title: (.*)$/title: "\1"/' $f
    #...
done

score 0 · Answer 5 · answered Aug 26 '19 at 07:16

This AWK based solution works for files with and without FrontMatter, doing nothing in the later case.

#!/bin/sh
# Strips YAML FrontMattter from a file (usually Markdown).

# Exit immediately on each error and unset variable;
# see: https://vaneyckt.io/posts/safer_bash_scripts_with_set_euxo_pipefail/
set -Ee

print_help() {
    echo "Strips YAML FrontMattter from a file (usually Markdown)."
    echo
    echo "Usage:"
    echo "    `basename $0` -h"
    echo "    `basename $0` --help"
    echo "    `basename $0` -i <file-with-front-matter>"
    echo "    `basename $0` --in-place <file-with-front-matter>"
    echo "    `basename $0` <file-with-front-matter> <file-to-be-without-front-matter>"
}

replace=false
in_file="-"
out_file="/dev/stdout"

if [ -n "$1" ]
then
    if [ "$1" = "-h" ] || [ "$1" = "--help" ]
    then
        print_help
        exit 0
    elif [ "$1" = "-i" ] || [ "$1" = "--in-place" ]
    then
        replace=true
        in_file="$2"
        out_file="$in_file"
    else
        in_file="$1"
        if [ -n "$2" ]
        then
            out_file="$2"
        fi
    fi
fi

tmp_out_file="$out_file"
if $replace
then
    tmp_out_file="${in_file}_tmp"
fi

awk -e '
BEGIN {
    is_first_line=1;
    in_fm=0;
}
/^---$/ {
    if (is_first_line) {
        in_fm=1;
    }
}
{
    if (! in_fm) {
        print $0;
    }
}
/^(---|...)$/ {
    if (! is_first_line) {
        in_fm=0;
    }
    is_first_line=0;
}
' "$in_file" >> "$tmp_out_file"

if $replace
then
    mv "$tmp_out_file" "$out_file"
fi

How to remove YAML frontmatter from markdown files?

5 Answers5

Linked