bash - parameter substitution until line break

Question

I have a .txt file that contains a text like this :

{remove}Some text in line 1
Some text in line 2
Some text in line 3
{remove} Some text in line 4

With Parameter expansion how can I delete all lines that contain the special tag {remove} ??

I tried this but it doesn't work !

text="$(cat "template.txt")"
echo "${text##\{remove\}*'\n'}"

Thank you

To remove lines, use `grep`: `text=$(grep -v '^{remove}' template.txt)`. — Alfe, Apr 14 '15 at 20:50
@Alfe, obviously that's possible. The question, as I read it, is whether this can be done with parameter expansions. That's what makes it an interesting question. :) — Charles Duffy, Apr 14 '15 at 20:51
(By the way -- I'd tend to suggest restricting this technique to smaller files; parameter expansions aren't always as efficient as they could/should be with large buffers, overwhelming even the fork/exec overhead of launching a tool like grep or sed with large enough content). — Charles Duffy, Apr 14 '15 at 21:03
@CharlesDuffy Yeah, well, considering that OP didn't explicitly state that this question is of academical interest and about solving an issue using a very special (and obviously unfit) approach (for an unexplained reason), I assumed that just solving the issue was the main interest ;-) — Alfe, Apr 14 '15 at 21:14
@Alfe, if all they wanted was to solve the problem the best way possible, why would anyone ask how to do something "with X" or "using X", ever? By definition, such qualifiers restrict the solution space; as such, they can never improve the quality of the best applicable answer given; they can only ever make it worse. — Charles Duffy, Apr 14 '15 at 21:41
...anyhow, getting exactly the answer one asked for is the appropriate reward for such a suboptimally filtered question. :) — Charles Duffy, Apr 14 '15 at 21:45
My experience is that some asking "how do I get this screw into the wall using that hammer" maybe just isn't reconsidering their choice of tool. — Alfe, Apr 15 '15 at 12:08
I'm not sure about "obviously unfit" -- for small enough buffers, the PE approach will be significantly faster to run (via avoiding fork/exec overhead). The approach given by Tim shares this advantage, but reduces the downside (scaling with input size in a manner clearly defined as O(n) rather than implementation-dependent). Whether the OP was considering these factors in asking the question... well, we'll only know if they tell us. — Charles Duffy, Apr 15 '15 at 13:29

Charles Duffy · Accepted Answer · 2015-04-14T20:56:50.827

3

This is a bit tricky, because you can't stop shell-pattern *s from being greedy.

What you can do is use an extglob to match only characters other than the newline:

shopt -s extglob
text=$(<template.txt)
echo "${text//'{remove}'*([!$'\n'])/}"

That will leave empty lines; if you want to remove those lines entirely, then you need to be sure that the string itself contains a trailing newline (or else the last line will never match the pattern):

shopt -s extglob
text=$(<template.txt)$'\n'
printf %s "${text//'{remove}'*([!$'\n'])$'\n'/}"

...using printf rather than echo above to avoid doubling that trailing newline.

edited Apr 14 '15 at 20:56

answered Apr 14 '15 at 20:53

Charles Duffy

280,126
43
390
441

nice +1, but this will indeed remove the content but still print blank line. – Tim Rijavec Apr 14 '15 at 20:56
@CharlesDuffy true, I posted that comment to soon :) – Tim Rijavec Apr 14 '15 at 20:58
1

a **gross** way to chomp the trailing newlines: `echo "$(echo "${text//'{remove}'*([!$'\n'])?($'\n')/}")"` -- does not require manually tacking a newline to the end of the string first. – glenn jackman Apr 14 '15 at 20:59
Thank you! It's working :-D Ps: I like your approach – GabryRome Apr 15 '15 at 13:25

Tim Rijavec · Answer 2 · 2015-04-14T21:02:09.940

2

You could do it this way. Just read string line-by-line and check if it contains the substring:

string="{remove}Some text in line 1
Some text in line 2
Some text in line 3
{remove} Some text in line 4"

while read -r line; do
if [[ $line != *"{remove}"* ]]; then
    printf '%s\n' "$line"
fi
done <<< "$string"

There are also other ways, as one that was commented by @Alfe .

edited Apr 14 '15 at 21:02

answered Apr 14 '15 at 20:53

Tim Rijavec

1,770
22
27

2

`printf '%s\n' "$line"`, or at least `echo "$line"`, would be improvements. To show a case where `printf` works but `echo` doesn't, consider a line with only `-n`. Whereas `echo $line` with no quotes will very, very much do the wrong thing with a line of `*`. – Charles Duffy Apr 14 '15 at 20:59
@CharlesDuffy you're absolutely right, I've already edited the post. – Tim Rijavec Apr 14 '15 at 21:03

bash - parameter substitution until line break

2 Answers2