Newlines in sed on Mac

For whatever reason, this is harder than it should have been. All I wanted to do was replace a particular expression with a newline character (0x10). My preference is typically to script such tasks, and the sed command is the perfect fit. This would be simple on just about any system, except for Mac OS X, where apparently all the standard advice is difficult to apply. Worse, the sed man page leads you to believe it’s a very simple matter of putting a newline in the replacement string. Of course, there’s no explanation as to how you are expected to do that. And not being a bash expert, I was at a loss.

Luckily enough I found a blog post that discussed, among other things, how to inject a newline using sed on Mac. Although the example is rather complicated given that he’s solving a different problem, the crux of the matter is this expression: $'\n/g'

$ echo 'foo bar baz quux' | sed -e 's/ /\'$'\n/g'
foo
bar
baz
quux

All that is really doing is taking advantage of the bash extquote option where $'string' quoting is performed on the enclosed string. In this case it’s a \n which expands to a newline character, followed by /g which goes through as-is. The baskslash (\) before the $'\n/g' tells sed to escape the newline character in the replacement string. I’m no expert here, but my understanding is that the argument to sed consists of two parts, the s/ /\ and the [newline]/g, where the latter resulted from the $'\n/g' evaluated by bash. Together this forms the sed expression s/ /\[newline]/g. How the \' doesn’t escape the quote and go through as-is to sed I’m not sure. Maybe someone can explain in the comments.

Thanks for reading!

About these ads
This entry was posted in Computing, HowTo and tagged , . Bookmark the permalink.

11 Responses to Newlines in sed on Mac

  1. iveney says:

    Damn it… I encounter this problem too. This is so hard to understand.

  2. Pingback: links for 2011-07-06 « Use You Imagination

  3. Anon A. Mous says:

    I’m having this problem on OsX 10.6 not just with sed but with bash in general. No matter how I try to get a script to output a newline, all it does is print \n.

  4. cbrinker says:

    The issue here is not with OSX per-se as it is with the default version of sed you get with OSX.

    /usr/bin/sed is the BSD version from May 10, 2005

    Optionally, you can install gnu sed or gsed via ports (currently 4.2.1 from June 2009) that behaves as you are accustomed to on linux.

    $ echo “a b c d” | gsed ‘s/ /\n/g’
    a
    b
    c
    d

  5. Pingback: You see, yesterday I referred to some work I had to do for university. That wasn’t quite true. - GhostLyrics' Journal

  6. Gordon says:

    There’s an interesting discussion in superuser about this if it helps. It can be done it seems, it just requires you to get bash to do the escaping: http://superuser.com/questions/307165/newlines-in-sed-on-mac-os-x

    These work:
    echo ‘foo bar’ | sed $’s/ /\\\n/g’
    nl=$’\n’; echo ‘foo bar’ | sed “s/ /\\$nl/g”

    Or you can use: tr ‘ ‘ ‘\n’

  7. SC says:

    You are awesome. Thanks!

  8. Glen says:

    Yes, this is a pain! I came across your blog when searching to solve the same problem. BTW, the following also works on OS X (the ‘y’ function does a global replace, and allows you to use \n as a replacement string):

    echo ‘foo bar baz quux’ | sed “y/ /\n/”

  9. Joe says:

    This worked for me to replace the Mac OSX csv line feeds with standard ones. Allowed me to import the .csv file I created from mac excel to mySQL.
    awk ‘{gsub(“\r”, “\n”); print $0;}’ file1.csv > file2.csv

  10. Xiao says:

    If all your sed usage tends to be substitution expressions (as mine tends to be), you can also use perl

    $ echo ‘foo bar baz quux’ | perl -pe ‘s/ /\n/g’
    foo
    bar
    baz
    quux

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s