Does lookbehind work in sed?

GNU sed does not have support for lookaround assertions. You could use a more powerful language such as Perl or possibly experiment with ssed which supports Perl-style regular expressions.

perl -pe 's/(?<=foo)bar/test/g' file.txt

Note that most of the time you can avoid a lookbehind (or a lookahead) using a capture group and a backreference in the replacement string:

sed 's/\(foo\)bar/\1test/g' file.txt

---

Simulating a negative lookbehind is more subtile and needs several substitutions to protect the substring you want to avoid. Example for (?<!foo)bar:

sed 's/#/##/g;s/foobar/foob#ar/g;s/bar/test/g;s/foob#ar/foobar/g;s/##/#/g' file.txt

• choose an escape character and repeat it (for example # => ##).
• include this character in the substring you want to protect (foobar here, => foob#ar or ba => b#a).
• make your replacement.
• replace foob#ar with foobar (or b#a with ba).
• replace ## with #.

---

Obviously, you can also describe all that isn't foo before bar in a capture group:

sed -E 's/(^.{0,2}|[^f]..|[^o].?)bar/\1test/g' file.txt

But it will quickly become tedious with more characters.