How do I git add only lines matching a pattern?

here's a way:

1. use git diff > patch to make a patch of the current diff.

2. use gawk to make a second patch only of +/- lines matching the pattern: remove - from deleted lines not matching the pattern, delete + lines not matching the pattern, modify the hunk header line numbers, output each modified hunk, but don't output any modified hunks that no longer have any changes in them.

3. use git stash save, apply patch, add -u, and stash pop to apply and stage the modified patch and leave the rest of the changes unstaged.

this worked for several test cases, it works on the entire diff at once (all files), and it's quick.

#!/bin/sh

diff=`mktemp`
git diff > $diff
[ -s $diff ] || exit

patch=`mktemp`

gawk -v pat="$1" '
function hh(){
  if(keep && n > 0){
    for(i=0;i<n;i++){
      if(i==hrn){
        printf "@@ -%d,%d +%d,%d @@\n", har[1],har[2],har[3],har[4];
      }
      print out[i];
    }
  }
}
{
  if(/^diff --git a\/.* b\/.*/){
    hh();
    keep=0;
    dr=NR;
    n=0;
    out[n++]=$0
  }
  else if(NR == dr+1 && /^index [0-9a-f]+\.\.[0-9a-f]+ [0-9]+$/){
    ir=NR;
    out[n++]=$0
  }
  else if(NR == ir+1 && /^\-\-\- a\//){
    mr=NR;
    out[n++]=$0
  }
  else if(NR == mr+1 && /^\+\+\+ b\//){
    pr=NR;
    out[n++]=$0
  }
  else if(NR == pr+1 && match($0, /^@@ \-([0-9]+),?([0-9]+)? \+([0-9]+),?([0-9]+)? @@/, har)){
    hr=NR;
    hrn=n
  }
  else if(NR > hr){
    if(/^\-/ && $0 !~ pat){
      har[4]++;
      sub(/^\-/, " ", $0);
      out[n++] = $0
    }
    else if(/^\+/ && $0 !~ pat){
      har[4]--;
    }
    else{
      if(/^[+-]/){
        keep=1
      }
      out[n++] = $0
    }
  }
}
END{
  hh()
}' $diff > $patch

git stash save &&
  git apply $patch &&
  git add -u &&
  git stash pop

rm $diff
rm $patch

refs:

git diff apply

unified diff format

gawk match groups to array

git add -u

I cranked out this experimental and poorly tested program in TXR:

Sample run: first where are we at in the repo:

$ git diff
diff --git a/lorem.txt b/lorem.txt
index d5d20a4..58609a7 100644
--- a/lorem.txt
+++ b/lorem.txt
@@ -2,10 +2,14 @@ Lorem ipsum dolor sit amet,
 consectetur adipiscing elit,
 sed do eiusmod tempor
 incididunt ut labore et dolore
-magna aliqua. Ut enim ad minim
+minim
+minim
 veniam, quis nostrud
 exercitation ullamco laboris
+maxim
+maxim
 nisi ut aliquip ex ea commodo
+minim
 consequat.  Duis aute irure
 dolor in reprehenderit in
 voluptate velit esse cillum

And:

$ git diff --cached  # nothing staged in the index

The goal is to just commit the lines containing a match for min:

$ txr addmatch.txr min lorem.txt
patching file .merge_file_BilTfQ

Now what is the state?

$ git diff
diff --git a/lorem.txt b/lorem.txt
index 7e1b4cb..58609a7 100644
--- a/lorem.txt
+++ b/lorem.txt
@@ -6,6 +6,8 @@ minim
 minim
 veniam, quis nostrud
 exercitation ullamco laboris
+maxim
+maxim
 nisi ut aliquip ex ea commodo
 minim
 consequat.  Duis aute irure

And:

$ git diff --cached
diff --git a/lorem.txt b/lorem.txt
index d5d20a4..7e1b4cb 100644
--- a/lorem.txt
+++ b/lorem.txt
@@ -2,10 +2,12 @@ Lorem ipsum dolor sit amet,
 consectetur adipiscing elit,
 sed do eiusmod tempor
 incididunt ut labore et dolore
-magna aliqua. Ut enim ad minim
+minim
+minim
 veniam, quis nostrud
 exercitation ullamco laboris
 nisi ut aliquip ex ea commodo
+minim
 consequat.  Duis aute irure
 dolor in reprehenderit in
 voluptate velit esse cillum

The matching stuff is in the index, and the nonmatching +maxim lines are still unstaged.

Code in addmatch.txr:

@(next :args)
@(assert)
@pattern
@file
@(bind regex @(regex-compile pattern))
@(next (open-command `git diff @file`))
diff @diffjunk
index @indexjunk
--- a/@file
+++ b/@file
@(collect)
@@@@ -@bfline,@bflen +@afline,@aflen @@@@@(skip)
@  (bind (nminus nplus) (0 0))
@  (collect)
@    (cases)
 @line
@      (bind zerocol " ")
@    (or)
+@line
@      (bind zerocol "+")
@      (require (search-regex line regex))
@      (do (inc nplus))
@    (or)
-@line
@      (bind zerocol "-")
@      (require (search-regex line regex))
@      (do (inc nminus))
@    (or)
-@line
@;;    unmatched - line becomes context line
@      (bind zerocol " ")
@    (end)
@  (until)
@/[^+\- ]/@(skip)
@  (end)
@  (set (bfline bflen afline aflen)
        @[mapcar int-str (list bfline bflen afline aflen)])
@  (set aflen @(+ bflen nplus (- nminus)))
@(end)
@(output :into stripped-diff)
diff @diffjunk
index @indexjunk
--- a/@file
+++ b/@file
@  (repeat)
@@@@ -@bfline,@bflen +@afline,@aflen @@@@
@    (repeat)
@zerocol@line
@    (end)
@  (end)
@(end)
@(next (open-command `git checkout-index --temp @file`))
@tempname@\t@file
@(try)
@  (do
     (with-stream (patch-stream (open-command `patch -p1 @tempname` "w"))
       (put-lines stripped-diff patch-stream)))
@  (next (open-command `git hash-object -w @tempname`))
@newsha
@  (do (sh `git update-index --cacheinfo 100644 @newsha @file`))
@(catch)
@  (fail)
@(finally)
@  (do
     (ignerr [mapdo remove-path #`@tempname @tempname.orig @tempname.rej`]))
@(end)

Basically the strategy is:

• do some pattern matching on the git diff output to filter the hunks down to the matching lines. We must re-compute the "after" line count in the hunk header, and preserve the context lines.

• output the filtered diff into a variable.

• obtain a pristine copy of the file from the index using git checkout-index --temp. This command outputs the temporary name it has generated, and we capture it.

• Now send the filtered/reduced diff to patch -p1, targetting this temporary file holding the pristine copy from the index. Okay, we now have just the changes we wanted, applied to the original file.

• Next, create a Git object out of the patched file, using git hash-object -w. Capture the hash which this command outputs.

• Lastly, use git update-index --cacheinfo ... to enter this new object into the index under the original file name, effectively staging a change for the file.

If this screws up, we can just do git reset to wipe the index, fix our broken scriptology and try again.

Just blindly matching through + and - lines has obvious issues. It should work in the case when the patterns match variable names in config files, rather than content. E.g.

Replacement:

-CONFIG_VAR=foo
+CONFIG_VAR=bar

Here, if we match on CONFIG_VAR, then both lines are included. If we match on foo in the right hand side, we break things: we end up with a patch that just subtracts the CONFIG_VAR=foo line!

Obviously, this could be made clever, taking into account the syntax and semantics of the config file.

How I would solve this "for real" would be to write a robust config file parser and re-generator (which preserves comments, whitespace and all). Then parse the new and original pristine file to config objects, migrate the matching changes from one object to the other, and generate an updated file to go to the index. No messing around with patches.