Remove partial duplicates consecutive lines but keep first and last
uniq is (sort of) the perfect tool for this, by default in uniq you can keep/show the first but not last line in set.
uniq has a -f flag which allows you to skip the first few fields.
From man uniq:
-f, --skip-fields=N
avoid comparing the first N fields
-s, --skip-chars=N
avoid comparing the first N characters
A field is a run of blanks (usually spaces and/or TABs), then non-blank characters. Fields are skipped before chars.
Example with uniq -c to show count see what uniq is doing:
-bash-4.2$ uniq -c -f 1 original_file
1 1447790360 99999 99999 20.25 20.25 20.25 20.50
9 1447790362 20.25 20.25 20.25 20.25 20.25 20.50
1 1447790388 20.25 20.25 99999 99999 99999 99999
1 1447790389 99999 99999 20.25 20.25 20.25 20.50
1 1447790391 20.00 20.25 20.25 20.25 20.25 20.50
3 1447790394 20.25 20.25 20.25 20.25 20.25 20.50
Not bad. Pretty close to what is wanted. And easy to do. But missing the last matching line in group . . . .
The grouping options in uniq are also interesting for this question . . .
--group[=METHOD]
show all items, separating groups with an empty line METHOD={separate(default),prepend,append,both}
-D, --all-repeated[=METHOD]
print all duplicate lines groups can be delimited with an empty line METHOD={none(default),prepend,separate}
Example, uniq by group . . .
-bash-4.2$ uniq --group=both -f 1 original_file
1447790360 99999 99999 20.25 20.25 20.25 20.50
1447790362 20.25 20.25 20.25 20.25 20.25 20.50
1447790365 20.25 20.25 20.25 20.25 20.25 20.50
1447790368 20.25 20.25 20.25 20.25 20.25 20.50
1447790371 20.25 20.25 20.25 20.25 20.25 20.50
1447790374 20.25 20.25 20.25 20.25 20.25 20.50
1447790377 20.25 20.25 20.25 20.25 20.25 20.50
1447790380 20.25 20.25 20.25 20.25 20.25 20.50
1447790383 20.25 20.25 20.25 20.25 20.25 20.50
1447790386 20.25 20.25 20.25 20.25 20.25 20.50
1447790388 20.25 20.25 99999 99999 99999 99999
1447790389 99999 99999 20.25 20.25 20.25 20.50
1447790391 20.00 20.25 20.25 20.25 20.25 20.50
1447790394 20.25 20.25 20.25 20.25 20.25 20.50
1447790397 20.25 20.25 20.25 20.25 20.25 20.50
1447790400 20.25 20.25 20.25 20.25 20.25 20.50
Then grep for line before and after every empty line and strip blank lines:
-bash-4.2$ uniq --group=both -f 1 original_file |grep -B1 -A1 ^$ |grep -Ev "^$|^--$"
1447790360 99999 99999 20.25 20.25 20.25 20.50
1447790362 20.25 20.25 20.25 20.25 20.25 20.50
1447790386 20.25 20.25 20.25 20.25 20.25 20.50
1447790388 20.25 20.25 99999 99999 99999 99999
1447790389 99999 99999 20.25 20.25 20.25 20.50
1447790391 20.00 20.25 20.25 20.25 20.25 20.50
1447790394 20.25 20.25 20.25 20.25 20.25 20.50
1447790400 20.25 20.25 20.25 20.25 20.25 20.50
Tah dahhh! Pretty good.
With awk one liner:
awk '{n=$2$3$4$5$6$7}l1!=n{if(p)print l0; print; p=0}l1==n{p=1}{l0=$0; l1=n}END{print}' file
The whole point is to manipulate few variables: n stores all fields except first in current line, l1 the same for previous line and l0 the whole previous line. The p is just a flag to mark if previous line was already printed.