Filter list based on date

Starting with:

gaps = {{{2009, 6, 29, 10, 41, 0.}, {2009, 6, 30, 15, 26, 
     0.}}, {{2009, 6, 30, 16, 52, 0.}, {2009, 7, 1, 6, 0, 
     0.}}, {{2009, 7, 1, 6, 0, 0.}, {2009, 7, 1, 6, 2, 0.}}};

data = {{{2010, 1, 1, 6, 15, 0.}, 0.04375}, {{2010, 1, 1, 6, 30, 0.}, 
    0.04375}, {{2010, 1, 1, 6, 45, 0.}, 
    0.04375}, {{2010, 1, 1, 7, 0, 0.}, 
    0.04375}, {{2010, 1, 1, 7, 15, 0.}, 
    0.04375}, {{2009, 6, 30, 7, 26, 0.}, 0.1}};

I recommend:

maint = Interval @@ Map[AbsoluteTime, gaps, {2}];

Cases[data, {date_, _} /; ! IntervalMemberQ[maint, AbsoluteTime@date]]

Or if you prefer a form with Select like wxffles shows you could write:

makeTest[gaps_] :=
 With[{maint = Interval @@ Map[AbsoluteTime, gaps, {2}]},
   ! IntervalMemberQ[maint, AbsoluteTime @ #[[1]]] &
 ]

Select[data, makeTest @ gaps]

This should be more convenient to use for multiple "gaps" lists.


This is what I believe Murta wanted to write:

removeRanges[data_, gaps_] :=
 Module[{absData, absGaps},
  absData = AbsoluteTime /@ data[[All, 1]];
  absGaps = Interval @@ Map[AbsoluteTime, gaps, {2}];
  Pick[data, ! IntervalMemberQ[absGaps, #] & /@ absData]
 ]

There is my way:

removeRanges[data_,gaps_]:=Module[{absData,absGaps},
    absData=AbsoluteTime/@data[[All,1]];
    absGaps=Interval@@Map[AbsoluteTime,gaps,{2}];
    Pick[data,Not@IntervalMemberQ[absGaps,#]&/@absData]
]

A little bit clean.


You could make a function to decide whether a given date is in maintenance:

Clear[inMaintenance];
inMaintenance[d_, maintenance_] :=
  Or @@ (0 >= DateDifference[#[[1]], d, "Minute"][[1]]
    DateDifference[#[[2]], d, "Minute"][[1]] & /@ maintenance)

This takes the DateDifference (in minutes) between your given date and each end of an interval. If their product is negative (or zero), then the date must be within the interval. If it's in within any of the given intervals (Or @@) then in must be in maintenance.

Then you can use this function to select from your instrument data:

Select[data, ! inMaintenance[#[[1]], maintenance] &]

As for making this fast, you could incorporate the answers from this question.