Trying to simulate cell level TTL in bigtable but whole column family data is getting removed by garbage collection
The issue is in your timestamp, you are most likely setting dates from the past. I would recommend you to set the dates with the date methods from javascript instead of manually setting them up like you are doing right now.
I did some tests with following code:
const Bigtable = require('@google-cloud/bigtable');
const bigtable = Bigtable();
const instance = bigtable.instance([instance]);
const table = instance.table([table]);
const now = new Date();
async function writeSimple() {
var now = new Date(Date.now());
var fiveMinutes = new Date(now.getTime() + 5 * 60000);
var anHour = new Date(now.getTime() + 60 * 60000);
var aDay = new Date(now.getTime() + 24 * 60 * 60000);
var twoDays = new Date(now.getTime() + 48 * 60 * 60000);
const rowsToInsert = {
key: "SUMEET",
data: {
payloads: {
'1': {
value: "NOTIFICATIONS_PAYLOAD_1",
timestamp: now,
},
'2': {
value: "NOTIFICATIONS_PAYLOAD_2",
timestamp: fiveMinutes,
},
'3': {
value: "NOTIFICATIONS_PAYLOAD_3",
timestamp: anHour,
},
'4': {
value: "NOTIFICATIONS_PAYLOAD_4",
timestamp: aDay,
},
'5': {
value: "NOTIFICATIONS_PAYLOAD_5",
timestamp: twoDays,
},
},
},
};
await table.insert(rowsToInsert);
console.log(`Successfully wrote row ${rowsToInsert.key}`);
}
and obtained a row like the following one:
2019/12/17 16:53:33 -creds flag unset, will use gcloud credential
----------------------------------------
SUMEET
payloads:1 @ 2019/12/17-16:30:34.343000
"NOTIFICATIONS_PAYLOAD_1"
payloads:2 @ 2019/12/17-16:35:34.343000
"NOTIFICATIONS_PAYLOAD_2"
payloads:3 @ 2019/12/17-17:30:34.343000
"NOTIFICATIONS_PAYLOAD_3"
payloads:4 @ 2019/12/18-16:30:34.343000
"NOTIFICATIONS_PAYLOAD_4"
payloads:5 @ 2019/12/19-16:30:34.343000
"NOTIFICATIONS_PAYLOAD_5"
And after the garbage collector passed (around 15 minutes later) I got the result that you desired:
2019/12/17 16:59:47 -creds flag unset, will use gcloud credential
----------------------------------------
SUMEET
payloads:3 @ 2019/12/17-17:30:34.343000
"NOTIFICATIONS_PAYLOAD_3"
payloads:4 @ 2019/12/18-16:30:34.343000
"NOTIFICATIONS_PAYLOAD_4"
payloads:5 @ 2019/12/19-16:30:34.343000
"NOTIFICATIONS_PAYLOAD_5"
I hope you find this useful!