How to manage the 5 seconds response timeout limit in Dialogflow / Api.ai?
You can extend the 5-second Intent limit up to 15 seconds by setting up multiple follow up events. Currently, you can only set up 3 follow-up events one after another (which can extend the timeout up to 15 seconds).
Here's an example of how you can do it in the fulfillment center:
function function1(agent){
//This function handles your intent fulfillment
//you can initialize your db query here.
//When data is found, store it in a separate table for quick search
//get current date
var currentTime = new Date().getTime();
while (currentTime + 4500 >= new Date().getTime()) {
/*waits for 4.5 seconds
You can check every second if data is available in the database
if not, call the next follow up event and do the
same while loop in the next follow-up event
(up to 3 follow up events)
*/
/*
if(date.found){
agent.add('your data here');//Returns response to user
}
*/
}
//add a follow-up event
agent.setFollowupEvent('customEvent1');
//add a default response (in case there's a problem with the follow-up event)
agent.add("This is function1");
}
let intentMap = new Map();
intentMap.set('Your intent name here', function1);;
agent.handleRequest(intentMap);To learn more about custom events please visit this page: https://dialogflow.com/docs/events/custom-events
I have just checked the Actions on Google documentation and the Fulfillment documentation pages, and indeed there is a 5-second timeout limit.
This may not be the nicest of the solutions and may not fit your case, but considering the given the strict 5-second window (we want to ensure a dynamic conversation without risking the user waiting for too long)
You start the computation with your first intent asynchronously and go back to the user and tell them to request the results in a few seconds, in the meantime when the computation is completed. It will be saved in a private space for the user, at which point the user will trigger a second intent that will request the results that in the meantime will have been pre-computed, so you can just fetch and return them.
Reduce the complexity of your code to make it faster; of you are using micro-service or nano service architecture such as firebase function, AWS lambda, or Kubernetes try to reduce dead start and cold start by initializing libraries inside the function instead of global scope,
If you have multiple API calls, try to make it in parallel instead of one after one to reduce. e.g. promise.all approach
You can also solve the problem through the database or context.
e.g user ask: what is my balance
Bot: I'm checking your balance. Ask in few seconds again
And fetch the time taking API in background and save the data in a high-speed database such as MongoDB (relatively higher than slow web service APIs ), and mark a flag in the context menu or database.
When the user asks again in a few seconds, check the flag if it is positive get the data from the high-speed database and give it to user
Tip: if you are on google assistant you can send push notifications when data fetch from API get complete
update:
Reply to comment: "Can you explain what do you mean with "initializing libraries inside the function instead of global scope"?"
For example in firebase functions case it actually got executed to containerized environment, and when you don't the call the function for a while it simply free your function's container from the memory, and when you call it again than it initialize the container again before the actual execution, that initialization is called cold start, so it takes little bit more time for the first call and subsequent call takes less, even execution time for the first call is the same but function can not strat execution until container initialization complete, initialization of container includes all the library and database connection initialization and all. This is all okay you can not get rid of cold start in micro/nano-service architecture but sometimes it takes more and more time and cause frustration and bad experience for the user, and services like dialogflow first call simply fails Everytime which is not good, here is more: services like firebase actually make separate container for each function for example if you have multiple functions firebase actually deploy each function in a separate container, so calling each function initialize only that function's container not all other functions container and here the real problem comes, you call one function and initialize everything in global scope regardless your function is using it or not, most developer do the mistake they initialize the database in global scope it means every function will must have it initialize in their cold start but not all of you function actually using database connection, so what we need to is initialize the database in each function's body separately and not outside of the function, infact what I do is I make a reusable function which checks if database is not already connected, connect it otherwise do nothing, this check is to avoid database initialization in every function call which may cause increase execution time.
I will try to add firebase functions code example later.
Update 2:
here is the code example
traditional way:
import * as functions from 'firebase-functions';
import * as admin from 'firebase-admin';
import * as _cors from 'cors';
import firestore from './../db'
import * as mongoose from "mongoose";
const defaultApp = admin.initializeApp(functions.config().firebase)
const dbURI = `mongodb://xxxxxx:[email protected]:123456/mydb`;
// const dbURI = `mongodb://localhost:27017/mydb`;
mongoose.connect(dbURI, {
useNewUrlParser: true, useUnifiedTopology: true
}).catch(e => {
console.log("mongo connection failed for reason: ", e);
})
var cors = _cors({ origin: true });// set these options appropriately According to your case,
// see document: https://www.npmjs.com/package/cors#configuration-options
// true means allow everything
// http example
export const addMessage = functions.https.onRequest((req, res) => {
const original = req.query.text;
admin.database().ref('/messages').push({ original: original }).then(snapshot => {
res.redirect(303, snapshot.ref);
});
});
export const signup = functions.https.onRequest(async (req, res) => {
... signup stuff using mongodb
res.send("user signed up");
})
//databse trigger example
export const makeUppercase = functions.database.ref('/messages/{pushId}/original')
.onWrite(event => {
const original = event.data.val();
console.log('Uppercasing', event.params.pushId, original);
const uppercase = original.toUpperCase();
return event.data.ref.parent.child('uppercase').set(uppercase);
});
//cors example
export const ping = functions.https.onRequest(async (req, res) => {
cors(req, res, () => {
res.send("this is a function");
})
})
In the above code, you may notice 4 functions
- HTTP trigger addMessage for adding a message in firebase DB
- HTTP signup function, uses MongoDB
- database trigger to make an entry uppercase, not using any database
- HTTP trigger ping function, which does not use any database
you may also notice two database initialization, firebase, and MongoDB
let's say when you call a function for the first time after a while and the function is cold, so it will initialize these two databases, not only once but for all four functions separately, let's say each database initialization takes 400 milliseconds, so these two would take 800 miles, so when you will call the first function to add a message it will initialize both db(800ms) then it will actually execute the function(lets say 150ms) so 800ms+150ms so it's gonna take approx 950ms for the first time, regardless it is not using mongodb it will initialize it because initialization is written in global scope
if you call signup function just after the addMessage function it will do the same 800ms for db init and then signup function execution lets says it takes 200ms so total 800+200=1000ms, you might be thinking that db is already initialized so why again, as I already mentioned in my initial answer that each function might live in separate container(not always but that true) it means signup function may doesn't know what is happening in addMessage function so it will initialize the db for it container too so first call would take more time then the subsequent calls
function 3 is a db trigger and it is not using database but when it is called it receives the handle to database and it uses that handle to make changes in the database, but in this case when function is cold and you make an entry in the db it actually initialize the function like any other function which means 800ms overhead is still there for the first time and this is the very reason most people hate db triggers but they don't know why it is happening (at this point I would like to mention there are few things other then cold start in their design and there are issues on github but believe me optimizing cold start will solve your problem 50%)
function 4 is nothing but a ping function but it will initialize the database too, 800ms overhead for nothing
now have a look of following code with some optimizations:
you may notice instead of initializing the db directly in global scope I registered a subroutine function in global scope named initMongodb containing db initialization logic, so when you call a firebase function it will not initialize database during cold start but it will just register this subroutine function in global scope so you will be able to access it any firebase function,
now if you observe the second function which is signup, you may have noticed I made the db initialization further conditional, because if function doesnt receive proper data to perform the signup what's the point in initializing the database, at this point I would like to mention that if database initialization is done once then in subsequent calls it would not actually initialize the database again, actually when firebase function execution completes it destroy all the variables in the that firebase functions scope but it keeps the global variables (until the next cold start) and you may notice I required the mongodb as varibale name mongoose and firebase as varibale named admin in global scope and initialization makes some changes in those variables and that all, and thats why initialization logic is conditional that if db is not initialized then initialize otherwise do nothing.
another point to be noted here is "do not" try to keep all the stuff inside the firebase function local scope(such as import of mongoose and initialization of mongoose and other DBs) it will make the overhead permanent and will import and initialize database every call from scratch since all the local variable gets destroyed after execution get completes so it is even more dangerous then cold start itself
and finally if you observe function 3 and 4, there will be no database initialization but this doesn't means that it would take the same time in cold start and subsequent call, still there are few things which happens during such as imports which loads library files from the disk to memory and all but this doesn't take that long as compare to db initialization (makes https/socket request to other computer on the internet) import is all happening in the same computer, still it is better to avoid unnecessary imports in production.
cold start optimized way (Recommended)
import * as functions from 'firebase-functions';
import * as admin from 'firebase-admin';
import * as _cors from 'cors';
import firestore from './../db'
import * as mongoose from "mongoose";
const dbURI = `mongodb://xxxxxx:[email protected]:123456/mydb`;
// const dbURI = `mongodb://localhost:27017/mydb`;
export functions initFirebase(){
if (admin.apps.length === 0) {
console.log("initializing firebase database");
admin.initializeApp(functions.config().firebase)
}else{
console.log("firebase is already initialized");
}
}
export function initMongoDb() {
if (mongoose.connection.readyState !== mongoose.STATES.connected
&& mongoose.connection.readyState !== mongoose.STATES.connecting) {
console.log("initializing mongoose");
mongoose.connect(dbURI, {
useNewUrlParser: true, useUnifiedTopology: true
}).catch(e => {
console.log("mongo connection failed for reason: ", e);
})
} else {
console.log("mongoose already connected: ", mongoose.STATES[mongoose.connection.readyState]);
}
}
var cors = _cors({ origin: true });// set these options appropriately According to your case,
// see document: https://www.npmjs.com/package/cors#configuration-options
// true means allow everything
// http example
export const addMessage = functions.https.onRequest((req, res) => {
initFirebase()
const original = req.query.text;
admin.database().ref('/messages').push({ original: original }).then(snapshot => {
res.redirect(303, snapshot.ref);
});
});
export const signup = functions.https.onRequest(async (req, res) => {
if(req.body.name && req.body.email && req.body.password){
initMongoDb();
... signup stuff using mongodb
res.send("user signed up");
}else{
res.status(400).send("parameter missing");
}
})
//database trigger example
export const makeUppercase = functions.database.ref('/messages/{pushId}/original')
.onWrite(event => {
const original = event.data.val();
console.log('Uppercasing', event.params.pushId, original);
const uppercase = original.toUpperCase();
return event.data.ref.parent.child('uppercase').set(uppercase);
});
//cors example
export const function3 = functions.https.onRequest(async (req, res) => {
cors(req, res, () => {
res.send("this is a function");
})
})
Update: a ping call to function on start of mobile app or on page load in web also works well
Inzamam Malik,
Web & Chatbot developer.
[email protected]