data.find method#
This method allows you to retrieve workflow data with the following features:. - Allows of server-side pagination - It allows filtering, sorting, and projecting fields in the results. - The results can be paginated using a cursor for efficient data retrieval.
Usage summary:#
const {data,nextCursor,error,totalCount}
= await server.dataStore.find({
filter,sort,projection,limit,after,getTotalCount
});
Scenario #1: List of Tasks along with Instance data#
let {data,nextCursor,error,totalCount} =
await server.dataStore.find( {
filter: {
name: 'Buy Used Car',
"items.type": 'bpmn:UserTask',
"items.status": 'wait'
},
sort: { "data.caseId": 1},
projection: { id: 1, data: 1, name: 1, _id:1,startedAt:1,status:1,
"items.dueDate":1, "items.type":1,"items.name":1,"items.status":1 ,"items.seq":1
}
}
);
[
{
_id: 67f877efc91a0c7f84500c25,
startedAt: 2025-04-11T02:01:19.697Z,
status: 'running',
items: {
seq: 2,
name: 'Buy',
status: 'wait',
type: 'bpmn:UserTask',
dueDate: 2025-04-16T02:01:19.743Z
},
name: 'Buy Used Car',
data: { caseId: 1755, starterUserId: null, startDate: null },
id: '2084fb04-999d-44dd-83f1-e8536e900f00'
},
{
_id: 67f877efc91a0c7f84500c27,
startedAt: 2025-04-11T02:01:19.842Z,
status: 'running',
items: {
seq: 2,
name: 'Buy',
status: 'wait',
type: 'bpmn:UserTask',
dueDate: 2025-04-16T02:01:19.882Z
},
name: 'Buy Used Car',
data: { caseId: 7637, starterUserId: null, startDate: null },
id: '999d34dd-83f1-4853-ae90-0f004fa8e990'
},
{
_id: 67f8780fc91a0c7f84500cab,
startedAt: 2025-04-11T02:01:51.108Z,
status: 'running',
items: {
seq: 2,
name: 'Buy',
status: 'wait',
type: 'bpmn:UserTask',
dueDate: 2025-04-16T02:01:51.129Z
},
name: 'Buy Used Car',
data: {
caseId: 8904,
starterUserId: 'user1',
startDate: null,
needsCleaning: 'No',
needsRepairs: 'No'
},
id: '8b9ab641-e7fa-4566-95b4-72965e17d5ed'
}
]
Scenario #2: List of Instances - No Items to display#
let {data,nextCursor,error,totalCount} =
await server.dataStore.find( {
filter: { name: 'Buy Used Car' },
sort: { endedAt: -1},
projection: { id: 1, data: 1, name: 1, _id:1,startedAt:1,status:1 }
});
[
{
_id: 67f877ecc91a0c7f84500c0e,
startedAt: 2025-04-11T02:01:16.492Z,
status: 'end',
name: 'Buy Used Car',
data: {
input1: '5',
input2: '15',
caseId: 1005,
starterUserId: null,
startDate: null
},
id: '5219ba0d-c573-4755-a4cb-60df22bd9ad8'
},
{
_id: 67f877eec91a0c7f84500c16,
startedAt: 2025-04-11T02:01:18.619Z,
status: 'end',
name: 'Buy Used Car',
data: {
caseId: 9441,
starterUserId: 'user1',
startDate: null,
needsCleaning: 'Yes',
needsRepairs: 'Yes'
},
id: 'f64af751-2aa9-42c8-a2b2-06fc437dd109'
},
{
_id: 67f877efc91a0c7f84500c1e,
startedAt: 2025-04-11T02:01:19.143Z,
status: 'end',
name: 'Buy Used Car',
data: {
caseId: 3797,
starterUserId: 'user1',
startDate: null,
needsCleaning: 'Yes',
needsRepairs: 'Yes'
},
id: 'e3b78d04-57e0-4985-9069-b51ca7e59d8e'
},
...
Scenario #3: List of Instances - with only the lastItem#
- last item created/started, regardless to status
let {data,nextCursor,error,totalCount} =
await server.dataStore.find( {
filter: { name: 'Buy Used Car' , "items.status": 'wait'},
sort: {"data.caseId": 1},
projection: { id: 1, data: 1, name: 1, _id:1,startedAt:1,status:1},
lastItem: { type:'bpmn:UserTask' ,status:'wait'}});
Scenario #4: List of Instances - with only the latestItem#
- latest item ended, status must be
end
let {data,nextCursor,error,totalCount} =
await server.dataStore.find( {
filter: { name: 'Buy Used Car' , "items.type":'bpmn:UserTask', "items.status": 'end'},
sort: {"data.caseId": 1},
projection: { id: 1, data: 1, name: 1, _id:1,startedAt:1,status:1},
latestItem: { type:'bpmn:UserTask' ,status:'end'}
});
find parameters#
All the parameters are optional
-
sortSort field can be specified as an object with field name as key and 1 (ascending) or -1 (descending) as values.Sort by `_id` in descending order by default.Examples: {"data.caseId:1} {"items.dueDate:1} {"items.startedAt:-1}
Note that MongoDB _id field is sequential, so descending order means most recently created instances -
filteris an object that specifies the criteria for filtering the results. Examples: "status":"end" "items.status":"wait" "items.type":"bpmn:UserTask" -
limitis a number that specifies the maximum number of documents to return in the result set. -
projectionis an object that specifies which fields to include or exclude in the returned documents. You need to list all fields that are part of output -
lastItemif only the last item is required, this specifies the condition of that item syntax is similar to filter but withoutitems.qualifier Examples: -
afteris a string that specifies the cursor for pagination, allowing you to retrieve results after a specific document. -
getTotalCountreturns the total number of records. (this requires additional processing, so you may call it only once.)
Important Note: projection must include all fields in the filter and sort.
Find Output#
find returns {data,nextCursor,error,totalCount}
- data: an array of instances fields as specified in the projection
but notice that
itemsfield (if requested) is a single object not an array. - nextCursor: value to be used for next call
- error: any error msg
- totalCount: if requested
Common Scenarios#
List Pending Tasks#
{
"filter": {
"name": "Buy Used Car", // name of the Process
"items.type": "bpmn:UserTask", // only User TaskS
"items.status": "wait" // only in wait state
},
"sort": {
"items.dueDate": 1 // sort by due date ,earlies first
}
}
List Recently Completed Task#
{
"filter": {
"name": "Buy Used Car",
"items.status": "end",
"items.type": "bpmn:UserTask"
},
"sort": {
"items.endedAt": -1
}
}
Example Code#
The example below shows how to call for find method and the output:
import { configuration, FindParams } from './';;
import { BPMNServer, Logger } from './';
import { EventEmitter } from 'events';
const logger = new Logger({ toConsole: true});
let name = 'find';
const server = new BPMNServer(configuration, logger, { cron: false });
main();
async function main() {
await findAggregation({ name: 'Buy Used Car' ,"items.status": 'wait' }); // has any item in wait status
console.log('--------------------------------------');
await findAggregation({ name: 'Buy Used Car' ,"status": 'end' }); // instances with end status
}
async function findAggregation(filter) {
// benchmark findInstances
console.time('find-instances call');
let insts=await server.dataStore.findInstances(filter, 'summary');
let lastIns=insts[insts.length-1];
console.log('findInstances:',insts.length,lastIns.startedAt,lastIns.data.caseId);
console.timeEnd('find-instances call');
// benchmark find)
// works perfect
let nextCursor = null;
let param:FindParams = {
getTotalCount: true, // get the total count of records
limit: 50, // limit to 10 records per page
filter: filter, // filter by process name
sort: { _id: 1 }, // sort by _id to get the earliest first
projection: { id: 1, data: 1, name: 1, _id:1,startedAt:1,status:1, // columns to return
items: {
$filter: { // filter items to only include UserTask types
input: "$items",
as: "item",
cond: {
$eq: ["$$item.type", "bpmn:UserTask"] // filter to include only UserTask items
}
}
}
}};
for(let i=0;i<100;i++) {
if (i>0)
param.after=nextCursor;
console.time('find-aggregation call');
let res=await server.dataStore.find(param);
if (i===0)
{
console.log('findAggregation total', res.totalCount,'total pages', Math.ceil(res.totalCount/param.limit));
console.log(JSON.stringify(param,null,2));
console.log('-----------------------------------------');
}
param.getTotalCount=false; // do not get the total count again
if (res.error) {
console.log('error',res.error);
return;
}
else if(!res.data)
return;
let first=res.data?res.data[0]:null;
let last=res.data?res.data[res.data.length-1]:null;
nextCursor=res.nextCursor;
console.timeEnd('find-aggregation call');
console.log('findAggregation page:',i+1,'length:',res.data.length,res.nextCursor,first.startedAt,first.data.caseId,last.data.caseId);//.data.length, res.data[0] );
if (i==0) {
console.log('Sample Instance Items:');
first.items.forEach(item=>{
console.log(` item:# ${item.seq} ${item.name} status:${item.status}`);
})
}
if(res.data.length<param.limit)
break;
}
}
Example Output#
