Node.js module for managing allocation of job IDs across a variable
number of instance IDs.
A typical use would be to allocate work across a number of worker
processes. Job allocations are stored in a SQLite or PostgreSQL database
which any worker (instance) can access.
You can add a new instance ID and jobs will start to be allocated to it.
You can mark an instance ID as unavailable and no further jobs will be
allocated to it. You can then wait until an instance ID has no jobs
allocated to it before removing it.
This is useful if your jobs are sticky, i.e. a job must remain
attached to the instance which started it. A job ID remains allocated to
an instance ID even if a new instance ID is added which would otherwise
have been allocated the job ID. You have to deallocate the job ID
explicitly to get it reassigned.
This is a new allocation persisted to the database in this call.
This is an allocation which already existed in the database before
the instance was made unavailable.
The allocation is removed from the database.
Allocator
The default algorithm for allocating a job to an instance is to hash the
job ID, treat the resulting digest as a 32 bit integer and use that as
an index into the list of available instances.
You can change the default algorithm by overriding the
_allocate
method.
Here’s an example which knows the ID of the instance on which it’s
running and only persists an allocation to the database if it’s for that
instance.
Since _allocate is only called when the allocation doesn’t already
exist in the database, if you call
allocate
for each job on every instance, this example can start a job on its
instance when first allocated.
const { Atributo } = require('atributo'), async = require('async'), assert = require('assert');class ExampleAtributo extends Atributo{ available(instance_id, cb) { // Remember out instance ID this._instance_id = instance_id; super.available(instance_id, cb); } allocate(job_id, cb) { super.allocate(job_id, (err, instance_id, persisted) => { if (persisted) { // first allocation on our instance so start job } cb
Installation
npm install atributo
SQLite
In the top-level directory you’ll find a file called
atributo.empty.sqlite3. This contains an empty copy of the database
atributo needs to store instance availablity and job allocations.
You should use a copy of this file in your application and pass its
location as db_filename when constructing
Atributo
objects.