You have Spouts (message sources) and Bolts (message processor). An Spout should have a start function.
function Spout() { this.start = function(context) { // ... context.emit(msg); // you can emit a message many times // .... }}
An spout emit message via its controller, in any of its methods.
A Bolt has a process method:
function MyBolt() { this.process = function(msg, context) { // Message process // and emit new message(s) context.emit(newmsg); }}
There is a topology builder:
// Objectsvar spout = new Spout();var downloader = new Downloader();var resolver = new Resolver();var harvester = new Harvester();// Setting Buildervar builder = ss.createTopologyBuilder();builder.setSpout("spout", spout);builder.setBolt("downloader", downloader).shuffleGrouping("resolver").shuffleGrouping("spout");builder.setBolt("resolver", resolver).shuffleGrouping
Start a topology:
topology.start();
Stop a topology (WIP):
topology.stop();
A topology worker can listen external messages, sent from other topology workers:
topology.listen(port);
If they are many topology workers, they can share its address using a central server. In a process,
create a topology server:
var tserver = ss.createTopologyServer();tserver.listen(port);
In each topology worker, connect to the topology server:
topology.connectToServer(port, host);
When a topology worker connects to the central server, it sends its address. The server shares that address
with the rest of the topology workers. See Distributed Web Crawler with Server/Workers
for a running example.
Development
git clone git://github.com/ajlopez/SimpleStorm.gitcd SimpleStormnpm installnpm test
Samples
Web Crawler sample shows
a local topology running a web crawler.
Distributed Web Crawler is a central server using
queues and many SimpleStorm topologies running in nodes.