A meteor is the visible event that occurs when a meteoroid or asteroid enters the earth’s atmosphere and becomes brightly visible (thanks to Garth for correcting me on this!). Our Meteor an implementation of a technique called Comet for using the HTTP protocol for persistent streaming data connections. The term ‘Comet’ was coined by Alex Russell in his post ‘Comet: Low Latency Data for the Browser‘.
In fact, this technology has been around for ages, it’s just difficult to implement well. Like AJAX before it, ‘comet’ only really became cool once someone gave it a name and showed the world in simple terms how it works. The main problem to be solved is one of scalability – most web servers are not designed to handle requests that take minutes or even hours to complete a response, and their threaded architectures collapse at the first sign of more than a few hundred simultaneously connected clients. As far as Apache is concerned, it’s all about getting requests answered and closed as quickly as possible, and that’s just not what’s needed to make streaming work.
The guys behind the ‘comet’ concept are working on a new event-driven server architecture called Cometd, but in the meantime we needed a solution to a pressing need, so we built our own. A perl-based HTTP server written from the ground up to support high concurrency and longevity of connections, as well as memory-cached data to allow event-driven broadcasting of the same information to thousands of clients in near-realtime with minimal resource overhead and no disk access.
Meteor is two servers in one. It listens on one port for event controllers, and on another for channel subscribers. Event controllers are clients that connect on the control port and use Meteor’s command protocol to inject events into named channels. Controllers can also issue commands to view the status of channels. Constructing a event controller is as easy as opening a socket and squirting a few simple text-based commands through it, trivial with most web programming platforms.
Subscribers are clients that connect on the subscriber port, and use the standard HTTP protocol to request a subscription to a particular channel or channels. A wide variety of querystring parameters can be included by the subscriber to indicate which interaction mode is desired.
Meteor then sends the events provided by the event controllers to the channel subscribers. All events are cached in memory so that the overhead required to send an event to a subscriber is minimal. In this way a few event controllers can do most of the hard work generating and formatting data, but without repetition, while Meteor handles the task of delivering the data to a large audience in near-real time.
Meteor Installation:
http://meteorserver.org/installation/
Categories: Web Development
