TornadIO2-0.0.4/0000755000076500000240000000000012041200342013470 5ustar sergestaff00000000000000TornadIO2-0.0.4/PKG-INFO0000644000076500000240000001334012041200342014566 0ustar sergestaff00000000000000Metadata-Version: 1.1 Name: TornadIO2 Version: 0.0.4 Summary: Socket.io 0.7+ server implementation on top of Tornado framework Home-page: http://github.com/MrJoes/tornadio2/ Author: Serge S. Koval Author-email: serge.koval@gmail.com License: Copyright (c) 2011, Serge. S. Koval. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. Description: ========= TornadIO2 ========= Contributors ------------ - `Serge S. Koval `_ Introduction ------------ This is python server implementation of the `Socket.IO `_ realtime transport library on top of the `Tornado `_ framework. TornadIO2 is compatible with 0.7+ version of the Socket.IO and implements most of the features found in original Socket.IO server software. Key features: - Supports Socket.IO 0.8 protocol and related features - Full unicode support - Support for generator-based asynchronous code (tornado.gen API) - Statistics capture (packets per second, etc) - Actively maintained What is Socket.IO? ------------------ Socket.IO aims to make realtime apps possible in every browser and mobile device, blurring the differences between the different transport mechanisms. It's care-free realtime 100% in JavaScript. You can use it to build push service, games, etc. Socket.IO will adapt to the clients browser and will use most effective transport protocol available. Getting Started --------------- In order to start working with the TornadIO2 library, you have to have some basic Tornado knowledge. If you don't know how to use it, please read Tornado tutorial, which can be found `here `_. If you're familiar with Tornado, do following to add support for Socket.IO to your application: 1. Derive from tornadio2.SocketConnection class and override on_message method (on_open/on_close are optional):: class MyConnection(tornadio2.SocketConnection): def on_message(self, message): pass 2. Create TornadIO2 server for your connection:: MyRouter = tornadio2.TornadioRouter(MyConnection) 3. Add your handler routes to the Tornado application:: application = tornado.web.Application( MyRouter.urls, socket_io_port = 8000) 4. Start your application 5. You have your `socket.io` server running at port 8000. Simple, right? Starting Up ----------- We provide customized version (shamelessly borrowed from the SocketTornad.IO library) of the ``HttpServer``, which simplifies start of your TornadIO server. To start it, do following (assuming you created application object before):: if __name__ == "__main__": socketio_server = SocketServer(application) SocketServer will automatically start Flash policy server, if required. If you don't want to start ``IOLoop`` immediately, pass ``auto_start = False`` as one of the constructor options and then manually start IOLoop. More information ---------------- For more information, check `TornadIO2 documentation `_ and sample applications. Examples ~~~~~~~~ Acknowledgment ^^^^^^^^^^^^^^ Ping sample which shows how to use events to work in request-response mode. It is in the ``examples/ackping`` directory. Cross site ^^^^^^^^^^ Chat sample which demonstrates how cross-site communication works (chat server is running on port 8002, while HTTP server runs on port 8001). It is in the ``examples/crosssite`` directory. Events and generator-based async API ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Example which shows how to use events and generator-based API to work with asynchronous code. It is in the ``examples/gen`` directory. Multiplexed ^^^^^^^^^^^ Ping and chat demo running through one connection. You can see it in ``examples/multiplexed`` directory. Stats ^^^^^ TornadIO2 collects some counters that you can use to troubleshoot your application performance. Example in ``examples/stats`` directory gives an idea how you can use these stats to plot realtime graph. RPC ping ^^^^^^^^ Ping which works through socket.io events. It is in the ``examples/rpcping`` directory. Transports ^^^^^^^^^^ Simple ping/pong example with chat-like interface with selectable transports. It is in the ``examples/transports`` directory. Platform: UNKNOWN Requires: simplejson Requires: tornado TornadIO2-0.0.4/setup.cfg0000644000076500000240000000007312041200342015311 0ustar sergestaff00000000000000[egg_info] tag_build = tag_date = 0 tag_svn_revision = 0 TornadIO2-0.0.4/setup.py0000644000076500000240000000147612041200031015205 0ustar sergestaff00000000000000#!/usr/bin/env python try: from setuptools import setup, find_packages except ImportError: from distribute_setup import use_setuptools use_setuptools() from setuptools import setup, find_packages try: license = open('LICENSE').read() except: license = None try: readme = open('README.rst').read() except: readme = None setup( name='TornadIO2', version='0.0.4', author='Serge S. Koval', author_email='serge.koval@gmail.com', packages=['tornadio2'], scripts=[], url='http://github.com/MrJoes/tornadio2/', license=license, description='Socket.io 0.7+ server implementation on top of Tornado framework', long_description=readme, requires=['simplejson', 'tornado'], install_requires=[ 'simplejson >= 2.1.0', 'tornado >= 2.2.0' ] ) TornadIO2-0.0.4/tornadio2/0000755000076500000240000000000012041200342015371 5ustar sergestaff00000000000000TornadIO2-0.0.4/tornadio2/__init__.py0000644000076500000240000000151712041200005017502 0ustar sergestaff00000000000000# -*- coding: utf-8 -*- # # Copyright: (c) 2011 by the Serge S. Koval, see AUTHORS for more details. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. """ tornadio2 ~~~~~~~~~ """ __version__ = (0, 0, 3) from tornadio2.conn import SocketConnection, event from tornadio2.router import TornadioRouter from tornadio2.server import SocketServer TornadIO2-0.0.4/tornadio2/conn.py0000644000076500000240000002214612041200005016701 0ustar sergestaff00000000000000# -*- coding: utf-8 -*- # # Copyright: (c) 2011 by the Serge S. Koval, see AUTHORS for more details. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. """ tornadio2.conn ~~~~~~~~~~~~~~ Tornadio connection implementation. """ import time import logging from inspect import ismethod, getmembers from tornadio2 import proto def event(name_or_func): """Event handler decorator. Can be used with event name or will automatically use function name if not provided:: # Will handle 'foo' event @event('foo') def bar(self): pass # Will handle 'baz' event @event def baz(self): pass """ if callable(name_or_func): name_or_func._event_name = name_or_func.__name__ return name_or_func def handler(f): f._event_name = name_or_func return f return handler class EventMagicMeta(type): """Event handler metaclass""" def __init__(cls, name, bases, attrs): # find events, also in bases is_event = lambda x: ismethod(x) and hasattr(x, '_event_name') events = [(e._event_name, e) for _, e in getmembers(cls, is_event)] setattr(cls, '_events', dict(events)) # Call base super(EventMagicMeta, cls).__init__(name, bases, attrs) class SocketConnection(object): """Subclass this class and define at least `on_message()` method to make a Socket.IO connection handler. To support socket.io connection multiplexing, define `_endpoints_` dictionary on class level, where key is endpoint name and value is connection class:: class MyConnection(SocketConnection): __endpoints__ = {'/clock'=ClockConnection, '/game'=GameConnection} ``ClockConnection`` and ``GameConnection`` should derive from the ``SocketConnection`` class as well. ``SocketConnection`` has useful ``event`` decorator. Wrap method with it:: class MyConnection(SocketConnection): @event('test') def test(self, msg): print msg and then, when client will emit 'test' event, you should see 'Hello World' printed:: sock.emit('test', {msg:'Hello World'}); """ __metaclass__ = EventMagicMeta __endpoints__ = dict() def __init__(self, session, endpoint=None): """Connection constructor. `session` Associated session `endpoint` Endpoint name """ self.session = session self.endpoint = endpoint self.is_closed = False self.ack_id = 1 self.ack_queue = dict() self._event_worker = None # Public API def on_open(self, request): """Default on_open() handler. Override when you need to do some initialization or request validation. If you return False, connection will be rejected. You can also throw Tornado HTTPError to close connection. `request` ``ConnectionInfo`` object which contains caller IP address, query string parameters and cookies associated with this request. For example:: class MyConnection(SocketConnection): def on_open(self, request): self.user_id = request.get_argument('id', None) if not self.user_id: return False """ pass def on_message(self, message): """Default on_message handler. Must be overridden in your application""" raise NotImplementedError() def on_event(self, name, args=[], kwargs=dict()): """Default on_event handler. By default, it uses decorator-based approach to handle events, but you can override it to implement custom event handling. `name` Event name `args` Event args `kwargs` Event kwargs There's small magic around event handling. If you send exactly one parameter from the client side and it is dict, then you will receive parameters in dict in `kwargs`. In all other cases you will have `args` list. For example, if you emit event like this on client-side:: sock.emit('test', {msg='Hello World'}) you will have following parameter values in your on_event callback:: name = 'test' args = [] kwargs = {msg: 'Hello World'} However, if you emit event like this:: sock.emit('test', 'a', 'b', {msg='Hello World'}) you will have following parameter values:: name = 'test' args = ['a', 'b', {msg: 'Hello World'}] kwargs = {} """ handler = self._events.get(name) if handler: try: if args: return handler(self, *args) else: return handler(self, **kwargs) except TypeError: if args: logging.error(('Attempted to call event handler %s ' + 'with %s arguments.') % (handler, repr(args))) else: logging.error(('Attempted to call event handler %s ' + 'with %s arguments.') % (handler, repr(kwargs))) raise else: logging.error('Invalid event name: %s' % name) def on_close(self): """Default on_close handler.""" pass def send(self, message, callback=None, force_json=False): """Send message to the client. `message` Message to send. `callback` Optional callback. If passed, callback will be called when client received sent message and sent acknowledgment back. `force_json` Optional argument. If set to True (and message is a string) then the message type will be JSON (Type 4 in socket_io protocol). This is what you want, when you send already json encoded strings. """ if self.is_closed: return if callback is not None: msg = proto.message(self.endpoint, message, self.queue_ack(callback, message), force_json) else: msg = proto.message(self.endpoint, message, force_json=force_json) self.session.send_message(msg) def emit(self, name, *args, **kwargs): """Send socket.io event. `name` Name of the event `kwargs` Optional event parameters """ if self.is_closed: return msg = proto.event(self.endpoint, name, None, *args, **kwargs) self.session.send_message(msg) def emit_ack(self, callback, name, *args, **kwargs): """Send socket.io event with acknowledgment. `callback` Acknowledgment callback `name` Name of the event `kwargs` Optional event parameters """ if self.is_closed: return msg = proto.event(self.endpoint, name, self.queue_ack(callback, (name, args, kwargs)), *args, **kwargs) self.session.send_message(msg) def close(self): """Forcibly close client connection""" self.session.close(self.endpoint) # TODO: Notify about unconfirmed messages? # ACKS def queue_ack(self, callback, message): """Queue acknowledgment callback""" ack_id = self.ack_id self.ack_queue[ack_id] = (time.time(), callback, message) self.ack_id += 1 return ack_id def deque_ack(self, msg_id, ack_data): """Dequeue acknowledgment callback""" if msg_id in self.ack_queue: time_stamp, callback, message = self.ack_queue.pop(msg_id) callback(message, ack_data) else: logging.error('Received invalid msg_id for ACK: %s' % msg_id) # Endpoint factory def get_endpoint(self, endpoint): """Get connection class by endpoint name. By default, will get endpoint from associated list of endpoints (from __endpoints__ class level variable). You can override this method to implement different endpoint connection class creation logic. """ if endpoint in self.__endpoints__: return self.__endpoints__[endpoint] TornadIO2-0.0.4/tornadio2/flashserver.py0000644000076500000240000000513012041200005020262 0ustar sergestaff00000000000000# -*- coding: utf-8 -*- # # Copyright: (c) 2011 by the Serge S. Koval, see AUTHORS for more details. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. """ tornadio2.flashserver ~~~~~~~~~~~~~~~~~~~~~ Flash Socket policy server implementation. Merged with minor modifications from the SocketTornad.IO project. """ from __future__ import with_statement import socket import errno import functools from tornado import iostream class FlashPolicyServer(object): """Flash Policy server, listens on port 843 by default (useless otherwise) """ def __init__(self, io_loop, port=843, policy_file='flashpolicy.xml'): """Constructor. `io_loop` IOLoop instance `port` Port to listen on (defaulted to 843) `policy_file` Policy file location """ self.policy_file = policy_file self.port = port sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM, 0) sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) sock.setblocking(0) sock.bind(('', self.port)) sock.listen(128) self.io_loop = io_loop callback = functools.partial(self.connection_ready, sock) self.io_loop.add_handler(sock.fileno(), callback, self.io_loop.READ) def connection_ready(self, sock, _fd, _events): """Connection ready callback""" while True: try: connection, address = sock.accept() except socket.error, ex: if ex[0] not in (errno.EWOULDBLOCK, errno.EAGAIN): raise return connection.setblocking(0) self.stream = iostream.IOStream(connection, self.io_loop) self.stream.read_bytes(22, self._handle_request) def _handle_request(self, request): """Send policy response""" if request != '': self.stream.close() else: with open(self.policy_file, 'rb') as file_handle: self.stream.write(file_handle.read() + '\0') TornadIO2-0.0.4/tornadio2/gen.py0000644000076500000240000000637112041200005016517 0ustar sergestaff00000000000000# -*- coding: utf-8 -*- # # Copyright: (c) 2011 by the Serge S. Koval, see AUTHORS for more details. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. """ tornadio2.gen ~~~~~~~~~~~~~ Generator-based interface to make it easier to work in an asynchronous environment. """ import functools import types from collections import deque from tornado.gen import engine, Runner, Task, Wait, WaitAll, Callback class SyncRunner(Runner): """Customized ``tornado.gen.Runner``, which will notify callback about completion of the generator. """ def __init__(self, gen, callback): """Constructor. `gen` Generator `callback` Function that should be called upon generator completion """ self._callback = callback super(SyncRunner, self).__init__(gen) def run(self): """Overloaded run function""" if self.running or self.finished: return try: super(SyncRunner, self).run() finally: if self.finished: self._callback() class CallQueue(object): __slots__ = ('runner', 'queue') def __init__(self): self.runner = None self.queue = deque() def sync_engine(func): """Queued version of the ``tornado.gen.engine``. Prevents calling of the wrapped function if there is already one instance of the function running asynchronously. Function will be called synchronously without blocking io_loop. This decorator can only be used on class methods, as it requires ``self`` to make sure that calls are scheduled on instance level (connection) instead of class level (method). """ @functools.wraps(func) def wrapper(self, *args, **kwargs): # Run method def run(args, kwargs): gen = func(self, *args, **kwargs) if isinstance(gen, types.GeneratorType): data.runner = SyncRunner(gen, finished) data.runner.run() else: return gen # Completion callback def finished(): data.runner = None try: args, kwargs = data.queue.popleft() run(args, kwargs) except IndexError: pass # Get call queue for this instance and wrapped method queue = getattr(self, '_call_queue', None) if queue is None: queue = self._call_queue = dict() data = queue.get(func, None) if data is None: queue[func] = data = CallQueue() # If there's something running, queue call if data.runner is not None: data.queue.append((args, kwargs)) else: # Otherwise run it run(args, kwargs) return wrapper TornadIO2-0.0.4/tornadio2/periodic.py0000644000076500000240000000473212041200005017543 0ustar sergestaff00000000000000# -*- coding: utf-8 -*- # # Copyright: (c) 2011 by the Serge S. Koval, see AUTHORS for more details. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. """ tornadio.flashserver ~~~~~~~~~~~~~~~~~~~~ This module implements customized PeriodicCallback from tornado with support of the sliding window. """ import time import logging class Callback(object): """Custom implementation of the Tornado.Callback with support of callback timeout delays. """ def __init__(self, callback, callback_time, io_loop): """Constructor. `callback` Callback function `callback_time` Callback timeout value (in milliseconds) `io_loop` io_loop instance """ self.callback = callback self.callback_time = callback_time self.io_loop = io_loop self._running = False self.next_run = None def calculate_next_run(self): """Caltulate next scheduled run""" return time.time() + self.callback_time / 1000.0 def start(self, timeout=None): """Start callbacks""" self._running = True if timeout is None: timeout = self.calculate_next_run() self.io_loop.add_timeout(timeout, self._run) def stop(self): """Stop callbacks""" self._running = False def delay(self): """Delay callback""" self.next_run = self.calculate_next_run() def _run(self): if not self._running: return # Support for shifting callback window if self.next_run is not None and time.time() < self.next_run: self.start(self.next_run) self.next_run = None return next_call = None try: next_call = self.callback() except (KeyboardInterrupt, SystemExit): raise except: logging.error("Error in periodic callback", exc_info=True) if self._running: self.start(next_call) TornadIO2-0.0.4/tornadio2/persistent.py0000644000076500000240000001420612041200005020142 0ustar sergestaff00000000000000# -*- coding: utf-8 -*- # # Copyright: (c) 2011 by the Serge S. Koval, see AUTHORS for more details. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. """ tornadio2.persistent ~~~~~~~~~~~~~~~~~~~~ Persistent transport implementations. """ import logging import time import traceback import tornado from tornado.web import HTTPError from tornado import stack_context from tornado.websocket import WebSocketHandler from tornadio2 import proto class TornadioWebSocketHandler(WebSocketHandler): """Websocket protocol handler""" # Transport name name = 'websocket' def initialize(self, server): self.server = server self.session = None self._is_active = not self.server.settings['websocket_check'] self._global_heartbeats = self.server.settings['global_heartbeats'] logging.debug('Initializing %s handler.' % self.name) # Additional verification of the websocket handshake # For now it will stay here, till https://github.com/facebook/tornado/pull/415 # is merged. def _execute(self, transforms, *args, **kwargs): with stack_context.ExceptionStackContext(self._handle_websocket_exception): # Websocket only supports GET method if self.request.method != 'GET': self.stream.write(tornado.escape.utf8( "HTTP/1.1 405 Method Not Allowed\r\n\r\n" )) self.stream.close() return # Upgrade header should be present and should be equal to WebSocket if self.request.headers.get("Upgrade", "").lower() != 'websocket': self.stream.write(tornado.escape.utf8( "HTTP/1.1 400 Bad Request\r\n\r\n" "Can \"Upgrade\" only to \"WebSocket\"." )) self.stream.close() return # Connection header should be upgrade. Some proxy servers/load balancers # might mess with it. if self.request.headers.get("Connection", "").lower().find('upgrade') == -1: self.stream.write(tornado.escape.utf8( "HTTP/1.1 400 Bad Request\r\n\r\n" "\"Connection\" must be \"Upgrade\"." )) self.stream.close() return super(TornadioWebSocketHandler, self)._execute(transforms, *args, **kwargs) def open(self, session_id): """WebSocket open handler""" self.session = self.server.get_session(session_id) if self.session is None: raise HTTPError(401, "Invalid Session") if not self._is_active: # Need to check if websocket connection was really established by sending hearbeat packet # and waiting for response self.write_message(proto.heartbeat()) self.server.io_loop.add_timeout(time.time() + self.server.settings['client_timeout'], self._connection_check) else: # Associate session handler self.session.set_handler(self) self.session.reset_heartbeat() # Flush messages, if any self.session.flush() def _connection_check(self): if not self._is_active: self._detach() try: # Might throw exception if connection was closed already self.close() except: pass def _detach(self): if self.session is not None: if self._is_active: self.session.stop_heartbeat() self.session.remove_handler(self) self.session = None def on_message(self, message): # Tracking self.server.stats.on_packet_recv(1) # Fix for late messages (after connection was closed) if not self.session: return # Mark that connection is active and flush any pending messages if not self._is_active: # Associate session handler and flush queued messages self.session.set_handler(self) self.session.reset_heartbeat() self.session.flush() self._is_active = True if not self._global_heartbeats: self.session.delay_heartbeat() try: self.session.raw_message(message) except Exception, ex: logging.error('Failed to handle message: ' + traceback.format_exc(ex)) # Close session on exception if self.session is not None: self.session.close() def on_close(self): self._detach() def send_messages(self, messages): # Tracking self.server.stats.on_packet_sent(len(messages)) try: for m in messages: self.write_message(m) except IOError: if self.ws_connection and self.ws_connection.client_terminated: logging.debug('Dropping active websocket connection due to IOError.') self._detach() def session_closed(self): try: self.close() except Exception: logging.debug('Exception', exc_info=True) finally: self._detach() def _handle_websocket_exception(self, type, value, traceback): if type is IOError: self.server.io_loop.add_callback(self.on_connection_close) # raise (type, value, traceback) logging.debug('Exception', exc_info=(type, value, traceback)) return True # Websocket overrides def allow_draft76(self): return True class TornadioFlashSocketHandler(TornadioWebSocketHandler): # Transport name name = 'flashsocket' TornadIO2-0.0.4/tornadio2/polling.py0000644000076500000240000002364012041200005017410 0ustar sergestaff00000000000000# -*- coding: utf-8 -*- # # Copyright: (c) 2011 by the Serge S. Koval, see AUTHORS for more details. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. """ tornadio2.polling ~~~~~~~~~~~~~~~~~ This module implements socket.io polling transports. """ import time import logging import urllib from tornado.web import HTTPError, asynchronous from tornadio2 import proto, preflight, stats class TornadioPollingHandlerBase(preflight.PreflightHandler): """Polling handler base""" def initialize(self, server): self.server = server self.session = None logging.debug('Initializing %s transport.' % self.name) def _get_session(self, session_id): """Get session if exists and checks if session is closed. """ # Get session session = self.server.get_session(session_id) # If session was not found, ignore it if session is None: raise HTTPError(401, 'Invalid session') # If session is closed, but there are some pending messages left - make sure to send them if session.is_closed and not session.send_queue: raise HTTPError(401, 'Invalid session') return session def _detach(self): """Detach from the session""" if self.session: if not self.server.settings['global_heartbeats']: self.session.stop_heartbeat() self.session.remove_handler(self) self.session = None @asynchronous def get(self, session_id): """Default GET handler.""" raise NotImplementedError() def post(self, session_id): """Handle incoming POST request""" try: # Stats self.server.stats.connection_opened() # Get session self.session = self._get_session(session_id) # Can not send messages to closed session or if preflight() failed if self.session.is_closed or not self.preflight(): raise HTTPError(401) # Grab body and decode it (socket.io always sends data in utf-8) data = self.request.body.decode('utf-8') # IE XDomainRequest support if data.startswith(u'data='): data = data[5:] # Process packets one by one packets = proto.decode_frames(data) # Tracking self.server.stats.on_packet_recv(len(packets)) for p in packets: try: self.session.raw_message(p) except Exception: # Close session if something went wrong self.session.close() self.set_header('Content-Type', 'text/plain; charset=UTF-8') self.finish() finally: self.server.stats.connection_closed() def check_xsrf_cookie(self): pass def send_messages(self, messages): """Called by the session when some data is available""" raise NotImplementedError() def session_closed(self): """Called by the session when it was closed""" self._detach() def on_connection_close(self): """Called by Tornado, when connection was closed""" self._detach() class TornadioXHRPollingHandler(TornadioPollingHandlerBase): """xhr-polling transport implementation""" # Transport name name = 'xhr-polling' def initialize(self, server): super(TornadioXHRPollingHandler, self).initialize(server) self._timeout = None # TODO: Move me out, there's no need to read timeout for POST requests self._timeout_interval = self.server.settings['xhr_polling_timeout'] @asynchronous def get(self, session_id): # Get session self.session = self._get_session(session_id) if not self.session.set_handler(self): # TODO: Error logging raise HTTPError(401) if not self.session.send_queue: self._bump_timeout() else: self.session.flush() def _stop_timeout(self): if self._timeout is not None: self.server.io_loop.remove_timeout(self._timeout) self._timeout = None def _bump_timeout(self): self._stop_timeout() self._timeout = self.server.io_loop.add_timeout( time.time() + self._timeout_interval, self._polling_timeout ) def _polling_timeout(self): try: self.send_messages([proto.noop()]) except Exception: logging.debug('Exception', exc_info=True) finally: self._detach() def _detach(self): self._stop_timeout() super(TornadioXHRPollingHandler, self)._detach() def send_messages(self, messages): # Tracking self.server.stats.on_packet_sent(len(messages)) # Encode multiple messages as UTF-8 string data = proto.encode_frames(messages) # Send data to client self.preflight() self.set_header('Content-Type', 'text/plain; charset=UTF-8') self.set_header('Content-Length', len(data)) self.write(data) # Detach connection from session self._detach() # Close connection self.finish() def session_closed(self): try: self.finish() except Exception: logging.debug('Exception', exc_info=True) finally: self._detach() class TornadioHtmlFileHandler(TornadioPollingHandlerBase): """IE HtmlFile protocol implementation. Uses hidden frame to stream data from the server in one connection. """ # Transport name name = 'htmlfile' @asynchronous def get(self, session_id): # Get session self.session = self._get_session(session_id) if not self.session.set_handler(self): raise HTTPError(401) self.set_header('Content-Type', 'text/html; charset=UTF-8') self.set_header('Connection', 'keep-alive') self.write('' + (' ' * 174)) self.flush() # Dump any queued messages self.session.flush() # If hearbeats were not started by `HandshakeHandler`, start them. if not self.server.settings['global_heartbeats']: self.session.reset_heartbeat() def send_messages(self, messages): # Tracking self.server.stats.on_packet_sent(len(messages)) # Encode frames and send data data = proto.encode_frames(messages) self.write( '' % proto.json_dumps(data) ) self.flush() if not self.server.settings['global_heartbeats']: self.session.delay_heartbeat() def session_closed(self): try: self.finish() except Exception: logging.debug('Exception', exc_info=True) finally: self._detach() class TornadioJSONPHandler(TornadioXHRPollingHandler): # Transport name name = 'jsonp' def initialize(self, server): self._index = None super(TornadioJSONPHandler, self).initialize(server) @asynchronous def get(self, session_id): self._index = self.get_argument('i', 0) super(TornadioJSONPHandler, self).get(session_id) def post(self, session_id): try: # Stats self.server.stats.connection_opened() # Get session self.session = self._get_session(session_id) # Can not send messages to closed session or if preflight() failed if self.session.is_closed or not self.preflight(): raise HTTPError(401) # Socket.io always send data utf-8 encoded. data = self.request.body # IE XDomainRequest support if not data.startswith('d='): logging.error('Malformed JSONP POST request') raise HTTPError(403) # Grab data data = urllib.unquote_plus(data[2:]).decode('utf-8') # If starts with double quote, it is json encoded (socket.io workaround) if data.startswith(u'"'): data = proto.json_load(data) # Process packets one by one packets = proto.decode_frames(data) # Tracking self.server.stats.on_packet_recv(len(packets)) for p in packets: try: self.session.raw_message(p) except Exception: # Close session if something went wrong self.session.close() self.set_header('Content-Type', 'text/plain; charset=UTF-8') self.finish() finally: self.server.stats.connection_closed() def send_messages(self, messages): if self._index is None: raise HTTPError(401) # Tracking self.server.stats.on_packet_sent(len(messages)) data = proto.encode_frames(messages) message = 'io.j[%s](%s);' % ( self._index, proto.json_dumps(data) ) self.preflight() self.set_header('Content-Type', 'text/javascript; charset=UTF-8') self.set_header('Content-Length', len(message)) self.set_header('X-XSS-Protection', '0') self.set_header('Connection', 'Keep-Alive') self.write(message) self._detach() self.finish() TornadIO2-0.0.4/tornadio2/preflight.py0000644000076500000240000000332312041200005017724 0ustar sergestaff00000000000000# -*- coding: utf-8 -*- # # Copyright: (c) 2011 by the Serge S. Koval, see AUTHORS for more details. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. """ tornadio2.preflight ~~~~~~~~~~~~~~~~~~~ Transport protocol router and main entry point for all socket.io clients. """ from tornado.web import RequestHandler, asynchronous class PreflightHandler(RequestHandler): """CORS preflight handler""" @asynchronous def options(self, *args, **kwargs): """XHR cross-domain OPTIONS handler""" self.preflight() self.finish() def preflight(self): """Handles request authentication""" if 'Origin' in self.request.headers: if self.verify_origin(): self.set_header('Access-Control-Allow-Origin', self.request.headers['Origin']) self.set_header('Access-Control-Allow-Credentials', 'true') self.set_header('Access-Control-Allow-Methods', 'POST, GET, OPTIONS') return True else: return False else: return True def verify_origin(self): """Verify if request can be served""" # TODO: Verify origin return True TornadIO2-0.0.4/tornadio2/proto.py0000644000076500000240000001447312041200005017113 0ustar sergestaff00000000000000# -*- coding: utf-8 -*- # # Copyright: (c) 2011 by the Serge S. Koval, see AUTHORS for more details. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. """ tornadio2.proto ~~~~~~~~~~~~~~~ Socket.IO protocol related functions """ import logging try: import simplejson as json json_decimal_args = {"use_decimal": True} except ImportError: import json import decimal class DecimalEncoder(json.JSONEncoder): def default(self, o): if isinstance(o, decimal.Decimal): return float(o) return super(DecimalEncoder, self).default(o) json_decimal_args = {"cls": DecimalEncoder} # Packet ids DISCONNECT = '0' CONNECT = '1' HEARTBEAT = '2' MESSAGE = '3' JSON = '4' EVENT = '5' ACK = '6' ERROR = '7' NOOP = '8' # socket.io frame separator FRAME_SEPARATOR = u'\ufffd' def disconnect(endpoint=None): """Generate disconnect packet. `endpoint` Optional endpoint name """ return u'0::%s' % ( endpoint or '' ) def connect(endpoint=None): """Generate connect packet. `endpoint` Optional endpoint name """ return u'1::%s' % ( endpoint or '' ) def heartbeat(): """Generate heartbeat message. """ return u'2::' def message(endpoint, msg, message_id=None, force_json=False): """Generate message packet. `endpoint` Optional endpoint name `msg` Message to encode. If message is ascii/unicode string, will send message packet. If object or dictionary, will json encode and send as is. `message_id` Optional message id for ACK `force json` Disregard msg type and send the message with JSON type. Usefull for already JSON encoded strings. """ if msg is None: # TODO: Log something ? return u'' packed_message_tpl = u"%(kind)s:%(message_id)s:%(endpoint)s:%(msg)s" packed_data = {'endpoint': endpoint or u'', 'message_id': message_id or u''} # Trying to send a dict over the wire ? if not isinstance(msg, (unicode, str)) and isinstance(msg, (dict, object)): packed_data.update({'kind': JSON, 'msg': json.dumps(msg, **json_decimal_args)}) # for all other classes, including objects. Call str(obj) # and respect forced JSON if requested else: packed_data.update({'kind': MESSAGE if not force_json else JSON, 'msg': msg if isinstance(msg, unicode) else str(msg).decode('utf-8')}) return packed_message_tpl % packed_data def event(endpoint, name, message_id, *args, **kwargs): """Generate event message. `endpoint` Optional endpoint name `name` Event name `message_id` Optional message id for ACK `args` Optional event arguments. `kwargs` Optional event arguments. Will be encoded as dictionary. """ if args: evt = dict( name=name, args=args ) if kwargs: logging.error('Can not generate event() with args and kwargs.') else: evt = dict( name=name, args=[kwargs] ) return u'5:%s:%s:%s' % ( message_id or '', endpoint or '', json.dumps(evt) ) def ack(endpoint, message_id, ack_response=None): """Generate ACK packet. `endpoint` Optional endpoint name `message_id` Message id to acknowledge `ack_response` Acknowledgment response data (will be json serialized) """ if ack_response is not None: if not isinstance(ack_response, tuple): ack_response = (ack_response,) data = json_dumps(ack_response) return u'6::%s:%s+%s' % (endpoint or '', message_id, data) else: return u'6::%s:%s' % (endpoint or '', message_id) def error(endpoint, reason, advice=None): """Generate error packet. `endpoint` Optional endpoint name `reason` Error reason `advice` Error advice """ return u'7::%s:%s+%s' % (endpoint or '', (reason or ''), (advice or '')) def noop(): """Generate noop packet.""" return u'8::' def json_dumps(msg): """Dump object as a json string `msg` Object to dump """ return json.dumps(msg) def json_load(msg): """Load json-encoded object `msg` json encoded object """ return json.loads(msg) def decode_frames(data): """Decode socket.io encoded messages. Returns list of packets. `data` encoded messages """ # Single message - nothing to decode here assert isinstance(data, unicode), 'frame is not unicode' if not data.startswith(FRAME_SEPARATOR): return [data] # Multiple messages idx = 0 packets = [] while data[idx:idx + 1] == FRAME_SEPARATOR: idx += 1 # Grab message length len_start = idx idx = data.find(FRAME_SEPARATOR, idx) msg_len = int(data[len_start:idx]) idx += 1 # Grab message msg_data = data[idx:idx + msg_len] idx += msg_len packets.append(msg_data) return packets # Encode expects packets in unicode def encode_frames(packets): """Encode list of packets. `packets` List of packets to encode """ # No packets - return empty string if not packets: return '' # Exactly one packet - don't do any frame encoding if len(packets) == 1: return packets[0].encode('utf-8') # Multiple packets frames = u''.join(u'%s%d%s%s' % (FRAME_SEPARATOR, len(p), FRAME_SEPARATOR, p) for p in packets) return frames.encode('utf-8') TornadIO2-0.0.4/tornadio2/router.py0000644000076500000240000001630712041200005017266 0ustar sergestaff00000000000000# -*- coding: utf-8 -*- # # Copyright: (c) 2011 by the Serge S. Koval, see AUTHORS for more details. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. """ tornadio2.router ~~~~~~~~~~~~~~~~ Transport protocol router and main entry point for all socket.io clients. """ from tornado import ioloop, version_info from tornado.web import HTTPError from tornadio2 import persistent, polling, sessioncontainer, session, proto, preflight, stats PROTOCOLS = { 'websocket': persistent.TornadioWebSocketHandler, 'flashsocket': persistent.TornadioFlashSocketHandler, 'xhr-polling': polling.TornadioXHRPollingHandler, 'htmlfile': polling.TornadioHtmlFileHandler, 'jsonp-polling': polling.TornadioJSONPHandler, } DEFAULT_SETTINGS = { # Sessions check interval in seconds 'session_check_interval': 15, # Session expiration in seconds 'session_expiry': 30, # Heartbeat time in seconds. Do not change this value unless # you absolutely sure that new value will work. 'heartbeat_interval': 12, # Enabled protocols 'enabled_protocols': ['websocket', 'flashsocket', 'xhr-polling', 'jsonp-polling', 'htmlfile'], # XHR-Polling request timeout, in seconds 'xhr_polling_timeout': 20, # Some antivirus software messed up with HTTP traffic and, as a result, websockets # to port 80 stop to work. If you enable this setting, TornadIO will try to send # ping packet and wait for response. If nothing will happen during 5 seconds, # TornadIO considers connection not working. 'websocket_check': False, # Starting from socket.io 0.9.2, client started verifying heartbeats for all transports. # Disable this if you're on 0.9.1 or lower, as this settings will significantly increase # your server load for clients with polling transports. 'global_heartbeats': True, # Client timeout adjustment in seconds. If you see your clients disconnect without a # reason, increase this value. 'client_timeout': 5 } class HandshakeHandler(preflight.PreflightHandler): """socket.io handshake handler""" def initialize(self, server): self.server = server def get(self, version, *args, **kwargs): try: self.server.stats.connection_opened() # Only version 1 is supported now if version != '1': raise HTTPError(503, "Invalid socket.io protocol version") sess = self.server.create_session(self.request) settings = self.server.settings # TODO: Fix heartbeat timeout. For now, it is adding 5 seconds to the client timeout. data = '%s:%d:%d:%s' % ( sess.session_id, # TODO: Fix me somehow a well. 0.9.2 will drop connection is no # heartbeat was sent over settings['heartbeat_interval'] + settings['client_timeout'], # TODO: Fix me somehow. settings['xhr_polling_timeout'] + settings['client_timeout'], ','.join(t for t in self.server.settings.get('enabled_protocols')) ) if self.server.settings['global_heartbeats']: sess.reset_heartbeat() jsonp = self.get_argument('jsonp', None) if jsonp is not None: self.set_header('Content-Type', 'application/javascript; charset=UTF-8') data = 'io.j[%s](%s);' % (jsonp, proto.json_dumps(data)) else: self.set_header('Content-Type', 'text/plain; charset=UTF-8') self.preflight() self.write(data) self.finish() finally: self.server.stats.connection_closed() class TornadioRouter(object): """TornadIO2 router implementation""" def __init__(self, connection, user_settings=dict(), namespace='socket.io', io_loop=None): """Constructor. `connection` SocketConnection class instance `user_settings` Settings `namespace` Router namespace, defaulted to 'socket.io' `io_loop` IOLoop instance, optional. """ # TODO: Version check if version_info[0] < 2: raise Exception('TornadIO2 requires Tornado 2.0 or higher.') # Store connection class self._connection = connection # Initialize io_loop self.io_loop = io_loop or ioloop.IOLoop.instance() # Settings self.settings = DEFAULT_SETTINGS.copy() if user_settings: self.settings.update(user_settings) # Sessions self._sessions = sessioncontainer.SessionContainer() check_interval = self.settings['session_check_interval'] * 1000 self._sessions_cleanup = ioloop.PeriodicCallback(self._sessions.expire, check_interval, self.io_loop) self._sessions_cleanup.start() # Stats self.stats = stats.StatsCollector() self.stats.start(self.io_loop) # Initialize URLs self._transport_urls = [ (r'/%s/(?P\d+)/$' % namespace, HandshakeHandler, dict(server=self)) ] for t in self.settings.get('enabled_protocols', dict()): proto = PROTOCOLS.get(t) if not proto: # TODO: Error logging continue # Only version 1 is supported self._transport_urls.append( (r'/%s/1/%s/(?P[^/]+)/?' % (namespace, t), proto, dict(server=self)) ) @property def urls(self): """List of the URLs to be added to the Tornado application""" return self._transport_urls def apply_routes(self, routes): """Feed list of the URLs to the routes list. Returns list""" routes.extend(self._transport_urls) return routes def create_session(self, request): """Creates new session object and returns it. `request` Request that created the session. Will be used to get query string parameters and cookies. """ # TODO: Possible optimization here for settings.get s = session.Session(self._connection, self, request, self.settings.get('session_expiry') ) self._sessions.add(s) return s def get_session(self, session_id): """Get session by session id """ return self._sessions.get(session_id) TornadIO2-0.0.4/tornadio2/server.py0000644000076500000240000000736212041200005017255 0ustar sergestaff00000000000000# -*- coding: utf-8 -*- # # Copyright: (c) 2011 by the Serge S. Koval, see AUTHORS for more details. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. """ tornadio2.server ~~~~~~~~~~~~~~~~ Implements handy wrapper to start FlashSocket server (if FlashSocket protocol is enabled). Shamesly borrowed from the SocketTornad.IO project. """ import logging from tornado import ioloop from tornado.httpserver import HTTPServer from tornadio2.flashserver import FlashPolicyServer class SocketServer(HTTPServer): """HTTP Server which does some configuration and automatic setup of Socket.IO based on configuration. Starts the IOLoop and listening automatically in contrast to the Tornado default behavior. If FlashSocket is enabled, starts up the policy server also.""" def __init__(self, application, no_keep_alive=False, io_loop=None, xheaders=False, ssl_options=None, auto_start=True ): """Initializes the server with the given request callback. If you use pre-forking/start() instead of the listen() method to start your server, you should not pass an IOLoop instance to this constructor. Each pre-forked child process will create its own IOLoop instance after the forking process. `application` Tornado application `no_keep_alive` Support keep alive for HTTP connections or not `io_loop` Optional io_loop instance. `xheaders` Extra headers `ssl_options` Tornado SSL options `auto_start` Set auto_start to False in order to have opportunities to work with server object and/or perform some actions after server is already created but before ioloop will start. Attention: if you use auto_start param set to False you should start ioloop manually """ settings = application.settings flash_policy_file = settings.get('flash_policy_file', None) flash_policy_port = settings.get('flash_policy_port', None) socket_io_port = settings.get('socket_io_port', 8001) socket_io_address = settings.get('socket_io_address', '') io_loop = io_loop or ioloop.IOLoop.instance() HTTPServer.__init__(self, application, no_keep_alive, io_loop, xheaders, ssl_options) logging.info('Starting up tornadio server on port \'%s\'', socket_io_port) self.listen(socket_io_port, socket_io_address) if flash_policy_file is not None and flash_policy_port is not None: try: logging.info('Starting Flash policy server on port \'%d\'', flash_policy_port) FlashPolicyServer( io_loop=io_loop, port=flash_policy_port, policy_file=flash_policy_file) except Exception, ex: logging.error('Failed to start Flash policy server: %s', ex) if auto_start: logging.info('Entering IOLoop...') io_loop.start() TornadIO2-0.0.4/tornadio2/session.py0000644000076500000240000003147612041200005017435 0ustar sergestaff00000000000000# -*- coding: utf-8 -*- # # Copyright: (c) 2011 by the Serge S. Koval, see AUTHORS for more details. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. """ tornadio2.session ~~~~~~~~~~~~~~~~~ Active TornadIO2 connection session. """ import urlparse import logging from tornado.web import HTTPError from tornadio2 import sessioncontainer, proto, periodic, stats class ConnectionInfo(object): """Connection information object. Will be passed to the ``on_open`` handler of your connection class. Has few properties: `ip` Caller IP address `cookies` Collection of cookies `arguments` Collection of the query string arguments """ def __init__(self, ip, arguments, cookies): self.ip = ip self.cookies = cookies self.arguments = arguments def get_argument(self, name): """Return single argument by name""" val = self.arguments.get(name) if val: return val[0] return None def get_cookie(self, name): """Return single cookie by its name""" return self.cookies.get(name) class Session(sessioncontainer.SessionBase): """Socket.IO session implementation. Session has some publicly accessible properties: `server` Server association. Server contains io_loop instance, settings, etc. `remote_ip` Remote IP `is_closed` Check if session is closed or not. """ def __init__(self, conn, server, request, expiry=None): """Session constructor. `conn` Default connection class `server` Associated server `handler` Request handler that created new session `expiry` Session expiry """ # Initialize session super(Session, self).__init__(None, expiry) self.server = server self.send_queue = [] self.handler = None # Stats server.stats.session_opened() self.remote_ip = request.remote_ip # Create connection instance self.conn = conn(self) # Call on_open. self.info = ConnectionInfo(request.remote_ip, request.arguments, request.cookies) # If everything is fine - continue self.send_message(proto.connect()) # Heartbeat related stuff self._heartbeat_timer = None self._heartbeat_interval = self.server.settings['heartbeat_interval'] * 1000 self._missed_heartbeats = 0 # Endpoints self.endpoints = dict() result = self.conn.on_open(self.info) if result is not None and not result: raise HTTPError(401) # Session callbacks def on_delete(self, forced): """Session expiration callback `forced` If session item explicitly deleted, forced will be set to True. If item expired, will be set to False. """ # Do not remove connection if it was not forced and there's running connection if not forced and self.handler is not None and not self.is_closed: self.promote() else: self.close() # Add session def set_handler(self, handler): """Set active handler for the session `handler` Associate active Tornado handler with the session """ # Check if session already has associated handler if self.handler is not None: return False # If IP address don't match - refuse connection if handler.request.remote_ip != self.remote_ip: logging.error('Attempted to attach to session %s (%s) from different IP (%s)' % ( self.session_id, self.remote_ip, handler.request.remote_ip )) return False # Associate handler and promote self.handler = handler self.promote() # Stats self.server.stats.connection_opened() return True def remove_handler(self, handler): """Remove active handler from the session `handler` Handler to remove """ # Attempt to remove another handler if self.handler != handler: raise Exception('Attempted to remove invalid handler') self.handler = None self.promote() self.server.stats.connection_closed() def send_message(self, pack): """Send socket.io encoded message `pack` Encoded socket.io message """ logging.debug('<<< ' + pack) # TODO: Possible optimization if there's on-going connection - there's no # need to queue messages? self.send_queue.append(pack) self.flush() def flush(self): """Flush message queue if there's an active connection running""" if self.handler is None: return if not self.send_queue: return self.handler.send_messages(self.send_queue) self.send_queue = [] # If session was closed, detach connection if self.is_closed and self.handler is not None: self.handler.session_closed() # Close connection with all endpoints or just one endpoint def close(self, endpoint=None): """Close session or endpoint connection. `endpoint` If endpoint is passed, will close open endpoint connection. Otherwise will close whole socket. """ if endpoint is None: if not self.conn.is_closed: # Close child connections for k in self.endpoints.keys(): self.disconnect_endpoint(k) # Close parent connections try: self.conn.on_close() finally: self.conn.is_closed = True # Stats self.server.stats.session_closed() # Stop heartbeats self.stop_heartbeat() # Send disconnection message self.send_message(proto.disconnect()) # Notify transport that session was closed if self.handler is not None: self.handler.session_closed() else: # Disconnect endpoint self.disconnect_endpoint(endpoint) @property def is_closed(self): """Check if session was closed""" return self.conn.is_closed # Heartbeats def reset_heartbeat(self): """Reset hearbeat timer""" self.stop_heartbeat() self._heartbeat_timer = periodic.Callback(self._heartbeat, self._heartbeat_interval, self.server.io_loop) self._heartbeat_timer.start() def stop_heartbeat(self): """Stop active heartbeat""" if self._heartbeat_timer is not None: self._heartbeat_timer.stop() self._heartbeat_timer = None def delay_heartbeat(self): """Delay active heartbeat""" if self._heartbeat_timer is not None: self._heartbeat_timer.delay() def _heartbeat(self): """Heartbeat callback""" self.send_message(proto.heartbeat()) self._missed_heartbeats += 1 # TODO: Configurable if self._missed_heartbeats > 2: self.close() # Endpoints def connect_endpoint(self, url): """Connect endpoint from URL. `url` socket.io endpoint URL. """ urldata = urlparse.urlparse(url) endpoint = urldata.path conn = self.endpoints.get(endpoint, None) if conn is None: conn_class = self.conn.get_endpoint(endpoint) if conn_class is None: logging.error('There is no handler for endpoint %s' % endpoint) return conn = conn_class(self, endpoint) self.endpoints[endpoint] = conn self.send_message(proto.connect(endpoint)) if conn.on_open(self.info) == False: self.disconnect_endpoint(endpoint) def disconnect_endpoint(self, endpoint): """Disconnect endpoint `endpoint` endpoint name """ if endpoint not in self.endpoints: logging.error('Invalid endpoint for disconnect %s' % endpoint) return conn = self.endpoints[endpoint] del self.endpoints[endpoint] conn.on_close() self.send_message(proto.disconnect(endpoint)) def get_connection(self, endpoint): """Get connection object. `endpoint` Endpoint name. If set to None, will return default connection object. """ if endpoint: return self.endpoints.get(endpoint) else: return self.conn # Message handler def raw_message(self, msg): """Socket.IO message handler. `msg` Raw socket.io message to handle """ try: logging.debug('>>> ' + msg) parts = msg.split(':', 3) if len(parts) == 3: msg_type, msg_id, msg_endpoint = parts msg_data = None else: msg_type, msg_id, msg_endpoint, msg_data = parts # Packets that don't require valid endpoint if msg_type == proto.DISCONNECT: if not msg_endpoint: self.close() else: self.disconnect_endpoint(msg_endpoint) return elif msg_type == proto.CONNECT: if msg_endpoint: self.connect_endpoint(msg_endpoint) else: # TODO: Disconnect? logging.error('Invalid connect without endpoint') return # All other packets need endpoints conn = self.get_connection(msg_endpoint) if conn is None: logging.error('Invalid endpoint: %s' % msg_endpoint) return if msg_type == proto.HEARTBEAT: self._missed_heartbeats = 0 elif msg_type == proto.MESSAGE: # Handle text message conn.on_message(msg_data) if msg_id: self.send_message(proto.ack(msg_endpoint, msg_id)) elif msg_type == proto.JSON: # Handle json message conn.on_message(proto.json_load(msg_data)) if msg_id: self.send_message(proto.ack(msg_endpoint, msg_id)) elif msg_type == proto.EVENT: # Javascript event event = proto.json_load(msg_data) # TODO: Verify if args = event.get('args', []) won't be slower. args = event.get('args') if args is None: args = [] ack_response = None # It is kind of magic - if there's only one parameter # and it is dict, unpack dictionary. Otherwise, pass # in args if len(args) == 1 and isinstance(args[0], dict): # Fix for the http://bugs.python.org/issue4978 for older Python versions str_args = dict((str(x), y) for x, y in args[0].iteritems()) ack_response = conn.on_event(event['name'], kwargs=str_args) else: ack_response = conn.on_event(event['name'], args=args) if msg_id: if msg_id.endswith('+'): msg_id = msg_id[:-1] self.send_message(proto.ack(msg_endpoint, msg_id, ack_response)) elif msg_type == proto.ACK: # Handle ACK ack_data = msg_data.split('+', 2) data = None if len(ack_data) > 1: data = proto.json_load(ack_data[1]) conn.deque_ack(int(ack_data[0]), data) elif msg_type == proto.ERROR: # TODO: Pass it to handler? logging.error('Incoming error: %s' % msg_data) elif msg_type == proto.NOOP: pass except Exception, ex: logging.exception(ex) # TODO: Add global exception callback? raise TornadIO2-0.0.4/tornadio2/sessioncontainer.py0000644000076500000240000001134312041200005021327 0ustar sergestaff00000000000000# -*- coding: utf-8 -*- # # Copyright: (c) 2011 by the Serge S. Koval, see AUTHORS for more details. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. """ tornadio2.sessioncontainer ~~~~~~~~~~~~~~~~~~~~~~~~~~ Simple heapq-based session implementation with sliding expiration window support. """ from heapq import heappush, heappop from time import time from hashlib import md5 from random import random def _random_key(): """Return random session key""" i = md5() i.update('%s%s' % (random(), time())) return i.hexdigest() class SessionBase(object): """Represents one session object stored in the session container. Derive from this object to store additional data. """ def __init__(self, session_id=None, expiry=None): """Constructor. ``session_id`` Optional session id. If not provided, will generate new session id. ``expiry`` Expiration time. If not provided, will never expire. """ self.session_id = session_id or _random_key() self.promoted = None self.expiry = expiry if self.expiry is not None: self.expiry_date = time() + self.expiry def is_alive(self): """Check if session is still alive""" return self.expiry_date > time() def promote(self): """Mark object as alive, so it won't be collected during next run of the garbage collector. """ if self.expiry is not None: self.promoted = time() + self.expiry def on_delete(self, forced): """Triggered when object was expired or deleted.""" pass def __cmp__(self, other): return cmp(self.expiry_date, other.expiry_date) def __repr__(self): return '%f %s %d' % (getattr(self, 'expiry_date', -1), self.session_id, self.promoted or 0) class SessionContainer(object): def __init__(self): self._items = dict() self._queue = [] def add(self, session): """Add session to the container. `session` Session object """ self._items[session.session_id] = session if session.expiry is not None: heappush(self._queue, session) def get(self, session_id): """Return session object or None if it is not available `session_id` Session identifier """ return self._items.get(session_id, None) def remove(self, session_id): """Remove session object from the container `session_id` Session identifier """ session = self._items.get(session_id, None) if session is not None: session.promoted = -1 session.on_delete(True) del self._items[session_id] return True return False def expire(self, current_time=None): """Expire any old entries `current_time` Optional time to be used to clean up queue (can be used in unit tests) """ if not self._queue: return if current_time is None: current_time = time() while self._queue: # Top most item is not expired yet top = self._queue[0] # Early exit if item was not promoted and its expiration time # is greater than now. if top.promoted is None and top.expiry_date > current_time: break # Pop item from the stack top = heappop(self._queue) need_reschedule = (top.promoted is not None and top.promoted > current_time) # Give chance to reschedule if not need_reschedule: top.promoted = None top.on_delete(False) need_reschedule = (top.promoted is not None and top.promoted > current_time) # If item is promoted and expiration time somewhere in future # just reschedule it if need_reschedule: top.expiry_date = top.promoted top.promoted = None heappush(self._queue, top) else: del self._items[top.session_id] TornadIO2-0.0.4/tornadio2/stats.py0000644000076500000240000001011712041200005017075 0ustar sergestaff00000000000000# -*- coding: utf-8 -*- # # Copyright: (c) 2011 by the Serge S. Koval, see AUTHORS for more details. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. """ tornadio2.stats ~~~~~~~~~~~~~~~ Statistics module """ from datetime import datetime from collections import deque from tornado import ioloop class MovingAverage(object): """Moving average class implementation""" def __init__(self, period=10): """Constructor. `period` Moving window size. Average will be calculated from the data in the window. """ self.period = period self.stream = deque() self.sum = 0 self.accumulator = 0 self.last_average = 0 def add(self, n): """Add value to the current accumulator `n` Value to add """ self.accumulator += n def flush(self): """Add accumulator to the moving average queue and reset it. For example, called by the StatsCollector once per second to calculate per-second average. """ n = self.accumulator self.accumulator = 0 stream = self.stream stream.append(n) self.sum += n streamlen = len(stream) if streamlen > self.period: self.sum -= stream.popleft() streamlen -= 1 if streamlen == 0: self.last_average = 0 else: self.last_average = self.sum / float(streamlen) class StatsCollector(object): """Statistics collector""" def __init__(self): self.periodic_callback = None self.start_time = datetime.now() # Sessions self.max_sessions = 0 self.active_sessions = 0 # Connections self.max_connections = 0 self.active_connections = 0 self.connections_ps = MovingAverage() # Packets self.packets_sent_ps = MovingAverage() self.packets_recv_ps = MovingAverage() # Sessions def session_opened(self): self.active_sessions += 1 if self.active_sessions > self.max_sessions: self.max_sessions = self.active_sessions def session_closed(self): self.active_sessions -= 1 # Connections def connection_opened(self): self.active_connections += 1 if self.active_connections > self.max_connections: self.max_connections = self.active_connections self.connections_ps.add(1) def connection_closed(self): self.active_connections -= 1 # Packets def on_packet_sent(self, num): self.packets_sent_ps.add(num) def on_packet_recv(self, num): self.packets_recv_ps.add(num) def dump(self): """Return current statistics""" return dict( # Sessions active_sessions=self.active_sessions, max_sessions=self.max_sessions, # Connections active_connections=self.active_connections, max_connections=self.max_connections, connections_ps=self.connections_ps.last_average, # Packets packets_sent_ps=self.packets_sent_ps.last_average, packets_recv_ps=self.packets_recv_ps.last_average ) def _update_averages(self): self.packets_sent_ps.flush() self.packets_recv_ps.flush() self.connections_ps.flush() def start(self, io_loop): # If started, will collect averages every second self.periodic_callback = ioloop.PeriodicCallback(self._update_averages, 1000, io_loop) self.periodic_callback.start() TornadIO2-0.0.4/TornadIO2.egg-info/0000755000076500000240000000000012041200342016723 5ustar sergestaff00000000000000TornadIO2-0.0.4/TornadIO2.egg-info/dependency_links.txt0000644000076500000240000000000112041200301022764 0ustar sergestaff00000000000000 TornadIO2-0.0.4/TornadIO2.egg-info/PKG-INFO0000644000076500000240000001334012041200301020014 0ustar sergestaff00000000000000Metadata-Version: 1.1 Name: TornadIO2 Version: 0.0.4 Summary: Socket.io 0.7+ server implementation on top of Tornado framework Home-page: http://github.com/MrJoes/tornadio2/ Author: Serge S. Koval Author-email: serge.koval@gmail.com License: Copyright (c) 2011, Serge. S. Koval. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. Description: ========= TornadIO2 ========= Contributors ------------ - `Serge S. Koval `_ Introduction ------------ This is python server implementation of the `Socket.IO `_ realtime transport library on top of the `Tornado `_ framework. TornadIO2 is compatible with 0.7+ version of the Socket.IO and implements most of the features found in original Socket.IO server software. Key features: - Supports Socket.IO 0.8 protocol and related features - Full unicode support - Support for generator-based asynchronous code (tornado.gen API) - Statistics capture (packets per second, etc) - Actively maintained What is Socket.IO? ------------------ Socket.IO aims to make realtime apps possible in every browser and mobile device, blurring the differences between the different transport mechanisms. It's care-free realtime 100% in JavaScript. You can use it to build push service, games, etc. Socket.IO will adapt to the clients browser and will use most effective transport protocol available. Getting Started --------------- In order to start working with the TornadIO2 library, you have to have some basic Tornado knowledge. If you don't know how to use it, please read Tornado tutorial, which can be found `here `_. If you're familiar with Tornado, do following to add support for Socket.IO to your application: 1. Derive from tornadio2.SocketConnection class and override on_message method (on_open/on_close are optional):: class MyConnection(tornadio2.SocketConnection): def on_message(self, message): pass 2. Create TornadIO2 server for your connection:: MyRouter = tornadio2.TornadioRouter(MyConnection) 3. Add your handler routes to the Tornado application:: application = tornado.web.Application( MyRouter.urls, socket_io_port = 8000) 4. Start your application 5. You have your `socket.io` server running at port 8000. Simple, right? Starting Up ----------- We provide customized version (shamelessly borrowed from the SocketTornad.IO library) of the ``HttpServer``, which simplifies start of your TornadIO server. To start it, do following (assuming you created application object before):: if __name__ == "__main__": socketio_server = SocketServer(application) SocketServer will automatically start Flash policy server, if required. If you don't want to start ``IOLoop`` immediately, pass ``auto_start = False`` as one of the constructor options and then manually start IOLoop. More information ---------------- For more information, check `TornadIO2 documentation `_ and sample applications. Examples ~~~~~~~~ Acknowledgment ^^^^^^^^^^^^^^ Ping sample which shows how to use events to work in request-response mode. It is in the ``examples/ackping`` directory. Cross site ^^^^^^^^^^ Chat sample which demonstrates how cross-site communication works (chat server is running on port 8002, while HTTP server runs on port 8001). It is in the ``examples/crosssite`` directory. Events and generator-based async API ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Example which shows how to use events and generator-based API to work with asynchronous code. It is in the ``examples/gen`` directory. Multiplexed ^^^^^^^^^^^ Ping and chat demo running through one connection. You can see it in ``examples/multiplexed`` directory. Stats ^^^^^ TornadIO2 collects some counters that you can use to troubleshoot your application performance. Example in ``examples/stats`` directory gives an idea how you can use these stats to plot realtime graph. RPC ping ^^^^^^^^ Ping which works through socket.io events. It is in the ``examples/rpcping`` directory. Transports ^^^^^^^^^^ Simple ping/pong example with chat-like interface with selectable transports. It is in the ``examples/transports`` directory. Platform: UNKNOWN Requires: simplejson Requires: tornado TornadIO2-0.0.4/TornadIO2.egg-info/requires.txt0000644000076500000240000000004412041200301021314 0ustar sergestaff00000000000000simplejson >= 2.1.0 tornado >= 2.2.0TornadIO2-0.0.4/TornadIO2.egg-info/SOURCES.txt0000644000076500000240000000073112041200301020603 0ustar sergestaff00000000000000setup.py TornadIO2.egg-info/PKG-INFO TornadIO2.egg-info/SOURCES.txt TornadIO2.egg-info/dependency_links.txt TornadIO2.egg-info/requires.txt TornadIO2.egg-info/top_level.txt tornadio2/__init__.py tornadio2/conn.py tornadio2/flashserver.py tornadio2/gen.py tornadio2/periodic.py tornadio2/persistent.py tornadio2/polling.py tornadio2/preflight.py tornadio2/proto.py tornadio2/router.py tornadio2/server.py tornadio2/session.py tornadio2/sessioncontainer.py tornadio2/stats.pyTornadIO2-0.0.4/TornadIO2.egg-info/top_level.txt0000644000076500000240000000001212041200301021441 0ustar sergestaff00000000000000tornadio2