Chaotic Mess ================ The most important are by far sendMessage and subscribe:: pub.subscribe(listener, topicName) pub.sendMessage(topicName, **data) Listener --------- In the subscribe example, the listener is any Python callable object: function, method, etc. Hence given the following definitions:: def function0(): pass class Foo: def method(self): pass @staticmethod def static(): pass @classmethod def classMeth(cls): pass foo = Foo() these are valid listeners:: function0 foo.method Foo.static Foo.classMeth Topic ------- In the subscribe example, the topicName is a string representing the name of the topic of the message, such as "user_input", "new_card", "game_over". It can contain any character a-z, A-Z, 0-9 and _&%$#@ and the hyphen. The dot "." and "/" are allowed but have special meaning, as explained in BEYOND. Topic Message Data ------------------- The **data** in pub.sendMessage() is a set of keyword arguments containing the data to send as part of the message. For instance, :: pub.sendMessage("game_over") pub.sendMessage("new_card", suit="hearts", rank=8) will send a message of topic "game_over" without any data, followed by a message of topic "new_card" with two pieces of data, namely a string representing the name of the suit of the card that was drawn, and an integer representing its rank. The sendMessage() can only accept message data arguments that are compatible with the topic. This is defined as the Message Data Specification (MDS). Message Data Specification (MDS) --------------------------------- Each topic has a Message Data Specification (MDS) that specifies what data is required vs optional, and what argument names are to be used. The **MDS is inferred for you by the first listener to be subscribed to a topic, or, by explicitely defining the topic via a Topic Tree Definition (TTD)**. Defining the MDS via the TTD is explained in a later section. Given the following callables:: def onCancelSave(): pass def onNewCard(arg1): pass def onGameOver(arg1, arg2=default): pass then the following subscriptions:: pub.subscribe( onCancelSave, "user_input") pub.subscribe( onNewCard, "new_card") pub.subscribe( onGameOver, "game_over") will create, assuming they are the first one for each of those topics, * a topic "user_input" with an empty MDS, ie no data can be sent with messages of this topic; * a topic "new_card" with an MDS of one required argument, named "arg1"; no other data can be sent in messages of this topic, and the data must be present; * a topic "game_over" with an MDS of one required message data named "arg1" and one optional named "arg2"; when not given, the listener will get the default value Note that the default value for an optional message data parameter is not part of the MDS; each listener can therefore decide what default value to use if the data is not provided. Any subsequent subscription of a listener to an already existing topic will succeed only if the listener matches the MDS for the topic. When this is not the case, pubsub raises a InvalidListener exception. Sending vs Broadcasting -------------------------- The pub.sendMessage() has a similar effect to broadcasting data: the topic is akin to the radio frequency of the broadcast. Indeed * all listeners subscribed to the topic will receive the message, as in broadcasting * the sender does not know which (if any) listeners have subscribed to a topic, as in broadcasting * a listener does not know the "emitter" or source of the broadcast * the order in which listeners receive the broadcast is undefined Unlike real broadcasting, a message is sent to each subscribed listener in sequence rather than in parallel (but the sequence order is undefined). Each listener must return (rather than raise an exception) in order for the broadcast to complete. Any return value of a listener is ignored. Your code must not make any assumptions about the order. In fact, you should design your application to be independent of this order. Types of errors ---------------- The main type of error you will encounter is when you attempt to subscribe a listener that does not have a signature (call protocol) compatible with the topic's MDS: a ListenerInadequte exception. Given :: def listener0(arg1, arg2=default0): pass def listener1(arg1): pass def listener2(arg1, arg2): pass def listener3(arg1=val, arg2=default3): pass pub.subscribe(listener0, "topic") then :: pub.subscribe(listener1, "topic") will fail since arg2, which might get sent, could not be received by the listener. Similarly, :: pub.subscribe(listener2, "topic") will fail because the second parameter is required, yet the MDS allows for that parameter to be missing from some messages, so listener2 is too "demanding". However, :: pub.subscribe(listener3, "topic") will succeed because eventhough listener3 doesn't require arg1 (it has a default value), the MDS will force you to always provide one. A second type of error is forgetting to include required data as part of the message. If the MDS for "new_card" is a required suit and an optional rank, then :: pub.sendMessage("new_card", rank=6) will raise an exception since suit, required, is missing. A third type of error is a silent one, unless you make use of a topic tree definition provider (TTDP, defined later): it occurs when you mistype the name of a topic. Without a topic tree definition provider, any new topic name causes the creation of a new topic. Therefore "new-card" would create a new topic. This can lead to hard-to-isolate bugs as the listeners subscribed to the topics having a typo in their name will never get a message! The strategy when a listener doesn't get a message should then be: * confirm that the message gets generated * confirm that the listener is subscribed to the correct topic name (no typo); this is easiest with `printTopicDocs(extra='L')` which will list the listeners of each topic:: from pubsub import pub from pubsub.utils.topictreeprinter import printTreeDocs printTreeDocs(pub, extra='L') Architecture and Design Guidelines ------------------------------------- * Look at examples. Typically GUI, controller, model * A listener should not return any data: the return value of a listener is not forwarded to the sender since senders and receivers have no notion of each other. * A listener should not raise any exceptions or let any escape. When this happens, the exception will interrupt the sendMessage. It is not a good idea to put an except clause around the sendMessage() since the sender should remain ignorant of what exceptions are possible. See ListenerExcHandler to trap exceptions. * Topic names should represent user or system domain events, not implementation details such as widgets or components. For instance, "dialog1OK" is likely a bad idea: it says nothing about the event (what is dialog 1 OK'ing?), and is dependent on the widget's name (which could change). A better name could be "fileSaveOK" if the dialog was asking the user to confirm a file save. * Use pubsub.utils.exportTopicTree() to get a readout of the topics and their MDS. Use pubsub.utils.printTopicTree() to get a printout of the topics and optional characteristics such as MDS and list of currently registered listeners. * Start all listener functions and methods with *psOn*, for instance ``def psOnCloseDocument()``. Example Development Process ---------------------------- The following represents a typical development process when using pubsub: - Design your application into independent modules or subpackages that don't import one another - Define basic event types exist in the application: 'user' (events from user interface), 'filesystem' (events from local filesystem), etc. These are your messaging topics. You may find it useful to use ``printTreeDocs`` from ``pubsub.utils.topictreeprinter``. - Define some data for each event type, and which data are optional/required - Implement your modules - Subscribe listeners with appropriate signature (according to data for each topic/event type) - Send messages with appropriate data - Handle messages in listeners, without making any assumptions about sender or order of receipt - Testing: no need for API, all you need to do is send messages! You can see a very informative view of an application before and after incorporatng pubsub, at `Steven Sproat's dev site`_ (click "expand all" and "show diffs side-by-side"). Steven says: *You can see how I removed some GUI logic from the Canvas class (a child of the GUI) and placed "controller" functions into my GUI that subscribed to pubsub topics and delegates to the appropriate classes.* .. _Steven Sproat's dev site: http://bazaar.launchpad.net/~sproaty/whyteboard/development/revision/286 Topic Tree Specification -------------------------------- *Topic Specification* can be used to have better control over your topic hierarchy. If you don't specify your application's topics, pubsub infers them from the first subscribed listener of each topic. E.g.:: def listener1(arg1, arg2=None): pass def listener2(arg1=None, arg2=None): pass pub.subscribe(listener1, 'topic.sub') pub.subscribe(listener2, 'topic.sub') Because listener1 is the first to be subscribed to 'topic.sub' topic, pubsub uses it to infer the specification of 'topic.sub': the specification is "messages of that topic *must* provide data for arg1, and *may* provide data for arg2". The second listener subscribed, listener2, is allowed to subscribe because it is compatible with the topic's specification created at the previous call. What if your intent was that arg1 is optional as well, i.e. the signature of listener1 is wrong (it should provide a default value for arg1)? Or what if per chance listener2 gets subscribed first (could happen if both are subscribed in different modules whose load order changes)? There are two ways to go about this: - Call ``pub.newTopic()`` to explicitly create the topic; this must happen before any listener subscribes to it - Use one or more *Topic definition providers* (TDP) Topic Definition Providers --------------------------- The base class that defines the *Topic Definition Provider* API is ``pubsubutils.ITopicTreeDefnProvider``. An implementation that uses a Python syntax to describe the topic tree, ie using Python classes and class variables, is available as ``pubsubutils.TopicTreeDefnSimple``. Example:: from pubsubutils import TopicTreeDefnSimple class MainTopicTree(TopicTreeDefnSimple): class topic1: '''Explain what this topic is for''' optArg1 = 'explain what this optional arg is for' class subtopic2: '''Explain what this subtopic is for''' reqArg1 = 'explain what this required arg is for' _required = 'reqArg1' class subsubtopic3: '''Explain what this subtopic is for''' reqArg2 = 'explain what this second required arg is for' reqArg3 = 'explain what this third required arg is for' optArg2 = 'explain what this second optional arg is for' _required = ('reqArg2', 'reqArg3') from pubsub import pub pub.addTopicDefnProvider( MainTopicTree() ) The use cases supported are: - Formalization of Topic tree: - You have not used the above yet; - You have a preliminary topic tree which gets created for you by pubsub based on listeners that you subscribe (default when ``pub.addTopicDefnProvider()`` not called); - You then want to formalize the tree such that can you specify clearly what arguments are required and optional, verify that there are no erroneous branches (due to a typo in a topic name), and document each topic and argument; - You want to keep expanding the tree, without always having to update formal topic tree specification To handle this, you would use the above example code. Note that a preliminary tree can be obtained from your application via ``pubsubutils.printTreeSpec()``. - Constrain the Topic tree: - Your application reaches a milestone, such that you want to "freeze" the tree; - You don't want to allow any other topics to be created; - If new topics are needed they will be few such that it's ok to require that they be first specified via a definition provider To handle this, you would add to the above example:: pubsubconf.setTopicUnspecifiedFatal(True) Then any attempt to work with a topic that does not exist will raise an ``pub.TopicUnspecifiedError``. - Allow application components to extend the tree: - Your application defines a basic tree - Your applications uses a plugin mechanism to support extensions to itself, or your application consists of several disjoint architectural components that communicate exclusively via pubsub (presumably, one of those components is the "core" part of the application). - The plugins or components should be able to add subtopics to the tree without the having to edit the main TDP of the tree. The ``pub.addTopicDefnProvider()`` can be called as many times as you wish. When an unknown topic is required, each TDP will be queried for it. This means that plugins need merely to call this function with their part of the topic tree. For instance:: class PluginATree(TopicDefnProviderSimple): class topic1: class subtopic2: class subsubtopic4: '''Describe new subtopic created by plugin''' optArg2 = 'explain what this second optional arg is for' class subtopic5: '''This is a new subtopic of topic1 so the plugin can...''' # no additional args class topic2: '''New root topic used only by plugin''' optArg1 = '...' This works because any topic that does not have a docstring is not considered to be a specification. Note that the representation of a Topic Tree Specification need only adhere to the ``ITopicTreeDefnProvider`` interface. Other implementations are possible, e.g. one using XML from an XML file would be useful. Notification ------------- Pubsub can call a specified handler everytime certain types of calls are made to it: - *subscribe*: whenever a listener subscribes to a topic - *unsubscribe*: whenever a listener unsubscribes from a topic - *deadListener*: whenever pubsub finds out that a listener has died - *send*: whenever the user calls sendMessage() - *newTopic*: whenever the user defines a new topic - *delTopic*: whenever the user undefines a topic A handler class is available already: ``pubsubutils.NotifyByPubsubMessage``. This handler takes each notification received and generates a pubsub message in the "pubsub.*" branch. Your application gives the handler *class* (whether or not it is NotifyByPubsubMessage) to pubsubconf at initialization (first time pubsub is loaded into your application) by calling ``pubsubconf.setNotifierClass``. Therefore, to use notification via this notifier, you must register one or more listeners for the various special topics. A default listener of all those topics is available, ``pubsubutils.DefaultLogger``, which registers its methods as listeners of the pubsub topics. These listeners merely print a message to stdout. A shortcut is available from pubsubutils, to do all of the above. For example:: #import pubsubconf # optional #...use pubsubconf functions... # optional import pubsubutils logger = pubsubutils.useDefaultLoggingNotification() # probably no point in calling pubsubconf functions after this You can make your own pubsub notification handler by deriving from ``pubsubutils.INotificationHandler`` and giving it to ``setNotifierClass``. .. _exception-handling: Exception Handling ------------------- How to recover from exceptions raised in listeners Multi-threading ---------------- The main challenge with using pubsub in multi-threaded application is insuring that the listener is called only from within its own thread, or provides some inter-thread synchronization mechanism. There are so many ways of doing the latter that it is unlikely that anything pubsub could provide would be of general use (that said, I'd be happy to be proven wrong). See the discussion on pubsub_dev, http://groups.google.com/group/pypubsub_dev/browse_thread/thread/7f414e82f62d64b7. Multi-processing ----------------- In progress: passing messages between python scripts spawned from a master application, to support transparent publish-subscribe of real multi-processing python applications (ie not limited by GIL). Consider: - mmap module: mentioned on http://bytes.com/forum/thread25421.html - pyprocessing package: http://pyprocessing.berlios.de/doc/index.html - pp package: http://www.parallelpython.com/ and possibly other items on http://wiki.python.org/moin/ParallelProcessing. Utilities --------- - Print: * current specification: ``pubsubutils.printTreeSpec()`` * current tree: ``pubsubutils.printTreeDocs()`` Miscellaneous -------------- other types of errors other: topic creation, listener validation, Listener, TopicObj arg1 messaging protocol migrating from old pubsub multithreaded apps pubsub in wxPython (old vs new pubsub, autopubsubsetupv1/setupv1; wxPy v2.8.11) multiple publisher domains