This section describes how to transition an application from using PyPubSub’s arg1 messaging protocol to using the kwargs protocol. It proposes some steps that will make such transition easier. These are suggestions that have worked for some people, and any feedback would be useful.
Real example transition data: 50 listeners, 60 topics, spread out over 5000 lines of code; required one hour of manual labor (for steps 1 to 4 in next subsection).
To verify which messaging protocol your application uses, get the return value of pub.getMsgProtocol(). If it is arg1, follow the suggested steps. If it is kwargs, there is nothing else to do: you are already there! For any other value, post to the PyPubSub help forum (see URL on Installation/Support page).
The first step assumes that the application can run before the step is executed. Each subsequent step assumes that the previous step resulted in a runnable application. If this assumption is wrong, the step will likely fail.
Pubsub provides some utility functions and settings that assist in the migration.
Note that only a subset of the steps involve changing your code.
The objective is to go from arg1 messaging protocol, which supports
pub.sendMessage('topic1')
pub.sendMessage('topic2', yourData)
where
to the kwargs messaging protocol, which supports
pub.sendMessage('topic1')
pub.sendMessage('topic2', argName=data)
where
listeners of topic1 and topic2 MUST be of the form:
listener() # for topic1
listener(argName[=anything]) # for topic2
listeners access the data directly (one or more argName’s)
Go to step 2. Should be simple :)
Second step is to change all the listeners to take one required parameter of the same name. This can be adjusted later but for now this will help catch errors.
Actions for step 2:
CHOOSE one argument name that all listeners will use. The name cannot be data as it is already used in sendMessage(). For instance, msg. This is a temporary measure, so something unique (ie, easy to search for) is sufficient. The remainder assumes you chose msg.
REPLACE the line
pubsubconf.setMsgProtocol( 'arg1' )
by the line
pubsubconf.transitionV1ToV3( 'yourSelectedParamName' )
CHANGE the first argument of all your listeners to what you chose in previous step, ‘yourSelectedParamName’.
RUN your application, and correct any errors:
REPEAT this step until there are no more errors.
Note
If your application runs without a console window and you don’t catch ListenerMismatchError somewhere in your main, you may not have time to see the exception traceback that the Python interpreter prints at exit. Start your GUI app from a console, or have a catch-Exception or catch-ListenerMismatchError and print the error in your GUI.
Third step is to add a keyword argument name in ALL your calls to pub.sendMessage(), and up the transition stage in pubsubconf.transitionV1ToV3() so that pubsub knows that you want it to check that all message sending will use a keyword argument:
pubsubconf.transitionV1ToV3( 'yourSelectedParamName', step=2 )
Actions for step 3:
ADD the ‘yourSelectedParamName’ keyword name to every pub.sendMessage() that sends data with the message. Don’t forget to do this in your unit tests as well. So each of your sendMessage should be of the form:
pub.sendMessage('topic', yourSelectedParamName=data)
REMOVE the message parameter of any all-topics listeners since ALL_TOPICS listeners cannot get any data (except a pub.Topic` or via **kwargs). Such listeners were subscribed as pub.subscribe(listener) or pub.subscribe(listener, pub.ALL_TOPICS) instead of the typical pub.subscribe(listener, topic). For those listeners that really need the data, you can bypass pubsub’s default behavior by using **kwargs as the message parameter, and access the data in your listener via kwargs[yourSelectedParamName].data.
CHANGE the transitionV1ToV3 call to be pubsubconf.transitionV1ToV3('msg', step=2). This will change the requirements imposed on listeners: in pubsub 3, listeners can have any signature: regular parameters, keyword arguments, and *args and **kwargs.
RUN your application, and correct any errors: the only ones should be listeners or sendMessage() calls that you forgot to adjust as per steps 9 and 10.
Note
In this step you will remove the ‘.data’ that appears in all listeners since that the next configuration setting gives the data directly to your listeners:
After step 4, you have a topic tree in which each topic has the same topic message specification (TMS): one argument (yourSelectedParamName), optional. You can see this by adding the following lines somewhere in your application, after all your subscriptions have been executed:
print pub.exportMyTopicTree()
The main shortcoming of your topic tree so far is that listeners can get any data via their msg argument, making it easy to send the wrong data to a listener. It is best to ‘divide’ your data into pieces that are specific to each topic, as you would if you had used pubsub version 3 from the start.
See the file tests/trans1to3/test_trans_step_5.py from source distribution for example of such steps.
To take full advantage of pubsub3, you should now split your listener argument (msg) into one or more arguments. It will be handy to refer to your topic tree printout for convenience.
SELECT a leaf topic (one that doesn’t have subtopics), and determine how the data currently given in associated sendMessage() calls could be divided among several keyword arguments. You may want to split off just one piece of data: for instance, if data was a pair (a,b) and b is only used by leaf topic, then split off b, as shown in the next step.
ADD relevant arguments to all sendMessage() for topic just affected. For instance,
sendMessage(‘topicA’, msg=(a,b)) # OLD sendMessage(‘topicA’, msg=(a,), b=b) # NEW
ADD those arguments to all listeners of the selected topic. Make the necessary adjustements to each listener’s code. Leave other topic listeners unchanged:
# OLD:
def listenerA(msg=None):
...use msg[0] and msg[1]...
# NEW:
def listenerA(msg=None, b=None):
...use msg[0] and b...
RUN your application and correct any errors resulting from above changes.
REPEAT from 16 until satisfied, first with other leaf topics, then making your way up the tree of topics, until your msg arguments are unused in the listeners’ code.
REMOVE all the unused msg arguments.
REMOVE default values from arguments that are required data. Note that all listeners of the associate topic must have the same argument names for both required and optional arguments. For instance to indicate that a is required:
# OLD:
def listenerA1(a=None, b=None) # both a and b are optional
def listenerA2(a=None, b=None) # both a and b are optional
# NEW:
def listenerA1(a, b=None) # only b is optional
def listenerA2(a, b=None) # must be same signature as listenerA1
The previous stage allows you to make it clear, on both the sending and receiving ends, what data is being sent with each message. However, thus far, you have left it up to pubsub3 to infer the TMS from the first listener subscribed. It is best to specify this explicitly, and to document your topics.
ADD a line, for printing the topic tree definition, for instance pub.exportMyTopicTree('MyTopicTree')
RUN your application so all the listeners get subscribed, and exit.
EDIT MyTopicTree.py:
ADD the following at the beginning of your application:
from MyTopicTree import MyTopicTree
pub.importMyTopicTree( 'MyTopicTree' )
RUN your application again. Correct errors, mostly related to some listeners no longer satisfying the TMAS for the topic they are subscribing to.
ADD a call to pub.setTopicUnspecifiedFatal() so that TMAS inference is turned off. This will cause subscription/sendMessage calls for a topic that is not specified in MyTopicTree to raise an exception.