Zeromq

fedmsg is "a library built on ZeroMQ using the PyZMQ Python bindings". So I thought it might help to learn a little bit more about ZeroMQ ( which is not a messaging queue).

  • Contexts - You usually have one context per process and which manages the sockets.
  • Socket - It can be configured and connect to other sockets to exchange messages. You have different types for the different patterns. fedmsg uses the publish/subscribe pattern (asynchronous), where a single process sends messages to multiple recipients.
  • Channel - It's the connection between two sockets.
  • Transport - fedmsg, in particular, uses the TCP protocol

The sockets used in fedmsg to publish and subscribe to messages is  PUB (publisher) and SUB (subscriber). Messages, which are JSON encoded, are published with a topic which can be used by subscribers to filter the messages. A topic in fedmsg can take the form

<topic_prefix>.<environment>.<modname>.<topic>

modname is the module trying to publish the message

Setting it up

Disclaimer: This is for a development instance and not production.

Installing fedmsg is pretty straightforward but configuring it can be a bit tedious.

sudo yum install fedmsg

If you go to /etc/fedmsg.d/ you should see all these files: base.py, endpoints.py, gateway.py, ircbot.py, logging.py, relay.py, and ssl.py

If you try to publish a message from the interpreter

>> import fedmsg
>>> fedmsg.publish(topic='testing', modname='test', msg={
... 'test': "Hello World",
... })

You might get this error AttributeError: 'FedMsgContext' object has no attribute 'publisher'

(Refer https://github.com/fedora-infra/fedmsg/issues/426)

To fix this you need to configure the endpoints in endpoints.py

So an endpoint is basically a string which specifies a transport (TCP) and an address to bind to.

<transport>://address

You can read more about it here http://www.fedmsg.com/en/stable/configuration/#endpoints

In endpoints.py add your endpoint to the config dictionary. The key has to be the <name of the application> .<host name> and the value is a list of all possible endpoints the socket can bind to. You can find out your hostname using hostname -s

You can view the configuration from the terminal using fedmsg-config

One possible entry could look like this:

"__main__.centos-s-1vcpu": ["tcp://127.0.0.1:5001"],

If we try to publish the message now it should work. But how can I check if it shows up on the fedmsg bus? Launch another Python interpreter and try:

>> for name, endpoint, topic, msg in fedmsg.tail_messages():
... print topic, msg

This should print out all the messages currently arriving. You could also use fedmsg-tail --really-pretty from the terminal. fedmsg.tail_messages() gets messages published on the sockets listed in endpoints.

But if you try to publish now you might get this

IOError: Couldn't find an available endpoint for name '__main__.centos-s-1vcpu-1gb-lon1-01'

That's because we specified one endpoint for this application in the config.py file. So if we change the config file to add another one.

"__main__.centos-s-1vcpu-1gb-lon1-01": ["tcp://127.0.0.1:5001", "tcp://127.0.0.1:5002"],

You should be able to publish messages now and see it

No handlers could be found for logger "fedmsg.crypto"
/usr/lib/python2.7/site-packages/fedmsg/core.py:441: UserWarning: !! invalid message received: {u'username': u'root', u'i': 3, u'timestamp': 1530124619, u'msg_id': u'2018-e4090d11-8944-4b30-9d25-0f59c549fab0', u'topic': u'org.fedoraproject.dev.test.testing', u'msg': {u'test': u'Hello World'}}
warnings.warn("!! invalid message received: %r" % e.msg)

You get this warning because validate_signatures in ssl.py is enabled by default. You can disable it for now for development.

references