Messaging with Fedora


keeping everything in sync, asynchronously











Aaron Coburn

SysAdmin/Programmer

acoburn@amherst.edu

Fedora is never just Fedora

Messaging







Hey, meet me at the Hangar for wings at 6

Application messaging: not so different



<?xml version="1.0" encoding="UTF-8"?>
<entry xmlns="http://www.w3.org/2005/Atom">
  <updated>2013-06-03T08:35:13.953Z</updated>
  <author>
    <name>fedoraAdmin</name>
    <uri>http://localhost:8080/fedora</uri>
  </author>
  <title type="text">purgeObject</title>
  <summary type="text">amherst:42</summary>
</entry>
      

Message brokers: decoupling components







Message brokers: distributing workloads




Writing a client in Python: lots of boilerplate



import stomp
import signal

class Listener(stomp.ConnectionListener):
  def on_message(self, headers, body):
    do_something(headers['pid'], headers['methodName'])

connection = stomp.Connection(host_and_ports=[('localhost', 61612)])
connection.set_listener('', Listener())
connection.start()
connection.connect()
connection.subscribe(destination='/queue/fedora.apim.update', id='my-id')

while True:
  signal.pause()
      

Message oriented middleware: no boilerplate






Message routes: pipelines





<route>
  <from uri="activemq:queue:fedora.apim.update" />
  <to uri="xslt:file:///path/to/stylesheet.xsl" />
  <to uri="solr://solr-host:8983/solr" />
</route>
      

Message routes: aggregating messages




<route id="fedora-aggregator">
  <from uri="activemq:queue:fedora.apim.update"/>
  <aggregate strategyRef="aggregatorStrategy"
             completionTimeout="10000">
      <correlationExpression>
        <simple>${header.pid}</simple>
      </correlationExpression>
      <to uri="seda:aggregated"/>
  </aggregate>
</route>
      

Message routes: filter and multicast


<route id="fedora-update-multicast">
  <from uri="seda:fedora.update"/>
  <choice>
    <when>
      <xpath>/foxml:digitalObject/foxml:objectProperties/foxml:property[
              @NAME = 'info:fedora/fedora-system:def/model#state' and
              @VALUE = 'Active']</xpath>
      <multicast>
          <to uri="vm:solr.update"/>
          <to uri="vm:jena.update"/>
          <to uri="vm:couchdb.update"/>
          <to uri="vm:oai.update"/>
      </multicast>
    </when>
    <otherwise>
      <to uri="seda:fedora.delete"/>
    </otherwise>
  </choice>
</route>
      

Putting it all together



Fedora Message



Aggregator
combine messages

Routing
delete or update?

Enhancing
retrieve full FoXML


Routing
active or inactive?

Multicast
pass to multiple handlers

Pipelines
modify FoXML

Solr update

Deploying routes: how does this all run?









Image Credits

http://activemq.apache.org
http://backbonejs.org
http://camel.apache.org
http://couchdb.apache.org
http://drupal.org
http://fedora-commons.org
http://fortawesome.github.io/Font-Awesome
http://imagemagick.org
http://jena.apache.org
http://jquery.com
http://karaf.apache.org
http://lucene.apache.org/solr
http://mysql.com
http://poi.apache.org
http://python.org
http://redis.io
http://shibboleth.net
http://stanbol.apache.org
http://wowza.com
http://www.mulgara.org
http://www.openlayers.org

Questions?







acoburn@amherst.edu








More information is available at acdc.amherst.edu/wiki