Skip to end of metadata
Go to start of metadata

 

 

Project Site: http://redis.io

Docker Image: 

Redis is an open source, in-memory data structure store, used as database, cache and message broker.

Packaging Redis?

The present Redis example was developed as a simple demonstration of the Service Fabric supporting non OSGi / Java artefacts using the Service Fabric 1.11 release and a previous implementation of Packager. This previous implementation of Packager, did not support Container Images or Docker, but rather managed software artefacts as native process directly on the host.

The interesting point is that the same example works today - without any changes - with the new Service Fabric 1.13 release.

Packager treats the software artefact as an OSGi bundle; hence proper encapsulation / isolation are provided by the OSGi bundle model. Specifically, the Redis Packager artefact was - and still is - a self-describing OSGi bundle and so the internal implementation of the Packaged Redis bundle is of no consequence to the Service Fabric itself.  Secondly, the Service Fabric and Packager are both based on stable, open, industry standard API's courtesy of the OSGi Alliance. Hence, backwards compatibility is a given (smile)

The only functional difference is that the REDIS URL will not be published in Entire's App Endpoint View.

The intent is still - at some point - to update the Redis example to demonstrate using the Fabric Ensemble Manager to control leader-elect; but we're in no hurry.

In the meantime, ...

Description

A simple Stock Market pricing application, demonstrates the Service Fabric's approach to dynamic deployment and configuration of a clustered Redis service.

The example consists of a client reading stock price values from a distributed Redis based service. 

The Systems structure is comprised of the following 3 system.parts

  • com.paremus.demo.redis
    The client retrieves stock price values and status of the Redis service and displays the resultant information in the users browser using Angular/JS. The client automatically rotates queries across available Redis cluster members. When the Redis service is writable this example also runs a background task randomly updating stock prices. Note that the client is pinned to a Fibre with a Feature Master and has a  cancellationCost of -1, which informs the Service Fabric Provisioner not to relocate the Redis Master should more attractive Master resources become available.

  • com.paremus.packager.pkg.redis.master.guard
    The Redis Master is the Redis node which holds the master copy of the data and is the only one which accepts write operations. The Redis Master is again pinned to Master and has a cancellationCost of -1.     

  • com.paremus.packager.pkg.redis.slave.guard
    Redis Slaves are Redis nodes which synchronize with the Master to provide a read-only view of the data, allowing queries to be load balanced over multiple machines. A Service Fabric Replication Handler is used to automatically scale the Slaves across available Fibres; one Slave per Fibre, staring with the second available Fibre. A Service Fabric Resource Contract is also used to ensure that a Slave will not be placed on the Master Fibreor any Fibre which already hosts two components of the Redis System.

The example is 'self-wiring'. By this we mean that that the client com.paremus.demo.redis is dynamically configured, and if necessary (i.e. due to failure) subsequently re-configured, with the required endpoint information (locations & ports) to access the Redis Master and Slaves. Likewise, com.paremus.packager.pkg.redis.master.guard and com.paremus.packager.pkg.redis.slave.guard are themselves dynamically configured, and if necessary subsequently re-configured.

Use of Service Property Updates

This example makes extensive use of the Service Property updates; a Paremus led addition to the OSGi Alliance Remote Service Administration specification. As shown, Service Properties may be tracked while working through the following example using the lab command on the appropriate fibre.

redis master - group redis-demo
% lsb | grep redis.guard
163  ACTIVE    com.paremus.packager.pkg.redis.guard:0
%
% lsb -b 163 -p
163  ACTIVE  com.paremus.packager.pkg.redis.guard:0
163    PUB:  198  0 -
163    PUB:  198  T: org.bndtools.service.packager.ProcessGuard
163    PUB:  198  P: package.type=redis
163    PUB:  198  P: version=2.6.14
163    PUB:  199  0 -
163    PUB:  199  T: org.bndtools.service.endpoint.Endpoint
163    PUB:  199  P: node.type=master
163    PUB:  199  P: redis.group=redis-demo
163    PUB:  199  P: service.exported.interfaces=*
163    PUB:  199  P: type=redis
163    PUB:  199  P: uri=redis://192.168.1.64:6379
163    PUB:  199  P: version=2.6.14
redis group changed to redis-dead
% lsb -b 163 -p
163  ACTIVE  com.paremus.packager.pkg.redis.guard:0
163    PUB:  198  0 -
163    PUB:  198  T: org.bndtools.service.packager.ProcessGuard
163    PUB:  198  P: package.type=redis
163    PUB:  198  P: version=2.6.14
163    PUB:  199  0 -
163    PUB:  199  T: org.bndtools.service.endpoint.Endpoint
163    PUB:  199  P: node.type=master
163    PUB:  199  P: redis.group=redis-dead
163    PUB:  199  P: service.exported.interfaces=*
163    PUB:  199  P: type=redis
163    PUB:  199  P: uri=redis://192.168.1.64:6379
163    PUB:  199  P: version=2.6.14

As the published properties are changed; the published Service ID (199), remains constant; so demonstrating the Service Properties have been updated rather than, as previously required by the specification, restarted with new properties.   

Example System

The Redis System ( https://www.bundlerepo.org/repos/examples/systems/redis.xml ), is shown.

To aid identification in the example - we allocate Redis Master to the fixed port 6379. In contrast, via the use of ( "${port#any}" ), the Fabric is allowed to dynamically choose which port to allocate to each deployed Slave. 

<system
    xmlns="http://schema.paremus.com/sf/1.1"
    name="redis-demo"
    boundary="fabric"
    repopath="
        https://www.bundlerepo.org/repos/paremus/index-nim.xml,
        https://www.bundlerepo.org/repos/oss/index-nim.xml,
	    https://www.bundlerepo.org/repos/examples/index-nim.xml
    ">

    <description>
       Deploys a Redis client application, as well as a highly
       available Redis group
    </description>
    <admin group="demo" />

    <!-- 
    The Redis client bundle is a simple web application for viewing
    the data stored in the Redis data store.
    -->

    <system.part category="osgi.active.bundle" name="com.paremus.demo.redis">
       <contract features="(master=true)" cancelationCost="-1" />
    </system.part>

    <!-- The Redis Master Node -->
    <system.part category="msf" name="com.paremus.packager.pkg.redis.master.guard">
       <config>
          <property name="redis.group" value="redis-demo" />
          <property name="port" value="6379" />
       </config>
       <contract features="(master=true)" cancelationCost="-1" />
    </system.part>

    <!-- Redis Normal Slave Nodes -->
    <system.part category="msf" name="com.paremus.packager.pkg.redis.slave.guard">
       <config>
          <property name="redis.group" value="redis-demo" />
          <property name="port" value="${port#any}" />
       </config>

       <!--
        We leave this property commented out because part of the
        demo involves switching it on to see the change
        -->
        <!-- <property name="can.become.master" value="true" /> -->
        <replication.handler name="scale" type="scalable">
           <property name="scaleFactor" value="1" type="float"/>
           <property name="minimum" value="1" type="integer"/>
       </replication.handler>
       <!-- <contract features="(!(master=true))" /> -->
       <!-- <contract features="(!(|(master=true)(fibre.composite.com.paremus.packager.pkg.redis.slave.guard &gt;2)))" /> -->
       <contract features="(!(|(master=true)(fibre.system.redis-demo &gt;2)))" />

    </system.part>

</system>

The Resource Contract and Replication Handler behaviors used are not mandatory.

The configuration presented is used to simplify the runtime topology for the purpose of the example. One might remove these constraints and collapse the Client / Master / Slave runtime onto single Fibre, or completely dispersed across multiple Fibres with the Fabric allowed to re-deploying Redis slave system.part instances should more attractive fibre resources appear.

 

Running the Example

Import Redis example in the usual fashion. Remember that both the Client and Master system.parts have a resource contract specifying a requirement master=true, so select a fibre and apply this label, the deploy the Redis system.

 

The Stock Market pricing example may now be viewed at the URL http://XXX:9000/paremus/demo/redis/index.html: where XXX is the IP address of the fibre labelled master=true.

The following screen capture indicates that the Stock Market service is running, healthy, updating and is currently retrieving data from the Redis Master on port 6379. If another port is displayed - whereas if any other port is shown, we know the update is from one of the Redis slaves.