Spurred by my belief in the Construct Infrastructure I took my official t-shirt with me in my backpack. In my role as Construct evangalist I’ll document the far reaching effects we’re having through a few photos. First is Rio de Janero in Brazil:
Provided my bag doesn’t get stolen there will be more to follow.
Part 2
And here it is. On Monday (8/9) I deployed Construct 0.8a to Machu Picchu in Peru. The Inkas love it. When I left the Inka chief was in deep discussion about named graph support and php-based web mashups over a hot cup of coca tea:
In part 1 of this post I described the new HTTP Port in Construct. In this part of the post I’ll explain how to write a web form and style the return values.
SPARQL Query
You can send a query to Construct with GET or POST. Here is how we do it in HTML with GET:
<textareaname=“q”cols=“64″rows=“10″>SELECT ?subject ?predicate ?object WHERE {?subject ?predicate ?object}</textarea><br/>URL of XSLT to apply (optional):<br/><inputtype=“text”name=“xsl”size=“56″value=“http://www.construct-infrastructure.org/stylesheets/default.xsl type=”submit” value=”Submit Query“>
</form>
the only thing the HTTP Port wants is the query field to be called either “q” or “query”. The returned XML will be sent back to your browser. Something like this:
To make the results look neater you can apply and XSL stylesheet. Send the URL for the XSL stylesheet as a field “xsl” in the form. This stylesheet must be web accessible. We’ve made a default example stylesheet you can use to get started.
Inserting N-3 RDF Data
You can insert new data into Construct using a form like this (we’re using POST this time for variety but GET works too):
<textareaname=“i”cols=“64″rows=“10″><http://www.pervasive-ontologies.org/ontologies/sensors/bluetooth#reading00:19:63:96:56:01@00:80:98:94:AE:4B@1201631502><http://www.pervasive-ontologies.org/ontologies/sensors/bluetooth#spotted>“00:19:63:96:56:01″.</textarea><br/>Expiry time for data (optional):<br/><inputtype=“text”name=“expiry”size=“15″value=“20000″><br/><inputtype=“submit”value=“Submit Data”>
</form>
in this form you send the data to be inserted as a field “i” or “insert”.
The returned data from an insert is an XML document with a status code and any messages given back from Construct. It looks something like this:
<response>
<status>OK</status>
<description>The data <http://www.pervasive-ontologies.org/ontologies/sensors/bluetooth#reading00:19:63:96:56:01@00:80:98:94:AE:4B@1201631502><http://www.pervasive-ontologies.org/ontologies/sensors/bluetooth#spotted>“00:19:63:96:56:01″. with given expiry 20000 was passed to the Construct data store without error.
</description>
</response>
Well, that is about it. You can make calls to Construct over HTTP from browsers or web applications using the HTTP Port. Have fun.
The latest release of Construct has the new HTTP Port enabled by default. The HTTP Port has similar functionality to the data port — you can use it to insert and query for data in Construct. The difference is that the data port works over raw TCP sockets, whereas the HTTP Port works over HTTP.
Now we can query Construct through a web browser and web apps… neat, huh?
these lines activate the HTTP Port on port 8888. When you have Construct running, open your web browser and go to http://localhost:8888/ and you should a simple set of forms for querying and inserting data using HTTP GET and HTTP POST. It looks something like this:
From this page you can send RDF data in N-3 format or SPARQL queries to Construct. Try clicking on one of the buttons to test it out.
This is the end of Part 1 on the HTTP Port. Part 2 of this post will explain how to build your own web forms and use of XSL stylesheets for your results.
This post contains a guide for building a data port and query service client proxy for Construct in a language of your choice. The purpose of proxy objects is to disguise as much as possible the distributed nature of Construct from the developer — taking care of locating and accessing Construct services over the network.
The following sections describe (at a high level) the steps involved in building proxies for the data port and query service. If you plan to implement a client proxy, take a look at the Java, Python, and Ruby code in the codebase for examples. See the Construct protocols on the tutorial page for protocol details.
Layer 1: Finding and Connecting to Services - Connect to the Construct Bonjour Proxy
The Construct Bonjour Proxy maintains a list of active Construct instances that it can see. Assuming you have set it up on your machine, it will be running on port 3826. Connect to this port to be sent a list of service descriptors.
Layer 2: Connecting to a Service
Assuming that some services are present, pseudocode for this process is a follows:
iterate through each available service instance
if the service descriptor contains the desired service, attempt to create a connection to it
if successful, return the connection. If not, continue the iteration
If no suitable services are available, you may wish to throw an exception.
Layer 3: Wrapping the socket in an appropriate client object
The final step is to provide code that can be wrapped round the socket to handle interaction with the services and client. Code is required for each Construct service that the client requires to access (the data port and the query service).
Retain a reference to the socket as it can be reused for multiple interactions. You should only need to obtain a new set of service descriptors when your connection fails.
Below are examples of the sort of functionality you may wish to provide in your proxy:
Get the response from the data port (using the correct protocol) and return true/false to indicate successful submission
add(String rdf, long expiryTime)
As above, but add the expiry time to the end of concatenated N-Triple string
add(LanguageSpecificConstruct model)
Convert the (Jena, rdflib etc.) model to N-Triples and continue as add(String rdf)
add(Model model, long expiryTime)
Convert the (Jena, rdflib etc.) model to N-Triples and continue as add(String rdf, long expiryTime)
Destructor
stop the process scanning for Construct instances
The Query Service proxy
Methods:
Constructor (boolean scan_localhost_only)
start the process to scan for Construct instances
query(String sparqlQuery)
Takes in a String containing a SPARQL query
Create a connection to the query service (see Layer 2), format the query using the correct protocol (see Construct protocols tutorial page) and send the query.
Get the response from the data port (using the correct protocol)
If an error occurred, return null
If successful, transform the RDF into the appropriate language specific construct representing a SPARQL result set (Jena, rdflib etc.)
In this post we’ll describe the construction and runtime operation of Construct Components. These are the building blocks that combine to form the Construct infrastructure. If you want to modify how Construct works internally (i.e. write a new data dissemination mechanism, change the internal data model, add a reasoner…) then this is the place to start.
The number and type of components in any given deployment is customisable. At runtime, each of the components defined in the construct.properties file is loaded and instantiated by the Component Manager. The Component Manager is also responsible for the correct shutdown of the components when the software is terminated.
In order to facilitate this process, components must implement two interfaces. The first interface is the ConstructComponent interface, which defines all the methods required for communication between a component and the Component Manager. The second interface is component specific, and is used to register the component with Construct, and allow other components to gain access to it. A default implementation of the ConstructComponent interface, the AbstractConstructComponent, is provided in order to simplify the process of creating new components. Developers may extend from this class if desired, or implement directly from the ConstructComponent interface.
The ConstructComponent Interface
The ConstructComponent interface extends the Runnable interface and defines 6 additional methods that all components must implement in order that the Component Manager can talk with them. Below is an overview of each of these methods.
shutdownComponent(): Called by the Component Manager in order to instruct the component to shutdown. Implementing components should ensure that this method performs any cleanup operations and then terminates.
getGraphicalInterface(): Gives access to the graphical interface defined by this component. The default value is null – meaning no interface provided
setGraphicalInterface(): Sets the graphical interface for this component. Null values are valid and indicate that no interface exists.
setupComponentLinks(): This template method is where the component should access the Component Registry in order to retrieve references to other components that it talks to. The Component Manager calls this method after instantiating all components, but before running their threads.
getLogger(): This method gives access to the logger for the component. Subclasses should call this method in order to report information, warnings, or errors.
Of these methods, a component should only require to call the setLogger(), getLogger() and setGraphicalInterface() methods. The rest of the methods are called by the Component Manager when required.
The AbstractConstructComponent Class
The AbstractConstructComponent class provides the expected implementation for the get/set methods defined within the ConstructComponent interface and provides a default (blank) implementation for the setupComponentLinks() method.
In order to keep the component thread alive, the run() method waits until a notification is triggered by a call to the shutdownComponent() method (which sets a flag to indicate that the component should terminate)
In addition to providing a default implementation for the methods defined in the ConstructComponent interface, the AbstractConstructComponent class defines two template methods for developer use: the onRun() method which is called when the component’s thread is started, and the onShutdown() method which is called when the component is requested to shutdown. For both methods, implementing classes should perform any necessary actions and ensure that the method returns normally.
The Component Manager
The Component Manager is responsible for loading, running, and stopping all the components in any given Construct deployment. The set of components to be started is defined in an external file (construct.properties) which is read by the Component Manager. The startup process is as follows:
Initialise all the components. For each component, this involves making a call to the Component Registry, which is responsible for ensuring that only one copy of a component exists at any given time. If the component does not exist, it is created (using reflection). If the component already exists, a reference to the existing component is returned.
Obtain and display the graphical interface for each component. This calls the getGraphicalInterface() method for each component, and adds the return value (if not null) to a tab in the main GUI.
Allow components to interact. Next, the setupComponentLinks() method of each component is called in order that they may obtain references to the other components that they interact with from the Component Registry. Should any requested component be unavailable, the initialisation process will be terminated.
Start the components. Finally, the thread for each component is started.
When shutting down a Construct deployment, the Component Manager iterates through all active components, calling their shutdownComponent() method. Once completed, the Component Manager removes the entry from the Component Registry.
The construct.properties File
The construct.properties file in the Construct root directory specifies the components they should be loaded, and the properties that should be passed to components on startup. The construct.properties file also contain properties that can be used to refine logging details. An example file construct.properties.tmp is included with the distribution. You should copy this file to construct.properties before running Construct.
Component entires in the file take one of two forms depending on whether the component requires properties on startup.
Components without properties are specified as follows:
In this post I’ll give a brief overview of the core components in a Construct node. It’ll be of interest to anyone adapting the way Construct works internally and developers of other middleware platforms. This is a good entry point into how we designed and built Construct.
If you want to build your own components I’m about to write some tutorials and blog posts so check back soon (or grab the RSS feed).
The Component Loader
The number and type of components in any given deployment of Construct is customisable. At runtime, each of the components defined in the construct.properties file is instantiated by the Component Loader, and indexed in a Component Registry to allow collaboration between components. The Component Loader is also responsible for the correct shutdown of the components when the software is terminated.
The Discovery Service
All components that interact directly with sensors or applications provide a self-describing manifest that contains information on how to interact with them. These manifests are maintained by the Discovery Service. This service provides the first point of contact for anyone that wishes to contact a Construct node. The Discovery Service advertises itself using Bonjour.
The Data Manager
The Data Manager is responsible for storing and maintaining an RDF model of all received data. It also contains a meta-data model which holds information describing the RDF statements stored in the first model. Currently this meta-model only stores information describing when the statement expires. The current implementation of this data store manager uses Jena to manage both the models.
The DataPort
The DataPort provides the gateway for adding new data to the Construct infrastructure. It takes the simple form of a TCP socket listening on a port for RDF expressed as a String in N-TRIPLE format. Multiple triples may be concatenated, and an expiry time may be appended to the end of the String if desired.
If you want to drop data directly into the DataPort go read the tutorial on our protocol format first.
The HttpPort
The HttpPort is a convenient way to query construct over HTTP. You can do this programmatically or via a web browser. If you ask the component loader to start the HTTP port (see the construct.properties file) and have Construct running try looking at http://localhost:8888/ to find a simple SPARQL query interface. This is a great way to interact with Construct from web applications.
The Query Service
The Query Service component is the interface that the framework provides to applications wishing to query the data store. It accepts a String from an application in the form of a SPARQL query and returns a String of results. The application can then decompose the returned String value into the variables that it queried for. Using the in-built Jena reasoner, the Query Service component first infers implicit triples from the data store contents (represented as RDF triples) so that it returns inferred results as well as explicit data to the application.
The Gossipping Service
The implementation of a gossiping algorithm takes the form of a protocol stack with three layers: the data-layer, which exchanges RDF with the data-manager; the gossiping layer, which drives the process; and the network layer which encapsulates the actual sending and receiving of raw packets of bytes.
The gossiping subsystem can be further decomposed into three components: the core gossiping protocol, the message buffer, and the peer membership manager. The core gossiping protocol is stateless. Gossiping is initiated periodically at each peer by sending their message buffer summary to a randomly selected peer from their contact list. When messages arrive they are identified and processed by type. The gossiping layer handles three types of messages: data messages, message buffer summaries, and individual message requests. Data messages are stored in the message buffer for future gossiping then passed up the protocol stack to the data-manager. Message buffer summaries contain a list of the message IDs present in the remote node’s buffer. On comparison with the local message buffer requests for messages that are not held locally are generated. These requests are fulfilled by return.
![[del.icio.us]](http://www.construct-infrastructure.org/wp-content/plugins/bookmarkify/delicious.png)
![[Digg]](http://www.construct-infrastructure.org/wp-content/plugins/bookmarkify/digg.png)
![[Google]](http://www.construct-infrastructure.org/wp-content/plugins/bookmarkify/google.png)
![[StumbleUpon]](http://www.construct-infrastructure.org/wp-content/plugins/bookmarkify/stumbleupon.png)
![[Windows Live]](http://www.construct-infrastructure.org/wp-content/plugins/bookmarkify/windowslive.png)
![[Yahoo!]](http://www.construct-infrastructure.org/wp-content/plugins/bookmarkify/yahoo.png)
![[Email]](http://www.construct-infrastructure.org/wp-content/plugins/bookmarkify/email.png)
