How the Importer works

The Importer is a simple mechanism that performs four basic steps to import from a source to destination within four corresponding components:

1. A Service which obtains a source.
2. An Importer that gets the data from the source.
3. An Updater that updates one or more destinations with the data.
4. A Handler that takes whatever action is required after the import has completed. Typical actions include handling individual update errors and informing the source provider of the status of the import.

Because there are a variety of ways to perform each of the four steps, the Importer is designed so each of the four corresponding components can be extended or replaced. The points where components can be extended are colored red in the following:

 

It isn't necessary to extend any of the components other than TableDataUpdater, however, because the Importer comes with the following stock implementations to cover common uses:

1. Both DirectoryWatch and Queue Services for monitoring directories and queues respectively for source data.
2. An Importer that extracts data from delimited text sources.
3. A Handler that saves records that were rejected by Updaters and provides a user interface for editing and re-submitting them.

The only component that needs to be extended is TableDataUpdater, and it only requires one override, getMatchingEntityWhereClause(), to help it identify unique destination records. ValidateValues() can also be overridden if necessary to customize the logic used to validate records before updating the destination. Otherwise, TableDataUpdater already implements basic validation and all the CRUD functionality required to update virtually any relational database destination 'out of the box'.

Extensible yet simple, the Importer provides both a foundation for new functionality as well as basic functionality required to import from text files to destination tables almost entirely through configuration.

Introducing "VeroMark", Object-RDF Mapping (ORM)

"VeroMark" semantic client user interface builder adds to WPF's already rich, XAML-based presentation system powerful UI control extensions as well as the ability to manage data retrieval, data bind UI elements to it, and navigate between pages entirely through configuration, thus enabling the construction of complete WPF user interfaces without touching compiled code. Instead of producing mock-ups and explaining what they imagine the business needs, picture your data and business analysts marking up their ideas directly as functioning UI screens, allowing your software developers to focus on more technical aspects.

As a part of eliminating the need for compiled code, VeroMark supports data binding to an RDF triples-based client model through an object-RDF mapping (ORM) mechanism, which decouples presentation from an underlying "VeroServe Data" data model such that the application's usability isn't limited by it. This article introduces "VeroMark" object-RDF mapping and illustrates key concepts using a simple example.

Building Blocks
WPF's built-in data binding technology is at the heart of how VeroMark connects the user interface elements to the client data and keep them in sync with one another. The client data model available in a page is "described" in XAML markup (this is the ORM) and made available for data binding via one or more EntityProviders. A user interface element is then data-bound to one of these EntityProviders (via an XAML Binding expression "Source"), which at run-time provides in-memory EntityValue object instance(s) that provide access to the actual data to be interacted with on the screen.


Figure 1 Logical relationships between data binding components

Conceptually, think of a description as a Type definition that includes information about how to access property values from within the client data, and an EntityValue as an instance of that Type that provides access to the described properties from a specific, run-time data context (e.g. some specific Id chosen on a search results page).

There is also an EntityValue derivative called Entity, which adds the ability to contain other EntityValue instances (think Composite pattern) and makes them accessible via its property indexer and their description Name. The containment hierarchy of Entity/EntityValue(s) returned by an EntityProvider is determined by the containment hierarchy of its associated description.

Binding Example
Here we show a small example based on the example Ontology. First, we create the description that defines our "BlogEntity". Each property description below defines the property name, the value type (Literal or Identifier), and a Sparql graph pattern for obtaining the unit of client data using a specific data context:

<Controller:EntityDescription x:Key="BlogEntityDescription">
    <Controller:EntityDescription.PropertyDescriptions>
        <Controller:EntityValueDescription Name="Type" ValueType="Utilities:Identifier">
           <Controller:EntityValueDescription.DataAccessor>
               <Controller:Template DefaultPattern="?BlogEntity rdf:type ?Type"/>
           </Controller:EntityValueDescription.DataAccessor>
        </Controller:EntityValueDescription>
        <Controller:EntityValueDescription Name="ReferenceName" ValueType="Utilities:Literal">
           <Controller:EntityValueDescription.DataAccessor>
               <Controller:Template DefaultPattern="?BlogEntity upo:hasReferenceName ?ReferenceName"/>
           </Controller:EntityValueDescription.DataAccessor>
        </Controller:EntityValueDescription>
        <Controller:EntityValueDescription Name="Location" ValueType="Utilities:Identifier">
           <Controller:EntityValueDescription.DataAccessor>
               <Controller:Template DefaultPattern="?BlogEntity upo:hasLocation ?Location"/>
           </Controller:EntityValueDescription.DataAccessor>
        </Controller:EntityValueDescription>
    </Controller:EntityDescription.PropertyDescriptions>
</Controller:EntityDescription>

Next, we create an EntityProvider instance that references the description (via the "DescriptionSource" property):

<Dsp:EntityProvider x:Key="BlogEntityProvider" PageModelManager="{StaticResource ModelManager}" RootVariable="BlogEntity">
    <Controller:DescriptionGenerator DescriptionSource="{StaticResource BlogEntityDescription}" IsSharedDescription="True">
        <Controller:Setter PropertyName="Name" Value="BlogEntity"/>
    </Controller:DescriptionGenerator>
</Dsp:EntityProvider>

And finally, we data-bind the Text property of a UI TextBox to the "ReferenceName" property we defined as part of the BlogEntity description:

Text="{Binding Source={StaticResource BlogEntityProvider}, Path=[ReferenceName].StringValue, Mode=OneWay}"

A WPF Binding Path starts from a particular object instance (that provided by the Binding Source) and follows the path of property relationships to the resulting property value. Properties preceded by the "." syntax refer to compile time properties, while those encased in square brackets ("[]") refer to property indexer with a signature Type-compatible with the value between the brackets.

We will cover EntityValue properties in more detail in a future article, but for now note that "StringValue" is a compile time property that gives us to access to the string form of the "ReferenceName" Literal in the client data.


Figure 2 Working Example showing "BlogEntity" properties bound to UI elements

In future articles we will elaborate on various aspects of VeroMark data binding including:

• View Model properties available for binding (via the XAML Binding expression "Path")
  
• Data context containment-inheritance
• Templates
  
• Binding converters
  
• Other data source "providers" (e.g. FindProvider)
  

Tabular Data from RDF

RDF data usually needs to be transformed into a tabular form for export to traditional relational database systems. While on the surface they may look quite different, RDF data and tabular data are just different ways of organizing the same thing.  Understanding a little about RDF, how it compares to tables, and how to query it is all that's needed to begin experimenting with generating tables from RDF data, which you can do right here from this posting.

RDF
An RDF data set is a collection of statements of interest about something. Each statement consists of three things, a subject, a predicate, and an object, and are collectively known as a triple. Triples are just like simple natural language statements. In the natural language statement "Jane Smith's haircolor's blonde.", "Jane Smith" is the subject, "haircolor" is the predicate, and "blonde" is the object. Together they make an assertion of fact about the world.

RDF Compared to Tables
In a table, like those within relational databases, the subject, predicate, and object are modeled as a row, column, and intersecting cell, respectively. For example, if we also made the statement that "John Doe's haircolor's brown" we could represent these statements in a simple table as:

person       haircolor   
---------    ---------
Jane Smith   blonde   
John Doe     brown  

Queries
A SPARQL SELECT query can be used to transform RDF data into a table. For example, a 'Persons' table can be obtained from some example RDF data by executing the following SPARQL SELECT query:

prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#>
prefix upo:  <http://mydomain.org/myupperontology/myversion/>
prefix o:  <http://mydomain.org/myontology/myversion/>

SELECT ?name ?haircolor ?country ?region
FROM <http://www.compass-point.net/data/ontology.ttl>
WHERE
{
    ?id upo:hasReferenceName ?name . 
    ?id rdf:type o:Person . 
    ?id upo:hasLocation ?loc . 
    ?loc rdf:type upo:Country . 
    ?loc upo:hasReferenceName ?country . 
    ?id o:hasHairColor ?hc . 
    ?hc upo:hasReferenceName ?haircolor . 
    ?loc o:countryIn ?reg . 
    ?reg upo:hasReferenceName ?region .
}

When executed against the RDF data, this query results in the following 'Persons' table:

--------------------------------------------------------------
| name         | haircolor | country | region                |
==============================================================
| "Jane Smith" | "Blonde"  | "China" | "Asia Pacific Region" |
| "John Doe"   | "Brown"   | "UK"    | "UK / Europe Region"  |
--------------------------------------------------------------

In the previous query, note the following:

• The variables declaration, the line  "SELECT ?name ?haircolor ?country ?region" defines the columns of the returned table as 'name', 'haircolor', 'country', and 'region'. 
• The portion after the word WHERE specifies a set of triple patterns that a SPARQL processer must find within the RDF data for there to be a variable match. For example, when taken by themselves the first two lines are two triple patterns that a SPARQL processor would use to find all subjects for which both the following statements have been made

1. A statement asserting it hasReferenceName
2. A statement asserting it is of type Person.

  In effect, a SPARQL SELECT query transforms RDF data into a table.


Kicking the Tires
You can try this out for yourself. Select and copy the following SPARQL SELECT query, paste it into this SPARQL Processor, choose "text output", and then click the "get results" button.

prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#>
prefix upo:  <http://mydomain.org/myupperontology/myversion/>
prefix o:  <http://mydomain.org/myontology/myversion/>

SELECT ?name ?type ?country
FROM <http://www.compass-point.net/data/ontology.ttl>
WHERE
{
    ?id upo:hasReferenceName ?name . 
    ?id rdf:type ?type . 
    ?type rdfs:subClassOf o:Party . 
    ?id upo:hasLocation ?loc . 
    ?loc o:countryIn o:UKR .     
    ?loc upo:hasReferenceName ?country
}

This query results in a "Parties" table populated with only those parties in the UK / Europe Region.

-----------------------------------------------------------------------------------
| name          | type                                                 | country  |
===================================================================================
| "MusPub Inc." | <http://mydomain.org/myontology/myversion/Publisher> | "France" |
| "John Doe"    | <http://mydomain.org/myontology/myversion/Person>    | "UK"     |
-----------------------------------------------------------------------------------

To create other queries, refer to the example RDF data when crafting your query and then modify both the parts after the word SELECT and within the brackets after the word WHERE. Note that the triple patterns within the WHERE brackets must be separated by a space-period-space and the FROM line must stay the same to access the example RDF data.



What's next
Subsequent articles will branch from here down both practical and conceptual paths, covering

• How to export from "VeroServe Data" to both Reporting and Integration Services, 
• How to make data bindings to RDF data in a "VeroMark" semantic client user interface,
• How "VeroServe Data" supports encapsulation of data, data structure, and ultimately business logic in one place through the use of RDF.

"Vero" what??

“VeroRight” is the code name identifying a suite of technologies that enable rapid construction of complete semantic data solutions. Unlike common solution development platforms that result in data being stuffed into rigid, quickly-aging models accessed by expensive, custom-programmed applications – causing business knowledge to be broken apart and buried within data, code, systems, tools, and people – “VeroRight” is designed from the ground up for:

• Changing data models and surrounding applications without corruption or excessive cost.
• Gathering knowledge in a single place instead of spreading it across data, code, and people.
• Deriving value from data in new, unanticipated ways today and in the future.
• Containing costs by reducing dependency on expensive software engineering.

With "VeroRight", a complete semantic data solution is mostly configured, not programmed, then deployed, enterprise-ready:

The “VeroRight” suite of technologies includes:

• “VeroMark”, the semantic client user interface builder
• “VeroServe”, a set of semantic data services
• The xServe set of enterprise infrastructure services:

• “LogServe” enterprise diagnostics service
• “ProfileServe” enterprise user profile service
• “PermissionServe” enterprise authorization service
• “IdentityServe” enterprise identity management and authentication service
• “ExecutionServe” distributed work execution service

Because data is modeled semantically, VeroRight solutions can deliver accuracy despite change -- which explains at least the first part of the name: Vero is Italian for truth.