I recently completed a high-profile deliverable for a major client. While I feel satisfaction for meeting the explicit goals of the project, I’m even happier that I’ve finally met goals that reflect some of my core motivations for working with information technology. To illustrate what I’ve done – and why – I’m going to use a specific event that happened over three years ago.

At the time I led a team that built and supported a web-based application that served the US-led coalition in Iraq. Like any web-based content management system, users were able to create, edit, and post content. In this case the most common item was a SIGACT – a report of significant activities (typically hostile) directed against the military or the civilian population. The SIGACT was an important piece of semi-structured data used for local situational awareness, analysis of effects-based operations, and as an overall metric of the success of the coalition itself.

One day, my boss came to me and asked how hard it would be to add a check box to the SIGACT report form that would indicate if an attack was "sectarian-related" – a reflection of the rising trend of attacks between rival religious groups throughout Iraq. I responded that technically it wasn’t a problem – a little HTML, some application logic, and a new column in a database table. Too easy, I thought. I was wrong.

The MVC changes for the report form itself were obvious and took about four hours to complete (we were using a web scripting language and relational database to power the application). What I didn’t immediately realize were the dependencies – technically explicit and socially expected – for that change. I spent the next few weeks discovering new management reports that needed modifying, search logic that had to be extended, even RSS feeds and web services that had to be altered.

I’ve always believed that users should have the ability to easily evolve knowledge models – and even before this incident I had explored collaborative technology and techniques for allowing users to add new properties and entity types – from simple content tags to OWL class and property editors. But I kept hitting my head against the same wall: any new form field (let alone a tag, class, or property) was almost useless unless the functional logic of the enterprise was flexible enough to take advantage of it. Invariably, as illustrated by my experience with the check box, that meant even simple changes had unforeseen technical impacts that led to a series of unplanned code re-writes.

But there was another issue with the check box – the semantics of the label itself. How did the coalition define "sectarian"? And what where the characteristics of an event that met the threshold of the "sectarian-related" label? After a few sessions with key leaders, we were able to arrive at a high-level definition of which groups were considered "sects". But deriving and agreeing upon the specific criteria that determined if a specific event was "sectarian-related" was almost impossible because of the differences in world view between those collecting the information and seemingly endless variety of consuming communities. Adding to the difficulty is that the environment in Iraq changed with a frequency that challenged our ability to collaboratively reach a consensus of meaning – let alone represent that meaning in our technology.

This is a common and pernicious problem: unless the human model (both structurally and logically) can be expressed, validated (qualitatively and quantitatively), and made functional within a technical infrastructure in a timely fashion, then the "impedance mismatch" between reality (as experienced by a user community) and the limitations of technology leads to inefficiency and frustration. In our specific case, this problem was exacerbated by the difficulty in deciding which roles were authorized to make such an assertion – and at which step in our overall reporting process that assertion should be made.

To resolve this issue, we explored two ideas – each at the extreme ends of the processing chain of a SIGACT. The first was to delegate the task of assessment to the soldier, typically in a field environment, creating the initial report. This individual would have the most comprehensive knowledge of the event at hand, and this seemed a logical way to capture the best data fidelity while also achieving an optimal distribution of the work load. But because the criteria for "sectarian-related" involved fuzzy qualitative judgments (that we didn’t have a way of formally expressing in our systems) we faced the real potential of variable data quality.

Then we considered using a specialist – a staff officer working exclusively on Iraq-wide sectarian issues (and therefore familiar with the qualitative aspects). While the consistency demands of the data could theoretically be met, we couldn’t be sure that the report would contain enough data necessary to make the judgment, not to mention the workload and impact on timely distribution that comes from concentrating a core function at the tail end of a processing chain. We eventually worked out a series of compromises that did provide tangible benefits – but only after we applied some quantitative analysis against historical data, adjusted and disseminated our qualitative definitions of our criteria, and added new fields to the initial report form itself (to better capture the details necessary to make that final judgment).

The result of this experience has shaped my career ever since. I’ve been energized (my friends call it "obsessed") with the pursuit of strategies that would allow users to both rapidly add knowledge model artifacts (like properties and classes) but to also express the criteria of those new elements so that they could be represented, validated, and utilized – by systems and humans. Furthermore, I started exploring how the context of user-system interaction could be functionally modeled in such a way as to reduce the downstream effects of knowledge model changes.

Here are three technologies I’ve recently successfully implemented as a result of these strategies:

Dynamic User Ontologies

Jim Hendler is often credited with the statement that "a little semantics goes a long way". For my users, simply being able to add and integrate new concepts and properties into their data model is a significant enabler. While this is a far cry from the utopia of a comprehensive and functional enterprise model, I’ve found that it suits the iterative process of technology and community co-development. New data models are frequently the precursor to new tasks models which lead to new work flow models, and so on.

Dynamic user ontologies also positively affect the architectural and development environment by forcing us to consider contextual interaction and presentation. If a user creates a date property, for example, the system should make the changes necessary so that any date-sensitive display elements (like a time graph) can identify, consume, and display that element. Similarly, if a user adds a new class and categorizes that class as dealing with a specific work task, our metadata management infrastructure can help us rapidly identify the effects of that change in non-obvious areas.

Now, I know what you’re thinking, and you’re right: I’ve made no mention of OWL restrictions or inference engines – so how can I claim that my users are building ontologies? That leads me to the second implementation:

Semantic Business Rules

While academics have published extensively about some of the programmatic limitations of description logics (e.g., OWL is not "Turing Complete"), it is only recently that industry has started exploring similar solutions. For the needs of my users, existing standards-based (e.g., SWRL) approaches weren’t sufficient for both technical and usability reasons. Instead, I worked backwards – extending a standard rule engine (JSR-94 compliant) to work with ontologies.

The immediate benefit of this approach is that my users can rapidly add classification and business rules based on the models they themselves have extended – and typically as part of the same step. As an example, users can extend the standard domain concept of a "Traveler" to create a customized sub-class of "Traveler Of Interest", and then write rules that use enterprise data ontology elements in the criteria to perform the classification. In the same rule, users can also choose to send an alert – or perform any function integrated into the rule engine – a powerful feature not typically found in ontology reasoning systems.

But the real power of this approach extends beyond the functional effect of the rule. Because a rules (and its representation) is simple, readable, and shareable, my community can now collaborate at a qualitative level like never before. Definitions are functional and accessible. Reasoning is explicit – and flexible. And the integration of logic and function allows for quantitative analysis of qualitative structures. Last – but not least – segmenting my reasoning allows me to avoid some of the performance overhead (and nasty side effects) of having to perform inference over all of my data every time my data changes – a limitation of most current inference engine implementations.

And that’s just for my end-users – imagine this power applied to enterprise middleware, business workflow, and other SOA infrastructure elements. This leads me to my final implementation:

Semantic Context Management Framework

I’m fond of using the Socratic Method in exploring the needs of my customers. During one particular memorable session, my current client finally became exasperated with me and said, "All I want to do is give the right user the right data in the right way at the right time and the right place to achieve the right effect". Although a big ungainly, I think his statement identifies both the elements – and the point – of context.

Much of my recent work has focused on using semantics for enterprise context management – performing service and data alignment for SOA middleware according to enterprise models. Our first implementation uses ontology models as the basis of federated search, with the semantic rule engine providing invaluable assistance in performing data translation and representation functions where ontology modeling and mapping alone isn’t sufficient. The bottom line is that we no longer have to create and tweak XSLTs as new services and data sources are brought into our environment, nor alter our queries as new data sources are mapped into our ontology.

My next step with enterprise context management is to apply semantic technology to the business process definition and execution environment as my client applies SCA (Service Composition Architecture) principles. This is a different challenge than the relatively non-dynamic nature of service and data models. It means we have to explore how business principles intersect with user roles, organizational metrics, and enterprise resources in creating context-aware dynamic business processes.

But that’s not all – the most exciting challenge for me, personally, is exploring how semantic technology can support user interaction. I’ve already seen interesting results by modeling display portlets and widgets in ontologies and then using display-time inferencing to present users with context-appropriate data (such as display labels). Allowing users to further define their own personal and customized processes (like DERI’s Semantic Pipes), promises to bring me even closer to my original goal of giving power to the user.

On a final note, I should mention that although I’m personally content at the moment, there are still questions I’m only beginning to address. How do I overcome the inability of current OWL standards to deal with probability? How can I better leverage text extraction to assist in automating both modeling and logic? How can I deal with inevitable social friction when different communities have competing views of the world? While there remains plenty of work, my real challenge is one of abundance: the growth of interest, standards, and solutions involving semantic technology have exploded recently. Assessing and incorporating these developments promises to be a pleasurable work, indeed.

–

"The wildest colts make the best horses when they come to be properly trained." – Plutarch

SemTechBiz is Less Than 2 Weeks Away

The Semantic Tech & Business Conference (SemTechBiz) is coming to San Francisco on June 3-7! Join us for case studies, innovative panels, tutorials, and keynotes that will provide you with practical advice, hands-on guidance, and breakthrough approaches to solving business problems with semantic technology. Passes go up $200 at the door. Sign up now and save !