David Ramel of ADTmag writes, “What the heck are you doing reading this article? You should be boning up on your Big Data developer skills. Well, if you like making the big bucks, that is. Yes, the Big Data skills shortage shows no signs of shrinking even after several years of hype. That means great opportunities for data developers. ‘By 2018, the United States alone could face a shortage of 140,000 to 190,000 people with deep analytical skills as well as 1.5 million managers and analysts with the know-how to use the analysis of big data to make effective decisions,’ stated a recent McKinsey Global Institute report. And where there’s hype, there’s money. ‘Salaries reported by those who regularly use Hadoop, NoSQL, and Mongo DB are all north of $100,000,’ claimed a recent report from the 2013-2012 Dice Salary Survey.” Read more

Enterprise NoSQL database platform provider MarkLogic has come into some cash: a $25 million round of growth capital from investors including Sequoia Capital, Tenaya Capital, Northgate Capital, CEO Gary Bloom and other corporate executives. Yesterday, at the company’s MarkLogic World 2013 conference, Bloom also prepared the audience to hear more today from company executives about MarkLogic’s next steps in semantics for its MarkLogic Server technology that ingests, manages and searches structured, semi-structured, and unstructured data.

“The way to think about this is that when we look at semantics, we didn’t … say we just want to check a box on semantics,” Bloom said, by working with partners on some low-hanging fruit – although it will be collaborating with them on various semantic enrichment capabilities. “We think semantics is critical technology, and more interesting I believe is that it is a critical technology that is both a search technology as well as a database technology.” Others in the marketplace will focus on changing their search engines to do semantics, but optimum results won’t come if all that’s being done is layering in semantics at the search level, he said.

Read more

Waqar Hasan, who in a past life was vp for data platforms at Yahoo, hasn’t lost his fascination with the power a business can gain when it knows what to do with its data – make that its Big Data. Now CEO of InsightsOne, Hasan and his company are focused on making predictive analytics accessible to the general B2C marketing organization via the cloud.

Among its early customers is online review site Angie’s List, which in mid-January selected the cloud-based predictive analytics solution to deliver a 1-to-1 consumer marketing experience to its members.

“We’re targeting B2C marketers to increase the relevance and profitability of their consumer interactions, by applying micro-segmentation on Big Data to extract all sorts of signals from the data and turn it to a more powerful predictor for the future – who will buy what and who is likely to churn,” Hasan says.

Read more

Rutrell Yasin recently wrote that a convergence of growing technologies, including semantic web technologies, will spur exciting Big Data projects in 2013. Yasin writes, “The maturity and convergence of four technologies will help government decision-makers derive more value from their big data projects in 2013, predicts Chris Biow, public sector CTO at MarkLogic, a developer of databases for big data applications. ‘Cloud computing, Hadoop and NoSQL databases are the three game-changing technologies that are being applied to big data,’ Biow said. ‘I think this is the year that government agencies get their hands around what each of them can and cannot do.’ Semantic technology is the fourth discipline to add to the equation, which can be used to extract facts from structured and unstructured data as well as handle relationships between data in a more flexible way than traditional relational databases, Biow said during an interview with GCN.” Read more

Courtesy: Flickr/zoetnet

As we close out 2012, we’ve asked some semantic tech experts to give us their take on the year that was. Was Big Data a boon for the semantic web, or is the opportunity to capitalize on the connection still pending? Is structured data on the web not just the future but the present? What sector is taking a strong lead in the semantic web space?

We begin with Part 1, with our experts listed in alphabetical order:

John Breslin, lecturer at NUI Galway, researcher and unit leader at DERI, creator of SIOC, and co-founder of Technology Voice and StreamGlider:
I think the schema.org initiative really gaining community support and a broader range of terms has been fantastic. It’s been great to see an easily understandable set of terms for describing the objects in web pages, but also leveraging the experience of work like GoodRelations rather than ignoring what has gone before. It’s also been encouraging to see the growth of Drupal 7 (which produces RDFa data) in the government sector: Estimates are that 24 percent of .gov CMS sites are now powered by Drupal.

Martin Böhringer, CEO & Co-Founder Hojoki:

For us it was very important to see Jena, our Semantic Web framework, becoming an Apache top-level project in April 2012. We see a lot of development pace in this project recently and see a chance to build an open source Semantic Web foundation which can handle cutting-edge requirements.

Still disappointing is the missing link between Semantic Web and the “cool” technologies and buzzwords. From what we see Semantic Web gives answers to some of the industry’s most challenging problems, but it still doesn’t seem to really find its place in relation to the cloud or big data (Hadoop).

Christine Connors, Chief Ontologist, Knowledgent:

One trend that I have seen is increased interest in the broader spectrum of semantic technologies in the enterprise. Graph stores, NoSQL, schema-less and more flexible systems, ontologies (& ontologists!) and integration with legacy systems. I believe the Big Data movement has had a positive impact on this field. We are hearing more and more about “Big Data Analytics” from our clients, partners and friends. The analytical power brought to bear by the semantic technology stack is sparking curiosity – what is it really? How can these models help me mitigate risk, more accurately predict outcomes, identify hidden intellectual assets, and streamline business processes? Real questions, tough questions: fun challenges!

Read more

Hadoop is on almost every enterprise’s radar – even if they’re not yet actively engaged with the platform and its advantages for Big Data efforts. Analyst firm IDC earlier this year said the market for software related to the Hadoop and MapReduce programming frameworks for large-scale data analysis will have a compound annual growth rate of more than sixty percent between 2011 and 2016, rising from $77 million to more than $812 million.

Yet, challenges remain to leveraging all the possibilities of Hadoop, an Apache Software Foundation open source project, especially as it relates to empowering the data scientist. Hadoop is composed of two sub-projects: HDFS, a distributed file system built on a cluster of commodity hardware so that data stored in any node can be shared across all the servers, and the MapReduce framework for processing the data stored in those files.

Semantic technology can help solve many of the  challenges, Michael A. Lang Jr., VP, Director of Ontology Engineering Services at Revelytix, Inc., told an audience gathered at the Semantic Technology & Business Conference in New York City yesterday.

Read more

Last week The Semantic Web Blog covered the launch of the SindiceTech Assisted SPARQL Editor as an open source project, noting that SparQLed also is part of SindiceTech’s commercial suite for large enterprises building private linked data clouds. This week, we’ll dive a little deeper into SindiceTech and its progress since the founders of the Sindice web of data search engine turned their attention to focusing on the commercial application of its technology as a real-time semantic warehousing infrastructure, which leverages cloud computing for integrating and normalizing the massive amounts of data the enterprise must deal with.

As SindiceTech founder and CEO Giovanni Tummarello explains, companies actually approached his team to help them make a reality of their visions to use RDF and SPARQL, as the best knowledge representation and querying technologies available, by providing the missing scalability and stability. Sindice.com was evidence that the technology the team had developed could answer these enterprises’ needs; currently there are about 700 million semantically marked-up web pages indexed in the Sindice.com search engine, with a live updated index of some 80 billion triples daily. Its database is over 5 terabytes.

Read more

Clark & Parsia’s Stardog lightweight RDF database is moving into release candidate 1.0 mode just in time for next week’s upcoming Semantic Technology & Business Conference in San Francisco next week. The product’s been stable and useable for awhile now, but a 1.0 nomenclature still carries weight with a good number of IT buyers.

The focus for the product, says cofounder and managing principal Kendall Clark, is to be optimized for what he says is the fat part of the market – and that’s not the part that is dealing with a trillion RDF triples. “Most people and organizations don’t need to scale to trillions of anything,” though scaling up, and up, and up, is where most of Clark & Parsia’s competitors have focused their attention, he says. “We’ve seen a significant percentage of what people are doing with semantic technology and most applications are not at a billion triples today.” Take as an example Clark & Parsia’s customer, NASA, which built an expertise location system based on semantic technology that today is still not more than 20 million triples. “You might say that’s a little toy but not if you are at NASA and need defined experts, it is a real, valuable thing and we see this all the time,” he says.

Read more

The Common Crawl Foundation is on the verge of adding to its Amazon Web Services (AWS) Public Data Set of openly and freely accessible web crawl data. It was back in January that Common Crawl announced the debut of its corpus on AWS (see our story here). Now, a billion new web sites are in the bucket, according to Common Crawl director Lisa Green, adding to the 5 billion web pages already there.

“When are you going to have new data is one of most frequent questions we get,” she says. The answer is that processing is underway now, and she hopes they’ll be ready to go this week.

Read more

Semantic Web Community: I’m disappointed in us!  Or at least in our group marketing prowess.  We have been failing to capitalize on two major trends that everyone has been talking about and that are directly addressable by Semantic Web technologies!  For shame.

I’m talking of course about Big Data and NoSQL.  Given that I’ve already given my take on how Semantic Web technology can help with the Big Data problem on SemanticWeb.com, this time around I’ll tackle NoSQL and the Semantic Web.

After all, we gave up SQL more than a decade ago.  We should be part of the discussion.  Heck, even the XQuery guys got in on the action early!

Check out this Google Trends diagram.

Semantic Web vs. NoSQL on Google Trends

NoSQL came out of nowhere in 2009, and now dominates much of the database conversation on the web.  Document stores like MongoDB and CouchDB, distributed, key-value stores such as Riak and Cassandra, and other weird stores like Hadoop-as-database (never understood that usage myself) now dominate the conversation as the alternative to traditional, SQL databases.

Read more