Business Intelligence Blog

Creating Better Customer Experiences and Increasing Big-Ticket Transactions using Aggregated Web Data

Jeff Hoffman — Wed, 17 Feb 2010 15:07:26 +0000

While value is the key word in this economy, consumers are thinking about more than functionality when buying big-ticket items such as electronics or kitchen appliances. They’re also thinking about aesthetics, energy efficiency, brand, delivery, price and reviews (consumer and analyst).

Consumers are most likely to shop across a variety of channels for durable, big-ticket items such as major appliances¹. By shopping in-store the consumer sees the item and gains valuable, time-saving knowledge. But let’s face it, consumers are surfing the web, checking online marketplaces, retail sites, price comparison sites and consumer reports to help them make an educated decision before they hand over their credit card. A major advantage of shopping with Search, Aggregation and Syndication (SAS), online marketplaces, retailers, or other content aggregators is the ability to quickly (price) comparison shop and read consumer and analyst reviews.

A Better, More Complete Experience

How can an automated web monitoring, collection and delivery process create a better customer experience and increase transactions for SAS, online marketplaces, retailers, or other content aggregators? Automating web monitoring, collection and delivery processes using intelligent Software Agents:

Increases marketplace “stickiness”, positioning a company as “the” source of information for a product or service.

Increases the quality and completeness of information that you provide by aggregating data from different kinds of sources – aggregated sentiment, pricing, availability and more.

Offers consumers personalized content they’re willing to sign-up for.

Benefits More Than Just Your Customers
Automating web collection and monitoring processes not only benefits your customer and their buying experience, it also benefits your business. Think about it, automation will increase employee productivity as resources can be shifted to more important business functions (versus manual data collection or programming).

If you’re a Search, Aggregation and Syndication, online marketplace, retailer, or content aggregator thinking about ways to improve your current offering, reduce production costs, or create new products, you owe it to yourself to learn more about the Connotate solution.

1. Retail Multichannel IT Road Map, Forrester, July 23, 2008, George Lawrie

The Apple iPad – A New Canvas for Publishers

Gina — Mon, 08 Feb 2010 13:56:51 +0000

As Apple unveiled the iPad last week I’m sure the Apple-loyalists lined up to reserve their very own precious iPad. But, what they get in return for the price is still to be seen, or better yet, realized.

The iPad is anticipated to provide a simple and convenient experience for consumers seeking high-value digital content. Already, Apple has designed an app for delivering books via the iPad (called iBookstore), but will they be able to compete with the Kindle or Netbook that’s already out there? If you own one of those other e-book readers, are you really going to put out another $500 for yet another cool tech device that serves the same function? I’m not so sure.

Publishers, Get Your Paintbrush, the Canvas is in Front of You

I believe the real opportunity may lie with the news and magazine publishers. The New York Times presentation of their news on the iPad looked pretty impressive. It had a nice layout, nice flow. Now that Apple – who has a pretty loyal following (more than 20% of Americans own at least one Apple product) — has created a canvas, it’s time for innovative publishers to start building their apps. A much larger canvas than was available via the iPhone and iPod, now might be the time for publishers to consider an alternate business model that may help and even speed the transition of consumers’ willingness to pay for digital content.

Many Limitations… Exercise Some Caution

I want to be fair here so let’s consider a few limitations of the iPad. Publishers are going to need to provide non-Flash versions of their content that consumers are willing to pay for.

There’s no browser choice. So make sure you build your iPad app with Safari in mind!

The iPad is truly a managed device so anything and everything that appears on it will be a direct choice/preference of the owner. Your iPad app will need to be something special. As mentioned in the Fox Business interview with Outsell’s Ned May the other day, the iPad is like a coffee table book. To me, it has the potential to be a coffee table book on steroids.

No More Magazine and Newspaper Clutter… Just One Device (Well, Maybe)

As I look at the pile of magazines and scattered newspaper sections sitting on my coffee table, the iPad may offer an attractive alternative. Rather than paying for my current subscriptions, I can just read these magazines and newspaper articles on my iPad. No more recycle bins overflowing from the page-worn periodicals I’ve read over the past few weeks!

But wait, I haven’t placed an order for my iPad yet… My newspaper and magazine publishers haven’t built their apps yet… Guess I’ll just have to deal with the paper clutter a little while longer.

Ready to design your iPad app? Contact Connotate to explore the possibilities.

Bringing “On Demand” Another Step Forward for Publishing

Gina — Mon, 01 Feb 2010 15:36:35 +0000

The recent news about Xerox partnering with On Demand Books so that self-published pieces and public titles that are no longer available in the stores can be printed “on demand” via the Espresso Book Machine kiosk is very exciting. We’ve certainly come a step forward in our thinking (and technologies) if you can stroll over to a kiosk, search for the title of a book and push the print button to secure your own bound copy!

But wait, why aren’t we bringing this concept even another step forward? If we have the data, why aren’t we letting the consumer walk up to the kiosk and select specific chapters to combine and print. You might think that’s a silly idea but in the world of textbooks and offering a unique curriculum, this might be the perfect way to offer a well-rounded course of study.

On Demand Should Include Personalization

Imagine, you walk up to the kiosk, refer to your syllabus and select chapter 3 from one textbook, chapters 6 and 7 from another textbook and so on. Once you’ve selected all the chapters you need to fulfill the reading and study outline for your course, you hit the print button and voila, a custom textbook is printed for you. Perhaps the syllabus also includes supplementary materials like industry reports and/or expert articles that can be selected and printed as an appendix. I’m pretty sure that McGraw Hill’s Higher Education division offers something like this already but, the textbooks are pre-designed by the professor so it’s already “assembled” per se once the student gets to the kiosk or online order form. There doesn’t seem to be any personalization element for the student.

Is it a Leap of Faith or Just a Matter of Rolling up Our Sleeves and Getting it Done?

Perhaps publishers are toying with this idea today but just haven’t taken the full leap of faith. If you have the materials already in a digital format, have the content in a repository, or have access to the materials via the web, this can indeed be done today.

And when the student pushes the order button, why do we have to print the book? Let’s push it to their Netbook, their Kindle or the soon to be available, iPad.

Perhaps this is something you’d like to build? To offer to universities? Let us help you explore the possibility.

So You Want to Be Considered “The Source”…

Gina — Mon, 25 Jan 2010 13:22:55 +0000

You have an idea… You want to be the most complete, most accurate source of healthcare information available.

You have a plan… You have identified the healthcare news sources that you merit most credible. You have identified the numerous healthcare organization sites that list professional names, contact info and specialties. You have identified the regulatory organizations that govern the profession. And you have other topics and data points relevant to the industry that you know you want to include in your offering.

You know there is a market for the data…

Now how do you quickly execute on that plan?

Well, you can see if you have the budget to go out and purchase lists from multiple vendors. And, you can hire an arsenal of IT resources to write scripts that can grab the data from the web sources that you identified in your plan (but the bad news is that the scripts will most likely break when the source web site changes). Then you can hire the QA team to assess the quality and massage the information pulled back from the scripts so that you can feed the good data into your final product. As I type this, I realize that this doesn’t sound like quick execution at all. Sounds costly, tedious and not necessarily very reliable, accurate or scalable!

With all that said, there IS good news. There is a solution out there that can help you realize your goals. The Connotate solution is a reliable, accurate and scalable solution that can serve as the automation platform to your web data mining, monitoring and aggregation efforts. An easy-to-use point-and-click interface empowers you to quickly navigate web sites down to the specific data you want to monitor and extract. And the data you bring back can be sent to a data repository or directly to a web page (or to other destinations but these were the most relevant to this example).

Whether you are an established business with a new product idea, a funded entrepreneur, or already have an offering that you’d like to make even more powerful, the Connotate solution can make the information access, monitoring and collection aspects of your project more efficient.

Want to see how easy it is to use? Watch our Agent Types Video and/or Sign-up for a FREE 2-week trial to see how easy it is to start building information assets from the sources you identified in your business plan!

The Future of Local News – The Smaller, Entrepreneurial Media Outlet

Jeff Hoffman — Tue, 20 Oct 2009 13:23:05 +0000

Forbes’ article San Diego News Shoot-Out offers a glimpse of what local news may look like in the not too distant future – entrepreneurial startups and a former monopoly vying for consumers’ attention.

How does a minimally staffed media company deliver enough content to effectively compete when good journalism is a slow, labor-intensive business?

Focusing on the local
Web content extraction & aggregation technologies - Corporate Web sites, social media sites, e-commerce sites, government sites, internal Web sites and other sources of digital media not typically used as editorial resources are undiscovered treasure troves from which publishers can automatically extract and deliver new sources of data and information.

After implementing Connotate’s solution, a unit of Reed Business was able to drastically reduce and re-assign the number of staff that monitors and aggregates content from the Web.

Web content aggregation technologies enable publishers to automate the acquisition of the most important portions of Web pages in a normalized format that’s easily transformed and organized into new ready-to publish editorial assets.

Delivering a Neighborhood Intelligence Portal using Web Data
As an example, a media outlet could use Web extraction technology to automate delivery of online intelligence about their area. What the local technology jobs available are, public school data, current year home sales, foreclosures, local interest rates, coupons and even babysitting services. Below is a “Neighborhood Intel” portal tab Connotate built for the New Brunswick, NJ – 08901 area. To select information on Brooklyn, Boston, Chicago, Miami Beach, Los Angeles, Denver, Dallas, or Detroit from a drop down menu click here (using Internet Explorer 7 or higher).

Publishers need to consider what can be achieved with technologies that enable the exchange of knowledge and information in unprecedented ways.

Whatever strategy media firms use to deliver news, Connotate can help automate the aggregation and delivery processes surrounding digital information workflow efforts. To get started building Agents now sign up for a free 2-week trial of Connotate’s On-Demand Library Beta.

One-Stop Shopping for NJ Governor Candidate Twitter Postings

Gina — Wed, 14 Oct 2009 12:50:05 +0000

Check out Connotate’s NJ Gubernatorial Candidate Twitter Posting application

Barely 3 weeks out from election day, I see the ad campaigns, read the newspaper articles and listen to talk-radio to keep myself up-to-date with the viewpoints surrounding the candidates and election. I want to know what each candidate stands for. I want to know what message they find most compelling to share with the voters. However, if I want to know how a candidate stands on an issue, or what issue is top-of-mind for them, I have to seek out their website (which in many cases is not easy to find) and take a click-stroll through their site to discover the information I’m looking for.

Thankfully, some of the candidates are using Twitter to promote themselves. This makes it easy for people who follow a candidate to keep up-to-date with what’s on a candidate’s mind. However, once again, I have to be checking/following up to 12 people’s (yes, contrary to the Star-Ledger’s Full Coverage of the New Jersey Governor’s Race, there are more than 3 candidates!) Twitter postings if I want to stay up-to-date with the electoral buzz.

Like most of you reading this, my day is pretty busy so I really don’t have much time to check all those individual Twitter postings. I’d much prefer to check the most recent 5 to 10 Tweets of ALL the candidates at any given time. With this in mind, Connotate is sharing a NJ Gubernatorial Candidate Twitter Posting application that will allow you to see the 5 most recent Tweets by all candidates any time you want. Our Agents check the candidate accounts for new postings every half hour (although they could be monitoring the accounts even more frequently). We were successful finding the Twitter accounts for 9 of the 12 candidates (trust me, some were not that easy to find!) and they are all being monitored by Connotate’s intelligent software Agents for new postings.

So, if you want to know what’s on a candidate’s mind, try the one-stop shopping our NJ Gubernatorial Candidate Twitter Posting application offers and take a look at their latest Twitter postings below. For the candidates who only posted one time or opened an account but didn’t post at all, shame on them! For the candidate’s who didn’t create a Twitter account or we couldn’t locate it, feel free to send us the account detail and we’ll be happy to add it to our NJ Gubernatorial Candidate Twitter Posting application.

Want to create your own Agent to monitor the candidate posts or news (or any other kind of web-based data) contact Connotate at 732-296-8844 or try out a free 2-week trial of Connotate’s On-Demand Library Beta

[So as not to show favoritism, Connotate’s NJ Gubernatorial Candidate Twitter Posting application results are in alphabetical order from left to right. We found the accounts for Chris Christie, Jon Corzine, Jason Cullen, Chris Daggett, Kenny Kap, Joshua Leinsdorf, Kostas Petris and Gary Steele. We could not find the Twitter accounts for Alvin Lindsay, David Meiswinkle, Greg Pason or Gary Stein.]

Intelligent Agents Keep a Watchful Eye on Green Initiatives… For You

Gina — Mon, 05 Oct 2009 13:22:23 +0000

Everything seems to be “green” these days… There’s “green investing”, “green energy goals”, “green collar jobs”, “green building certification”, “green funds”, “green companies”, “green cars”… And let’s not forget about that “green” that fuels the economy…

Keeping on top of all the “green” developments can be overwhelming. Every time I read the news, check out the latest hot topics in an industry publication or surf the blogs, I come across some kind of “green” story. Like many, I try to do my part. I’m as “green aware” as I can be. My “green” efforts are reflected in many aspects of my life and I have to admit that I like to keep up on the new programs that promote “green” grants, stimulus packages, and guidelines.

Green Resources are Abundant

I recently went to a seminar focused on green collar jobs and green building initiatives. You can imagine the lengthy reference list of online sources that was given to me (as a printed hand-out if you can believe it) so that I could stay on top of “green” initiatives and announcements. Potentially, there were hundreds of resources for me to monitor but no two sources were identical in navigation or layout or format.

So, I went through the lengthy list of “green” resources given to me at the seminar and identified two sources that reported headlines and news that mattered to me. Unfortunately, the site didn’t have any kind of RSS feed option or subscription option. Now, I don’t know about the rest of you but, I’m pretty busy so there is no way that I’m going to remember to go and check that site every few days for new developments, headlines or initiatives. I decided to train an Agent on these 2 sources and have any new postings sent to me automatically. In a matter of 2 minutes (really, in barely 2 minutes), I trained an Agent to monitor the State and Local Climate and Energy and the National Action Plan for Energy Efficiency pages and send me any new stories that get posted to these pages. In this case, I just want the posting date and linked headline to the full story but, I could have trained my Agent to send me the entire story as well.

Anybody Can Stay “Green Aware”

I don’t have a programming background… I don’t know how to write code…. But, I do know how to point-and-click so I had the skill-set required to monitor the EPA site each day for new “green” news that focuses on energy efficiency. Now I just sit back and wait for my Agents to shoot me an email with any new headlines as they hit the EPA web pages.

Now this is a pretty simple example of what an Agent can do in the “green” arena. Imagine that there are hundreds (thousands) of sources out there like the two pages I’m monitoring. Imagine you’re interested in “green” investments or new “green” job postings – you can have Agents monitoring all the sources for you and keep you “green” aware.

Create Your Own Green Agents

Want to train your own Agents or subscribe to the ones that I’ve created? Try Connotate’s On Demand Community Trial free for 2 weeks. See how easy it is to train Agents to monitor almost any web source and report changes back to you.

Leveraging Web Data to Anticipate Real Estate Market Trends

Jeff Hoffman — Tue, 29 Sep 2009 13:31:13 +0000

In early September, prominent banking analyst Meredith Whitney said home prices have another leg to drop, perhaps another 25% over the next two years as high unemployment levels continue. While S&P/Case-Shiller shies away from predictions, David Blitzer, chairman of the index committee, thinks Ms. Whitney’s estimate is too negative. Moody’s expects a drop of about 10% from current levels.

How could web data help you determine which (if any) of these predictions is accurate before Commerce Department figures are released?

Monitoring, collecting, and aggregating precise real estate data points from the web can help you anticipate market trends as part of your day-to-day routine.

Imagine the unique insight you’d gain by automatically extracting and aggregating thousands of data points every 24 hours from home builder, MLS, for sale by owner, and/or foreclosure websites. In 1 place you’d be able to view average price, average price per square foot, total count of inventory, data on employment, interest rates, rental prices, sentiment and other quantitative or qualitative variables you deem important. This information could be further segmented by price point and region.

As examples, the 2 screen shots below are the result of Connotate’s intelligent software Agents collecting thousands of real estate data points from 8 U.S. Homebuilders every 24 hours. This data gives unique insight into new home sale pricing trends before Commerce Department figures or Earnings are released.

The intelligent Agent technology interacts with data sources the way you and I would, but on an automated basis — and at a disciplined and scalable pace (that would be impossible for us to do ourselves). Agents are smart, tireless and effective, and may provide you with unique insight.

Those firms equipped with the tools to transform the infinite Web (and internal sources) into a focused set of actionable, highly personalized and differentiated data will be the clear winners in this turbulent and competitive market and well beyond.

For more information on creating real-time feeds of real estate data (or any other kind of data feed) contact Connotate at 732-296-8844 or try out a free 2-week trial of Connotate’s On-Demand Library Beta.

What can Fantasy Football Sites teach Content Oriented Businesses and Intelligence Professionals?

Jeff Hoffman — Tue, 22 Sep 2009 17:47:52 +0000

The participation in and popularity of fantasy football has exploded over the last decade. By some estimates the industry grosses more than $1 billion a year.

In early September I began gearing up for a twelve team fantasy football league that uses an auction style draft. With little spare time, I wasn’t looking forward to the annual task of manually collecting and aggregating predicted stats, point totals and other data on the top 200 players/defenses from disparate sources and monitoring news sites for player news and updates that might make them more or less valuable.

ESPN.com came to the rescue eliminating the tedious tasks that drain 40% or more of my time preparing for the fantasy football auction. ESPN provided not only player’s aggregated point totals for the season based on my league’s scoring but also links to really useful, up-to-date player information and alerts about breaking news.

This illustrates two important lessons for content oriented businesses and intelligence professionals:

Content oriented businesses can differentiate themselves by offering a one-stop solution that is comprehensive (yet actionable), timely and relevant to their consumers needs.
Being able to focus your time strategizing versus collecting and aggregating data from the web can be the difference between being in the game versus prospering.

In all the economic gloom many people have forgotten that we live in the most extraordinary technological age in history. While my fantasy competitors had access to the same ESPN data as I did, in the real world, your competition is (or may soon be) using intelligent Agent technology to monitor, extract, aggregate and normalize data, text and graphics from tens of thousands of Web Sites or internal sources– effortlessly, for competitive advantage.

To learn more about how intelligent Agent technology can help you gain a competitive advantage in the real world visit Connotate.com.

Interested to see an example of how I used intelligent Agents to help me with fantasy football last year? If so, please view the chart below (in Internet Explorer 7). In summary Agents collected and graphed data on the number of touches (defined as the number of handoffs and receptions combined) by player helping me figure out who to start and/or pick-up. My take away from the example below was to sit both Hightower and James towards the end of the season because the Cardinals were increasingly passing the ball.

Unfortunately the data source I used to collect this data last year isn’t available….so if anyone knows where I can find data on the number of handoffs and receptions combined by player for this season please send me a comment and let me know.