Identity Resolution Daily

By Mike Betron, Infoglide Software Director of Marketing

We talked a week ago about the rapidly emerging market space called Big Data. One statistic that opened my eyes is Gartner’s prediction that the volume of new data generated by enterprises will grow by 650% in the next five years, and 80% of that will be unstructured data!

The 451Group’s definition of Big Data describes a growing need for non-traditional processes that can treat massive amounts of data as a whole, thereby making it impossible to use many traditional tools and techniques. Data is voluminous, complex, and very dynamic, yet business drivers demand that it be captured, managed, and harnessed to benefit the organization.

While entity resolution (ER) software is technologically mature, the evolving requirements for managing Big Data fit ER perfectly. For example, Infoglide’s Identity Resolution Engine (IRE) scales to meet Big Data requirements, and together with its flexibility in handling ambiguous unstructured and structured data with missing elements makes it an ideal solution for wringing value from the “data deluge” we increasingly find ourselves in.

One of the unique problems associated with Big Data is its multiple disparate sources that include email, Word documents, spreadsheets, and social media such as IM, newsfeeds, Facebook, and LinkedIn, just to name a few. Again, entity resolution systems like IRE now include support for multiple data forms and have created special ways to incorporate social media.

So, while Big Data presents a daunting challenge for many organizations, flexible technologies like entity resolution represent a key element of any solution.

Share This

By the Infoglide Team

Hays Daily News: Making the move to electronic records a natural fit for clinic

“Beginning in 2015, providers who have not successfully demonstrated meaningful use will face cuts in the amount of Medicare reimbursement they receive. It will begin with 99-percent payment in 2015, and drop to 97 percent by 2017, according to information from the Centers for Medicare and Medicaid Services. ‘So if your practice has not implemented an EHR and have meaningful use, you’re going to get reimbursed less dollars for the same service as someone who does,’ Brull said.”

GIGaom: Jeff Jonas Video on How Data Makes Corporations Dumb

“‘Information is being created faster than organizations can make sense of it,’ he says. The gap between the growth of information and understanding is widening because the tools for understanding are not scaling as fast as the growth in data and information.  ‘As computers are getting faster and the world is getting more sensors, the organizations have been getting dumber,’ he said. ‘The percentage of what is knowable is on a decline.’”

Share This

[Post from Infoglide] OYSTER: A Configurable ER Engine

“Now that I have finished the four-part series on linking methods, I would like to talk about one of my pet projects, OYSTER.  It stands for Open sYSTem Entity Resolution, a project to build a configurable, open-source entity resolution.  Although I am somewhat hesitant to announce a system that is not yet available to readers, it does exist and has been a valuable teaching tool in my ER class.  A run-time version (Java JAR file) will available soon on the ERIQ website, and the source code should be available on Source Forge by the end of the year.”

DATAMONITOR: Bad data costing US businesses $700 billion a year

“Madan Sheina, author of the report and an Ovum lead analyst, said: ‘Bad data is a growing problem for businesses due to the sheer volume and pace at which it is now moved between organisations. We now estimate that bad data costs US companies 30 per cent of their revenues – a massive $700 billion per year and a figure that is set to increase.’”

thestar.com: Watchdog warns criminals, terrorists could abuse new payment methods

“‘FINTRAC anticipates that the FATF will publish a public report on this work later in 2010,’ it said. Over the past few years, prepaid cards and Internet payment services have only been identified in a minority of domestic money laundering and terrorist financing cases. In 2008-2009, for instance, Internet-based payment services were involved in roughly 4 per cent of all disclosed cases, FINTRAC said in its report.”

Share This

By the Infoglide Team

The Miami Herald: Medicare’s fraud hot line begins to root out billing scams

“By September, Feliberto Ramos was arrested on fraud charges accusing him and his company, Miracle Group Rehabilitation Center, of falsely billing the federal healthcare program $3.1 million over just three months. Medicare paid Ramos $1.9 million for rehab services never provided to angry beneficiaries.”

OCDQ:Data, data everywhere, but where is data quality?

“Data matters because everything—and not just the rows in our relational databases and spreadsheets, but also our status updates from Facebook and Twitter, our blog posts, and even most of our daily conversations—is data. The growing challenge is can we extract meaningful insights from these vast and veritable oceans of unrelenting data volumes, and use those insights to make better decisions in near real-time in order to positively impact the various aspects of our lives.”

eBusiness Tweets: Microsoft entering the electronic medical record (EMR) software market

“You would think Microsoft would be in such a promising industry, but you won’t find a Microsoft EHR available. The primary reason why is that EHRs are highly specialized, and Microsoft’s main products (Dynamics, CRM, and SharePoint) don’t come anywhere near the needs of physician practices. It would be very difficult for Microsoft to build an EHR from scratch and introduce it to the market. so what should Microsoft do to enter the industry? Acquire a current player.”

Share This

By Mike Shultz, Infoglide Software CEO

Andrew White of Gartner recently posed a question about whether master data management (MDM) is dead. He didn’t actually suggest that the demise of master data management is imminent. He was challenging whether our current terminology adequately clarifies the current reality about MDM and associated product areas.

Certainly the terms describing many markets and types of products are being associated with MDM. Jackie Roberts of DATAForge pointed out that the definition of MDM now seems to include “data integrity, data quality, entity resolution, matching, data integration, governance, metrics and analysis.”

While entity resolution was mentioned in her list, our obsessive focus on entity resolution (aka identity resolution) leads to the conclusion that, rather than being subsumed, its role is growing. Wayne Eckerson at TDWI seems to agree that identity resolution is a critical component of the recent MDM acquisitions. In his post about the acquisitions by Informatica and IBM of Siperian and Initiate Systems, respectively, he described the two transactions this way:

“You could say that Siperian is mostly MDM, but with identity resolution and other capabilities, whereas Initiate is mostly about identity resolution, but with MDM and other capabilities.”

Identity resolution is becoming an integral part of many product areas. Within MDM itself, creating a single-entity view is best done with an identity resolution engine. Data mining is greatly enhanced by the addition of entity resolution. Dan Power of Hub Solution Designs wrote about how key identity resolution is to data matching. We’ve talked about how social CRM can resolve identities of individuals across multiple disparate data sources using identity resolution, as well as “rationalize multiple variations and errors and anomalies that block finding existing customers within their systems”.

Although identity resolution technology has been years in the making, it has only recently risen into the consciousness of most analysts and customers. Because of its ability to bring enhanced clarity to ambiguous data, advanced identity resolution is now beginning to have a significant impact across many data-centered disciplines.

Share This

[Post from Infoglide] Attacking Subscription Fraud with Identity Resolution

“In March 2006, the Communications Fraud Control Association (CFCA) estimated that annual global fraud losses in the telecom sector were between $54 billion and $60 billion, and the losses continue to be substantial. Many types of fraud have been identified, but by far the most prevalent is subscription fraud.”

ITBusinessEdge: Analyst: SAP Missed Out During Recent MDM Acquisition Spree

“SAP, on the other hand, has had a lot of issues in the past couple of years. They haven’t made a direct MDM acquisition since they acquired A2i years and years ago, which was a PIM vendor and they’ve just been working off of that architecture and been trying to improve it.”

Liliendahl On Data Quality: Data Quality Tools Revealed

“Data matching is the ability to compare records that are not exactly the same but are so similar that we may conclude, that they represent the same real world object.”

BeyeNETWORK: Master Data Management: Moving Forward…

“So now that MDM has been around for a while, and the master data terminology has drifted into our standard vocabulary, it might be worth stepping back and asking a different question:  Is MDM the revolutionary approach to organizational data consolidation and enterprise information management or is it devolving into yet another  (of many) data management tools?”

Share This

[Post from Infoglide] Master Data Movement

“I read with interest yesterday’s article at SeekingAlpha which discusses rumors swirling around the MDM software industry.  According to the article, sources suggest that two deals are very near completion.  The first of those rumored transactions would see Informatica picking up MDM provider Siperian.  On the heels of their acquisitions of Identity Systems and AddressDoctor, the Siperian purchase could not be totally unexpected – but would most certainly create some ripple effect worth watching.”

[Post from Infoglide] Connecting the Dots: We May Be Closer Than We Think

“Paul Rosenzweig, former Deputy Assistant Secretary for Policy at the Department of Homeland Security, recently posted an intriguing piece on Harvard National Security Journal about connecting the dots regarding the Christmas Bomber. He makes a strong case that a decision to stop research on data analytic tools in 2003 has contributed to the problem analysts face today in making sense of the massive and manifold data sources they sift through.”

Forrester Blog: Introducing The MDM Market’s Newest 800lb Gorilla: Informatica Acquires Siperian!

“In the short term, I’m sure Informatica will be more than happy to continue to collect revenue from Oracle while keeping this partnership alive, but don’t expect future negotiated contracted terms to remain very reasonable as Informatica gains traction with its MDM strategy. No matter how often Oracle says how happy they are to maintain a friendly state of co-opetition with strategic partners, I don’t anticipate they will want to run the risk of a competitor pulling the rug out from under its aggressive MDM strategy.”

News8Austin: Community forum poses questions about Fusion Center

“According to department officials, sharing information with neighboring jurisdictions as well as state and federal agencies ensures that crime history and other information is shared outside the city limits. The department said it the center will be one that ‘analyzes information in order to best detect, respond and hopefully prevent criminal and terrorist activity — as well as other public safety hazards.’”

Ramon Chen: Informatica + Siperian Acquisition = Premier MDM Platform

“As expected, Informatica has announced that it has acquired Siperian (disclosure, my former company) for $130M… If predictions are correct, this will be a relative ‘bargain’ when compared with the upcoming IBM and Initiate Systems tie up which is expected to be 4 to 5x Initiate’s $90M annual revenues.”

Share This

Technology writer Chris Calnan’s story opened with a comment about Infoglide that nicely sums up the evolution of the broader market for identity resolution and entity analytics: “The market may have finally caught up with Infoglide Software Corp.’s technology.”

While identity resolution technology has evolved rapidly over the past decade, its market visibility only emerged fairly recently. It was barely two years ago in mid-2007 when Gartner analyst Mark Beyer dubbed it “entity resolution and analysis” and pointed out that it “was previously an obscure, but gradually developing, technology that has come to the forefront as a result of world events and market forces.” Gartner singled it out as an “On the Rise” technology within operational business intelligence.

That first Gartner “hype cycle” showed entity resolution and analysis entering at the earliest stage. A year later in mid-2008, a broader report on data management  depicted it significantly higher on the curve in the opinion of the Gartner analyst team. In both reports, its estimated time to “mainstream adoption” was 2-5 years, the second fastest category.

At the end of 2008, noted consultant and speaker Jill Dyché of Baseline Consulting issued her predictions for 2009. Along with predictions about SaaS, data governance, BI, and MDM, she said that “Identity Resolution will get its due.” Rob Karel of Forrester had written several months before about Informatica’s acquisition of one of the two closest Infoglide competitors (IBM EAS being the other one). Identity Systems was acquired from Nokia for $85 million.

As we progressed further into 2009, the most meaningful indicator of identity resolution’s growing importance surfaced: an escalating identification with the space by other companies. IBM, Infoglide, and Informatica were joined by Initiate Systems, Intelligent Search, and Netrics, each of whom began incorporating messaging around identity and entity resolution.

For our customers and for us, this is all good news.  Our evolving space becoming better known and more highly valued will provide more alternatives for customers while increasing our own visibility. The future of identity resolution looks bright, and we all win.

[Distributed earlier this week in our quarterly publication, Identity Resolution Quarterly]

Share This

By the Infoglide Team

information management: Multi-Entity MDM Enablement

“Most efforts, however, are executed in surroundings inhibited by existing infrastructure (legacy applications, tools, hardware and integration), dispersed organizational structures and suboptimal processes. This reality introduces challenges in architecting and deploying efficient and effective multi-entity MDM solutions.”

BAM INTEL: BAM’s Thinking on the New DHS Standards

“Public Fusion Centers must be seen by citizens and policy-makers to play a direct role in the response to disasters as well as intelligence gathering. They cannot remain in the intelligence-sharing role only and not take some of the spotlight when their good work prevents or lessens the impact of America’s next disaster.”

newsday.com: OPINION: Revolution right in your doctor’s hand

“For doctors and their patients (in other words, all of us), the electronic health record is a far more revolutionary idea than those that brought us the ability to download a song, post a video online or read and send e-mails when you’re on a camping trip. While those other innovations indirectly enhance the quality of life, they are designed for entertainment or business purposes. The EHR directly improves quality of life because the end result of its design is better health.”

SmartData Collective: Data May Require Unique Data Quality Processes

“All data quality projects can appear the same from afar but ultimately can be as different as stars and planets. One of the biggest ways they vary is in the data itself and whether it is chiefly made up of name and address data or some other type of data.”

Share This

By Robert Barker, Infoglide Senior VP & Chief Marketing Officer

A continual theme at IdentityResolutionDaily is maintaining the privacy and confidentiality of data at all times. Two recent posts concerned fusion centers and citizen profiling, but the same issues apply to virtually any application of entity resolution technology. The fact is that, in some cases, anonymous identity resolution is a requirement for more sensitive identity resolution implementations.

The strong emphasis in data management for the last decade or so has been to implement data warehouses, data marts, and master data management. When bundled with associated processes like data extraction, transformation, and cleansing, these methods have been widely accepted as the best approach to solve any data problem. Here at IdentityResolutionDaily, we tend to talk about this over-handling of data as “data deterioration.”

A more basic approach is simply working with data sources undisturbed in their native environments. New principles suggest that you should perform scoring analyses as close to the source as possible. By exploiting existing security layers already in place, the need to add new layers of security is obviated.

Of course, for key sources of operational data, existing IT policies may deny direct access. In other cases, it may be necessary or preferable to move data for other reasons. For example, achieving desired performance parameters may dictate working with an extracted subset of the data rather than the entire data store.

The point I’m making is not to forbid moving data or creating data marts under any circumstances. Rather, I’m suggesting that the most rational approach is the following:

1. Develop solutions that adapt easily to multiple, disparate, remote data sources.
2. Default to leaving data where it lives whenever and wherever possible.
3. Provide the appropriate levels of entity anonymity within the solution and with the least possible intrusion to the enterprise.

Share This