Identity Resolution Daily

By Robert Barker, Infoglide Senior Vice President & Chief Marketing Officer

Andy Hayler recently commented on the “scale of the problem that Master Data Management (MDM) is attacking.” He specifically cited a survey of IT leaders recently conducted by Information Difference and reported on by ITPro that indicates a majority of senior IT executives believe their data quality is poor, yet many believe the sheer number of data sources they have to deal with is a big obstacle. In fact, “13 percent of companies had over 100 systems storing customer data, while 11 percent had over 100 systems storing product data.”

Imperfect data can be so daunting that it leads to paralysis. Maybe the bigger obstacle is not the number of data sources – it’s the mindset. Contrary to common belief, complete cleansing and integration of all data into a warehouse is not mandatory before you can gain value from MDM. Cleansing and combining data no longer has to be the first step in MDM. Newer technology exists that not only reaps value from un-cleansed data but has the ability to create a virtual data warehouse that allows an organization to access all its data in real time.

Identity resolution technology can search across multiple, disparate databases to find associated records without needing to merge and purge. And a sophisticated identity resolution solution can do this in real time without performance issues. Additionally, identity resolution is not “one size fits all” but supports flexible and highly configurable solutions that work with data in specific domains.

The task of cleaning and combining over 100 sources of data seems overwhelming, which is probably why those companies still have scores of data sources. Identity resolution addresses this issue directly.

Using the right technology before applying ETL processes, you can incorporate identity resolution into existing applications in order to combine similar identities, find hidden relationships, and enhance automated decision-making. Demonstrating ROI early will lead to continuing organizational support for additional MDM projects.

Share This

[Post from Infoglide] The Importance of Identity Resolution to MDM

“In a recent post on the Hub Solution Designs blog, I wrote about the importance of integration to Master Data Management (MDM). Today, I’d like to dig into the importance of identity resolution to MDM.”

b-eye.com - Business Intelligence Network: Loss Prevention in the Retail Environment

“Leading retailers are using business intelligence to implement enterprise-wide loss prevention solutions that take into account the wide variety of factors that contribute to economic loss.”

DataFlux Community of Experts: Data Quality Strategy

“I was just creating a data quality strategy for a customer going through a conversion from multiple sources to one integrated database. While doing the conversion, new software will be introduced and propagated out to the organization. Not an easy task for any organization!”

Wired Danger Room: FBI Data-Mining Slashed After G-Men Dis Congress

“Earlier today, a House appropriators voted to pull $11 million to expand a controversial FBI data-mining project, after the Bureau repeatedly stiff-armed Congressmen and their gumshoes in the Government Accountability Office.”

b-eye.com - Business Intelligence Network: The Dirty Job of Data Governance

“Hence, this inaugural edition of the Jill Dyché Data Governance newsletter, courtesy of our buddies at the Business Intelligence Network (BeyeNETWORK.com). We’re encountering some new clients who’ve asked us to do cleanup duty on their failed data governance efforts, and that’s taught us a lot about how to deliver data governance, data management, data quality, master data management (MDM) and data integration initiatives the right way. Through this monthly newsletter, I’ll be sharing some of those hard-won lessons with you.”

Data Governance and Data Quality Insider: Data Quality Events – Powerful and Cozy

“For those of you who enjoy hobnobbing with the information quality community, I have a couple of recommendations for you. These events are your chance to rub elbows with different factions of the community. In the case of these events, the crowds are small but the information is powerful.”

Share This

By Dan Power, President and Founder, Hub Solution Designs

In a recent post on the Hub Solution Designs blog, I wrote about the importance of integration to Master Data Management (MDM). Today, I’d like to dig into the importance of identity resolution to MDM.

In a previous life, I worked for Dun & Bradstreet, and I had a chance to learn quite a bit about D&B’s technology for identity resolution, which it calls Entity Matching.

But the general question of identity resolution, or matching, is critical to a successful MDM initiative. There’s a good article by Roger Wolter and Kirk Haselden of Microsoft which says “the matching accuracy of MDM tools is one of the most important purchase criteria.”

I agree with that. If your goal is to build an MDM solution, with a central hub, integrated data quality, robust middleware, and useful external content, all driven by a data governance organization composed of experts from the business and from IT, then you’re going to need highly accurate matching tools.

The consequences of false matches (showing one customer record as a duplicate of another when that isn’t the case) or missed matches (not grouping two customer records together when they should be) will strongly affect the level of trust your MDM solution receives from the organization as a whole.

And depending on your industry, too many false positive or false negative matches could have serious business impacts, and in a medical or law enforcement MDM application, could even have life and death consequences.

The crucial question is “are the built-in matching tools provided by your MDM hub vendor good enough, or are you better off integrating a purpose-built identity resolution engine with your MDM hub?”

There’s one good way to answer that question - do a “bake off” or side-by-side comparison of the MDM hub with the dedicated matching tool, and see if the difference in results warrants integrating a matching tool with your MDM solution.

I know the first time that the business users of your new MDM solution find records grouped together that shouldn’t be, or find a large number of duplicates that to the human eye are obviously the same, their level of trust in your MDM hub will drop, and it may be very difficult to get it back again.

Dan Power is president of Hub Solution Designs, Inc., a consulting firm specializing in master data management and data governance. He has 20 years of experience in management consulting, enterprise applications and strategic alliances at companies such as D&B, Deloitte & Touche, Computer Sciences Corporation, eCredit and Parson Consulting. He speaks frequently at technology conferences, has written several articles for DM Review magazine, and regularly advises clients on developing and implementing high impact MDM strategies.

Share This

[Post from Infoglide] Identity Resolution Daily - One Year Later . . . or Thereabouts

“Well, here we are feeling like the stereotypical bad husband. We can’t believe we missed our first year anniversary. I guess a couple dozen long stemmed roses are in order.”

b-eye.com - Business Intelligence Network: Master Data Management Readiness

“The selection and purchase of a core master data management system should be the culmination of a series of preparatory steps that are described in this article.”

newsday.com: NYPD arrests 15 suspected in car insurance fraud

“The 13 people were arrested Tuesday on insurance fraud and grand larceny charges stemming from at least five wrecks. Two suspects are still at large. Police say the ring targeted car insurance companies, taking advantage of a state law that entitles people involved in a crash to $50,000 worth of medical benefits. . . . Police are investigating whether the clinics were in on the scam.”

DataFlux Community of Experts: CIO Forum

“This time, I am on my way to be a panel member at a local CIO Forum. . . . Data integration, and data quality issues are going to be on the top of the list for this discussion.”

supermarket.co.za: ‘E-Fencing’ Is Growing Wave In Organized Crime

“These losses are clearly affecting retailers’ bottom lines. According to the survey, organized crime accounts for approximately $30 billion in retail losses every year. As a result, retailers are also spending more to ward off these detrimental incidents. In fact, the average retailer spends approximately $230,000 per year on labor costs, the study said. This price tag can hit $1 million a year at larger chains that are constant organized retail crime (ORC) targets.”

Data Governance and Data Quality Insider: Get Smart about Your Data Quality Projects

“With all due respect to Agent Maxwell Smart, there is a mini battle between good and evil, CONTROL and KAOS, happening in many busy, fast-growing corporations. It is, of course, with information quality. Faster growing companies are more vulnerable to chaos because by opening up new national and international divisions, expanding through acquisition, manufacturing offshore, and doing all the other things that an aggressive company does, it leads to more misalignment and more chaotic data. While a company may have a strong desire to ‘own the world’ or at least their market, they may wind up owning chaos and disorder instead - in the form of disparate data.”

Share This

Well, here we are feeling like the stereotypical bad husband. We can’t believe we missed our first year anniversary. I guess a couple dozen long stemmed roses are in order.

We started this blog on June 5th, 2007. It was, at the time, a bit of a leap of faith for us. Our goals were to have a voice in the conversation, be seen as a thought leader in the area of identity/entity resolution, and improve our search engine optimization.

Looking back, we think we’ve accomplished all that and more. We really appreciate all of you who have continued to read our postings and get our e-newsletter, those of you who have entered into the conversation with us through your comments, and especially our partners who have participated in Identity Resolution Daily and given their own voices to this community.

This past year, we’ve seen the term identity resolution become much more mainstream, and we’ve seen the value of identity resolution become understood. There was certainly evidence of this with the recent acquisition of Identity Systems by Informatica, which we chronicled here extensively.

We’re excited to have played a role in the evolution and growth of this market, and we look forward to things to come.

Share This

Homeland Security Watch: Homeland Security & Technology Panel Event

“Greg Nojeim, Director of the Project on Freedom, Security, and Technology at the Center for Democracy and Technology offered insightful warnings about the unintended consequences of technology when it is not developed or deployed with privacy protections at the initial stages.”

b-eye.com - Business Intelligence Network - Blog: Jill Dyche: At Night She Sells Oranges: Sarah Mclachlan Eats Our Lunch

“You may have seen Baseline’s latest YouTube video, one of several in our ongoing Get in the Game series. To film it we took advantage of the actor’s strike in L.A. and ended up producing several videos on the cheap. We were pretty proud of our opportunism, which let us deliver a funny and tongue-in-cheek send-up of master data management. The video illustrates how customer identity resolution —a key part of the MDM stack—is a problem that can often be found in your own backyard, as it were.”

Fraud, Phishing and Financial Misdeeds: NRF Survey shows Organized Retail Crime activity is growing!

“When I stated that this activity hurts all of us, the reason is that retailers have to make up the $30 billion they are losing to this activity somewhere. This normally equates to higher prices, or in extreme circumstances (especially in tight economic times) cutting payroll. Simply stated, people might be losing their jobs because of this activity.”

Andy on Enterprise Software: Data Quality stories

“The most entertaining thing I saw was a presentation by a consultancy firm on some real-life data quality situations they had encountered at clients, showing just how tricky data quality algorithms can be.”

National Retail Federation: The Most Famous Sheriff You Never Heard Of: Sheriff Grady Judd of Florida’s Polk County Sheriff’s Office will speak about high-profile cases of organized retail crime

“Sheriff Judd and I have something in common—we truly believe in the value of educating the public about organized retail crime. During Sheriff Judd’s keynote at the opening session of the LP show, he will address the importance of partnering with local retailers and others in the private sector. He will also share his story about uncovering the organized retail crime ring and about how his team worked with local retailers to track the criminals.”

[Note: While not exactly on topic, we felt that anyone who reads or writes blogs should be aware of the following story.]
Concurring Opinions: The Associated Press, Copyright, and the Blogosphere

“The Associated Press, one of the nation’s largest news organizations, said that it will, for the first time, attempt to define clear standards as to how much of its articles and broadcasts bloggers and Web sites can excerpt without infringing on The A.P.’s copyright.”

Share This

[Post from Infoglide] Integrating Identity Resolution into MDM Solutions: Real-Time or Batch?

“Dan Power at Hub Solution Designs recently challenged ‘MDM project team leaders, program managers, and data champions’ to think about the importance of integration. In particular, he asks about MDM integration in three contexts: real-time, business modeling, and two-way hub communication.”

Jeff Jonas: Smart Systems Flip-Flop

“Certainty often shifts with observations over time. And this is good.”

DataFlux Community of Experts: Integration in a Big Way!

“I am so excited! I get to work on an integration project that is huge. I mean really, really big — and it affects an entire organization. . . . After the project plan is solidified, roles and responsibilities are identified. Now a role is not necessarily a person! For example, there could be a data architect, business architect, technical architect, and a business intelligence architect role. Are you confused? I was, until we dug into the responsibilities of each role.”

Evolution of Security: Why is ID Important for Security?

“Last week we announced on our Web site a plan to begin REQUIRING ID from travelers on June 21st. This plan includes enabling our officers to refuse entry into the area beyond the security checkpoint anyone that does not cooperate with us to establish his or her identity. The exclusive reason to do this is to ensure people are who they say they are and are not gaming the system by using a boarding pass in a fake name; a well-known endeavor of professionals and college kids alike that could potentially circumvent the no-fly list.”

b-eye.com - Business Intelligence Network: An Update on Knowledge Management in the Federal Government

“A survey of the attendees of this year’s Knowledge Management Conference reveals that knowledge management in the federal government is alive and well.”

Share This

By Robert Barker, Infoglide Senior Vice President & Chief Marketing Officer

Dan Power at Hub Solution Designs recently challenged “MDM project team leaders, program managers, and data champions” to think about the importance of integration. In particular, he asks about MDM integration in three contexts: real-time, business modeling, and two-way hub communication.

Additionally, Jill Dyché recently pointed out that Identity Resolution supports and enhances five of eight core MDM functions outlined in her book on CDI, and we agree wholeheartedly. Given that Identity Resolution (IR) is integral to MDM, we can ask the first of Dan’s integration question: how should IR be integrated with MDM? More specifically, should this integration be batch or real-time?

The answer is, of course, it depends. Instead of asking “what will we be giving up by not providing for any real-time capabilities?” maybe we should ask “is the value of the MDM application sufficiently enhanced to justify the incremental infrastructure cost of using real-time data sources?” Sometimes the answer is yes, and sometimes it’s no.

Let’s consider two examples in the area of fraud detection. In a bank or retail setting, discovering a hidden relationship between two fraudulent people and keeping a fraudulent transaction from being consummated may save thousands of dollars in apprehension costs over catching it a day later, so a real-time approach is easily justified. On the other hand, in a case like insider securities trading, the acts are not preventable and the data connecting someone “in the know” with the person who makes a transaction based on a tip persists forever and can be caught and prosecuted months after it occurs.

Regardless of whether real-time or batch integration is selected, having multiple modes of integration available to match project requirements is important. In some cases, simple point-to-point XML data exchange may be enough, but in others it can be imperative that IR decisions be smoothly integrated into existing business processes. That may require using BPM, queuing, web services and other broadly supported methods of integration, again depending on the specific application context.

The takeaway: it’s essential to include multiple integration capabilities among the criteria you use in evaluating Identity Resolution technology.

Share This

DailyRecord.co.uk: Insurance firms cracking down on fraud

“With fraudulent claims on the up, companies are cracking down on false claims. According to RSA, data from any false policy claim will now be shared with other companies through the Insurance Fraud Bureau (IFB). John Beadle, RSA’s counter fraud manager, said: ‘The reality is that insurance fraud adds a significant amount to overall claim costs and it’s the honest policyholders who are the true victims. ‘Fraud adds five per cent onto their insurance bills.”

DMReview.com: Quality Control - Sanjay Rao transforms Identity Systems’ focus from identity matching to data quality

“With backgrounds in both finance and software engineering, Rao immediately took over as general manager at the identity matching pure-play, making him the highest-ranking person with direct hands on the company. As the inside and outside face of the organization, Rao has since overseen eight quarters of strong growth, including almost doubled revenue and profitability. Most lately, Rao has spearheaded Identity Systems’ push into the broader data quality market, as he recently explained to DM Review Editorial Director Jim Ericson.”

b-eye.com - Business Intelligence Network - Blog: Claudia Imhoff: First Steps in Operational BI

“Operational BI is becoming quite the buzz today. There are countless articles, tips, vendors with operational BI offerings, case studies, etc., available today. Yet, I still get asked the very basic question of how do you get started. What are the first steps? Well, here are my thoughts on how to get started…”

ITWeb: Informatica reports record first quarter results

“Significant milestones achieved since January 2008 include: * Announced definitive agreement to acquire Identity Systems, a pioneer in identity resolution and matching technology.”

CIO - Blog: Why Storage and Data Management Will Kick Your Butt

“So what say these IT professions? The ‘Crunch Time’ study concludes that most IT organizations ‘are not equipped to handle periods of intense data flow,’ despite the majority of respondents saying they know that the data tsunamis are coming in the next year.”

Share This

[Post from Infoglide] Mistaken Identity Resolution Part V: Identity Resolution vs. Data Quality

“In this series of posts on Mistaken Identity Resolution we have compared identity resolution with other market spaces that it’s sometimes confused with, such as Master Data Management (MDM), data integration, and data warehousing. With Informatica’s recent acquisition of Identity Systems, now’s a good time to address the confusion between identity resolution and data quality.”

The Daily News Tribune: Shoplifting a matter of opportunity

“LaRocca said the theft of items that are specifically targeted can be associated with the economic status of the country, however a direct correlation is difficult to make. ‘The economy is just one of a number of different triggers. It is not the driving force behind it,’ LaRocca said. ‘The largest factor is always the opportunity.’ . . . Last year, retailers nationwide lost $40.5 billion worth of merchandise due to theft. . . . That monetary loss, he said, is unfortunately passed on to honest consumers who must pay higher prices. According to National Retailers Federation, consumers pay 1.5 cents more per dollar spent because of the toll shoplifting takes on retail business. Furthermore, when merchandise is stolen, no one is paying sales tax to the state, he said.”

Government Computer News: Big Brother may listen in Britain

“The British government’s Home Office reportedly is considering building a massive database of virtually all the electronic communications generated by residents of the United Kingdom, including voice and data communications along with Web site views and other online traffic.”

Andy on Enterprise Software: A Burning Platform

“I was amused by a piece regarding data quality in which a data quality initiative at a chemical manufacturer was kicked off only after a warehouse burnt down and the company discovered that they had no way of tracing which customers would be affected.”

STORES Knowledge Series: Refunds Management: Balancing Customer Service and Loss Prevention (Free Webinar)

“Retailers are continually challenged with balancing good customer service and loss prevention. Good customer service breeds loyalty and has a strong impact on top-line revenue. Refunds management practices and tools provide retailers with a well-managed solution that can provide top- and bottom-line revenue and profit. Join us as we explore the refunds challenge, tools and practices in place today that can offer a way to manage the refund process, and explore emerging tools that can set your stores apart from the competition.”

PogoWasRight.org: Department of Homeland Security Information Sharing Strategy

“The Department’s Information Sharing Strategy provides strategic direction and guidance for all Department of Homeland Security information sharing efforts.”

Forbes.com: Supermarket group stocked $1.4M into 1Q lobbying

“The Food Marketing Institute, the trade group for food retailers and wholesalers, spent nearly $1.4 million in the first quarter to lobby on food safety, public health and other issues, according to a disclosure report. . . . FMI also lobbied on retail pharmacy and Medicaid drug reimbursement, credit card interchange fees, identity theft, organized retail crime, and pension, wage and tax issues, according to the report filed April 21 with the House clerk’s office.”

Share This