Identity Resolution Daily

smart (enough) systems: How to address decision making challenges - ownership

“You want to manage the decision so that the value of the customer, their retention risk and more is considered and so that the person who REALLY made the decision was the person who works on how to handle the best customers. Only taking control of decisions, being explicit about how they are made and using automation to deliver the decision in every channel, every time is really going to work.”

Evolution of Security: ID Update and Word on the Blog

“Now that the new ID requirement is almost one week old, we wanted to provide additional stats.”

Hi-Tech-Blog: Economy Down Shoplifting Up

“The rise in retail thefts is also being blamed on the presence of organized crime, which account for up to $30 billion in losses every year, according to the FBI. These rings of professional thieves can easily move stolen items through the Internet, where consumers go to look for deals.”

Andy on Enterprise Software: MDM Fashions

“There are some good points in the DM Review article on MDM by William McKnight. In particular, I like that he clearly highlights some issues that some parts of our industry are still in denial over:”

Homeland Security Watch: Technology Task Force Presents 7 Recommendations to Chertoff

“Before he left, Chertoff charged the HSAC membership with one more task: ‘What are the ten tasks for the next Administration to take up and accomplish over its first year or two?’ It seemed odd to charge this group with something so trite. However, he explained, rightly, that it is important that efforts be made to preserve the institutional knowledge of the Department into and through its first ever Presidential transition. I’d like to know what you think should make the top ten list. Comment below.”

Hub Solution Designs: Building a Data Governance Organization

“I’m often asked how many data stewards a company needs. A rule of thumb I’ve used for quite a while is ‘one data steward per $1 billion in annual revenue’, but of course that can’t be a hard and fast rule. Depending on the company, the degree of automation, what data quality tools are available, and the major processes around the enterprise, that number is going to vary.”

The First Post: More snooping please, we love it

“Freeborn Englishmen like nothing more than the state’s jackboot on their windpipes, says David Cox”

Share This

[Post from Infoglide] The Importance of Identity Resolution to MDM

“In a recent post on the Hub Solution Designs blog, I wrote about the importance of integration to Master Data Management (MDM). Today, I’d like to dig into the importance of identity resolution to MDM.”

b-eye.com - Business Intelligence Network: Loss Prevention in the Retail Environment

“Leading retailers are using business intelligence to implement enterprise-wide loss prevention solutions that take into account the wide variety of factors that contribute to economic loss.”

DataFlux Community of Experts: Data Quality Strategy

“I was just creating a data quality strategy for a customer going through a conversion from multiple sources to one integrated database. While doing the conversion, new software will be introduced and propagated out to the organization. Not an easy task for any organization!”

Wired Danger Room: FBI Data-Mining Slashed After G-Men Dis Congress

“Earlier today, a House appropriators voted to pull $11 million to expand a controversial FBI data-mining project, after the Bureau repeatedly stiff-armed Congressmen and their gumshoes in the Government Accountability Office.”

b-eye.com - Business Intelligence Network: The Dirty Job of Data Governance

“Hence, this inaugural edition of the Jill Dyché Data Governance newsletter, courtesy of our buddies at the Business Intelligence Network (BeyeNETWORK.com). We’re encountering some new clients who’ve asked us to do cleanup duty on their failed data governance efforts, and that’s taught us a lot about how to deliver data governance, data management, data quality, master data management (MDM) and data integration initiatives the right way. Through this monthly newsletter, I’ll be sharing some of those hard-won lessons with you.”

Data Governance and Data Quality Insider: Data Quality Events – Powerful and Cozy

“For those of you who enjoy hobnobbing with the information quality community, I have a couple of recommendations for you. These events are your chance to rub elbows with different factions of the community. In the case of these events, the crowds are small but the information is powerful.”

Share This

By Dan Power, President and Founder, Hub Solution Designs

In a recent post on the Hub Solution Designs blog, I wrote about the importance of integration to Master Data Management (MDM). Today, I’d like to dig into the importance of identity resolution to MDM.

In a previous life, I worked for Dun & Bradstreet, and I had a chance to learn quite a bit about D&B’s technology for identity resolution, which it calls Entity Matching.

But the general question of identity resolution, or matching, is critical to a successful MDM initiative. There’s a good article by Roger Wolter and Kirk Haselden of Microsoft which says “the matching accuracy of MDM tools is one of the most important purchase criteria.”

I agree with that. If your goal is to build an MDM solution, with a central hub, integrated data quality, robust middleware, and useful external content, all driven by a data governance organization composed of experts from the business and from IT, then you’re going to need highly accurate matching tools.

The consequences of false matches (showing one customer record as a duplicate of another when that isn’t the case) or missed matches (not grouping two customer records together when they should be) will strongly affect the level of trust your MDM solution receives from the organization as a whole.

And depending on your industry, too many false positive or false negative matches could have serious business impacts, and in a medical or law enforcement MDM application, could even have life and death consequences.

The crucial question is “are the built-in matching tools provided by your MDM hub vendor good enough, or are you better off integrating a purpose-built identity resolution engine with your MDM hub?”

There’s one good way to answer that question - do a “bake off” or side-by-side comparison of the MDM hub with the dedicated matching tool, and see if the difference in results warrants integrating a matching tool with your MDM solution.

I know the first time that the business users of your new MDM solution find records grouped together that shouldn’t be, or find a large number of duplicates that to the human eye are obviously the same, their level of trust in your MDM hub will drop, and it may be very difficult to get it back again.

Dan Power is president of Hub Solution Designs, Inc., a consulting firm specializing in master data management and data governance. He has 20 years of experience in management consulting, enterprise applications and strategic alliances at companies such as D&B, Deloitte & Touche, Computer Sciences Corporation, eCredit and Parson Consulting. He speaks frequently at technology conferences, has written several articles for DM Review magazine, and regularly advises clients on developing and implementing high impact MDM strategies.

Share This

[Post from Infoglide] Identity Resolution Daily - One Year Later . . . or Thereabouts

“Well, here we are feeling like the stereotypical bad husband. We can’t believe we missed our first year anniversary. I guess a couple dozen long stemmed roses are in order.”

b-eye.com - Business Intelligence Network: Master Data Management Readiness

“The selection and purchase of a core master data management system should be the culmination of a series of preparatory steps that are described in this article.”

newsday.com: NYPD arrests 15 suspected in car insurance fraud

“The 13 people were arrested Tuesday on insurance fraud and grand larceny charges stemming from at least five wrecks. Two suspects are still at large. Police say the ring targeted car insurance companies, taking advantage of a state law that entitles people involved in a crash to $50,000 worth of medical benefits. . . . Police are investigating whether the clinics were in on the scam.”

DataFlux Community of Experts: CIO Forum

“This time, I am on my way to be a panel member at a local CIO Forum. . . . Data integration, and data quality issues are going to be on the top of the list for this discussion.”

supermarket.co.za: ‘E-Fencing’ Is Growing Wave In Organized Crime

“These losses are clearly affecting retailers’ bottom lines. According to the survey, organized crime accounts for approximately $30 billion in retail losses every year. As a result, retailers are also spending more to ward off these detrimental incidents. In fact, the average retailer spends approximately $230,000 per year on labor costs, the study said. This price tag can hit $1 million a year at larger chains that are constant organized retail crime (ORC) targets.”

Data Governance and Data Quality Insider: Get Smart about Your Data Quality Projects

“With all due respect to Agent Maxwell Smart, there is a mini battle between good and evil, CONTROL and KAOS, happening in many busy, fast-growing corporations. It is, of course, with information quality. Faster growing companies are more vulnerable to chaos because by opening up new national and international divisions, expanding through acquisition, manufacturing offshore, and doing all the other things that an aggressive company does, it leads to more misalignment and more chaotic data. While a company may have a strong desire to ‘own the world’ or at least their market, they may wind up owning chaos and disorder instead - in the form of disparate data.”

Share This

Homeland Security Watch: Homeland Security & Technology Panel Event

“Greg Nojeim, Director of the Project on Freedom, Security, and Technology at the Center for Democracy and Technology offered insightful warnings about the unintended consequences of technology when it is not developed or deployed with privacy protections at the initial stages.”

b-eye.com - Business Intelligence Network - Blog: Jill Dyche: At Night She Sells Oranges: Sarah Mclachlan Eats Our Lunch

“You may have seen Baseline’s latest YouTube video, one of several in our ongoing Get in the Game series. To film it we took advantage of the actor’s strike in L.A. and ended up producing several videos on the cheap. We were pretty proud of our opportunism, which let us deliver a funny and tongue-in-cheek send-up of master data management. The video illustrates how customer identity resolution —a key part of the MDM stack—is a problem that can often be found in your own backyard, as it were.”

Fraud, Phishing and Financial Misdeeds: NRF Survey shows Organized Retail Crime activity is growing!

“When I stated that this activity hurts all of us, the reason is that retailers have to make up the $30 billion they are losing to this activity somewhere. This normally equates to higher prices, or in extreme circumstances (especially in tight economic times) cutting payroll. Simply stated, people might be losing their jobs because of this activity.”

Andy on Enterprise Software: Data Quality stories

“The most entertaining thing I saw was a presentation by a consultancy firm on some real-life data quality situations they had encountered at clients, showing just how tricky data quality algorithms can be.”

National Retail Federation: The Most Famous Sheriff You Never Heard Of: Sheriff Grady Judd of Florida’s Polk County Sheriff’s Office will speak about high-profile cases of organized retail crime

“Sheriff Judd and I have something in common—we truly believe in the value of educating the public about organized retail crime. During Sheriff Judd’s keynote at the opening session of the LP show, he will address the importance of partnering with local retailers and others in the private sector. He will also share his story about uncovering the organized retail crime ring and about how his team worked with local retailers to track the criminals.”

[Note: While not exactly on topic, we felt that anyone who reads or writes blogs should be aware of the following story.]
Concurring Opinions: The Associated Press, Copyright, and the Blogosphere

“The Associated Press, one of the nation’s largest news organizations, said that it will, for the first time, attempt to define clear standards as to how much of its articles and broadcasts bloggers and Web sites can excerpt without infringing on The A.P.’s copyright.”

Share This

[Post from Infoglide] That’s the Ticket: Addressing a Different Kind of Lottery Fraud

“Type ‘lottery fraud’ into Google, and you get lots of returns warning about the dangers of Nigerian lottery scams and other phishing attacks on unsuspecting, and/or perhaps overly optimistic, consumers. But there are less well known types of lottery fraud also being perpetrated, and some of them hurt consumers as well.”

b-eye.com - Business Intelligence Network - Blog: Jill Dyche: Doublin’ Down at the Data Governance Conference

“In a prior blog I bemoaned having to miss Informatica World in Vegas this week. (Regular readers of the Jillblog know that what happens in Vegas gets blogged about almost immediately.) But I feel much better now, since the Wilshire Data Governance Conference in San Francisco has already exceeded my expectations.”

Washington Business Journal: Retail survey reveals growing theft problems

“Retailers are battling an increase of organized retail crime, according to the National Retail Federation’s (NRF) 2008 Organized Retail Crime report published Wednesday.”

Informatica: Data Quality - Ensuring Trust in Your Data

“TDWI once published a study that poor data quality was costing companies an estimated $600B annually in the US alone. And yet, industry analysts estimate the data quality tools market to be between $300-400M annually. The demand for better data quality is clearly there, but the size of the data quality tools market does not appear to reflect that same demand. Could it be that traditional Data Quality tools are not delivering enough value for customers?”

PogoWasRight.org: Few Canadians are ‘very confident’ their personal data is safe with retailers, banks and governments

“Of the three types of organizations, Canadian retailers fared the worst, with less than one per cent (O.5 per cent) of consumers saying they are very confident retailers can protect their customers’ on-line personal and private information.”

Andy on Enterprise Software: The dust clears (a bit)

“I have wondered for some time about Informatica’s intentions in the MDM space. There had been some market rumours about them possibly buying Siperian, but it seems as if their not so secret meetings with Siperian were actually to form a partnership, which was just announced.”

Share This

DailyRecord.co.uk: Insurance firms cracking down on fraud

“With fraudulent claims on the up, companies are cracking down on false claims. According to RSA, data from any false policy claim will now be shared with other companies through the Insurance Fraud Bureau (IFB). John Beadle, RSA’s counter fraud manager, said: ‘The reality is that insurance fraud adds a significant amount to overall claim costs and it’s the honest policyholders who are the true victims. ‘Fraud adds five per cent onto their insurance bills.”

DMReview.com: Quality Control - Sanjay Rao transforms Identity Systems’ focus from identity matching to data quality

“With backgrounds in both finance and software engineering, Rao immediately took over as general manager at the identity matching pure-play, making him the highest-ranking person with direct hands on the company. As the inside and outside face of the organization, Rao has since overseen eight quarters of strong growth, including almost doubled revenue and profitability. Most lately, Rao has spearheaded Identity Systems’ push into the broader data quality market, as he recently explained to DM Review Editorial Director Jim Ericson.”

b-eye.com - Business Intelligence Network - Blog: Claudia Imhoff: First Steps in Operational BI

“Operational BI is becoming quite the buzz today. There are countless articles, tips, vendors with operational BI offerings, case studies, etc., available today. Yet, I still get asked the very basic question of how do you get started. What are the first steps? Well, here are my thoughts on how to get started…”

ITWeb: Informatica reports record first quarter results

“Significant milestones achieved since January 2008 include: * Announced definitive agreement to acquire Identity Systems, a pioneer in identity resolution and matching technology.”

CIO - Blog: Why Storage and Data Management Will Kick Your Butt

“So what say these IT professions? The ‘Crunch Time’ study concludes that most IT organizations ‘are not equipped to handle periods of intense data flow,’ despite the majority of respondents saying they know that the data tsunamis are coming in the next year.”

Share This

[Post from Infoglide] Mistaken Identity Resolution Part V: Identity Resolution vs. Data Quality

“In this series of posts on Mistaken Identity Resolution we have compared identity resolution with other market spaces that it’s sometimes confused with, such as Master Data Management (MDM), data integration, and data warehousing. With Informatica’s recent acquisition of Identity Systems, now’s a good time to address the confusion between identity resolution and data quality.”

The Daily News Tribune: Shoplifting a matter of opportunity

“LaRocca said the theft of items that are specifically targeted can be associated with the economic status of the country, however a direct correlation is difficult to make. ‘The economy is just one of a number of different triggers. It is not the driving force behind it,’ LaRocca said. ‘The largest factor is always the opportunity.’ . . . Last year, retailers nationwide lost $40.5 billion worth of merchandise due to theft. . . . That monetary loss, he said, is unfortunately passed on to honest consumers who must pay higher prices. According to National Retailers Federation, consumers pay 1.5 cents more per dollar spent because of the toll shoplifting takes on retail business. Furthermore, when merchandise is stolen, no one is paying sales tax to the state, he said.”

Government Computer News: Big Brother may listen in Britain

“The British government’s Home Office reportedly is considering building a massive database of virtually all the electronic communications generated by residents of the United Kingdom, including voice and data communications along with Web site views and other online traffic.”

Andy on Enterprise Software: A Burning Platform

“I was amused by a piece regarding data quality in which a data quality initiative at a chemical manufacturer was kicked off only after a warehouse burnt down and the company discovered that they had no way of tracing which customers would be affected.”

STORES Knowledge Series: Refunds Management: Balancing Customer Service and Loss Prevention (Free Webinar)

“Retailers are continually challenged with balancing good customer service and loss prevention. Good customer service breeds loyalty and has a strong impact on top-line revenue. Refunds management practices and tools provide retailers with a well-managed solution that can provide top- and bottom-line revenue and profit. Join us as we explore the refunds challenge, tools and practices in place today that can offer a way to manage the refund process, and explore emerging tools that can set your stores apart from the competition.”

PogoWasRight.org: Department of Homeland Security Information Sharing Strategy

“The Department’s Information Sharing Strategy provides strategic direction and guidance for all Department of Homeland Security information sharing efforts.”

Forbes.com: Supermarket group stocked $1.4M into 1Q lobbying

“The Food Marketing Institute, the trade group for food retailers and wholesalers, spent nearly $1.4 million in the first quarter to lobby on food safety, public health and other issues, according to a disclosure report. . . . FMI also lobbied on retail pharmacy and Medicaid drug reimbursement, credit card interchange fees, identity theft, organized retail crime, and pension, wage and tax issues, according to the report filed April 21 with the House clerk’s office.”

Share This

By Robert Barker, Infoglide Senior Vice President & Chief Marketing Officer

In this series of posts on Mistaken Identity Resolution we have compared identity resolution with other market spaces that it’s sometimes confused with, such as Master Data Management (MDM), data integration, and data warehousing. With Informatica’s recent acquisition of Identity Systems, now’s a good time to address the confusion between identity resolution and data quality.

A Gartner study done several years ago estimated that poor quality customer data costs U.S. businesses an estimated $611 billion dollars a year. So obviously data quality is a very important component of data management.

Data quality is defined by Whatis.com as “the reliability and effectiveness of data… maintaining data quality requires going through the data periodically and scrubbing it. Typically this involves updating it, standardizing it, and de-duplicating records to create a single view of the data, even if it is stored in multiple disparate systems.”

Identity resolution is defined by wikipedia as the process that “analyzes all of the information relating to individuals and/or entities from multiple sources of data, and then applies likelihood and probability scoring to determine which identities are a match and what, if any, non-obvious relationships exist between those identities.”

So while both data quality and identity resolution seek to create a unified view of the data and determine which entities are the same, identity resolution takes the process further by also determining which entities are related.

While both technologies can de-dupe data records, identity resolution adds powerful matching functions to data quality, so numerous patterns that otherwise would go undetected are uncovered quickly and accurately. These data patterns include multi-cultural name and address matching, character insertions/deletions, nicknames, abbreviations, transpositions, repetitions, etc. And perhaps the most valuable reason for incorporating identity resolution into data quality is being able to automate the decision process and integrate it into existing business processes. Specific rules can be applied to the intelligence gathered from search, relationship, and identity results, and an explicit action executed based on business requirements.

Perhaps instead of asking how data quality and identity resolution differ, the better question to ask is, “Are you risking poor data quality if identity resolution is absent?”

Share This

Note: There will be no post on Monday as we will be observing Memorial Day. Enjoy your holiday weekend, and we hope to ’see’ you back here on Wednesday.

[Post from Infoglide] Partners in Crime Fighting

“eBay and other online auction services have taken some heat from retailers and Congress because organized retail crime rings and other shoplifters are using these types of services for e-fencing. It’s a bit of a dilemma for these sites because if they do the right thing, and reduce e-fencing on their sites, they lose revenue, which is certainly something no business, public or private, wants to do.”

The 451 Group (requires free registration): An MDM acquisition: the next deal for Informatica?

“Through a combination of homegrown and acquired software, Informatica now has all the requisite pieces for a complete MDM offering – data access, data analysis and profiling, data correction, cleansing, data transformation and now identity management – but not an actual MDM platform itself. It therefore seems logical to us that once the company has digested Identity, a deal scheduled to close in May, it will start eyeing an MDM target to provide a complete offering.”

WKYC.com: Shoplifting: A $40 billion enterprise

“A National Retail Federation study indicates 75 percent of businesses have found their shoplifted items for sale on the internet. . . . Bokovitz said increased incidents of shoplifting are directly related to the state legislature’s decision to drop progressive sentencing on theft related crimes. The penalties for shoplifting no longer increase with the number of times you commit the crime. So who really pays the price? ‘Me and you, the consumer, everybody who plays by the rules because our costs just went up,’ Bokovitz said.”

Netrics HD: The Monk Factor

“It’s great to see that simply collecting customer data is no longer good enough - the data needs to be usable to benefit the business. ‘Refining customer data quality and access to customer data have emerged as two of the top marketing investment priorities of b-to-b CMOs this year. Half of b-to-b marketers plan to put more resources against creating marketing databases, cleaning up customer data, improving sales force automation and CRM integration, according to Forrester Research in its “B2B CMO Investment Priorities for 2008” report.’”

PogoWasRight.org: DHS to monitor access to IT systems

“Government employees, contractors and consultants with access to Homeland Security Department computer systems are among those whose names and personal information will be kept in a newly created database, according to a notice posted in the Federal Register.”

DMReview: Master Data Management and Service-Oriented Architecture

“One of the many ‘hidden benefits’ of a successful master data management (MDM) strategy is the contribution it can make to your enterprise’s move toward a service-oriented architecture (SOA).”

Share This