Identity Resolution Daily

By Ram Anantha, Infoglide Director of Product Management

Master Data Management (MDM) is gaining a lot of traction as evidenced by the number of vendors (SAP, Siperian, D&B/Purisma, Oracle, Initiate Systems, and a host of others) and end customers (such as AIG, Grange Insurance, McGraw-Hill, Merck, Merrill Lynch, and Nationwide) who shared their experiences in deploying MDM solutions at the MDM Summit in New York City this week.

Identity Resolution clearly plays a key role in the MDM space. Many of the conference sessions highlighted how Identity Resolution has become a key ingredient within an overall MDM solution. Without the ability to accurately match and resolve various attributes of an entity or identity, MDM solutions can’t succeed.

Some of the key requirements for MDM Identity Resolution were in evidence at this week’s Summit:

• Strong, flexible customer data matching algorithms must provide a real-time view of customer entity – must handle anomalies, variations, tokenization, etc.
• Tunable algorithms must minimize false positives and false negatives.
• Solutions must permit correction of easy mistakes on the fly, such as values that have been typed in a wrong field (e.g., Social Security Number used for Driver License Number).
• Deployments must handle an unlimited number and size of data sources.
• Matching accuracy must be controlled using attribute weighting.
• Matching decisions should be as automated as much as possible.
• The matching process must be able to leverage external reference data sources.

For you MDM practitioners out there, let us know if this list agrees with your experiences.

Share This

We’re celebrating the holiday and will be back on Monday. Have a fun, safe Fourth!

Share This

Due to President’s Day, we will not be publishing our standard link post today. Please check back with us on Wednesday, February 20th.

Share This

ars technica: TSA answers our question, changes policy after blog comments

“When the TSA launched its Evolution of Security blog last month in an effort to engage in open communication with travelers, my colleague Jon Stokes expressed skepticism and characterized as laughable the prospect of TSA acting on feedback from the public. Having written about the ineptitude of TSA in the past myself, I was emphatically inclined to agree with Jon. It looks like we might have passed judgment prematurely, because it only took TSA one week to start making broad policy changes in response to feedback received through the blog.”

Roseville & Rocklin Today: Organized Retail Crime Ring Busted

“Dave Kemp spotted suspicious activity by a shopper within the Longs store on Stanford Ranch Road in Rocklin and obtained the license plate number of the car that the shopper and two others left in. He suspected that the shoppers were involved in ORC based upon the methods of filling shopping carts; specifically, the type of products being taken and the method of hiding the products in the shopping cart.”

cnet: How will Real ID affect you?

“The Real ID law is touted by Homeland Security officials as an anticrime and antiterror measure, but is steadfastly opposed by some state governments on privacy and sovereignty grounds. Computer scientists also have raised concerns about how its creation of a national interlinked database would work in practice. . . . Real ID will require states to share detailed information about anyone with a state ID card or driver’s license, perhaps through a network called AAMVAnet, which the Department of Transportation is paying to expand in hopes of supporting the massive amount of data that will be exchanged. Databases owned by Social Security and U.S. Citizenship and Immigration Services will also be integrated.”

casino.co.uk: Is Casino Fraud Increasing?

“There has been a flurry of police investigations and arrests on the subject of Casino fraud, although surprisingly it has been a number of casino employees whom have been arrested. So is Casino fraud on the increase or is this just a spate of fraud cases hitting the media?”

b-eye.com - Business Intelligence Network: What is Master Data?

“The enormous interest in master data management (MDM) that has appeared in the past couple of years has not yet generated a great deal of methodological progress. Hopefully, as data professionals, consultants, and vendors grapple with the complex issues involved, the situation will improve. A central problem, however, is that there is little agreement about what master data is.”

[Note: Look for our upcoming blog post on Identity Resolution vs. Master Data Management, a part of our series on Mistaken Identity Resolution.]

Share This

[Post from Infoglide Software] Mistaken Identity Resolution

“Sometimes identity resolution is hard for folks to correctly categorize. That’s because it resembles a number of other technologies.”

KCRA.com: 3 Generations Accused Of Lodi Shoplifting

“A grandmother, mother and grandchildren were all arrested earlier this week on suspicion of shoplifting at a Lodi Target store, police said.”

PogoWasRight.org: Webcasts from Data Privacy Day 2008 Available Online

“Webcasts of the Jan. 28 conference at Duke University, Data Privacy in Transatlantic Perspective: Conflict or Cooperation?, are available online for free viewing with Real Player.”

beye.com - Business Intelligence Network: Wouldn’t It Be Cool? Our Business Intelligence and Data Integration Wish List for 2008

“We recently jotted down our industry predictions for 2008. But more than once during this process, one of us would say, ‘Naah. That will never happen.’ Then we’d move on to more practical predictions. But our wheels were still turning. We kept brainstorming about what would be cool to see in the world of business intelligence (BI) and data integration, however unrealistic. So we made a list of capabilities, features, and fabulous functions that we (and, more to the point, our clients) would love to see in the coming year. Or the next year, for that matter.”

ABC: Computers can recognise Mr Average

“The researchers have developed an imaging technique that merges several photos of the same person into one composite. They found when composites of different people were shown to a computer it was able recognise all of them. Such a method could improve face recognition programs already in use to identify suspects at airports, for example, or as part of immigration and employment verification.”

Share This

It is usually nice to be mistaken for someone famous. It is especially nice if they are talented, athletic, or attractive.

You might resemble a rock legend and use that talent to bring joy to Parkinson’s Disease patients. Or you might be a look-alike for a superstar quarterback. Or maybe people mistake you for another quarterback’s top fan.

Then again, you could be employed as Saddam Hussein’s look-alike.

Or “you” might be a product that resembles another “product” (er, illegal substance).

Or you could be the computer programmer saddled by an unfortunate name in Mike Judge’s cult-classic movie “Office Space”:

Michael Bolton: There was nothing wrong with it… until I was about 12 years old and that no-talent a** clown became famous and started winning Grammys.
Samir: Hmm… well why don’t you just go by Mike instead of Michael?
Michael Bolton: No way. Why should I change? He’s the one who s*cks.

That occasionally happens to identity resolution (AKA entity resolution and analytics) technologies, such as our Identity Resolution Engine(tm) and IBM’s Entity Analytic Solutions.

Sometimes identity resolution is hard for folks to correctly categorize. That’s because it resembles a number of other technologies. So when you explain it to the uninitiated, if you see their light bulb begin to glow, it’s likely the wrong light bulb.

They interrupt to say, “Oh, so it’s the same thing as _______,” where the blank is filled with one or more of the following:

• customer data integration (CDI)
• data cleansing
• data integration
• data profiling
• data quality
• data warehousing
• de-duping/record linkage
• master data management (MDM)

But it’s not the same thing.

That is a natural reaction. When faced with something new, we like to align it with an existing concept, file it in a familiar folder, or check a pre-positioned box on our mental grid. And it’s always nice to have it already covered by other technologies that we are currently deploying or that are in our roadmap.

But in the case of identity resolution, that would be a mistake. Identity resolution differs significantly from most of those other technologies.

The good news is that identity resolution is also very complementary to existing solutions and can resolve unique problems left unresolved by its cousins. And it typically doesn’t bring the same discomfort to an organization, its infrastructure, and its pocketbook that they do.

So what is identity resolution? How does it compare to those other technologies? What unique value does it bring to the table? Well, that’s a big topic that we’ll be posting on repeatedly in coming months.

In the meantime, please let us know your impressions of identity resolution’s relatives by clicking the “Comments” link below and leaving a reply.

• Does your enterprise use them?
• How long did they take to deploy?
• Have they met your expectations?
• What pains were you trying to address?
• What pains remain?

Or just tell us who people commonly mistake you for. We always like to hear from our readers.

Share This

By Johnny Norris, Director UK Markets

[Note: Today’s post is from our offices ‘across the pond’ in the UK.]

News this week from ComputerworldUK tells us that large merchants have seen an increase of over 10% over twelve months in fraud through their online businesses. Notably, a considerable piece of this issue has to do with the use of multiple identities (33% of large retailers saw this) and the difficult problem of internal theft.

One of the issues facing multi-channel or Internet retailers is that online fraud is often quite different from store based problems. It’s seen by perpetrators as anonymous, and it is considerably easier to appear to hide identity. Combine this with the ability to ‘pass on’ goods via online auction sites and retailers are facing attack from external and internal sources (often combined). According to the CyberSource research, many retailers are willing to invest in technology to tackle the issue, but most solutions are based on exception and don’t always tackle the huge identity issue.

But, there is good news! Traditionally, store based retailers in the UK have not collected much in the way of customer or returns data (certainly as compared to US retailers), but the online/multi-channel boom is changing that. Retailers now face the issue of what information to store and how to deal with the deluge of it. Fraud analysts haven’t increased in number, so retailers must find a way to harvest ALL their information sources (whether it be employee or customer data) and provide their fraud/security teams with prioritized, high quality information on those using multiple identities, or even working in an organized way. Organized Retail Crime (ORC) wasn’t mentioned in the article, but this is a key area and often the way it can be detected is to identify links or relationships between customers or customers and employees.

Identity resolution technology (AKA entity resolution and analysis technology) that can identify the use of multiple identities and detect fraudulent relationships (or ORC) is now in existence, but not many companies offer the full range of elements. Importantly, this technology can be used to look at the problem of e-fencing via online auction sites as well. Looking for the subtle matches between stock that has gone missing and, say, an online vendor with details a lot like an employee may be a critical step in stemming this fast growing fraud problem.

Can retailers accept a 10% year on year rise in fraud? We think they shouldn’t have to.

Share This

Fox Business: High Shrink Takes a Toll on the Bottomline of Retail Enterprises

“Aberdeen, a Harte-Hanks Company (NYSE: HHS: 13.90, +0.17, +1.23%), surveyed 680 retail enterprises in October 2007 and found that on an average, 60% of retailers surveyed recorded year-over-year shrink or write-off of at least 1.75% of their total inventory (as a percent of sales), further compounding existing theft and fraud-related complexities in retail. Another alarming trend is that almost a third of retailers (30%) have reported between 2% to 6% invalid transactions (unauthorized and fraudulent customer transactions as a percentage of total transactions). This indicates that retailers may be faced with high transaction fraud incidence due to organized retail crime, data loss, and dishonest employees. . . . Cash asset protection, real-time video and transaction data intelligence, and access control are some of the pillars that enable retailers across different sub-segments to build an agile and responsive loss prevention environment.”

Yahoo! News: New ID rules may complicate air travel

“Millions of air travelers may find going through airport security much more complicated this spring, as the Bush administration heads toward a showdown with state governments over post-Sept. 11 rules for new driver’s licenses. By May, the dispute could leave millions of people unable to use their licenses to board planes, but privacy advocates called that a hollow threat by federal officials.”

PogoWasRight.org: De: Citizen’s rights activists intend to prevent EU plans to record passengers’ flight details

“German Arbeitskreis Vorratsdatenspeicherung (Work Group on Data Retention) has announced a constitutional appeal if the German government agrees to back plans by the EU Commission to retain Passenger Name Records for 13 years.”

WashingtonTechnology: Web extra: States face Real ID privacy dilemma

“‘If you centralize it, it would be easier to manage in terms of security and privacy,’ said Harold Kocken, business solution manager of BearingPoint’s national motor vehicle solutions group. ‘There would be fewer people with the possibility of interfering with the database.’ . . . The verification grant program will expand a pilot project run by the National Association for Public Health Statistics and Information Systems. Named the Electronic Verification of Vital Events, it is operating with three states that can originate queries and nine states that can respond to the queries. Using the system, operators type information for verification, such as someone’s full name and date of birth, and the system automatically processes the query to determine if there is a match with the original birth certificate. The response is either ‘match’ or ‘no match.’ It attaches notes about suspected misspellings and near matches.”

Share This

Concurring Opinion: Privacy and the 2008 Election

“Polling on issue like privacy is a challenge, which makes interpreting survey results difficult, said privacy law expert Daniel Solove, a professor at George Washington University and author of several privacy-related books, including the newly published ‘The Future of Reputation.’“

PogoWasRight.org: Flawed New Jersey Online Dating Bill Still Seeking Ratification

“The flawed part of the bill comes in the fact that to satisfy the bill’s ‘Criminal Background screening’ all a site has to do is a simple name search via a regularly updated government public records database or a database maintained by a private vendor. Any user of the Internet and social networking or dating sites knows that many users don’t provide their real names. The bill makes no provisions for dating services to check backgrounds via information that like social security numbers and to corroborate the data to be sure the member is who they claim to be.”

Bloomberg.com: U.S. Asked to Exempt Canadian Flights From Watch List Checks

“Already, flights between Canadian cities that cut through U.S. airspace to save time are exempt from the Department of Homeland Security’s planned `Secure Flight Program,’ Canadian Transport Minister Lawrence Cannon said today in a statement. Canada filed a written request for the broader exemption.”

Share This

Few would find a problem with large contributions coming to political candidates and committees from a single household. But over $200,000 in contributions since 2004 from a single middle-class household might raise some eyebrows, especially when the head of that household is reportedly a mail carrier.

The Paw family of Daly City, California was highlighted in a Wall Street Journal article. The Paws were noted for their apparent connections with Norman Hsu of New York, a one-time fugitive and major campaign fundraiser. Hsu’s status as a “bundler” meant that he brought big money to campaigns. While all of the details still aren’t clear, it is undeniable that Hsu faces Federal fraud charges for his business dealings and campaign funding activities.

Campaigns are required to report contributions they receive to the Federal Elections Commission (FEC). That data is publicly available through several sources, including public interest groups, political websites, and the FEC itself.

But campaign data is like all data. It suffers from the same problems that we see in industries such as retail, insurance, and banking — it is “noisy”. Variations in formatting and mistakes in typing cause straight “equality” comparisons to fail. And when you can’t count on the data to agree, you can’t make decisions.

Traditional, exact searches of the public sources yield conflicting results that range from variations in the spelling of contributor names and their employers to differing cities. Exact matching yields only a portion of the contributions that were apparently from their household.

You cannot count on common database queries or even “similarity” measures like soundex to reveal matches across variations such as:

• Multipart names, nicknames, and misspellings
• Address abbreviations (e.g., “Parkway” versus “Pkwy.”) and misspellings “6300 Bridgepoint” versus “6300 Bridge Point”)
• City misspellings and miscodings
• Zip code misspellings and formatting (e.g., “78730″ versus “78703″ versus “787303824″)

Advanced similarity search algorithms, however, can resolve most of these inconsistencies.

And in the cases in which field-by-field similarity comparisons fail, all is not lost. Advanced relationship detection and identity resolution techniques can see past inconsistencies and still connect the dots. So contributions attributed to residents of a specific address but related cities can still be resolved to the same individuals. And the connections can be exposed, analyzed, and highlighted.

Here’s a scenario for how identity resolution software might have automatically highlighted the Hsu connection for further scrutiny:

• Similarity search would be applied to group contributions by identical household to find the highest contributors.
• The total for the household would be above average. It would be flagged as a household of interest.
• Relationships of household members to others would be examined.
• Some contributions report variations of “Next Components, Ltd.” as an employer.
• Another person related to “Next Components, Ltd.” is Norman Hsu.
• Hsu repeatedly made contributions to the same committees on the same days as the household.

Identity resolution technology, like that of Infoglide Software’s Identity Resolution Engine™, could have detected this as an interesting pattern warranting further investigation. It cannot unequivocally determine that these patterns indicate that there is criminal behavior involved vs. just an innocent coincidence where two co-workers have similar political leanings. But for organizations and individuals seeking potentially dirty needles in a haystack of dirty data, it could narrow down the search significantly.

All of this is revealed from the data, if enough brute force or hindsight — or the right technology — is brought to bear.

It does not take sophisticated technology to tell some parts of the story, however. When the scandal broke, campaigns had to field uncomfortable questions, return hundreds of thousands of dollars in contributions, and find new contributors to replace the old.

Too bad the campaigns didn’t use technology to help them “know their contributors” and avoid a black eye. But others can learn from their woes and protect their organizations from similar problems.

Other industries faces analogous, if not identical, issues.

• Retailers unwittingly hire bad employees with connections to known bad check writers, hand cash-equivalent refunds to shoplifters, and overlook organized retail crime (ORC) rings in their midst.
• Insurers are defrauded by individuals who game their application process or collude to cash in on fraudulent claims.
• Bankers are challenged with knowing a sea of customers well enough to avoid painful consequences and public embarrassment related to Bank Secrecy Act (BSA) and other compliance regulations.

It’s a good thing that the solution to these problems exists. Is your organization using it? Or is it waiting for an Hsu-sized black eye before it takes action?

Share This