Identity Resolution Daily

[Post from Infoglide] Mistaken Identity Resolution Part V: Identity Resolution vs. Data Quality

“In this series of posts on Mistaken Identity Resolution we have compared identity resolution with other market spaces that it’s sometimes confused with, such as Master Data Management (MDM), data integration, and data warehousing. With Informatica’s recent acquisition of Identity Systems, now’s a good time to address the confusion between identity resolution and data quality.”

The Daily News Tribune: Shoplifting a matter of opportunity

“LaRocca said the theft of items that are specifically targeted can be associated with the economic status of the country, however a direct correlation is difficult to make. ‘The economy is just one of a number of different triggers. It is not the driving force behind it,’ LaRocca said. ‘The largest factor is always the opportunity.’ . . . Last year, retailers nationwide lost $40.5 billion worth of merchandise due to theft. . . . That monetary loss, he said, is unfortunately passed on to honest consumers who must pay higher prices. According to National Retailers Federation, consumers pay 1.5 cents more per dollar spent because of the toll shoplifting takes on retail business. Furthermore, when merchandise is stolen, no one is paying sales tax to the state, he said.”

Government Computer News: Big Brother may listen in Britain

“The British government’s Home Office reportedly is considering building a massive database of virtually all the electronic communications generated by residents of the United Kingdom, including voice and data communications along with Web site views and other online traffic.”

Andy on Enterprise Software: A Burning Platform

“I was amused by a piece regarding data quality in which a data quality initiative at a chemical manufacturer was kicked off only after a warehouse burnt down and the company discovered that they had no way of tracing which customers would be affected.”

STORES Knowledge Series: Refunds Management: Balancing Customer Service and Loss Prevention (Free Webinar)

“Retailers are continually challenged with balancing good customer service and loss prevention. Good customer service breeds loyalty and has a strong impact on top-line revenue. Refunds management practices and tools provide retailers with a well-managed solution that can provide top- and bottom-line revenue and profit. Join us as we explore the refunds challenge, tools and practices in place today that can offer a way to manage the refund process, and explore emerging tools that can set your stores apart from the competition.”

PogoWasRight.org: Department of Homeland Security Information Sharing Strategy

“The Department’s Information Sharing Strategy provides strategic direction and guidance for all Department of Homeland Security information sharing efforts.”

Forbes.com: Supermarket group stocked $1.4M into 1Q lobbying

“The Food Marketing Institute, the trade group for food retailers and wholesalers, spent nearly $1.4 million in the first quarter to lobby on food safety, public health and other issues, according to a disclosure report. . . . FMI also lobbied on retail pharmacy and Medicaid drug reimbursement, credit card interchange fees, identity theft, organized retail crime, and pension, wage and tax issues, according to the report filed April 21 with the House clerk’s office.”

Share This

By Robert Barker, Infoglide Senior Vice President & Chief Marketing Officer

In this series of posts on Mistaken Identity Resolution we have compared identity resolution with other market spaces that it’s sometimes confused with, such as Master Data Management (MDM), data integration, and data warehousing. With Informatica’s recent acquisition of Identity Systems, now’s a good time to address the confusion between identity resolution and data quality.

A Gartner study done several years ago estimated that poor quality customer data costs U.S. businesses an estimated $611 billion dollars a year. So obviously data quality is a very important component of data management.

Data quality is defined by Whatis.com as “the reliability and effectiveness of data… maintaining data quality requires going through the data periodically and scrubbing it. Typically this involves updating it, standardizing it, and de-duplicating records to create a single view of the data, even if it is stored in multiple disparate systems.”

Identity resolution is defined by wikipedia as the process that “analyzes all of the information relating to individuals and/or entities from multiple sources of data, and then applies likelihood and probability scoring to determine which identities are a match and what, if any, non-obvious relationships exist between those identities.”

So while both data quality and identity resolution seek to create a unified view of the data and determine which entities are the same, identity resolution takes the process further by also determining which entities are related.

While both technologies can de-dupe data records, identity resolution adds powerful matching functions to data quality, so numerous patterns that otherwise would go undetected are uncovered quickly and accurately. These data patterns include multi-cultural name and address matching, character insertions/deletions, nicknames, abbreviations, transpositions, repetitions, etc. And perhaps the most valuable reason for incorporating identity resolution into data quality is being able to automate the decision process and integrate it into existing business processes. Specific rules can be applied to the intelligence gathered from search, relationship, and identity results, and an explicit action executed based on business requirements.

Perhaps instead of asking how data quality and identity resolution differ, the better question to ask is, “Are you risking poor data quality if identity resolution is absent?”

Share This

[Post from Infoglide Software] Mistaken Identity Resolution Part IV: Identity Resolution vs. Data Warehousing

“Thus far this series of posts on ‘Mistaken Identity Resolution’ has contrasted Identity Resolution with Master Data Management (MDM) and Data Integration. What about one of the more established data concepts - data warehouses? How are they different? And how do they work together? If at all?”

Ovum: Informatica gets new identity, reports steady Q1 growth

“Informatica has agreed to buy Identity Systems, a subsidiary of Nokia, for around $85m in cash. . . . Informatica believes that Identity’s software will give it a ‘differentiated’ cross-language identity matching capabilities. That said, InfoGlide and IBM (from its SRD acquisition) also have identity resolution engines. Arguably these two products are more complete in that they use rules engine and workflow processes to automate decision making.”

Vindy.com: Organized retail theft to equal racketeering

“A Cincinnati lawmaker introduced legislation Tuesday cracking down on organized retail theft rings that target stores for everything from baby formula to small appliances. Republican Sen. Bill Seitz said such crime is costing retailers billions of dollars annually in lost sales and state and local governments some $88 million in sales tax revenues. Seitz’s SB 320 would add ‘organized retail theft’ to the list of offenses Ohio law categorizes as ‘corrupt activity,’ ranking it with racketeering.”

b-eye.com - Business Intelligence Network - Blog: Shawn Rogers: The stack is moving….

“Generally when we use the term stack in the business intelligence world we are talking about the big 4: IBM, Microsoft, Oracle and SAP along with all their recent acquisitions Cognos, BO etc. All of these companies claim to provide a platform to enable every type of business intelligence you might need to have. It’s interesting to me that one of the largest platforms out there is often ignored but continues to grow into our business world through the internet. Google is much more than a search engine. Over the past couple years the company has branched out to become a business solution platform. Some of the players in the BI space are already taking advantage of it.”

KNXV-TV: Website tells you how long security lines are at Sky Harbor Airport

“The next time you fly you may want to click on the TSA website in order to save some time or prepare to wait a little longer in security lines. . . . The government owned site allows a user to click on various airports, terminals, and security checkpoints to see how long you should expect to wait in line before making it to the gate.”

DataFlux Community of Experts: Master Data Discovery and Data Mapping

“In my experience, if you follow processes associated with customer data you start to discover master data residing in other unexpected places, e.g. Access databases, Excel spreadsheets etc. . . . Having discovered disparate master data, you will most likely find that even if copies of master data reside in different systems the data may be named differently across those systems. So, the next challenge is for each master data entity (e.g. customer, product, employee, asset etc.) to map disparate master data in identified systems to commonly defined master data described using the shared business vocabulary. This is a difficult and time-consuming exercise to do manually. Fortunately, there are tools now available to help explore systems looking for master data and to automatically map that disparate data to commonly defined enterprise wide definitions of master data.”

Homeland Security Watch: Homeland Secretary Offers 10-year Vision, in 4 Parts

“Last week DHS Secretary Michael Chertoff delivered a speech at Yale University entitled ‘Confronting The Threats To Our Homeland.’ Citing the five-year mark for DHS and the three-year mark for his tenure as its head, he explained that such an occasion warrants not just one speech, but four. And the first — the one he delivered at Yale — offers insight into the way he views the ‘challenges and threats that we face over the next five and ten years relating to homeland security in the broadest sense.’”

Share This

By Robert Barker, Infoglide Senior Vice President & Chief Marketing Officer

Thus far this series of posts on “Mistaken Identity Resolution” has contrasted Identity Resolution with Master Data Management (MDM) and Data Integration. What about one of the more established data concepts - data warehouses? How are they different? And how do they work together? If at all?

The differences are pretty simple. In 1990 Bill Inmon defined a data warehouse as “a subject-oriented, integrated, time-variant and non-volatile collection of data in support of management’s decision making process.” What was a new concept in 1990 has passed through Gartner’s hype cycle to become an expected component of IT infrastructure for most large- and many medium-sized organizations. Data is transferred into a data warehouse and is expected to reside there indefinitely (and without changing) to support mostly analytical, but increasingly operational, activities. So a data warehouse is a repository for data. Identity resolution technologies are software that operate on data.

So is the relationship between identity resolution and warehousing complementary, beneficial, or problematic? At times the presence of a well-managed data warehouse can ease the identity resolution process by providing one reliable data source that can be similarity searched with other “dirtier” data sources. At other times, the cleansing methodology that is often used to prepare data for warehousing is problematic because it can hide or even obliterate data variances caused by fraudsters. These data variances are exactly what identity resolution uses to resolve multiple identities into one and uncover hidden relationships.

To further confuse the matter, in certain cases, identity resolution technologies like our Identity Resolution Engine(tm) (IRE) operate on data in remote, disparate databases, acting as a virtual data warehouse by combining data “on the fly.” In other cases, IRE works perfectly well on existing data warehouses.

So the answer to our previous question about identity resolution and data warehouses is the famous one you often get from consultants: “it depends.” However, by carefully determining the right points in existing processes to implement identity resolution, data warehousing and identity resolution can be both complementary to each other and beneficial to existing systems.

Make sense? Or maybe not? Let us know what you think.

Share This

[Post from Infoglide Software] Mistaken Identity Resolution Part III: Identity Resolution vs. Data Integration

“In this blog, we will pick up our conversation from our Mistaken Identity Resolution series and discuss how identity resolution and data integration are related or unrelated.”

HeraldTribune.com: You may be paying $400 to $600 a year to offset shoplifting costs

“But Dan Doyle, the 50-year-old vice president of loss prevention and human resources for Bradenton-based Bealls Inc., has spent the past few years talking about it — with everyone who would listen: other retailers, law enforcement and even the public. . . . ‘There was this fear that the media would uncover our dirty little secrets,’ Doyle said. ‘But now we want the public to know about it, because they’re paying for it. They pay $400 to $600 a year more for merchandise because of retail theft.’ The extra money is not just the cost of markups to cover the losses, but also the cost of insurance, electronic tags and in-store cameras. ‘All of that stuff comes at a price,’ Doyle said. ‘We want the public to be angry about it.’”

Homeland Security Watch: Where the Candidates Stand on HLS Part III

“To learn about McCain’s views on combating terrorism, you’ll find it among, again not surprisingly, his positions on continuing the Iraq War. The net of all this is that McCain is no slouch when it comes to counterterrorism. Parsing brawn and brains is the tough part. A closer look reveals his campaign’s priorities and perspectives on the mission of securing the homeland, but its an incomplete picture.”

Forbes.com: Retail group to lobby on crime matters

“The National Retail Federation hired Alston & Bird LLP to lobby the federal government on organized retail crime and related consumer protection issues. Retail crime has been a top issue for retail trade groups and their members. The NRF said last year that the problem is getting worse as it released a May 2007 survey showing that more than 75 percent of retailers said they were a victim of crime within the past year.”

PogoWasRight.org: To be anonymous or not to be, that is the privacy question

“‘A total surveillance is not only inevitable and irreversible, but also irresistible,’ Jeff Jonas, distinguished engineer and chief scientist at IBM Entity Analytics, said during a panel on surveillance at a Legal Futures Conference at Stanford on Saturday.”

Share This

By Ram Anantha, Infoglide Software

In this blog, we will pick up our conversation from our Mistaken Identity Resolution series and discuss how identity resolution and data integration are related or unrelated.

Data integration is the process of combining data residing at different sources and providing the user with a unified view of this data. Data integration resolves the complex issues that arise from data fragmentation including poor data quality, inconsistencies in the structure and meaning of data, and inadequate data governance. One of the common approaches to integrating data is to bring it all together into a single data warehouse. In this approach, data from disparate sources and locations within an enterprise are loaded into the centralized structure of a data warehouse or dedicated database. Typically, in a data integration exercise, the variations and anomalies in the data present across the various data repositories are combined into one representation.

Since data integration involves presenting a unified view of the data, identity resolution techniques can effectively be used for this purpose. Identity resolution systems have sophisticated matching and linking algorithms that can match two records from different data sources and determine the likelihood of whether they are the same or not.

In contrast to data integration though, identity resolution systems do not require a centralized data warehouse to operate on. Instead, these systems can connect to disparate, multiple databases and operate on the data in its native format without the need to cleanse the data.

Additionally, as pointed out here, identity resolution technologies can be differentiated from data integration technologies because they actually derive significant value from the inconsistency of data drawn from multiple sources.

For example, you may have two individuals with the same exact name and the same exact address. Data integration technologies would see this as the same person and combine them, in effect ‘losing’ one of the records. Identity resolution technology would score these two records as having a very high likelihood of being the same entity but would not remove either record. Then when subsequent data gets entered into the system that shows that one individual has a birth date of 1-23-64 and the other has a birth date of 2-5-88 it becomes clear that these are actually two separate individuals, and it’s likely a parent-child relationship.

In conclusion, while identity resolution technologies are capable of doing much more than just data integration, they can be used for this purpose. Their greatest value though is in being able to create a single view of an entity (person, product, etc.) while maintaining forensic data that may be useful. This allows the view of the entities to evolve as new data is entered. Many organizations can benefit from this ability to provide a unified view of their data that is capable of adapting to new information.

Share This

[Post from Infoglide Software] Mistaken Identity Resolution Part II: Identity Resolution vs. Master Data Management (MDM)

“In a recent post, we talked about how the relatively new technology of identity resolution is often confused with other areas. One of those, master data management (MDM), is receiving lots of press attention these days. So how specifically does identity resolution (which Gartner terms “entity resolution and analysis”) differ from MDM?”

The Denver Post: Bill regulating “e-fencing” killed

“A bill to regulate Internet auction sales of items commonly stolen and resold — known as ‘e-fencing‘ — was effectively killed Wednesday when a legislative committee voted to table it.”

WSAZ.com: Insurance Fraud and More in Court

“A man was in court Thursday charged with scamming his insurance company, but this case of fraud turned into much more and it left police wondering what he was thinking.”

ZDNet: Surveillance society needs privacy safeguards

“Writing in the Washington Post, Frank Baitman, of the British-based Petards Group, which develops advanced surveillance systems, says that the time has come for some federal legislation on how surveillance images are captured, stored and shared.”

Homeland Security Watch: Where the Candidates Stand on HLS

“That’s how we’ll start a mini-series here on HLSWatch to take a look at how the candidates of both parties stack up on homeland security.”

Share This

By Bob Barker, Infoglide Software Senior Vice President & CMO

In a recent post, we talked about how the relatively new technology of identity resolution is often confused with other areas. One of those, master data management (MDM), is receiving lots of press attention these days. So how specifically does identity resolution (which Gartner terms “entity resolution and analysis”) differ from MDM?

Rather than being a specific technology or set of technologies, MDM is an overarching discipline that has a clear goal: achieve consistency of computing across varying IT systems by sharing master reference data. MDM seeks to manage data through various means to achieve this desired consistency.

MDM is a very broad term spanning multiple data domains, use cases, and industries. Gartner correctly points out that only two data domains have yet emerged to become viable - “customer” and “product” - and their emergence has been driven by the consistency requirements of specific industries, i.e. “customer” is driven primarily by financial services and healthcare and “product” is driven by manufacturing. Thus MDM currently has two identifiable submarkets called “customer data integration” (CDI) and “product information management” (PIM), each being served by multiple software vendors who help create more consistent customer and product data.

So how does identity resolution differ from MDM?

• Rather than demanding data consistency, identity resolution technologies derive significant value from the inconsistency of data drawn from multiple sources.
• Identity resolution solutions like Identity Resolution Engine (IRE) resolve multiple identities into one and frustrate those who purposely alter their identity in order to fool existing computer systems.

Literally billions of dollars lost each year due to fraud could be saved by implementing identity resolution applications to support retail, financial services, and other industries.
Is there an implicit antagonism between MDM and identity resolution? The answer, of course, is definitely not. Each is applied to solve a specific problem and at a different point to achieve a positive result.

MDM identifies and manages master reference sets of data to increase consistency and thus productivity within transaction systems. Identity resolution finds fraudsters and highlights their hidden relationships in real time by analyzing data systems that capture and hold identity data, cross referencing them with watchlists and other data sources. (MDM output can even serve as a data source for identity resolution!).

So what do they have in common? Both MDM and identity resolution are important contributors to a stronger bottom line.

Share This

ars technica: TSA answers our question, changes policy after blog comments

“When the TSA launched its Evolution of Security blog last month in an effort to engage in open communication with travelers, my colleague Jon Stokes expressed skepticism and characterized as laughable the prospect of TSA acting on feedback from the public. Having written about the ineptitude of TSA in the past myself, I was emphatically inclined to agree with Jon. It looks like we might have passed judgment prematurely, because it only took TSA one week to start making broad policy changes in response to feedback received through the blog.”

Roseville & Rocklin Today: Organized Retail Crime Ring Busted

“Dave Kemp spotted suspicious activity by a shopper within the Longs store on Stanford Ranch Road in Rocklin and obtained the license plate number of the car that the shopper and two others left in. He suspected that the shoppers were involved in ORC based upon the methods of filling shopping carts; specifically, the type of products being taken and the method of hiding the products in the shopping cart.”

cnet: How will Real ID affect you?

“The Real ID law is touted by Homeland Security officials as an anticrime and antiterror measure, but is steadfastly opposed by some state governments on privacy and sovereignty grounds. Computer scientists also have raised concerns about how its creation of a national interlinked database would work in practice. . . . Real ID will require states to share detailed information about anyone with a state ID card or driver’s license, perhaps through a network called AAMVAnet, which the Department of Transportation is paying to expand in hopes of supporting the massive amount of data that will be exchanged. Databases owned by Social Security and U.S. Citizenship and Immigration Services will also be integrated.”

casino.co.uk: Is Casino Fraud Increasing?

“There has been a flurry of police investigations and arrests on the subject of Casino fraud, although surprisingly it has been a number of casino employees whom have been arrested. So is Casino fraud on the increase or is this just a spate of fraud cases hitting the media?”

b-eye.com - Business Intelligence Network: What is Master Data?

“The enormous interest in master data management (MDM) that has appeared in the past couple of years has not yet generated a great deal of methodological progress. Hopefully, as data professionals, consultants, and vendors grapple with the complex issues involved, the situation will improve. A central problem, however, is that there is little agreement about what master data is.”

[Note: Look for our upcoming blog post on Identity Resolution vs. Master Data Management, a part of our series on Mistaken Identity Resolution.]

Share This