Unlocking Personalization and Product Equity With Data

Last week I had the pleasure of joining practitioners across Product, Research, and Engineering at The Product Equity Summit, hosted by The Aspen Institute and Tech Accountability Coalition in Washington, D.C. The 2.5 days were filled with insights about Data, Algorithmic Fairness, Trust & Safety and more! It was a pleasure to hear and learn from leaders from major players like LinkedIn, Adobe, TikTok, Paypal, Pinterest, Meta, Google and more! 

During this summit, I led a workshop about the intersection of Product Equity, Data and Personalization. As part of the workshop, I introduced a framework we developed at Everi AI called “The Personalization Framework.” We leverage this framework to think about how we build datasets for personalization use cases, but it also applies to product development. 

I want to share that framework with all of you and continue in the spirit of co-design and collaboration to build AI that is people-centric, data-driven, and product-oriented.

Personalization as predictability at scale 

After  7+ years working in one of the top consumer social media companies, I developed an understanding that one of the biggest unlocks for growth is personalization. At Everi, when we say personalized AI, we mean AI products that deliver tailored, predictable experiences for every customer. 

Tailored experiences serve individual customer needs, and most of the time, this is done through one product that can simply understand and deliver against the nuance of different customer segments. Examples can include a playlist recommendation, a “twin” ML lens, a loan recommendation, or a color matched skin care product. For personalized consumer AI, you need to ensure the product offering predictably works for each customer segment. The call to action of personalization we’re talking about here is simple: create a product that provides personal utility and make sure it works.  

If your product doesn’t predictably work for everyone, it's broken. This is a core belief at Everi. This is not a radical belief. Anyone who has been remotely involved with shipping products knows that product development is iterative. Sometimes teams ship broken, unfinished or partial versions of their product because they want to ship quickly. But this doesn’t mean we should maintain broken product experiences for customers. 

Data Quality and Consumer AI

Let’s take an example from AI as a use case for building predictable products. 

The quality of the prediction or decision of a model is determined by the quality of the input data. As the saying goes, “garbage in, garbage out.” Below is a simplified view of thinking about how training data can impact model output:

At Everi, we think about three pillars of dataset development:

(1) Data Collection - how, where, and what data is sourced

(2) Data Annotations - how and where we label data. This also includes schema design and metadata insights.

(3) Data Analysis - analyzing and preparing data for training. For the sake of brevity, we’ll bucket feature engineering here. 

When you have poor data development across these three inputs, you end up with ineffective training data. The dataset is ineffective because it introduces gaps or biases that show up in the model output. These gaps and biases can have a negative impact on revenue. For example, in Q1 of 2022, Unity Technologies reported poor quality data ingestion that led to $110 million revenue loss because of inaccurate predictive ML algorithms for ad targeting. In other cases, poor data quality can lead to more harmful outcomes for end users and companies alike. In 2017, a poor classification error with object detection contributed to a fatal crash with Uber’s self-driving car. Regardless of the example, the ability to scale a predictable product decreases when the training dataset is poor quality and ineffective. For your customers, poor quality data limits the predictability of a product working for them.

To address this issue, our team at Everi created a simple framework that helps you think about how to improve personalization when curating datasets. Use this framework to think through how to leverage data, people, and product outcomes to create predictable product experiences for your customers. 

The Personalization Framework 

The Personalization Framework is a product development framework to help unlock personalization, predictability and product growth. 

In this framework we define personalization as: Creating a tailored product experience that predictably works as intended, for every customer.

This does not mean that the product or model is right or works 100% of the time. A former Product Leader at Snap shared with us “Good products are predictable products.” In a world where 71% consumers expect a personalized experience, we’ve taken his insight one step further at Everi. For us, good products are personalized products, and personalized products are predictable products.

The 3 Fits of Personalization 

Every product leader and founder knows about “product market fit.” If you don’t know, product market fit is the degree to which your product satisfies a strong market demand. In other words, do you have a unique product offering that people desperately want? We applied this thinking to our framework, and developed the “Three Fits” of The Personalization Framework.

Before we jump into the “Three Fits,” I want to preface this by sharing that every product starts with a problem you’re facing and known user demand for the solution. This is the beginning of your data collection.

There are three channels to The Personalization Framework: Product, Data and People. We define the framework by how these three channels interact with each other: 

Data People Fit (DPF)

Data People Fit means you have representative data for your customer personas and that data is labeled correctly. When you achieve data people fit, you’ve prepared or acquired data that accurately represents the end consumer (“the people”) for your product. If you’re familiar with the pillars of data quality, when you’ve achieved DPF you can address these questions: Is my data accurate? Is my data complete?

At Everi, we have an asterisk for Data People Fit: collect representative data in a fair and ethical way. There are plenty of debates on what is ethical and whether ethical collection is scalable. At Everi, Data People Fit means that people are aware of and consent to the collection of their data for the purposes of its use. We believe that you can optimize the scalability of personalization by prioritizing the preferences of “people” or the end user.

Product Data Fit (PDF)

Product Data Fit means you have relevant data for your product's use cases. When you achieve product data fit, you’ve aligned your input data to the expected outcomes for your product. Relevant data also includes modality. Access to relevant data is not just about acquiring new data. Sometimes you have access to raw or proprietary data that can be made relevant once annotated or filtered for the appropriate use case.  When you’ve achieved PDF you can address the questions: Is my data relevant? Can I make the data I have relevant?

Product People Fit (PPF)

Product People Fit means you have achieved predictable outcomes for your end customer. When you achieve product people fit (PPF), you demonstrate an understanding of your end user and you have built a product that resonates with and predictably works for all of them. Product People Fit is about quality outcomes and should be measured against KPIs for various customer segments in production. PPF does not mean that your product works 100% of the time, but it does mean that your product predictably works as expected across your customer segments. When you’ve achieved PPF you can address these questions: Does my product work as expected? Does my product work predictably across my customer segments?

The “So What” Of It All

The consequence of poor Data People Fit (DPF), Product Data Fit (PDF) and Product People Fit (PPF) is that you don’t have accurate, relevant, or complete data to develop personalized products or tailored experiences for your end consumers. As a result, you may fail to penetrate a market, you may have a lower NPS score, or you may simply fail to deliver a “good product” for a group of customers. Again, a good product is a personalized product, and a personalized product is a predictable product. 

At Everi, we believe personalization is the utility of AI. Without personalization, you’re creating interesting products, but not effective products, especially not for growth markets. As more agents and consumer-facing products are developed, promising personalization and utility for customers around the world, we have an opportunity to get ahead of data quality issues and improve personalization by improving predictability across customer segments. 

Product Equity and Personalization

Personalization and Product Equity intersect when we start to consider the use cases of people who fall outside of the middle distribution of AI products. More often than not BIPOC, women, queer folx, immigrants, multilingual speakers, and economically disadvantaged people. As a result, these groups receive less access to AI products that work for them. That means less accurate diagnostic solutions, less delight with playful lens technology, less productivity through voice assistants etc.

Not every issue with product equity is about data, especially as it applies to algorithmic fairness. That said, we know that data is one of the major keys to unlocking personalization. There are ~8 billion people in the world, with less than 1 billion of those being in the United States and Europe. As we develop consumer AI for scale, it's important for companies to build products that predictably work for global markets. 

The Everi Opportunity 

Everi AI is a marketplace and curation platform for high quality datasets. Our mission is to help companies train and fine tune AI models for personalization and internationalization. We partner with companies to build an AI enabled future that works for Everi one, Everi where. 

If you'd like to partner or learn more about our work, contact us at [email protected] and subscribe to this newsletter for additional thoughts and updates!