From Craftsmanship to Algorithms: The Evolution of Personalisation
Tracing the Journey Across Time: Past, Present, and Future
Personalisation, a bit of history
You would be hard-pressed to find a dense treatment of this topic. A quick Google search will reveal a Wikipedia page ( not a top-ranked search result, by the way! ). Additionally, there are some scattered articles, mainly in form of blogs and papers from industry advisories & consultancies. While some of this content is pretty thorough, most of it is very superficial. The motivation for this post is to aggregate some key insights from these sources and add my own observations and notes.
Let’s start with a definition(my take on it, a bit different from the ‘wiki‘ definition):
Personalisation is a process of selecting, creating, communicating, and promoting a product or service that closely matches a customer’s needs and wants, at the point in time they wish to utilise it.
The Oxford English Dictionary tracks the origin of the word ‘Personalisation’ to around the 1860s (see here) . Interestingly, the usage trend (see graph below) in written English shows that it really picked up pace in the post-World War II (roughly the 1960s), which coincides with the boom in modern information age and proliferation of marketing and advertisement. More on that later.
Industrialisation, Mass Production & Marketing and Minimal Personalisation
Historically speaking (mostly within the bounds of Western written history ), in the pre-industrialised world, products were handcrafted with limited or no use of machines, using locally or regionally supplied materials, and marketed directly to local consumers. Local traders and storekeepers could address customers needs directly, often with personalised pricing and modifications. The loop from idea to production to marketing was very short but also limited in scale. One can still find examples of this today in some places such as small and local bespoke tailoring, catering, footwear setups.
The Western Industrial Revolution, starting in 18th century, brought about a major transition from pre-industrial hand production methods to mechanised production at scale. This massively improved the access to goods for the masses. However, one of the consequences was that goods were now produced in a more standardised form for the masses. Supply chains and demand shifted from local to nationwide and global.
Take Ford Model T as an example: these were produced in identical ( nearly ) form by the hundreds of thousands in the early 1900s. Henry Ford famously said “Customers could have any colour car they liked, as long as it was black”. This kind of mass-produced product had a net positive effect in improving consumers’ lives from a utilitarian point of view. And because these products immediately improved the consumers’ lives (and status), having identical products was not an issue… initially.
As time progressed into the tail end of the industrial age, the utility of mass-produced products started to be taken for granted, and there was a demand for more/different. Ironically, the same technologies (communication and transport tech like radio, telegram, rail and road networks) that helped proliferate and mass-market these products also made them undifferentiated. Everyone could easily compare their belongings to others, thanks to the widespread dissemination of information.
The reach of these mass-marketed products was massive, but at the cost of personalisation. Radio, TV, Newspapers all offered blunt instruments for personalised direct marketing to the right customers. For example, advertising slots during specific times of the day, advertising around important events like games, or running adverts on local radio and TV channels all made some inroads into the customer population, but were still very rough and inaccurate.
Direct mail and personalised letters were one way to directly market to the existing customers. This technique was pioneered by English pottery businessman Josiah Wedgewood, and later extensively used by Sears, Roebuck and Co in America with their direct mail product catalogue .
Systematic Use of Data in Personalisation
However, there was one major gap in the above approach: the information available for personalisation was extremely limited. At most, only the customer's name could be used to address them in the mail. This was still hugely successful, even with just use of customer’s name to personalise the letter!
This lack of key information was addressed in the next stage in personalised marketing evolution; the collection, organisation and analysis of customer & market data. One of the earliest example of this in modern times was George B Waldron at Mahin's advertising agency, who utilised tax registers, city directories, and census data to create customer groups for marketers to target ( in the early 1900s). This can be considered one of the very first example of customer segmentation.
This type of data driven approach really picked up pace in the mid to late 20th century, with numerous efforts aimed at developing a better understanding of customers & markets through data. This coincided very well with arrival of early electronic computers, which made number crunching easier. The enhanced ability to group, segment, and understand people helped narrow down marketing targets, thus facilitating better personalisation. However, the channels ( radio, TV and newspapers) used for their massive reach still lacked precision for the same reasons as mentioned before.
Until this point, people were only on the receiving end of mass media as consumer of the information. While the understanding of markets & customers was driven by data, the depth of this understanding remained shallow, because there were no scalable means for people to reveal themselves and their behaviours and preferences directly to the world.
Path to Deeper Insights on Users and Markets: The Impact of Personal Computers, the Internet and E-commerce
Things changed rapidly after the arrival of desktop publishing( thorough personal computers in the 1980s) and proliferation of internet (through internet search & browser technology) starting in the 1990s.
People could search, browse and publish on the internet using their personal computers, and every single action on internet could potentially produce signature of choices one makes and preferences they have.
Technologies such as cookies were developed to track users browsing activities. Concurrently, search engines like Google and AOL offered another door for understanding population trends through the analysis of search terms combined with location and time data. These developments offered unprecedented opportunities to gain insights into consumer interests and behaviours on a global scale.
During the early days of the internet, interactions were primarily with static content, mainly for information seeking and sharing purposes. This put an upper ceiling to understanding people and behaviours, as the most one could know about someone interacting with a static page was the category, tag or type of content it belonged to!
Around the same time, in mid to late 1990s, e-commerce took off. eBay’s predecessor AuctionWeb and Amazon came to the scene around 1995, accompanied by payment processor systems like PayPal sometime around 1998. I believe this was another major step towards potentially better and deeper understanding of people at scale, a feat previously unattainable with brick-and-mortar stores alone. As these e-commerce platform grew, so did their repositories of 1st-party data on consumer spending behaviours, offering insights on a much more granular product level, spanning nationwide and sometimes global markets.
Data-Driven Personalisation at Scale: Algorithmic Recommendations
All of this data in vast repositories is great, but there was no systematic, algorithmic approach to achieve direct 1:1 personalisation. This changed with the introduction of algorithmic recommendation system.
Amazon emerged as a pioneer (among others) in using the extensive 1st-party data they possessed to personalise interactions in a 1:1 manner. As early as 1999, they began implementing product recommendations based on a customer's past purchases and interactions within their online store. Pretty sure most of us have interacted with their well-known “People who bought this item also bought these” feature, or its many variations. Its a great example of a data driven 1:1 personalisation, presenting customers with highly personalised product suggestions right on the edge, very close to where the customer is in time and mental space ( right in the middle of process of buying something). Some more interesting insights into their recommendation systems can be found here.
Netflix and Spotify are other examples of massive-scale 1:1 personalisation within the realm of media products. Despite the differences in products, the core concept remains consistent: a large product catalogue boiled down to a small set of products which are highly aligned with the customer’s individual taste, preferences and needs.
Path to Deeper Insights on Users and Markets, with the Arrival of Social Media, Mobile Devices and Interactive Internet
Moving on, the first decade of 21st century saw the rise of social media networks, internet enabled mobile devices and increasingly interactive interfaces (web and mobile apps).. This was yet another step towards people sharing more & richer signals from their lives. For instance, mobile devices with GPS immediately opened up precise location data, while social media networks encouraged users to share their lives through text, photos and videos. Additionally, the concept of a “connection” or “friend” in social media enabled these platforms to build detailed social graphs capturing potentially deep signal on how people are connected together.
This era led to explosive growth in the amount of collected and processed data as hundreds of millions of people connected through internet and mobile devices to each other and to organisations. But what advancements in personalisation of products and experiences emerged during this time. Did all this data from the last two decades has actually lead to better personalisation? what are some examples of truly impressive product personalisation in recent times?
The simple answer is that there haven't been many significant advancements in product and service personalisation. While personalisation has become more widespread, and the effectiveness of retail product recommendations may have improved, no groundbreaking innovations have emerged. Additionally, there is much less differentiation in the extent and richness of personalisation offered by a massive retailer like Amazon compared to a smaller, tech-savvy online retailer. It appears we have been in a plateau from the 2010s leading into the 2020s.
Artificial Intelligence and the Personalisation Paradigm Shift
There is hope for the next step in the evolution of personalisation. The resurgence of artificial intelligence technologies, particularly large language models (text-only and multi-modal) emerging in the 2020s, has the potential to be the catalyst for this change. These models are built on the vast amounts of data produced by humanity over the last two decades and earlier. In their current form, these models demonstrate glimpses of "general" intelligence capabilities (though not close to AGI), such as understanding and generating media, identifying and understanding objects in images and sounds, and generating code and instructions.
I believe AI could play multiple roles in the personalisation space. One of the first roles could be that of a "Reducer," a tool to simplify, streamline, and accelerate existing personalisation paradigms. For example, in personalised communication, a well-tuned AI agent can write emails and letters specifically tailored in style and substance for an individual, faster, cheaper, and with less process complexity compared to the traditional methods of generating hyper-personalised letters!
Another interesting area where AI could add value to personalisation is in enriching the data used to describe and understand customers, products, and markets. Large language models can be considered as representations of "world knowledge," capturing what works and what doesn't (though hallucinations remain a significant issue in these recent models). This capability means that AI can not only interpret the data collected for a user or a process but also extend and enhance it. For example, imagine a travel agency aiming to personalise the booking experience for customers based on a revealed destination city. An AI system could extrapolate a great deal from this simple information (a customer X is interested in destination Y). It could suggest what to pack, which financial instruments (credit cards, travel checks, cash, etc.) would be best suited, and more. AI essentially fills in the gaps, reducing the need to maintain an overly complex set of data about customers and their various needs related to a product or service.
Finally, my thoughts on the ultimate future for personalisation in the age of AI.
Reflecting on the evolution of personalisation up to this point in the "information age," it has become widely adopted and highly data-driven, with the latest technologies used to streamline and enhance personalisation capabilities. However, the basic paradigm remains the same: the direction is still from producers (and marketers) to users. Companies decide what data to collect, formulate and execute personalised marketing strategies, develop features they believe will be useful for their customers, set prices, and so on.
.
I believe AI would ultimately allow a “"role upgrade” for the users, where their own needs, wants and preferences could be translated directly into personalised products and experiences, effectively making them the producers!
Consider an example: if someone wants a highly personalised money management tool to help control their budget and daily spending, they currently have to search the market for an existing product. Whatever they choose will come with pre-defined features that may adapt to their needs over time, but they remain largely dependent on the decisions made by the tool's producer.
An alternative would be a highly "bespoke" money management tool created specifically for the individual. This unique application would be accessible only to them, allowing them to tune, change, and customise it to their specific needs. Realising this vision requires AI to mature to a point where it can take rough requirements from a user and create a comprehensive, end-to-end experience. While we are not there yet, this potential future highlights the transformative power of AI in personalisation.
If you’ve found this post interesting, please subscribe, share and leave a comment.