Fighting Listing Fatigue
Clustering duplicates on Divar, and the evidence that ended it
Year
'23
Company
Divar
Role
Product Designer
One in ten listings in a property search was an apartment you'd already seen
Divar is Iran’s largest classifieds platform. Its real-estate category carries about 4.8 million serious home-seekers a month, roughly 60% of the national property market, which makes it, for most Iranians, the way you find a home.
By an internal hand-labelled review of listings in spring 2023, roughly 10% of listings were duplicates: the same property advertised simultaneously by multiple competing agents. We needed to get it to under 5%.
Only one kind of duplicate was actually mine to solve
Not all duplicates are the same problem. Divar’s moderation already caught the first two of three cases automatically:
Duplicate Type
What happens today
Same person, re-posting
Posting rejected by the moderation bot
Same agency, different agent
Posting rejected
Different agencies, same property
Survives moderation, clutters the feed
The bet: group them, don't remove them
Iran's property market has no exclusivity: any agent can work any listing. Some of them are genuinely working the property. Others copied the listing off Divar and changed a few details. Nothing visible separates the two.
So suppressing the extras won’t sort legitimate from opportunistic.
Design Direction: One Property, One Card, All Agents
After a team brainstorm I proposed adapting an approach used by Hungarian portal ingatlan.com: collapse the duplicates under a single representative card. The user sees one clean entry per property and every agent keeps their listing.
figure 01 - ingatlan.com hides its duplicates in an expandable card
The representative slot holds serious weight in design. Whoever holds it gets the user’s click, so the rule that assigns it is the fairness policy; and it's the thing that changed most across the rollout.
Risk I flagged early
If agents worked out what made two listings match, they could publish deliberately wrong details to stay out of a cluster, trading listing accuracy for visibility. That prediction shaped the fraud work below.
The clustering model: detecting signals and building the cluster
Working with data science and data analysis, I specified what the pipeline had to produce: what counts as the same property, how a cluster surfaces, which listing represents it. Data science owned the model itself and set the precision and recall that made those rules safe to ship. Before anything reached users we backtested its output against two weeks of hand-labelled listings from the pilot neighborhood.
figure 02 - Visual Similarity and Metadata match results in forming a cluster with a single representative listing
Fraud guard: the bounce-back heuristic
To catch agents gaming the system, I proposed adding a behavioral signal to the pipeline: a home-seeker who returns from contacting an agent in under 25 seconds probably never connected. Repeated short bounce-backs on one listing flag it for human review as a probable fake.
Three Iterations: what changed, and what forced it
Impact: target reached but …
Duplicates fell from 10% to 4% of listings in a result list. That clears the target, and it is a real improvement. It is also a measure of what the model did, not of what a home-seeker felt.
Before clustering home-seekers were spending the same amount of attention as after clustering; they were just spending some of it twice on the same property. However now, they would spend it on three more distinct ones.
figure 06 - average total listing views is the same before and after clustering, however, now the attention is spent on +3 distinct listings
Where the leads went
Concentrating home-seekers onto one card per property does something to the agents on the other side of it:
figure 07 - 80% of the clusters contain 2-4 listings. The representative listing takes ~3 in 4 leads in these clusters and its grip loosens only slightly as clusters grow while each individual runner-up keeps getting smaller.
A clustered listing loses roughly 40% of its leads, concentrated on the non-representative ones, which is where the dissatisfaction came from.
Did agents actually game clustering?
I had predicted they would publish wrong details to escape clustering. We went looking, and what we found was not what I expected.
The boost-abuse theory came back null. If clustering pushed agents to pay their way out, boost usage should have diverged between clustered and unclustered listings. It didn't: about 50% of clustered listings used a boost at least once against 48% of unclustered ones, with an average of 1.5–1.7 boosts in both groups. No meaningful difference.
Half right. I'd predicted agents would falsify details to escape a cluster. They did, and the field they chose was location: registering a listing in the adjacent neighborhood took it outside the match. We responded by opening whole districts at once instead of rolling out neighborhood by neighborhood. Whether agents falsified other fields at scale was never measured here, I handed that to a project already underway on listing quality and fake information, which owned the data.
Two things clustering made possible
Neither of these was a goal. Both came out of the structure rather than the feature.
A rule could be relaxed. Divar had previously blocked a single agency from publishing more than one listing for the same property. Once duplicates collapsed visually, that restriction stopped being necessary. Opening it up gave agents inside one agency room to compete rather than queue.
Properties got better described. Grouping several agents' listings for one property made the information about it more complete; a side effect we observed but never measured.
What I Learned: we fixed the problem that annoyed people least
After the second wave of agent complaints, I ran a perception survey (908 participants, 3.2% margin of error) to re-check the underlying assumption that duplicates were the primary pain point:
figure 08 - Users perceive encountering duplicate, fake, and unavailable listings at broadly similar rates.
figure 09 - Users report fake listings more than twice as annoying
Although users reported encountering duplicates and fake ads in an almost same measure, they reported fake listings far more annoying. (73.8% for fakes vs. 32.7% for duplicates)
The OKR had been scoped around all three listing-quality problems at once (fake, unavailable, and duplicate listings) without weighting them by how much harm each one actually did. Most of engineering effort landed on the one that mattered least.
So we took it down. The benefit was real but small: fewer repeats for home-seekers who, by their own account, weren't very bothered by them. The costs kept growing, agent complaints, and fake listings on the rise. After three months, we removed clustering.
Scoping this again, I would rank perceived harm per problem type first and build against that order, and I would run the perception survey before writing the OKR rather than after the complaints arrived.
The limit we already knew about
Telling a real duplicate from a fake dressed up to survive clustering needs an owner-side source of truth; deed, exact location, postal code. We were pursuing one, but it ran through the government and the real-estate union, where trust-building moves on its own schedule. So we built behavioral proxies instead: the bounce-back heuristic, the quality nudge.
I'd make the same call again; waiting indefinitely wasn't an option. What I'd change is the framing. This was a temporary measure against a pending data source, and saying so in the goal would have set expectations a proxy could actually meet.
TL;DR
Divar's real-estate feed had a duplicate problem: the same property, listed by competing agents, cluttering the first page of feed for 1 in 5 home-seekers.
The bet
Group the duplicates, don't remove them. Iran's property market has no agent exclusivity, so suppressing the extras wouldn't sort legitimate listings from opportunistic ones. It would just reward whoever posted fastest.
The system
An ML clustering pipeline, a rule deciding which listing represents the cluster, a loss-aversion notification for the agents it affected, and a behavioral fraud heuristic. Built as an interim layer while an owner-side verification source was still stuck in negotiation.
The result
Duplicates fell from 10% to 4% of a result list, and home-seekers opened three more distinct properties for the same number of clicks. Wasted attention converted into real supply.
The takeaway
It worked, and it was the wrong problem. A 908-person survey showed duplicates bothered home-seekers least of the three listing-quality problems, and as agent complaints and fake listings grew, the feature was withdrawn. Next time I'd rank problems by harm before building anything.