Part of an overhaul in its search infrastructure
Micro-blogging site Twitter has launched personalised searching on its site to filter accurate Tweets, images, and videos.
The company said that it used two tools — Blender and Earlybird at the core — so that its infrastructure could support two major features: relevance-filtering of search results and the identification of relevant images and photos.
Earlybird is a real-time reverse index used for real-time search. Twitter said that Blender improved search latencies by 3x, and gave it 10x throughput.
Today, we are indexing an average of 2,200 TPS while serving 18,000 QPS (1.6B queries per day).
Since the acquisition of Summize in 2008, Twitter has invested heavily in search. Its search team has expanded from three to 15 engineers.
Twitter said that it has scaled its real-time search engine by two orders of magnitude.
About personalised search, Twitter said that it is most powerful when users personalise it by choosing interesting accounts to follow.
It said, "Our ranking function accesses the social graph and uses knowledge about the relationship between the searcher and the author of a Tweet during ranking."
Recently, US President Barack Obama selected Twitter’s chief Dick Costolo among other Web leaders to join his National Security Telecommunications Advisory Committee.