This paper has presented a novel algorithm for searching for rare resources in unstructured P2P networks. Existing protocols such as Flooding and Random Walk can effectively locate popular resources while they are limited by very low hit rate for rare resources. For users, the utility of getting rare resources is no less than that of getting popular ones. Thus high hit rate for rare resources will improve the system’s efficiency. According to different capability of different nodes, this paper explores a three-grade balanced tree to distribute index replicas of rare resources uniformly across a very small portion of nodes, which is easily deployable and lightweight in overhead. Both mathematical analysis and simulation results show that it improves the hit rate for rare resources from less than 1% to more than 90%.