A method for estimating response relevance with respect to a received query includes receiving a set of user feedback items, a set of historical feedback data, and a set of context data, creating a user profile model according to the set of historical feedback data, wherein the user profile model indicates a weighting attribute based on the set of historical feedback data, weighting the set of user feedback items according to the created user profile model, creating a response relevance estimation model based on the weighted set of user feedback items, the received set of context data, and the received set of historical feedback data, and ranking one or more responses according to the created response relevance estimation model. The method may further include adjusting the user profile model and the response relevance estimation model responsive to receiving additional data.