Sometime back I was face with the above question. What is more important – a feature that the product manager deems is super important or the algorithm which is used behind the screens which is directly affected by the inclusion of the feature.

I will give you an example. Our product has the functionality to compare two ideas or “things”. This is called as pairwise voting. It is much like hot or not for ideas. Traditionally hot or not kind of voting does not contain any other option to muddy the voting results. However in our case the product manager thought it was important to have a “neither” button in which the user does not select either of the options for whatever reason. It is reasonable to assume that the “neither” button us important because there may truly be instances when the user does not like either of the options. However, there is a twist in the situation. The votes collected from pairwise voting are used to compute the score to rank ideas on the “top scoring ideas” leaderboard. As long as up/down/star votes are being collected the algorithm to compute the score and rank is pretty staright forward. However, when the “neither” votes are considered in the equation the computation of the score is not so straight forward. “Neither” votes can skew the results greatly and infact we observed that users began to use it as a pseudo gaming mecahnism by skipping until they saw the ideas they wanted to see.

So after this experience I began to wonder what is more important – a feature like “neither button” or the integrity of the leaderboard ranking which uses an algorithm? Of course there are ways to get around things like these, for example, by keeping the button intact but not taking into consideration the vote from a “neither” click.