Given the point you beautifully put up there, honesty as a 'virtue' deserves far more serious attention (both academic and social) than what it is getting at the present time. To answer the latter part of your question, I definitely have wondered why we lie all the time, and I have found it very interesting that to think about honesty de facto comes with thinking about lying. Lying is such an intriguing behaviour if you come to think about it.
Being trained in biology, I always default to thinking of emotions and behaviours from an evolutionary perspective. As theorized by some evolutionary biologists, we might have been hardwired for some degree of dishonesty and intentional deceit. In their view, a partial-honesty model of communication in animals is the one that is the most evolutionarily stable, and it works when the stakes of lying/dishonesty are both high and low. This model has also been validated by computer simulations . Hence, talking from a biological standpoint, we lie often because we cannot help it. While the game theory may tell us that in the long run, we would be better off if all of us told as much less lies to each other as possible, it doesn't omit the utility of occasional lies. Hence, it would be wise to posit that it is better not to lie until and unless we need to do so for life ( or any other serious output).
However, that doesn't solve anything. I lie for many reasons. I lie so that I can make other people form untrue beliefs about myself or any other person/object/idea/incident. Borrowing the ideas of my favourite Philosopher Sam Harris, I either lie through commission, or I lie through omission. In the first case, I am active in my intent to deceive. I am lying by commission when I falsify my skills and credentials in my CV with an aim of landing a job. On the other hand, lying by omission is more passive. When I deliberately miss out to list the job from which I got fired on my CV, I am lying by omission. While both the lies are condemnable, Harris says, the former is ethically more violent .
Another interesting point about honesty/lying is that we seem to be confused, or at least negligent about the difference between honesty to the self vs honesty to the others. While being always honest with others might always not be pragmatic and at times even counterproductive, being honest to the self is almost always revealing and empowering. Only when we are the truest to ourselves can we assess the outer world more clearly and objectively and tune our behaviour for the best output in any given area.
It would be utopian to say that we must strive not to lie. But we can always aim to build societies where we need to lie as less possible. For this, there should be more rigorous philosophical research. We can dig more into the mathematical models of social-setting game theory revolving around (dis)honest communications. Not all of us lie in a similar manner. If we use proper theories and tools, maybe it is possible for us to determine the domains in our personal/social/emotional/psychological lives where we lie more often. Once we distinguish such lie-prone domains, then we could move on to demarcate the issues/ideas/incidents we tend to lie more about as individuals/groups. If we could identify these multi-level parameters involved in (dis)honesty, maybe we could better learn how to cultivate rather than just celebrate honesty as a virtue.