Curiosity leads to starting small talks about you, what you haven’t done and what you have done and not everybody is happy for you when they watch you prospering.
This is the main reason why you should not involve people who know you in your business. Friends, family, life partners and you name it; they are the same ones who don’t take anything you do seriously just like they always say that a prophet is not without honour except in his own home town.
These rumours will either build or break you depending on how you choose to use or apply it because somewhere somehow, it has to apply either positively or negatively. I am personally a kind of a person who takes criticism positively and use it to my advantage because if you come to think of it, unless they talk about it you will never know what they actually think of, which will hinder your growth at large.