Advantage/Disadvantage should only apply to combat actions (attack, dodge, block, and shoot). We had even decided that spellcasting only got 1 die for a singular action and was at disadvantage for a primary action (we may have not been playing with this, but we should be). All other actions need a talent or item to gain an advantage like danger sense.
I definitely would not want actions like tracking, hearing checks, or barter to automatically get advantage. You could even argue that shoot should not get the advantage bonus for a singular action.