Often the very best jokes are laden with insight, and one of my favorite such jokes belongs to Demetri Martin, who says, “It’s interesting that ‘cologne’ rhymes with ‘alone’.” Zing!

The word cologne evokes images of greasy dudes in Ed Hardy t-shirts, teenage boys with more scents than sense, and clueless divorcees getting back into the “dating scene.” This was my presumption when Fashionista asked me to look into the state of men’s cologne, yet once I ventured down this particularly pungent little rabbit hole, I realized just how misguided that presumption was. Beginning with the name.

“Cologne is a marketing word that Americans have been taught to use,” says Chandler Burr, perfume critic for the New York Times. “The word ‘cologne’ in the U.S. means nothing. You should just call it ‘scent’.”

So what’s cologne? “Cologne is a very specific recipe,” Burr adds. “It’s a citrus with an aromatic effect; it’s basically lemon, bitter orange, grapefruit, bergamot, and then maybe nutmeg or cinnamon added, maybe some other spice.”

What we’re looking for then is men’s “scent,” or “fragrance,” which unlike cologne (perfumes based in alcohol were not invented until 1370, in Hungary), has been around as long as civilization.

As for where men’s fragrances are now, there’s no one answer: Sales are both up and down—and like the scents themselves, the devil is in the details. “The fragrance market has been in decline for several years,” says Karen Grant, Vice President and Senior Industry Analyst for Beauty at National Purchase Diary, a group that tracks department store sales.

“Men’s fragrances were picking up around the time of the early metrosexual trend: 2001 – 2002. Then we saw a dropoff…. It’s been in decline ever since.” Throughout the 2000s, a celebrity would pop up with a fragrance from time to time, Diddy for instance, which would help drive sales. But these surges were ephemeral, and now the celebrity trend has faded completely. Last year, the sales of high-end men’s fragrances (these are the scents sold at Sephora and in the big department stores, not, say, Walmart) hovered around $779 million, a 10% drop from 2008, which was 8% less than 2007.

But just because the big stores are selling less, doesn’t mean people aren’t buying fragrances. “The one area of the market where we’re seeing the most stability is in what we would call these ‘niche’ fragrances. Where it’s very special. Those could be fragrances at Barneys or Saks, or even the higher end.”

While the big market, designer brand fragrances (“commercial…shit” Burr calls it), like Acqua di Gio, are still the bestsellers, they have also taken the biggest hit. They’re losing younger customers to deodorant sprays, like Axe, and older more sophisticated clientele to the niche brands like Le Labo and Frederic Malle.

“We don’t sell much of Acqua di Gio. We don’t sell a lot of Tom Ford,” says Bettina O’Neill, VP of Cosmetics and Fragrances at Barneys New York (who, in fact, only carries the former as part of an arrangement with the Armani brand). “We find our customers don’t want to smell like anyone else. More so men than women.”