It is sometimes be difficult to dress down, but it can be a necessity to avoid looking like a sore thumb in a situation where it is needed to blend in. This can be particularly challenging if you are in a casual work environment, but yourself is used to dressing formal.
Make More of Your Blazers
I consider the blazer one of the most versatile pieces you can have. It can keep you warm, it can add some flare and it can compliment the proportions of your body by for example adding some masculinity.

Layer your blazer with a sweater instead of shirt and tie

An open, less formal shirt and a pair of bottle green corduroy trousers fits well with this houndstooth blazer
Make Use of Your Ties!
I love ties, and it’s sad to see them becoming less common. When I first started wearing ties to the office at the beginning of my style journey, my colleagues would often ask "What’s the occasion?". But I never felt like a tie needed an occasion - I just thought it worked well with what I was wearing that day. Over time, people got used to seeing me in a tie, especially on Tie Tuesday - though I never managed to convince anyone else to join in.
If you put a tie on while wearing a suit and shirt, it will of course elevate your outfit to a quite formal level. But if you pair it with less formal garments, it can give you a chic and elegant look that makes your expression look well planned.

Using an overshirt can soften the impression compared to wearing a blazer

Wearing jeans together with a tie can be a relaxed and cool looking combination