In most instances you shouldn’t need to soundproof your chimney. This is because chimney breasts are usually made from a double layer of very dense fire bricks. This means the chimney breast naturally has more mass than the alcoves and is therefore better able to block sound. Learn more about soundproofing a chimney.
If you are unsure if the chimney breast needs soundproofing, do an ear test by putting your ear to the alcoves and then to the chimney when you can hear the unwanted noise and compare the difference.