Advice for DIY hidden in-wall speakers?

I would like to install my speakers inside my walls and was considering hiring a contractor to create recessed shelves that are in the same wall my television is mounted to, and rear speakers inside the walls opposite the television.

I own a Denon AVR reciever/amp along with a full 5.1 Klipsch speaker system. I was considering simply seating the speakers in the recessed shelf, and purchasing a thin fabric to cover the recess, and have the wiring for the speakers go down the inside of the wall, into the crawl space of my home, and converge back up into an electronics closet at the center of my home, where the Denon AVR will reside.

Does anyone have experience with this kind of setup? Any advice? Am I dumb?