No it'll be fine. What you want to do is connect the pipes up so they run the same way as if the box was still there but replace the box with some copper or plastic pipe. Does that make sense?
It won't do any damage as it's just to reduce noise and if you run a shrick inlet manifold you actually have to remove this damper from the system.
nick