Modular systems are not practical from a manufacturing or design standpoint. First, for a modular system to work you'd have to make a spot to accommodate the largest supported format. Secondly, how many card types should a given camera support? Thirdly, modular systems create extra power drain, (every connector you add does this) which could effect battery life. Fourthly, SD, CF and XQD all use different interconnects (what goes between the reader and the source), which would make that somewhat difficult. I believe SD and CF are based on IDE connections and XQD is a PCI-E or SATA type connection, which is why the theoretical top speed is much faster than the former two.
I know many outspoken people seem to think that we need more modular parts in cameras, but I think that would only hinder design. We need to move towards one standard format for all camera systems. I think XQD brings the best of SD and CF under one roof, size, speed and build quality. Right now they are more expensive, but if all cameras used XQD (and lower performing cards came out) then it would become cheap fast.