One of the most important objectives of Identity Management (IdM) Systems is to provide end user privacy. However, these concepts rarely extend beyond the application layer. In the IST SWIFT project a special attention is given to cross-layer Identity Management support, and in this paper we show why applying only IdM solutions is insufficient to preserve user privacy if network mechanisms are not considered. We present a solution to retain user privacy by using network pseudonyms closely coordinated with the IdM framework proposed by the SWIFT project. We include these concepts in the IdM framework and present the necessary architecture and functional mechanisms required to support the privacy extensions.