Deep learning-based Point Cloud Representation Deep-PCR

Project: Deep learning-based Point Cloud Representation

Acronym: Deep-PCR

Main Objective:
The scope of this project is the design, development, implementation and assessment of a DL-based static PC compressed representation targeting both decoding and enhancement consumptions. This will pioneer DL-based solutions that simultaneously accommodate efficient fidelity-to-original PC coding, with compression efficiency improvements over available solutions, and effective enhanced-from-original PC processing using the same compressed domain stream. The expected outcomes are DL-based PC coding and enhancement solutions with the following features: i) efficient PC geometry and color codecs, with and without scalability capabilities, providing the highest fidelity to the original PC for the available rate; and ii) effective PC geometry and color denoising and super-resolution processors, generating enhanced PCs using the same compressed domain stream as the fidelity codec. These outputs should offer the best user experience for multiple application domains, considering different fidelity-to-original requirements. The project will advance the knowledge in the domain and disseminate its achievements with top international conference and journal publications.

Reference: PTDC/EEI-COM/1125/2021

Funding: FCT

Approval Date: 15-10-2021

Start Date: 01-01-2022

End Date: 31-12-2024

Team: Fernando Manuel Bernardo Pereira, Nuno Miguel Morais Rodrigues, André Filipe Rodrigues Guarda, Abdelrahman Seleem Mohamed Seleem, Mohammadreza Ghafari

Groups: Multimedia Signal Processing – Lx, Multimedia Signal Processing – Lr

Partners: IT

Local Coordinator: Fernando Manuel Bernardo Pereira

United Nations Strategic Development Goal - Industry, Innovation and Infrastructure

Associated Publications

3Papers in Journals
A. Seleem, A. Guarda, Nuno M. M. Rodrigues, F. Pereira, Adaptive bridge model for compressed domain point cloud classification, Eurasip Journal on Image and Video Processing, Vol. 2024, No. 13, pp. - - -, June, 2024,
| Abstract | BibTex
A. Seleem, A. Guarda, Nuno M. M. Rodrigues, F. Pereira, Deep Learning-based Compressed Domain Multimedia for Man and Machine: A Taxonomy and Application to Point Cloud Classification, IEEE Access, Vol. 11, No. -, pp. 128979 - 128997, November, 2023,
| Abstract | Full text (PDF 1 MB) | BibTex
A. Guarda, M. Ruivo, L. Coelho, A. Seleem, Nuno M. M. Rodrigues, F. Pereira, Deep Learning-Based Point Cloud Coding and Super-Resolution: a Joint Geometry and Color Approach, IEEE Transactions on Multimedia, Vol., No., pp. 1 - 13, November, 2023,
| Abstract | BibTex

12Papers in Conferences
M. Ghafari, A. Guarda, Nuno M. M. Rodrigues, F. Pereira, Learning-Based Point Cloud Decoding with Independent and Scalable Reduced Complexity, IEEE International Conference on Image Processing ICIP, Abu Dhabi, United Arab Emirates, October, 2024,
| Abstract | Full text (PDF 604 KBs) | BibTex
D. Mari, A. Guarda, Nuno M. M. Rodrigues, S. Milani, F. Pereira, Point Cloud Geometry Scalable Coding with a Quality-Conditioned Latents Probability Estimator, IEEE International Conference on Image Processing ICIP, Abu Dhabi, United Arab Emirates, October, 2024,
| Abstract | Full text (PDF 697 KBs) | BibTex
M. Ghafari, A. Guarda, Nuno M. M. Rodrigues, F. Pereira, Deep Learning-based Point Cloud Geometry Coding with Attention Models, IEEE International Symposium on Multimedia (ISM), Laguna Hills, CA, United States, December, 2023,
| Abstract | BibTex
A. Guarda, Nuno M. M. Rodrigues, F. Pereira, Point Cloud Geometry and Color Coding in a Learning-Based Ecosystem for JPEG Coding Standards, IEEE International Conference on Image Processing ICIP, Kuala Lumpur, Malaysia, October, 2023,
| Abstract | Full text (PDF 577 KBs) | BibTex
M. Ruivo, A. Guarda, F. Pereira, Learning-Based Rate Control for Learning-Based Point Cloud Geometry Coding, IEEE International Conference on Image Processing ICIP, Kuala Lumpur, Malaysia, October, 2023,
| Abstract | BibTex
A. Seleem, A. Guarda, Nuno M. M. Rodrigues, F. Pereira, Deep Learning-Based Compressed Domain Point Cloud Classification, IEEE International Conference on Image Processing ICIP, Kuala Lumpur, Malaysia, October, 2023,
| Abstract | BibTex
M. Ruivo, A. Guarda, F. Pereira, Learning-based Point Cloud Geometry Coding Rate Control, IEEE Data Compression Conference - DCC, Snowbird, UT, United States, March, 2023,
| Abstract | BibTex
L. Coelho, A. Guarda, F. Pereira, Deep Learning-based Point Cloud Joint Geometry and Color Coding: Designing a Perceptually-Driven Differentiable Training Distortion Metric, IEEE International Conference on Multimedia Big Data BigMM, Naples, Italy, December, 2022,
| Abstract | Full text (PDF 890 KBs) | BibTex
A. Seleem, A. Guarda, Nuno M. M. Rodrigues, F. Pereira, Impact of Conventional and Deep Learning-based Point Cloud Geometry Coding on Deep Learning-based Classification Performance, IEEE International Symposium on Multimedia (ISM), Naples, Italy, December, 2022,
| Abstract | Full text (PDF 1006 KBs) | BibTex
F. Pereira, Deep Learning-based Extended Reality: Making Humans and Machines Speak the Same Visual Language, ACM - Multimedia, Lisbon, Portugal, October, 2022,
| Abstract | BibTex
F. Pereira, Deep Learning-based Point Cloud Coding for Immersive Experiences, ACM - Multimedia, Lisbon, Portugal, Vol., pp. -, October, 2022,
| Abstract | BibTex
M. Ruivo, A. Guarda, F. Pereira, Double-Deep Learning-Based Point Cloud Geometry Coding with Adaptive Super-Resolution, IEEE European Workshop on Visual Information Processing - EUVIP, Lisbon, Portugal, September, 2022,
| Abstract | Full text (PDF 568 KBs) | BibTex

Project / Deep-PCR

Project: Deep learning-based Point Cloud Representation