Optimized Dynamic Point Cloud Compression

Project overview

MSCA-IF-EF-ST - Standard EF

Grant Agreement ID: 836192

Start date: 23 November 2020

End date: 22 November 2021

Funded under: H2020-EU.1.3.2.

Overall budget: € 112,466.88

Objectives: 3D point clouds are receiving increased attention due to their potential for many important applications, such as real-time 3D immersive telepresence. Compared to traditional video technology, 3D point cloud systems allow free viewpoint rendering, as well as mixing of natural and synthetic objects. However, this improved user experience comes at the cost of increased storage and bandwidth requirements as point clouds are typically represented by the geometry and colour of millions up to billions of 3D points. For this reason, major efforts are being made to develop efficient point cloud compression schemes. The task, however, is very challenging due to the irregular structure of point clouds. To standardize these efforts, the Moving Picture Experts Group (MPEG) launched in January 2017 a call for proposals for 3D point cloud compression technology. In October 2017, the responses were evaluated and the first test model for lossy compression of dynamic point clouds (TMC2) was established. This test model defines a first “common core” algorithm for collaborative work towards the final standard. The aim of OPT-PCC is to contribute to these efforts by developing algorithms that optimize the rate-distortion performance of the test model. OPT-PCC’s objectives are to:

O1: build analytical models that accurately describe the effect of the geometry and colour quantization of a 3D point cloud on the bit rate and distortion;

O2: develop fast search algorithms that optimize the allocation of the available bit budget between the geometry information and colour information;

O3: implement a compression scheme for dynamic 3D point clouds that outperforms the state-of-the-art in terms of rate-distortion performance. The target is to reduce the bit rate by at least 20% for the same reconstruction quality;

O4: provide multi-disciplinary training to the researcher in algorithm design, metaheuristic optimisation, computer graphics, and leadership and management skills.

Participants

Researcher: Prof. Hui Yuan (hui.yuan@dmu.ac.uk, huiyuan@sdu.edu.cn)

Hui Yuan received the B.E. and Ph.D. degrees in telecommunication engineering from Xidian University, Xi’an, China, in 2006 and 2011, respectively. In April 2011, he joined Shandong University, Jinan, China, where he was a Lecturer from April 2011 to December 2014, Associate Professor from January 2015 to October 2016 and has been a Full Professor since September 2016. From January 2013 to December 2014 and November 2017 to February 2018, he was a Postdoctoral Fellow (funded by the Hong Kong Scholar Project) and a Research Fellow, respectively, with the Department of Computer Science, City University of Hong Kong. Since November 2020 he has been a Marie Curie Fellow at De Montfort University. His current research interests include video/image/immersive media processing, compression, adaptive streaming, and computer vision.

Supervisor: Prof. Raouf Hamzaoui (rhamzaoui@dmu.ac.uk)

Raouf Hamzaoui received the M.Sc. degree in mathematics from the University of Montreal, Montral, QC, Canada, in 1993, the Dr.rer.nat. degree from the University of Freiburg, Freiburg im Breisgau, Germany, in 1997, and the Habilitation degree in computer science from the University of Konstanz, Konstanz, Germany, in 2004. He was an Assistant Professor with the Department of Computer Science, University of Leipzig, Leipzig, Germany, and with the Department of Computer and Information Science, University of Konstanz. In 2006, he joined De Montfort University, Leicester, U.K., where he is a Professor in Media Technology. His research interests include image and video coding, multimedia communication systems, error control systems, and machine learning.

Supervisor: Prof. Shengxiang Yang (syang@dmu.ac.uk)

Shengxiang Yang is Professor of Computational Intelligence and Director of the Centre of Computational Intelligence (CCI), De Montfort University. Before joining the CCI in July 2012, he worked at Brunel University, University of Leicester, and King's College London as a Senior Lecturer, Lecturer, and Post-doctoral Research Associate, respectively. Shengxiang's main research interests lie in evolutionary computation. He is particularly active in the area of evolutionary computation in dynamic and uncertain environments. Shengxiang has also published on the application of evolutionary computation in communication networks, logistics, transportation systems, and manufacturing systems, etc.

Supervisor: Associate Prof. Ferrante Neri (Ferrante.Neri@nottingham.ac.uk)

Ferrante Neri is an associate professor with the School of Computer Science at the University of Nottingham and member of the Computational Optimisation and Learning (COL) Research Group. He is a scholar in Optimisation and Computational Modelling. He is also the author of Linear Algebra for Computational Sciences and Engineering.

Secondment supervisor: Prof. Peter Eisert (peter.eisert@hhi.fraunhofer.de)

Peter Eisert is Head of Vision & Imaging Technologies Department at Fraunhofer HHI and Chair of Visual Computing at Humboldt University. He has published more than 150 conference and journal papers and obtained several awards. He is Associate Editor of the International Journal of Image and Video Processing and on the Editorial Board of the Journal of Visual Communication and Image Representation. He is leading numerous industrial as well as national and international research projects. His research interests include 3D image analysis and synthesis, face processing, machine learning, computer vision, and computer graphics.

Deliverables

D0: Data Management Plan (M6)

D1: Coder source code (M10)

D2: Report on analytical models (M5)

D3: Report on the bit allocation solution (M9)

D4: Report on experimental results (M11)

D5.1: Patent application (M8)

D5.2: Proposal to the MPEG 3DG standardization committee (M12)

D6: Career Development Plan (M12)

Related publications

1. Q. Liu, H. Yuan, J. Hou, H. Liu, R. Hamzaoui, "Model-based encoding parameter optimization for 3D point cloud compression," in: Proc. APSIPA ASC 2018, 10th Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, Honolulu, Nov. 2018.

2. Q. Liu, H. Yuan, J. Hou, R. Hamzaoui, H. Su, "Model-based joint bit allocation between geometry and color for video-based 3d point cloud compression," IEEE Transactions on Multimedia, DOI: 10.1109/TMM.2020.3023294.

3. Q. Liu, H. Yuan, R. Hamzaoui, H. Su, "Coarse to fine rate control for region-based 3D point cloud compression," in: Proc. IEEE International Conference on Multimedia & Expo Workshops (ICMEW), London, Jul. 2020.

4. Q. Liu, H. Yuan, R. Hamzaoui, H. Su, J. Hou, H. Yuang, Reduced reference perceptual quality model with application to rate control for video-based point cloud compression, IEEE Transactions on Image Processing, vol. 30, pp. 6623-6636, 2021, doi: 10.1109/TIP.2021.3096060.

5. H. Yuan, R. Hamzaoui, F. Neri, S. Yang, Model-based rate-distortion optimized video-based point cloud compression with differential evolution, in: Proc. 11th International Conference on Image and Graphics (ICIG 2021), Haikou, China, August 2021.

6. H. Yuan, R. Hamzaoui, F. Neri, S. Yang, T. Wang, Global rate-distortion optimization of video-based point cloud compression with differential evolution, in: Proc. 23rd International Workshop on Multimedia Signal Processing (IEEE MMSP 2021), Tampere, Oct. 2021.

Activities

ICME 2020 Workshop on 3D Point Cloud Processing, Analysis, Compression, and Communication (PC-PACC), July 2020

Online seminar: Hui Yuan, "Model-based Joint Bit Allocation between Geometry and Color for V-PCC", 16 December 2020.

3D Point Cloud Processing, Analysis, and Communication Workshop at 11th International Conference on Image and Graphics (ICIG 2021), Haikou, 6-8 August 2021. Postponed to 27 November 2021 (15:30-17:40)

Point Cloud Compression and Rendering, Nuffield Research Placement, Summer 2021

Research seminar at University of Nottingham, 10 November 2021

Links

Cordis website

Twitter

MPEG-PCC Project