Hardware Implementation of Hyperspectral Data Compression
The invention comprises a real-time wideband compressor system for multi-dimensional data that is composed of parallel compression engines and a central high-speed network switch. The CSA developed this hardware system for on-board, real-time compression of hyperspectral data prior to transmission from spacecraft to the Earth.
The invention includes two hyperspectral data compression techniques for use on board a satellite, for real-time data compression. These operate on the 2-D focal plane frame of hyperspectral data, the cross-track line of ground samples (pixels), each with its own spectral vector, sensed at a given moment. The two data compression techniques are based on innovative hyperspectral data compression techniques that were previously developed for software implementation: Successive Approximation Multi-Stage Vector Quantization (SAMVQ) and Hierarchical Self-Organizing Cluster Vector Quantization (HSOCVQ).
A compression engine is a stand-alone autonomous machine that implements either the SAMVQ or the HSOCVQ technique to compress a sub-set of hyperspectral imagery. A codevector trainer is the key component of a compression engine. In the development of a real-time data compressor, the implementation of vector distance calculation is critical, as it dominates the overall performance of the compression system. Two innovative architectures for computing vector distance are provided.
The invention has application for any sensor that collects three-dimensional data, in which the spectral dimension and one spatial dimension form a 2-D focal plane frame for data and the other spatial dimension varies with time (as in the case of a satellite moving over the Earth or a conveyor system moving under a sensor array), and for which the rate of data collection exceeds the capabilities of the available data transmission and/or data analysis. This system for real-time data compression and transmission provides a solution for rapid, real-time hyperspectral data analysis and spectral/spatial feature recognition.
The Business Opportunity
The invention has direct application to the collection of hyperspectral datasets from spaceborne or airborne remote sensing platforms. The technology will interest companies that provide hyperspectral data collection systems or data collection services to end-users.
The invention not only overcomes these limitations, but requires smaller computations, which contributes to minimizing the computational cost of performing a simulation. The proposed algorithm has application in such simulators, in that it facilitates both speed and accuracy in simulating complex dynamic systems in real-time.
The technology will also be of interest to companies and R&D organizations that are developing real-time hyperspectral monitoring and analysis systems for process control, inspection and quality control.
Technology Transfer Details
These technologies are available for licensing.
The business opportunity may be referred to by its CSA case ID: 50690-50691-50692
- Qian, S.-E., "A decadal R&D of near lossless data compression on-board satellites at Canadian Space Agency," Journal of Applied Remote Sensing, vol. 4, pp. 041797.1-22, Oct. 2010 (doi: 10.1117/1.3515313).
- Qian, S.-E., "More efficient satellite data transmission," SPIE Newsroom, article-2940, pp. 1-2, May 2010 (doi: 10.1117/2.1201005.002940)
- Qian, S.-E., "Study of hyperspectral and multispectral images compression using vector quantization in development of CCSDS international standards," Proceedings of SPIE, Vol. 7477A, pp. 23.1-11, September 2009.
- Qian, S.-E., M. Bergeron, J. Lévesque, P. Oswald, C. Black and A. Hollinger, "Considerations of Data Handling System of a Spaceborne Imaging Spectrometer with Onboard Data Compression," Canadian Journal of Remote Sensing, Vol. 34, Supplement 1, pp. 12-28, April 2008.
- Zarrinkhat, P. and S.-E. Qian, "Enhancement of Resilience to Bit-Errors of Compressed Data On-board a Hyperspectral Satellite using Forward Error Correction," Proceedings of SPIE Conference on Satellite Data Compression, Communications and Processing, Vol. 7084, pp. 07.1-9, August 2008.
- Qian, S.-E. and A. Hollinger, "Current Status of Satellite Data Compression at Canadian Space Agency," Invited paper in Proceedings of SPIE, Satellite Data Compression, Communications and Archiving III, Vol. 6683, pp. 04.01-12, August 2007.
- Qian, S.-E., "Fast vector quantization algorithms based on nearest partition set search," Institute of Electrical and Electronics Engineers (IEEE), Transactions on Image Processing, Vol. 15, No. 8, pp. 2422-2430, August 2006.
- Qian, S.-E., "Near Lossless Data Compression Techniques and Their Evaluation Using Remote Sensing Applications," Chapter 10 of book Recent Advances in Hyperspectral Signal and Image Processing by Trivandrum, Kerala: Research Signpost, 2006.
- Qian, S.-E., M. Bergeron, I. Cunningham, L. Gagnon and A. Hollinger, "Near Lossless Data Compression On-board a Hyperspectral Satellite," Institute of Electrical and Electronics Engineers (IEEE), Transactions on Aerospace and Electronic System, Vol. 42, No. 3, pp. 851-866, July 2006.
- Qian, S.-E., J. Lévesque and R. A. Neville, "Effect of Removing Random Noise of Radiance Data Using Smoothing on Data Compression Onboard a Hyperspectral Satellite," WSEAS Trans. on Systems, Issue 1, Vol. 5, pp. 219-224, January 2006.
- Qian, S.-E., M. Bergeron, A. Hollinger and J. Lévesque, "Effect of Anomalies on Data Compression Onboard a Hyperspectral Satellite," Proceeding of SPIE, Satellite Data Compression, Communications and Archiving, Vol. 5889, pp. (02)1-11, August 2005.
- Qian, S.-E., M. Bergeron, J. Lévesque and A. Hollinger, "Impact of Pre-processing and Radiometric Conversion on Data Compression Onboard a Hyperspectral Satellite," Institute of Electrical and Electronics Engineers (IEEE), Proceedings of the International Geoscience and Remote Sensing Symposium, Vol. 2, pp. 700-703, July 2005.
- Qian, S.-E., A. Hollinger, M. Bergeron, I. Cunningham, C. Nadeau, G. Jolly and H. Zwick, "A Multi-disciplinary User Acceptability Study of Hyperspectral Data Compressed Using Onboard Near Lossless Vector Quantization Algorithm," International Journal of Remote Sensing, Vol. 26, No. 10, pp. 2163-2195, May 2005.
- Hu, B., S.-E. Qian, D. Haboudane, J. R. Miller, A. Hollinger and N. Tremblay, "Retrieval of crop chlorophyll content and leaf area index from decompressed hyperspectral data: the effects of data compression," Journal of Remote Sensing Environment, Vol. 92, Issue 2, pp. 139-152, August 2004.
- Qian, S.-E., "Hyperspectral data compression using a fast vector quantization algorithm," Institute of Electrical and Electronics Engineers (IEEE), Transanctions on Geoscience and Remote Sensing, Vol. 42, No. 8, pp. 1791-1798, August 2004.
- Qian, S.-E., M. Bergeron and A. Hollinger, "Evaluation of Near Lossless Feature of Two On-board Data Compression Algorithms: SAMVQ and HSOCVQ," WSEAS Transactions on Systems, Vol. 3, Issue 5, pp. 2153-2158, July 2004.
- Harvey, W., L. Gagnon, E. Faier, S.-E. Qian, P. Oswald and P. Melanson, "A Real-time Data Compressor for Onboard Hyperspectral Imagery," ASTRO 2002, Ottawa, November 2002.
- Qian, S.-E., A. Hollinger, M. Dutkiewicz, H. Tsang, H. Zwick and J. R. Freemantle, "Effect of Lossy Vector Quantization Hyperspectral Data Compression on Retrieval of Red Edge Indices," Institute of Electrical and Electronics Engineers (IEEE), Transactions on Geoscience and Remote Sensing, Vol. 39, No. 7, pp. 1459-1470, July 2001.
- Hu, B., S.-E. Qian and A. Hollinger, "Impact of lossy data compression using vector quantization on retrieval of surface reflectance from CASI imaging spectrometry data," Canadian Journal of Remote Sensing, Vol. 27, No. 3, pp. 1-19, February 2001.
- Qian, S.-E., A. Hollinger, D. Williams and D. Manak, "Vector quantization using spectral index based multiple sub-codebooks for hyperspectral data compression," Institute of Electrical and Electronics Engineers (IEEE), Transactions on Geoscience and Remote Sensing, Vol. 38, No. 3, pp. 1183-1190, May 2000.
- Manak, D., S.-E. Qian, A. Hollinger and D. Williams, "Efficient Hyperspectral Data Compression using vector Quantization and Scene Segmentation," Canadian Journal of Remote Sensing, Vol. 24, No. 2, pp. 133-143, June 1998.
- Qian, S.-E., A. Hollinger, D. Williams and D. Manak, "Fast 3D data compression of hyperspectral imagery using vector quantization with spectral-feature-based binary coding," Journal of Optical Engineering, Vol. 35, No. 11, pp. 3242-3249, November 1996.
- Number: 6,798,360
- Number: 7,551,785
- Number: 7,801,365
- Number: 7,251,376
- Status: Issued patents
- Source: United States Patent and Trademark Office (USPTO)
- Number: 1642238 (A1)
- Number: EP1642237 (A1)
- Status: Issued patents
- Number: EP1665077 (A2)
- Status: Pending patent application
- Source: European Patent Office (EPO)
- Date modified: