|Publication number||US9275648 B2|
|Application number||US 12/747,148|
|Publication date||Mar 1, 2016|
|Filing date||Dec 18, 2008|
|Priority date||Dec 18, 2007|
|Also published as||CA2708861A1, CN101903944A, CN101903944B, EP2229677A1, EP2229677A4, EP2229677B1, US20100292994, WO2009078681A1|
|Publication number||12747148, 747148, PCT/2008/7522, PCT/KR/2008/007522, PCT/KR/2008/07522, PCT/KR/8/007522, PCT/KR/8/07522, PCT/KR2008/007522, PCT/KR2008/07522, PCT/KR2008007522, PCT/KR200807522, PCT/KR8/007522, PCT/KR8/07522, PCT/KR8007522, PCT/KR807522, US 9275648 B2, US 9275648B2, US-B2-9275648, US9275648 B2, US9275648B2|
|Inventors||Hyun Kook LEE, Dong Soo Kim, Sung Yong YOON, Hee Suk Pang, Jae Hyun Lim|
|Original Assignee||Lg Electronics Inc.|
|Export Citation||BiBTeX, EndNote, RefMan|
|Patent Citations (49), Non-Patent Citations (3), Classifications (10), Legal Events (1)|
|External Links: USPTO, USPTO Assignment, Espacenet|
This application is the National Phase of PCT International Application No. PCT/KR2008/007522, filed on Dec. 18, 2008, which claims priority under 35 U.S.C. 119(e) to U.S. Provisional Application No. 61/014,441, filed on Dec. 18, 2007 and U.S. Provisional Application No. 61/118,647, filed on Nov. 30, 2008.
The present invention relates to an apparatus for processing a signal and method thereof. Although the present invention is suitable for a wide scope of applications, it is particularly suitable for encoding and decoding audio signals using spectral data of signal.
Generally, in processing an audio signal using signal characteristics, the audio signal is processed based on characteristics between signals from different bands.
Conventional art is insufficient to process an audio signal effectively based on characteristics between signals from different bands.
The present invention is directed to an apparatus for processing a signal and method thereof that substantially obviate one or more of the problems due to limitations and disadvantages of the related art.
An object of the present invention is to provide an apparatus for processing a signal and method thereof, by which an audio signal can be processed based on characteristics between signals from different bands.
Another object of the present invention is to provide an apparatus for processing a signal and method thereof, by which spectral data on a different band can be obtained in a manner of selecting appropriate spectral data from a plurality of spectral data of a specific band.
A further object of the present invention is to provide an apparatus for processing a signal and method thereof, by which a bitrate can be minimized despite processing such a signal having a different characteristic as a speech signal, an audio signal and the like by a scheme appropriate for the corresponding characteristic.
The present invention provides the following effects or advantages.
First, the present invention decodes a signal having a speech signal characteristic as a speech signal and decodes a signal having an audio signal characteristic as an audio signal. Therefore, the present invention can adaptively select a decoding scheme that matches each signal characteristic.
Secondly, the present invention obtains spectral data of a different band by selecting the most appropriate spectral data from transferred spectral data, thereby increasing a reconstruction rate of an audio signal.
Thirdly, the present invention selects spectral data using start band information transferred from an encoder. Therefore, the present invention increases accuracy in selecting spectral data but decreases complexity required for carrying out an operation.
Fourthly, the present invention omits a transfer of spectral data corresponding to a partial band, thereby reducing bits required for a spectral data transfer considerably.
The accompanying drawings, which are included to provide further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the principles of the invention.
In the drawings:
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be apparent from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims thereof as well as the appended drawings.
To achieve these and other advantages and in accordance with the purpose of the present invention, as embodied and broadly described, a signal processing apparatus according to the present invention includes a copy band determining unit, a band extension information receiving unit and a target band generating unit. And, the target band generating unit includes a time dilatation/compression unit and a decimation unit. Moreover, the target band generating unit can further include a filtering unit.
The copy band determining unit receives spectral data corresponding to a low frequency band in a frequency band including the low frequency band and a high frequency band. The copy band determining unit then determines a copy band based on frequency information of the copy band corresponding to a partial band of the low frequency band.
The band extension information obtaining unit obtains side information for generating a target band from the copy band. In this case, the side information can be obtained from a bitstream and can include gain information, harmonic information and the like.
The target information generating unit generates spectral data of a target band corresponding to the high frequency band using the spectral data of the copy band. In this case, the copy band can exist above the low frequency band. It is able to generate the high frequency band using the copy band existing on the low frequency band. In the same way, it is also possible to generate the low frequency band using the copy band existing on the high frequency band.
The target band generating unit includes the time dilatation/compression unit and the decimation unit and is able to further include the filtering unit. In particular, the copy band can be obtained from the bitstream or can be obtained by filtering the received spectral data.
In this case, frequency information of the copy band indicates at least one of a start frequency, a start band and index information indicating the start band. And, the spectral data of the target band can be generated using at least one of gain information corresponding to a gain between the spectral data of the copy band and the spectral data of the target band, and harmonic information of the copy band. The spectral data of the low frequency band can be decoded by one of the audio signal and the speech signal.
The present invention is applicable to core coding of AAC, AC3, AMR and the like or future core coding. The following descriptions mainly refer applications on downmix signal but are not limited.
It is understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are intended to provide further explanation of the invention as claimed.
Mode For Invention
Reference is made to the preferred embodiments of the present invention in detail, examples of which are illustrated in the accompanying drawings.
Terminologies in the present invention can be construed as the following references. Terminologies not disclosed in this specification can be construed as concepts matching the idea of the present invention. It is understood that ‘coding’ can be construed both as encoding or decoding in a specific case. ‘Information’ in this disclosure can generally mean values, parameters, coefficients, elements and the like and its meaning can be construed as different occasionally, by which the present invention is not limited.
The multi-channel encoding unit 110 receives a plurality of channel signals (hereinafter named a multi-channel signal) and then generates a downmix signal by downmixing the multi-channel signal. The multi-channel encoding unit 110 generates spatial information required for upmixing the downmix signal to the multi-channel signal. In this case, the spatial information can include channel level difference information, inter-channel correlation information, channel prediction coefficient and downmix gain information and the like.
Meanwhile, this downmix signal can include a signal in a time-domain (e.g., residual data) or information of a frequency-transformed frequency domain (e.g., scale factor coefficient, spectral data).
The partial band encoding unit 120 generates a narrowband signal and band extension information from a broadband signal.
In this case, an original signal including a plurality of bands is named a broadband signal and at least one of a plurality of the bands is named a narrowband signal. For instance, in a broadband signal including two bands (a low frequency band and a high frequency band), either one of the bands is named a narrowband signal. Moreover, a partial band indicates a portion of the whole narrowband signal and shall be named a copy band in the following description.
The band extension information is the information for generating a target band using the copy band. And, the band extension information can include frequency information, gain information, harmonic information and the like. In a decoder, the broadband signal is generated from combining the target band with the narrowband signal.
If a specific frame or segment of a downmix signal (narrowband downmix signal DMXn) has a large audio characteristic, the audio signal encoding unit 130 encodes the downmix signal according to an audio coding scheme. In this case, the audio signal may comply with AAC (advanced audio coding) standard or HE-AAC (high efficiency advanced audio coding) standard, by which the present invention is not limited. Moreover, the audio signal encoding unit 130 may correspond to an MDCT (modified discrete transform) encoder.
If a specific frame or segment of a dowrunix signal (narrowband downmix signal DMXn) has a large speech characteristic, the speech signal encoding unit 140 encodes the downmix signal according to a speech coding scheme. In this case, the speech signal can include G. 7XX or AMR-series, by which examples of the speech signal are not limited. Meanwhile, the speech signal encoding unit 140 can further use a linear prediction coding (LPC) scheme. If a harmonic signal has high redundancy on a time axis, it can be modeled by linear prediction for predicting a present signal from a past signal. In this case, if the linear prediction coding scheme is adopted, it is able to increase coding efficiency. Moreover, the speech signal encoding unit 140 can correspond to a time domain encoder.
Thus, the narrowband downmix is encoded per frame or segment by either the audio signal encoding unit 130 or the speech signal encoding unit 140.
And, the multiplexer 150 generates a bitstream by multiplexing the spatial information generated by the multi-channel encoding unit 110, the band extension information generated by the partial band encoding unit 120 and the encoded narrowband downmix signal.
In the following description, the detailed configuration of the partial band encoding unit 120 is explained with reference to
If a received broadband signal is not spectral data, the spectral data obtaining unit 122 generates spectral data in a manner of converting a downmix to a spectral coefficient, scaling the spectral coefficient with a scale factor and then performing quantization. In this case, the spectral data includes spectral data of broadband corresponding to a broadband downmix.
The copy band determining unit 124 determines a copy band and a target band based on the spectral data of the broadband and generates frequency information for band extension. In this case, the frequency information can include a start frequency, start band information or the like. In the following description, the copy band and the like are explained with reference to
Spectral data corresponding to a broadband signal is the spectral data corresponding to the total band sfb0 to sfbn−1 including a first band and a second band. Spectral data corresponding to a narrowband downmix DMXn is the spectral data corresponding to the first band and include the spectral data of the 0th band sfb0 to the spectral data of the (i−1)th band sfbi-1. In particular, the narrowband spectral data are transferred to a decoder, while the spectral data of the rest of the bands sfb1 to sfbn−1 are not transferred thereto.
Thus, the decoder generates the band that does not carry the spectral data. And, this band is called a target band tb. Meanwhile, a copy band cb is a scale factor band of spectral data used in generating the spectral data of the target band tb. The copy band includes portions sfbs to sfbi−1 of the bands sfb0 to sfbi−1 corresponding to the narrowband downmix. A band, from which the copy band cb starts, is a start band sb and a frequency of the start band is a start frequency. In other words, the copy band cb can be the start band sb itself, may include the start band and a frequency band higher than the start band, or can include the start band and a frequency band lower than the start band.
According to the present invention, an encoder generates narrowband spectral data and band extension information using broadband spectral data, while a decoder generates spectral data of a target band using spectral data of a copy band among narrowband spectral data.
Referring to (A) of
Referring to (C) of
The relevance of the target band, the copy band and the start band according to the various embodiments of the present invention are previously explained. The rest of the elements are explained with reference to
As mentioned in the foregoing description, the copy band determining unit 124 determines a copy band, a target band and a start band, sb of the copy band. The start band can be variably determined per frame. This can also be determined according to a characteristic of a signal per frame. In particular, the start band can be determined according to whether a signal is transient or stationary. For example, a start band can be determined as a low frequency when a signal is transient since the signal has less harmonic components than when it is stationary.
Meanwhile, the start band can be determined as a numerical value of brightness of sound using a spectral centroid. For instance, if a sound is relatively high (when high-pitched tone is dominant), a start band can be formed in high frequency band. If a sound is relatively low (when low-pitched tone is dominant), a start band can be formed in low frequency band. Although the start band is determined variably per frame, it is preferable to form the start band by considering the trade-off between sound quality and bitrate.
The copy band determining unit 124 outputs a narrowband downmix DMXn or the spectral data of the narrowband excluding the spectral data of the target band. This narrowband downmix is inputted to the audio signal encoding unit or the speech signal encoding unit described in
The copy band determining unit 124 generates start band information that indicates start frequency information on a start frequency from which the copy band cb starts or a start band information of the copy band cb. The start band information can be represented not only as a substantial value but also as index information. When the start band information is represented as the index information, the start band information corresponding to the index is stored in a table and can be used in a decoder. The start band information is forwarded to the band extension information transferring unit 129 and is then included as band extension information.
The gain information obtaining unit 126 generates gain information using the spectral data of the target band and the copy band. In this case, the gain information can be defined as an energy ratio of target band to copy band and can be defined as the following formula.
In Formula 1, ‘gi’ indicates a gain and ‘i’ indicates a current target band.
This gain information can be determined for each target band as previously shown. The gain information is forwarded to the band extension information transferring unit 129 and is then included as the band extension information as well.
The harmonic component information obtaining unit 128 generates harmonic component information by analyzing a harmonic component of the copy band. The harmonic component information is forwarded to the band extension information transferring unit 129 and is then included as the band extension information as well.
The band extension information transferring unit 129 outputs band extension information having the start band information, gain information and harmonic component information included therein. This band extension information is inputted to the multiplexer described with reference to
Thus, the narrowband downmix and the band extension information are generated by the above-described method. In the following description, a process for generating a broadband downmix in a decoder using band extension information and a narrowband downmix is explained.
The demultiplexer 210 extracts a narrowband downmix DMXn, band extension information and spatial information from a bitstream. If a narrowband downmix signal has more audio characteristic, the audio signal decoding unit 220 decodes the narrowband downmix signal by an audio coding scheme. In this case, as mentioned in the foregoing description, an audio signal can comply with AAC or HE-AAC standard. If the narrowband downmix signal has more speech characteristic, the speech signal decoding unit 230 decoded the narrowband downmix signal by a speech coding scheme.
The partial band decoding unit 240 generates a broadband signal by applying the band extension information to the narrowband downmix, which will be explained in detail with reference to
The multi-channel decoding unit 250 generates an output signal using the broadband downmix and the spatial information.
The band extension information receiving unit 242 extracts start band information, gain information and harmonic component information from the band extension information, which are sent to the copy band determining unit 244 and the target band information generating unit 246.
The copy band determining unit 244 determines a copy band using a narrowband downmix DMXn and start band information. In this case, if the narrowband downmix DMXn is not spectral data of a narrowband, it is converted to spectral data. Moreover, the copy band may be equal to or different from a start band. If the copy band is different from the start band, from a band corresponding to the start band information to a band having spectral data are determined as the copy band. Spectral data determined by the copy band are forwarded to the target band information generating unit 246.
The target band information generating unit 246 generates spectral data of a target band using the spectral data of the copy band, the gain information and the like. Data of target band can be generated by the following formula.
sd(target_band)=g i ×sd(copy_band) [Formula 2]
In Formula 2, ‘g’ indicates a gain of a current band, ‘sd(target_band)’ indicates spectral data of target band, and ‘sd(copy_band)’ indicates spectral data of copy band.
In case of the former embodiment shown in (A) of
Meanwhile, the number of spectral data of target band Nt may differ from the number of spectral data of copy band Nc. This case is explained as follows.
Referring to (A) of
Referring to (A) of
Referring now to
The target band information generating unit 246 is able to generate spectra data by combination of a time dilatation/compression step and a decimation step. In this case, the time dilatation/compression step may include a step of dilating a time-domain signal in a temporal direction and this dilatation step can use a phase vocoder scheme. The decimation step may include a step of compressing a time-dilated signal into an original time. It is able to apply the time dilatation/compression step and the decimation step to target band spectral data.
The signal reconstructing unit 248 generates a broadband signal using the target band spectral data and the narrowband signal. In this case, the broadband signal may include spectral data of a broadband or may correspond to a signal in a time domain.
An audio signal processing method according to the present invention can be implemented in a computer-readable program in combination with some hardware and can be stored in a recordable medium. Multimedia data having the data structure of the present invention can also be stored in the computer-readable recordable medium. The recordable media includes all kinds of storage devices which are capable of storing data readable by a computer system. The recordable media include ROM, RAM, CD-ROM, magnetic tapes, floppy discs, optical data storage devices, and the like for example and also include carrier-wave type implementations (e.g., transmission via Internet). Bitstream generated by the encoding method can be stored in a computer-readable recordable media or transmitted via wire/wireless communication network.
Accordingly, the present invention is applicable to encoding/decoding of an audio/video signal.
While the present invention has been described and illustrated herein with reference to the preferred embodiments thereof, it will be apparent to those skilled in the art that various modifications and variations can be made therein without departing from the spirit and scope of the invention. Thus, it is intended that the present invention covers the modifications and variations of this invention that come within the scope of the appended claims and their equivalents.
|Cited Patent||Filing date||Publication date||Applicant||Title|
|US5857168 *||Apr 14, 1997||Jan 5, 1999||Nec Corporation||Method and apparatus for coding signal while adaptively allocating number of pulses|
|US5956674||May 2, 1996||Sep 21, 1999||Digital Theater Systems, Inc.||Multi-channel predictive subband audio coder using psychoacoustic adaptive bit allocation in frequency, time and over the multiple channels|
|US6680972 *||Jun 9, 1998||Jan 20, 2004||Coding Technologies Sweden Ab||Source coding enhancement using spectral-band replication|
|US7139702 *||Nov 13, 2002||Nov 21, 2006||Matsushita Electric Industrial Co., Ltd.||Encoding device and decoding device|
|US7205910 *||Jul 29, 2003||Apr 17, 2007||Sony Corporation||Signal encoding apparatus and signal encoding method, and signal decoding apparatus and signal decoding method|
|US7359854 *||Apr 10, 2002||Apr 15, 2008||Telefonaktiebolaget Lm Ericsson (Publ)||Bandwidth extension of acoustic signals|
|US7630882 *||Dec 8, 2009||Microsoft Corporation||Frequency segmentation to obtain bands for efficient coding of digital media|
|US7831434 *||Jan 20, 2006||Nov 9, 2010||Microsoft Corporation||Complex-transform channel coding with extended-band frequency coding|
|US7864843 *||Jun 4, 2007||Jan 4, 2011||Samsung Electronics Co., Ltd.||Method and apparatus to encode and/or decode signal using bandwidth extension technology|
|US7885819 *||Feb 8, 2011||Microsoft Corporation||Bitstream syntax for multi-process audio decoding|
|US7953605 *||Oct 6, 2006||May 31, 2011||Deepen Sinha||Method and apparatus for audio encoding and decoding using wideband psychoacoustic modeling and bandwidth extension|
|US8036903 *||May 4, 2007||Oct 11, 2011||Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V.||Analysis filterbank, synthesis filterbank, encoder, de-coder, mixer and conferencing system|
|US8265940 *||Jun 30, 2006||Sep 11, 2012||Siemens Aktiengesellschaft||Method and device for the artificial extension of the bandwidth of speech signals|
|US20030093271||Nov 13, 2002||May 15, 2003||Mineo Tsushima||Encoding device and decoding device|
|US20040028125||Jun 27, 2001||Feb 12, 2004||Yasushi Sato||Frequency interpolating device for interpolating frequency component of signal and frequency interpolating method|
|US20040131203||May 23, 2001||Jul 8, 2004||Lars Liljeryd||Spectral translation/ folding in the subband domain|
|US20050004793 *||May 4, 2004||Jan 6, 2005||Pasi Ojala||Signal adaptation for higher band coding in a codec utilizing band split coding|
|US20050246164 *||Apr 15, 2005||Nov 3, 2005||Nokia Corporation||Coding of audio signals|
|US20060147048 *||Feb 9, 2004||Jul 6, 2006||Koninklijke Philips Electronics N.V.||Audio coding|
|US20070038439 *||Apr 14, 2004||Feb 15, 2007||Koninklijke Philips Electronics N.V. Groenewoudseweg 1||Audio signal generation|
|US20070271095||Aug 18, 2005||Nov 22, 2007||Shuji Miyasaka||Audio Encoder|
|US20070282599 *||Jun 4, 2007||Dec 6, 2007||Choo Ki-Hyun||Method and apparatus to encode and/or decode signal using bandwidth extension technology|
|US20080077412 *||Sep 24, 2007||Mar 27, 2008||Samsung Electronics Co., Ltd.||Method, medium, and system encoding and/or decoding audio signals by using bandwidth extension and stereo coding|
|US20080109215 *||Jun 26, 2006||May 8, 2008||Chi-Min Liu||High frequency reconstruction by linear extrapolation|
|US20080129350 *||Nov 6, 2007||Jun 5, 2008||Yuhki Mitsufuji||Frequency Band Extending Apparatus, Frequency Band Extending Method, Player Apparatus, Playing Method, Program and Recording Medium|
|US20080300866 *||May 31, 2006||Dec 4, 2008||Motorola, Inc.||Method and system for creation and use of a wideband vocoder database for bandwidth extension of voice|
|US20090006103 *||Jun 29, 2007||Jan 1, 2009||Microsoft Corporation||Bitstream syntax for multi-process audio decoding|
|US20090125300 *||Oct 26, 2005||May 14, 2009||Matsushita Electric Industrial Co., Ltd.||Scalable encoding apparatus, scalable decoding apparatus, and methods thereof|
|US20090271204 *||Nov 4, 2005||Oct 29, 2009||Mikko Tammi||Audio Compression|
|CN1430777A||May 23, 2001||Jul 16, 2003||编码技术瑞典股份公司||Improved spectral translation/folding in subband domain|
|CN1527995A||Nov 7, 2002||Sep 8, 2004||松下电器产业株式会社||Encoding device and decoding device|
|EP1768451A1||Jun 13, 2005||Mar 28, 2007||Matsushita Electric Industrial Co., Ltd.||Acoustic signal encoding device and acoustic signal decoding device|
|JP2000075899A||Title not available|
|JP2001521648A||Title not available|
|JP2002041089A||Title not available|
|JP2003216190A||Title not available|
|JP2004053940A||Title not available|
|JP2004078183A||Title not available|
|JP2004198485A||Title not available|
|JP2007110565A||Title not available|
|JPH08263098A||Title not available|
|JPH11175098A||Title not available|
|KR100566630B1||Title not available|
|KR20050122267A||Title not available|
|KR20060131745A||Title not available|
|RU2131169C1||Title not available|
|RU2214048C2||Title not available|
|WO1998020709A1||Oct 31, 1997||May 14, 1998||Srs Labs, Inc.||Multi-channel audio enhancement system for use in recording and playback and methods for providing same|
|WO2001091111A1||May 23, 2001||Nov 29, 2001||Coding Technologies Sweden Ab||Improved spectral translation/folding in the subband domain|
|1||Korean Office Action dated Feb. 8, 2011 corresponding to Korean Application No. 10-2009-0090705.|
|2||*||Portnoff, M.; , "Implementation of the digital phase vocoder using the fast Fourier transform," Acoustics, Speech and Signal Processing, IEEE Transactions on , vol. 24, No. 3, pp. 243-248, Jun. 1976.|
|3||Wolters et al., "A Closer Look into MPEG-4 High Efficiency AAC," Audio Engineering Society Convention Paper, 115th Convention, New York, NY, USA, Oct. 10-13, 2003, XP008063876.|
|International Classification||G10L19/008, G10L21/038, G10L19/00, G10L19/02, G10L19/12, G10L19/24|
|Cooperative Classification||G10L19/24, G10L19/008, G10L21/038, G10L19/0204|
|Jun 14, 2010||AS||Assignment|
Owner name: LG ELECTRONICS INC., KOREA, REPUBLIC OF
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:LEE, HYUN KOOK;KIM, DONG SOO;YOON, SUNG YONG;AND OTHERS;SIGNING DATES FROM 20100518 TO 20100519;REEL/FRAME:024533/0059