Harman/Lexicon QuantumLogic/QLS Surround Processing

amirm

Banned
Apr 3, 2010
15,813
25
0
Seattle, WA
As you have probably heard, Harman demonstrated the QuantumLogic system (QLS) at CEDIA 2011. This was done in their mobile truck and is a precursor of what is to come in the Lexicon MP-20 processor.

The demonstration started surprisingly and pleasantly so by Dr. Gil Soulodre who was the chief inventor of the algorithm. He has 18 patents in his name and 30 years in the audio industry in addition to being an AES fellow and winner of a Technical Emmy.

The system is more ambitious than the ones coming from it. The cornerstone of the system is stream and reverb extraction algorithm. Stream extraction is aimed and finding specific components in the source and creating a separate channel from it. This can for example be a predominant instrument. Reverb extraction is exactly that. It identifies the reverberations and separates from the source of the same. These reverbs then can be played in a different channel than the remaining “dry” component of the same and by doing so, increase the sense of spaciousness.

One key component of the system is that the sum total of all the streams gives you back the original track unchanged. Such a reversible decomposition is critical in preserving the full fidelity of the presentation and proof point that nothing is added to the sound that was not there in the source soundtrack/music.

Next step is speech and mono detection. After this, the output is fed to an “Aesthetic Engine” which is a euphemism for putting the listener in the driver’s seat as to where each one of the extracted streams should be steered to and how much. Indeed, the elevator pitch for this system can be that instead of the music producer/engineer deciding where everything goes, you get to do that!

The demonstration started with them extracting the vocals from the music and playing each stream separately. And then summing them together. The latter was most impressive to me as it sounded like the original proving the lossless aspect of it. The separated streams achieved pretty high level of extraction. Yes, there was small amount of bleed through but since the final goal is to still play all of these streams together (as opposed to just playing the one stream by itself) that was not a harmful effect.

The process can drive 5 additional height channels/speakers (see the picture of the back of the room below). In the truck configuration, they were in a smaller circle than the main speakers due to space limitations. As you see, they used pretty low cost speakers for that. Someone in the audience commented about small tonal change when the height effect was triggered. This was no doubt due to the small sonic differences between the main (JBL) speakers and the height. That said, I did not hear the difference and think that low cost ceiling speakers can be used to satisfactorily play this role. The notion of using identical speakers up in the ceiling is probably not very practical :).

The next demonstration was extracting the reverb. The result was quite effective, playing the “dry” non-reverb version separate from the reverb track. The A/B comparison against the original was pretty striking.

They then brought on Nathan Kunkel who proceeded to play some of the music he had mixed. It should be noted that he had time with the system to decide what settings he should use. He said that after it was all said and done, he found one set of dials that worked across all of his music and indeed, they played them that way. The effect was pretty remarkable. The level of liveliness shot up through the roof. In a dance piece, it took a dull front speaker only track and distributed it across the whole room making you want to get up and dance :D.

The final demo was the Matrix soundtrack. By then I had moved to the back of the room and standing next to the wall, I did not get anything out of it. But my colleague who was sitting up front loved the enhancement.

I should note that in one of the streams they played back, there was fair bit of pre-echo distortion. This is due to use of large window transforms. That distortion vanished however when the track was played with the rest of the music.

As you see in the screen snapshots below, the system has a lot of controls. I asked how the final system would work and they said it would have a few simple dials that you could turn. One would be how far the streams spread. Imagine the minimum being your current front speakers for stereo and maximum being completely spread into the rear speakers. Similar control determines the height of how much the content is pulled up into the height speakers. Another control would manage what happens to reverb.

The engineer in me marvels at what they have done, but more importantly, why no one else thought of it! :) Certainly currently state of signal analysis allows such capabilities with the challenge being the reversible nature of it and making it computationally efficient to run on consumer gear while it is doing everything else (in case of Lexicon MP20, 16 channels of EQ).

I think this will make a fun system and lots of tweaking to be had to tailor the music and movie soundtracks to our liking. I suspect in some cases “off” would also be a good setting :). But hopefully that gets used a lot less than it is with current systems.

surround speakers:


Stream Extraction:


Extracting Voice:


UI Control (for PC prototype, not final product)


Architectural Logic Flow
 

Roger Dressler

Industry Expert
Aug 4, 2011
129
2
93
Oregon
One key component of the system is that the sum total of all the streams gives you back the original track unchanged. Such a reversible decomposition is critical in preserving the full fidelity of the presentation and proof point that nothing is added to the sound that was not there in the source soundtrack/music. ...

The separated streams achieved pretty high level of extraction. Yes, there was small amount of bleed through but since the final goal is to still play all of these streams together (as opposed to just playing the one stream by itself) that was not a harmful effect. ...

I should note that in one of the streams they played back, there was fair bit of pre-echo distortion. This is due to use of large window transforms. That distortion vanished however when the track was played with the rest of the music.
While reconstruction of the original mix is apparently lossless, the whole point of QLS is not to reconstruct the original. Altering relative levels and spatial relationships is the order of the day.

The challenge QLS faces will be familiar to Amir, I think, in that lossy audio codecs run into unmasking problems when the masking signal is reproduced from a different speaker than the coding artifacts it is intended to hide. Small changes in overall channel balance (sitting outside the sweet spot) or in a room with uneven frequency response, can expose coding artifacts that were inaudible under well controlled listening conditions. This is exacerbated by the tendency to run the audio bitrates down so low that the NMR (noise to mask ratio) is barely sufficient under ideal conditions, so the signal is fragile. That would not be expected to be the case with QLS, but the more range the adjustment options have, the more it stresses the NMR of the QLS extraction process. Harman may have to decide how much to limit the tweakability range to prevent exposing flaws under any conditions. Not an easy job! Just consider that Neo:6 and even the much improved Neo:X, which do not alter the gains of the source signal components at all (it only maps them to different speakers), nonetheless inflict audible side effects due to the "frequency slicing" technique they use.

If QLS, when used judiciously, remains sonically pure, that will truly represent a high bar. Can't wait to hear it!

Indeed, the elevator pitch for this system can be that instead of the music producer/engineer deciding where everything goes, you get to do that! ...

The engineer in me marvels at what they have done, but more importantly, why no one else thought of it! :) Certainly currently state of signal analysis allows such capabilities with the challenge being the reversible nature of it and making it computationally efficient to run on consumer gear while it is doing everything else (in case of Lexicon MP20, 16 channels of EQ).

I think this will make a fun system and lots of tweaking to be had to tailor the music and movie soundtracks to our liking.
The concept of allowing users to drastically remix music for themselves has been discussed many times in the AES literature, but always as a use case for object-based audio delivery. Audio BIFS in MPEG-4 supports it, a spec left to collect dust on the shelf. Maybe it is a solution looking for a problem. After all, for decades we have been fighting to achieve playback systems that optimally and faithfully render the original intentions of the creative folks who toiled over their mixes. So this is a bit of a different philosophy, different even from the more modest surround upmixers which simply aimed to better envelop the listener while keeping the original art intact.

I'm not saying consumer remixing is a bad thing. So many times I've dreamed of "fixing" some stupid (IMHO) mixing choices! All this just to say I can see why no one has done it before.
 
Last edited:

audioguy

WBF Founding Member
Apr 21, 2010
2,778
61
445
Near Atlanta, GA but not too near!
I'm not saying consumer remixing is a bad thing. So many times I've dreamed of "fixing" some stupid (IMHO) mixing choices! All this just to say I can see why no one has done it before.

That was the one issue I brought up as you can only image how bad a user (see "user influence" box in Amir's diagram) could make it sound. They replied that they will put limits on user control so that won't happen. As I've said elsewhere, I was blown away by the sound they presented (and I'm not a JBL horn fan :D)
 
Last edited:

amirm

Banned
Apr 3, 2010
15,813
25
0
Seattle, WA
Thanks for chiming in Roger. Is it possible to have you give an overview of DTS and Dolby technology in this area? Either here or another thread would be of great benefit in understanding the current landscape there.

As to MPEG-4 object system, it was designed by people who didn't connect what people needed and efficiency loss in the system when elements are transmitted. I say this with much chagrin as my group at Microsoft designed and held the patents for object oriented video in MPEG-4! :).
 

amirm

Banned
Apr 3, 2010
15,813
25
0
Seattle, WA
The "two Kevin's" video reminded me that the QLS is already shipping or should I say "rolling" in the form of Ferrari FF! :D Given the constraints of power consumption and space, that is a great sign of its feasibility and completeness relative to the same arriving at home:



http://www.harman.com/EN-US/Newscenter/Pages/QuantumLogicIntro.aspx

"HARMAN Introduces Revolutionary QuantumLogic™ Surround Technology With Ferrari

01-March-2011
GENEVA MOTOR SHOW – 01 March 2011 – HARMAN, the premium global audio and infotainment group, has launched the world’s first production vehicle, the Ferrari FF, featuring its innovative QuantumLogic surround sound technology. Developed entirely by HARMAN’s audio experts, QuantumLogic technology transforms any stereo or multi-channel audio source into an astounding 7.1 channel surround sound experience. QuantumLogic technology sets a new benchmark in audio signal processing excellence.


The patented QuantumLogic technology forms an integral part of the newly developed JBL® Professional surround sound system for the FF, Ferrari’s innovative four-seater Gran Turismo. The premium branded audio system offers unparalleled listening enjoyment, with a dynamic, accurate spatial quality never before thought possible in a vehicle.


“The world premiere of the JBL Professional QuantumLogic surround sound system in the new FF represents a new milestone in our partnership with the legendary Italian sports car manufacturer,” says Dinesh C. Paliwal, Chairman, President and CEO of HARMAN. “With our uncompromising determination to capture every detail of the sound experience, together we have reached new heights in audio signal processing and sound quality. The HARMAN JBL Professional brand represents decades of experience in reproducing audio in the world’s best concert halls and performance venues, and QuantumLogic helps brings that immersive, surround sound experience to the interior of the Ferrari.”


Engineers from the two companies share a passionate dedication to performance excellence, and strove to exceed the expectations of the discerning clientele of the Ferrari and JBL Professional brands. This challenge has been brilliantly achieved with the implementation of HARMAN’s QuantumLogic surround technology in the all-new JBL Professional sound system for the FF. QuantumLogic surround technology gives audio tracks a controlled sound quality by introducing revolutionary new audio algorithms to extract signal streams and impulse responses from the original recording. Individual voices and instruments, as well as embedded reverberant spatial information, are identified then re-authored into a precise multi-channel soundstage. Unique to QuantumLogic is its “aesthetic engine”, which scientifically combines the individual signal streams and patented spatial filter bank technology with psycho-acoustic modeling for transparent digital processing and perfect acoustic reconstruction. The result is a stunningly immersive playback system with refined clarity and detail.


The HARMAN automotive acoustic specialists have spent years creating the JBL Professional audio system for the FF that is custom tuned to meet the unique environmental characteristics of this world-class sports car. A total of 15 speakers and a 1280-watt, JBL Professional Class-D high-performance amplifier provide rich and powerful sound levels with excellent dynamics. Individual audio components are honed to the nearest millimeter through computerized simulations before being seamlessly integrated into the car’s instrument panel, doors, and quarter panels behind the rear seats. In the B pillar of the car, acoustic specialists mounted two HARMAN patented, ultra slim EDPL (Electro Dynamic Planar Loudspeakers), ensuring perfect surround sound.


Additionally a subwoofer box with a 335 mm x 190 mm racetrack Carbon Rohacell drive unit is integrated into the trunk area capable to produce an extensive, rich bass response. Only advanced materials have been chosen to perfectly match the target frequency range of all JBL Professional loudspeaker components: CMMD (Ceramic Metal Matrix Diaphragm) for tweeters, Kevlar for midranges, Black Kevlar for mid-woofers and Rohacell for woofers. Sound tuning was carried out using HARMAN’s AuraVox technology which greatly enhanced the system performance.


In addition to their landmark audio systems, the new Ferrari features a HARMAN hard-disk-based GPS navigation system with map and arrow display. The hard disk offers access times are much quicker than conventional alternatives with data on a DVD. Bird’s-eye perspectives and an automatic intersection zoom feature are included, as are dynamic route and destination calculation and a large range of points of interest (POI). HARMAN has integrated “one shot” directions through its voice recognition engine, allowing for more natural interaction with the navigation system. Verbal route guidance is available in a choice of five languages. HARMAN also expands the functionality of the infotainment system with a number of connectivity options, including USB iPod support via the Apple ID chip; Bluetooth™ connectivity for hands free dialing as well as phone book and contact access; and streaming music to the JBL Professional audio system from Bluetooth-enabled smartphones or devices, thanks to an A2DP wireless protocol. The DVD player is designed to play video DVDs, audio DVDs and CDs in MP3 and WMA format and an integral USB port allows connection to external media. Satellite radio support will also be available for US vehicles.


Close cooperation for maximum performance


The engineers from Ferrari and HARMAN share a passion for technical perfection and precision engineering, as well as a highly developed sense of individuality. Since 2008, HARMAN has created five made-to-measure JBL Professional sound systems for sports cars from Maranello, including the Scuderia Spider 16M, 458 Italia and Ferrari California models.

> More from HARMAN at the 2011 Geneva Motor Show"
 

NorthStar

Member
Feb 8, 2011
24,308
1,283
435
Vancouver Island, B.C. Canada
The Art of Natural Sound Envelopment...

Most interesting stuff to me; and we are lucky to have people like Roger and Kalman (eventually) pitching in. :cool:

I wanna read about it, and hear it. :) ...From my own ears.
 

Andre

New Member
Apr 7, 2011
1
0
0
Southern California
While reconstruction of the original mix is apparently lossless, the whole point of QLS is not to reconstruct the original. Altering relative levels and spatial relationships is the order of the day.

Roger, I agree. I think the lossless property of QLS, while interesting to consider, is a bit overstated. Energy-preservation matrices in other matrix systems (eg. TriField) have been around for a while.

But the main reason I think it's overstated is that there are many extraction flaws that can exist while still preserving the lossless aspect: steering, leakage, and other directionality-related flaws, as well as timbre alterations (you mention DTS Neo, which is an excellent example of timbre flaws).

Also, it's interesting that you bring up the unmasking example from perceptual coding. I wonder if there are complementary distortions due to QLS that cancel each other out when you sum it back to the original, but exist in an audible way when things are separated out and coming from you from different directions. Also if the user or algorithm alters levels of different channels, some of these flaws can become more audible. From reports of hearing single singers extracted so far, people haven't reported audible flaws, but it is a fairly limited set of signals they've tried, and it's in a controlled demo, too.

With the high-level description of spatial slicing, I wonder too how their perceptual model is done. Audibility of directionality depends on many different cues which differ across the spectrum and direction of sound. Certain other aspects of spatial hearing like envelopment use different cues. This model is important for them to correctly place extracted streams in the right places so it sounds good or realistic. On the one hand, I wonder how sophisticated their perceptual model is, but on the other hand, with software upgrades, their perceptual model could be continually refined for better and better sound as time goes by.
 

Roger Dressler

Industry Expert
Aug 4, 2011
129
2
93
Oregon
Also, it's interesting that you bring up the unmasking example from perceptual coding. I wonder if there are complementary distortions due to QLS that cancel each other out when you sum it back to the original, but exist in an audible way when things are separated out and coming from you from different directions.
Amir mentioned the pre-echo, so there is one example.

Also if the user or algorithm alters levels of different channels, some of these flaws can become more audible. From reports of hearing single singers extracted so far, people haven't reported audible flaws, but it is a fairly limited set of signals they've tried, and it's in a controlled demo, too.
Luckily, hopefully, QLS will not be asked to perform such parlor tricks in real use. Total isolation of a singer is fun, like a ping pong ball bouncing between the first stereo system speakers. But it is only to prove a point about the systems' capabilities, it is not intended to be heard that way in the long run. Rather, it will allow a great deal of freedom in how to portray the complete mix with presumably fewer tradeoffs. So I'm all for it on principle.

With the high-level description of spatial slicing, I wonder too how their perceptual model is done. Audibility of directionality depends on many different cues which differ across the spectrum and direction of sound. Certain other aspects of spatial hearing like envelopment use different cues. This model is important for them to correctly place extracted streams in the right places so it sounds good or realistic. On the one hand, I wonder how sophisticated their perceptual model is, but on the other hand, with software upgrades, their perceptual model could be continually refined for better and better sound as time goes by.
Since they separate the direct sounds from the reverb/ambience, let's look at them individually. The direct sounds are presented in stereo or 5.1 mixes as correlated signals. They (mostly) come from one speaker alone or a pair of speakers as in a phantom image. If QLS can identify those positions in the spatial plane, it can then do as it wishes to alter that signal.

While technologically not QLS, I think this demo video illustrates some of the same concepts and effects. The whole video is fascinating, but the part describing shotgun mics at 3:55 is relevant to “spatial slice” decomposition.

[video]http://harpex.net/demovideo.mp4[/video]

As the video also shows, the shotgun mic pointed to the rear of the room hears only the room ambience. When funneled through a single mic it of course is just another direct sound. Picking up 2 or 3 or 4 different spatial slices around the back of the room would retain more and more of the decorrelated nature of the reverb when played from multiple speakers. And if that is not sufficient, additional decorrelation can be applied. I’m just saying that it is not so important to follow a human perceptual model in order to impart spatial reverb/ambience. Just sufficiently decorrelated signals coming from a sufficient number of speakers so as to remain diffuse in the listening room.
 

Roger Dressler

Industry Expert
Aug 4, 2011
129
2
93
Oregon
Thanks for chiming in Roger. Is it possible to have you give an overview of DTS and Dolby technology in this area? Either here or another thread would be of great benefit in understanding the current landscape there.
When it comes to explaining DTS technology, I have to leave that to others. My comments are based on "black box" perceptions and what I have read about their inner workings. For example, this quote about Neo:X comes from the WSR Issue 158, July/August 2011 article entitled: "DTS® Neo:X—Advancing The Art And Science Of Holosonic® Spherical Surround™ Sound":
According to Frederick Kitson, Ph.D, Executive Vice President and Chief Technology Officer, DTS Neo:X separates input frequencies into sub-bands and then creates additional output channels to match the user’s loudspeaker layout. It is the result of dissecting in real time the power response of each discrete channel into 64 frequency bands, based on correlated and uncorrelated signals.

Neo:6 is well described in their patent.

Both Neo's are multiband systems, so whereas QLS can detect and isolate a given instrument across the entire spectrum, Neo:6 would look only at the loudest signals in a given frequency band and steer them accordingly. That could result in the louder parts of a piano being placed in one speaker but softer parts clouded by other instruments might be directed elsewhere. I think the theory was that those louder sounds would mask the "spatial cognitive dissonance" of hearing those softer bits from other locations. They probably do that, sometimes, but the continuously shifting warping it inflicts to the spectrum is quite disturbing, to me.

Neo:X is more advanced, and it sounds so. But as to its specific technology, it might be based on Neural, or on further work by Jean-Marc Jot or JJ Johnston. Not sure.

Pro Logic II (and IIx) is dumb-old single-band logic steering, just executed very well, thanks to Jim Fosgate's inventive persistence.
 
Last edited:

amirm

Banned
Apr 3, 2010
15,813
25
0
Seattle, WA
Thanks Roger. I would think that general algorithms for mass market CE products have to occupy a very small CPU footprint or else, they won't be easy to broadly license. Whereas what Harman is doing doesn't nearly have that kind of constraint so it can be more ambitious.

On Neo:X I will ask JJ next time I run into him and report back what I hear.
 

Orb

New Member
Sep 8, 2010
3,010
1
0
I wonder if this will help to corner the soundbar speaker market, which IMO is where the general consumer is more likely to spend their cash if they work very well.
It will be very interesting to see if this does become implemented into a soundbar solution, and the effect this will have sonically and on consumer preference.

Amir, JJ has been working on soundbars more recently (and going back a few years), might be worth also getting his view on whether he sees this fitting in that market.

CHeers
Orb
 

amirm

Banned
Apr 3, 2010
15,813
25
0
Seattle, WA
This is the opposite of what you need in a sound bar. This technology takes 2 channels and spreads it across many. The stuff you need for soundbar takes many channels and stuffs them into a few in the sound bar. Now there may be applications for having a height channel above the sound bar and such....
 

Orb

New Member
Sep 8, 2010
3,010
1
0
LOL yes I was in stereo thinking mode and forgot films in good ol 5.1 channel structure :)
Bloody audiophiles :)
Still, when you catch up with JJ see if he has anything to share about what he has been doing looking at soundbars.

Ok, nothing to at look here move along all.
Phew wonder if I got away with this brain fudge moment..... :)
Cheers
Orb
 

Orb

New Member
Sep 8, 2010
3,010
1
0
This is the opposite of what you need in a sound bar. This technology takes 2 channels and spreads it across many. The stuff you need for soundbar takes many channels and stuffs them into a few in the sound bar. Now there may be applications for having a height channel above the sound bar and such....

Just to add, the modern soundbars studies are looking to provide an alternative to the traditional 5.1 setup, which means we do have many speakers or will in modern soundbars.
As mentioned not really useful for films, but thinking further may help if the soundbar all-in-one system is also used to do traditional music.
Still, would be interesting to know more from a JJ perspective as there has been R&D into this type of technology by him, and who knows whether in future development would be better for films as well.

Cheers
Orb
 

About us

  • What’s Best Forum is THE forum for high end audio, product reviews, advice and sharing experiences on the best of everything else. This is THE place where audiophiles and audio companies discuss vintage, contemporary and new audio products, music servers, music streamers, computer audio, digital-to-analog converters, turntables, phono stages, cartridges, reel-to-reel tape machines, speakers, headphones and tube and solid-state amplification. Founded in 2010 What’s Best Forum invites intelligent and courteous people of all interests and backgrounds to describe and discuss the best of everything. From beginners to life-long hobbyists to industry professionals, we enjoy learning about new things and meeting new people, and participating in spirited debates.

Quick Navigation

User Menu

Steve Williams
Site Founder | Site Owner | Administrator
Ron Resnick
Site Co-Owner | Administrator
Julian (The Fixer)
Website Build | Marketing Managersing