Computational photography has now become the standard in the smartphone world: from portrait to tone mapping, passing through night and long exposure modes, there is no product that does not use machine learning to improve performance in conditions where the small sensor alone , would be in trouble.
Computational photography today could go to unprecedented levels, but there is a limit related to the processing power of the processor used: no one wants to wait a few minutes before seeing the photo taken.
Most processors today are fast enough to give the appearance of working in real time in many situations, from portrait mode to night mode: after shooting, if you open the gallery, the photo is ready to be distributed. To be able to achieve this ideal condition for the user, manufacturers have had to compromise: you work exclusively with compressed YUV or RGB data, to lighten the bit stream.
Despite this, in the vast majority of cases, what you see on the screen while shooting is not at all the result that you will then see when you open the photo taken at the end of its processing.
With the arrival of more powerful processors, manufacturers have begun to raise the bar: someone managed to provide an on-screen preview of portrait mode or bokeh video, others managed, thanks to fast sensors, to work directly in the domain. RAW. This means that night or portrait modes are processed starting from the 12 or 14 bit RAW file, and not from a reduced file with compressed color data.
None of the processors that push smartphones today have the processing speed of ML models that can work in the RAW domain if the data arrives at the rate of a video, that is 30 or 60 frames per second. That’s why, a bit like Huawei did in 2017 when he pulled the first NPU out of the hat to be able to manage the night mode on the photos, OPPO has decided to create an ad hoc processor to be able to manage the same type of work, but at the video level, therefore with a considerably higher calculation frequency. The Marisilicon X.
At the time, Huawei inserted its NPU into the Kirin processor, she made it; OPPO, which relies on Qualcomm for the SoC, had to build a separate processor that works in conjunction with the Snapdragon’s Spectra. The Specta and the Hexagon DSP of the Snapdragon 8 Gen. 1 cannot reach, for the processing of the images alone, those 10 TOPS that would be used to process the frames at a speed of 30 per second without consuming excessively. OPPO’s Marisilicon X instead reaches 18 TOPS, with an efficiency of 11.6 TOPS per watt consumed.
Made by TSMC at 6 nanometers, the Marisilicon is able to apply noise reduction on a 4K video at 40 fps and, thanks to the fusion of multiple frames starting from RAW data at different exposures, it is able to reach 20 bits of dynamic range on the video, dynamic which is then used to save the file in HDR format.
We have already tried and appreciated the Marisilicon with 4K videos when we wrote the review of the Find X5 Pro, but on the occasion of a recent trip we thought to go further, trying to quantify the visible difference, if any, between the videos. made with the Marisilicon and those without the Marisilicon.
How to do? Simple, the Marisilicon works only and exclusively if you use the OPPO application to record videos, and it never works with third-party apps. We then took the Find X5 Pro and recorded small clips in 4K using the OPPO app and one of the most popular third-party video apps, ProCam X, then extracted a series of frames under different conditions.
The comparison: with Marisilicon and without Marisilicon
The first scene is the port of Ibiza: the version without Marisilicon X has a reduced dynamic range which leads to burning all the highlights, even getting the white balance wrong. If you look at the small frame it doesn’t seem like a disaster, but careful vision puts us in the same situation as comparing a photo taken at night with a normal smartphone and a photo taken at night using the night mode of a top of the range.
The Marisilicon really has an edge with the same sensor: excellent definition, little noise, natural scene and well-managed lights.
The second scene also highlights important differences: while in the previous case the subject was static, here we photographed two dancers on the beach. The scene is incredibly difficult, flames in total darkness with moving subjects.
ProCam X, which in addition to not being able to take advantage of the Marisilicon X, cannot even use the RGB exposure sensor mounted on the back of the Find X5 Pro, and this means terribly red photos and incorrect white balance. Added to this is an excess of noise amplified by compression, which makes the dancers invisible, and the total absence of detail on the writing.
The frame extracted from the clip with Marisilicon, above, is decidedly more balanced and faithful, especially if we think about how the scene looks to our eyes. In addition to making the dancers clearly visible, she manages to give us some more details on the edges of the writing. You can’t obviously work miracles: the scene would require a much larger sensor.
We captured the third scene instead with one goal: to verify the effective noise reduction. As you can see in the railing on the right, and especially on the wall, the Marisilicon X has a definition and a cleanliness of the picture that the image extracted from the video recorded by ProCam X is unable in any way to have.
Here a fourth scene, lights, smoke, people. Here, too, a similar result.
OPPO’s success is Android’s defeat
There is no denying it, there are differences. Although some may consider the need to record a quality video at night superfluous, and it is subjective, it takes very little to realize that in these conditions the Marisilicon has an edge.
However, we are faced with a processor that was built specifically by a phone manufacturer which, to work at its best, it needs its application. The same app is the only one able to also take advantage of the RGB exposure meter sensor on the back, which deserves the credit for being able to manage well in white balance that often at night, with artificial lights, goes into crisis.
As we have already seen in many other situations, there is scope for innovation, but this does not come without sacrifice. Who creates a smartphone by adding “his”, to increase the quality, it must then collide with an ecosystem that does not follow it and does not facilitate itand many may think that it is not convenient to follow the path followed by OPPO.
If all smartphone manufacturers started to “invent” dedicated processors, we would probably have dozens of really different phones on the market, each with particularities that let it be appreciated: the one designed to sound good, the one for photos, the one for videos. We have also seen this in the past, when LG made phones: it was the only one to use dedicated DACs for audio, but those DACs were used solely and exclusively by the pre-installed playback app. A bit like what happens on the Find X5 with the Marisilicon X.
What about those who make photo apps, who find themselves in the paradoxical situation of never being able to compete with the stock apps of the best phones because the latter can count on perfect hardware and software integration? Today there is no application on the Google store that be able to record videos with the same quality as the stock OPPO: maybe they add more functions, or more controls, but the quality is the most important thing. What incentive can a developer with this premise have, knowing they can’t even tap into the same APIs that manufacturers are reluctant to release?
Google’s CameraX project, which was supposed to bring some order, seems to have failed: it is impossible to find a common thread for all the photo and video customizations of the producers. The consequence is that after so many years the modest photo and video quality on apps like Instagram or TikTok remains the most frequent reason that pushes someone to switch from Android to iPhone.
Unfortunately, Android keeps everyone on the same level: in a few years, when the processors of the phones will be more powerful, all the processing processes that today require a Marisilicon X will be processed directly by the SoC, and this means that quality night shooting , now exclusive to “OPPO”, will become democratic as have become the portrait mode and the photo at night.
We will have better mid-range phones, more and more complete but also more and more equal to each other, but we will not necessarily have different high-end phones. The history of Android phones teaches that innovation is possible, but only if we leave the right path: not everyone today is willing to take risks.
#Marisilicon #differences #OPPOs #success #Androids #defeat