Resources

Expansion in Mobile, Wearables, IoT and Automotive in the coming years

Sept 14, 2016, via MIPIAlliance on Feb 17, 2017: Mobile Technology Expands to New Horizons by Linley Gwennap – MIPI DevCon 2016 keynote (26 min)

Singularity coming in the next decades

Jan 8, 2017, Mint: My Future Prediction – Masayoshi Son (4 min)

Oct 25, 2016, ARM: SoftBank Group CEO Masayoshi Son – The Journey to 1 Trillion IoT Chips – ARM TechCon 2016 keynote (4 min)

Nov 24, 2016, ARM: A vision for vision and imaging technologies by Jem Davies | ARM TechCon 2016 (20 min)

GPU

ARM Mali GPU IP offers high performing, energy efficient graphics processing across a large and growing number of mobile and consumer devices, including entry-level mass market smartphones, visually stunning high-performance smartphones, Android™ OS-based tablets, SmartTVs and wearables.

Graphics processors:

More information about Mali GPUs

MALI-G71 ENABLES COHERENT COMPUTING - May 30, 2016 by The Linley Group — Source: Linley mobile chip report: Mali-G71 enables coherent computing: … With its new Bifrost design, ARM aims to gain more share in the high-end graphics segment and to address the requirements of performance-intensive tasks such as mobile gaming, augmented reality (AR), and virtual reality (VR). To enable these applications, the Mali-G71 GPU can scale from 4 to as many as 32 shader cores—twice the maximum number in the current Mali performance leader, the T860/880. It’s also a more efficient design, requiring 20% less power and 40% less die area than the T880 under the same process and performance conditions. … The new Vulkan 3D graphics API mandates coherent shared memory. Before Bifrost, Mali GPUs provided only partial GPU/ CPU coherence. … As Figure 2 shows, Bifrost GPUs upgrade to the full Ace interface, so the GPU can act as a cache-coherent master peer to the CPUs. … ARM’s cache-coherent system employs the CoreLink cache-coherent interconnect (CCI) to link the CPU, GPU, and any I/O-coherent masters. The CCI-550 is a 128-bit bus that supports up to six Ace interfaces. The CoreLink network interconnect (NIC) links a Mali video processor (VPU) and display processor (DPU) into the memory system to access TrustZone-protected external DRAM. … Until recently, the high-end mobile-GPU market remained a stronghold for Imagination’s PowerVR licensable cores and Qualcomm’s in-house Adreno GPU; the new Mali G71 enables licensees to achieve performance levels high enough to match or even beat those processors. …

Video

Mali Video processors provide advanced video technology combined with power-efficiency and one of the smallest chip sizes in the market to offer creators of mass market mobile devices support for a wide range of video technologies. Designed to scale from 1080p60 on a single core to 4k120 on the full eight cores, all variations offer stunning HD visual quality to end users.

Video processors:

Display

As the resolution and quality of displays increase, efficient processing of the display output becomes even more important in a media system. Having a dedicated display processor that not only controls output to the screen but also enables the GPU to offload basic tasks such as multi-layer composition or image post-processing improves visual quality and reduces overall SoC power consumption.

Display processors:

More information about Mali Display

June 17, 2016, Graphics & Multimedia blog – Graphics & Multimedia – ARM Community: Happy 10th Birthday Mali! … Back in early 2006 Mali was just a twinkle in ARM’s eye, it wasn’t until June of that year that ARM announced the acquisition of Norwegian graphics company Falanx and ARM Mali was born. …

May 3, 2017, Graphics & Multimedia blog – Graphics & Multimedia – ARM Community: Mali-Cetus preview – Driving display

… This time last year we brought you a sneak preview of our latest video processor, Mali-V61, way ahead of launch and back when it was still known by its codename, Egil. This year is no different, and we’re so excited about our upcoming display processor, built on a brand new, innovative architecture, that we can’t wait another minute to give you the details. …

This time last year we brought you a sneak preview of our latest video processor, Mali-V61, way ahead of launch and back when it was still known by its codename, Egil. This year is no different, and we’re so excited about our upcoming display processor, built on a brand new, innovative architecture, that we can’t wait another minute to give you the details.

…

Imaging and computer vision

Through the acquisition of Apical, ARM is able to provide cutting-edge imaging and computer vision IP that is based on fundamental research into human vision and visual processing. The technology enables electronic devices to capture visual inputs and produce high quality user viewing experiences, and increasingly to understand and learn from their environments, just like we do. ARM Mali Camera and Assertive Display IP powers many cameras and displays today, from smartphones to embedded and IoT devices.

Mali Camera, State-of-the-art camera subsystems [on ARM site as of/since May 15, 2016]: … Mali-C71 is the first product in the Mali Camera series of Image Signal Processors (ISP) from ARM’s Imaging & Vision Group. Mali-C71 was developed for the emerging smart automotive market, delivering key visual information to the driver for clear and convenient viewing. The Mali-C71 also supports the requirements of SIL3 and ASIL-D functional safety for a variety of automotive applications from all-round vehicle awareness, mirror replacement, through to night vision improvement. …
Assertive Display, Sunlight viewability and power saving [on ARM site as of/since Jan 19, 2012]
Computer vision: Real-time, Scalable, Accurate, Predictable

May 26, 2016, Graphics & Multimedia blog – Graphics & Multimedia – ARM Community: Seeing the Future With Computer Vision … A particularly exciting aspect of welcoming Apical to ARM is their Computer Vision interpretation engine which takes data from video and a variety of sensors and produces a digital representation of the scene it’s viewing. This allows, for example, security staff to monitor the behaviour of a crowd at a large event or in public areas and raise an alert before overcrowding becomes an issue. It also opens the doors for vehicles and machines to begin to be able to process their surroundings independently and apply this information to make smart decisions and make their behaviour contextually aware. …

Nov 26, 2016, Charbax: Interview with Jem Davies, ARM VP of Technology, Imaging and Vision Group, after his keynote at ARM Techcon 2016. He talks about the upcoming development at ARM in the field of Computer Vision, after the acquisition by ARM of Apical, adding their ISP technology, local tone-mapping Display Engine to fit inside Mali’s Display Processor, and the Computer Vision does object recognition in a fixed function dedicated engine. The Computer Vision engine is configurable to recognize people, objects, places, enabling a new visual level of smart technology. (7 min)

February 22, 2017, Graphics & Multimedia blog – Graphics & Multimedia – ARM Community: Watch your weight: demonstrating machine learning and computer vision on your smartphone at Mobile World Congress

… A key part of the Apical portfolio was Spirit, a piece of highly efficient silicon designed for object detection. What it detects depends on the model set, typically people. Spirit can detect and track any number of people in a scene, all from a full HD, full frame rate camera feed, detecting at a wide variety of sizes and all whilst consuming minimal power and overhead on other processors in the system. As well as knowing where the people are Spirit and its accompanying software can tell which direction they are facing, track them as they move, identify them from a database of known faces and generally understand the context of what they are doing. You can find a video of Spirit in action here. …

[here: Jul 28, 2014, Apical Imaging: Spirit track record thumbnails 1]

… The other two key products from Apical (now the Imaging & Vision Group within ARM) include Assertive Display and Mali Camera. Assertive Display is a display management subsystem which cleverly applies local tone mapping to adjust display clarity; this extends the indoor viewing experience to allow your phone screen to be viewed just as easily in bright sunny conditions. Mali Camera is an HDR[High-dynamic-range imaging]-capable Image Signal Processor (ISP) that cleans up and enhances images and video coming from multiple mobile sensors. …

March 11, 2013, Martin Stanford: Apical at MWC 2013 Sky News’ Martin Stanford talks to Mike Tusch from Apical about their solution [Assertive Display] for a brighter, more visible screen when using your smartphone or tablet outdoors.

More technical information: Assertive Display intro high-level (Apical presentation of 2012, available from Jan 18, 2014) … Early adopters, Shipping products (announced) …

… Assertive Display core IP is associated with 6 separate patents …

Oct 27, 2016, Charbax: Apical Computer Vision at ARM, HDR10 on cheap displays and more Judd Heape, Senior Director, Marketing in IVG (Imaging & Vision Group) at ARM talks about Apical that was acquired by ARM in May 2016, enabling Computer Vision in a new area on the SoC, already shipped with over 500 million ARM Processors, here demonstrating HDR-10 video through Assertive Display. HDR10 HEVC video stream decoded by Mali-V550 video processor to AFBC[ARM Frame Buffer Compression]-compressed 10-bit YUV, Displayed at Full HD through Mali-DP650 display processor. With Assertive Display technology for excellent image clarity under all viewing conditions, such as under bright sunlight. (8 min)

From Mali Technologies

ARM Frame Buffer Compression

The ARM Frame Buffer Compression (AFBC) protocol reduces the overall system-level bandwidth and power cost of transferring spatially coordinated image data throughout the system by up to 50%. This enables increasingly complex SoC designs to be created within the thermal limit of a mobile device. AFBC is a lossless image compression protocol and format, which minimizes the amount of data transferred between IP blocks within a SoC. The lossless compression ratios achievable with AFBC are comparable with other leading standards but with the added benefit of fine-grained random access, which importantly allows AFBC to be applied throughout other IP blocks within your SoC design. AFBC is available in all ARM Mali Video Processors, ARM Mali Display Processors and recent ARM Mali Graphics Processing Units (GPUs). AFBC is also available as a licensable IP for use with other IP blocks in a system that uses an ARM Mali GPU or ARM Mali Video processor.

May 3, 2017, Graphics & Multimedia blog – Graphics & Multimedia – ARM Community: Mali-C71 driving next-generation image processing for automotive

The automobile is no longer a metal box with four wheels that gets you from A to B. As technology has evolved, vehicles have become a dynamic platform for innovation which is changing the driver experience and taking us closer to the reality of truly autonomous driving. Advanced Driver Assistance Systems (ADAS) capabilities are integral to this evolution. New ADAS applications such as mirror replacement, driver drowsiness detection and pedestrian protection systems drive the need for enhanced image processing within the vehicle. Enabling these technologies requires an increasing number of cameras in vehicles, with most high end mass market vehicles expected to contain up to twelve cameras within the next few years, according to Strategy Analytics.

However, this type of complex camera technology must quickly process and analyze images under the most extreme conditions and be specifically designed for stringent automotive safety standards. Simply integrating standard camera technologies used in smartphones, or even consumer video cameras, is not an option. To address these challenges, today we’re launching the ARM Mali-C71 image signal processor (ISP), the first product in our Mali Camera family. The Mali-C71 was designed from the ground up for ADAS System on Chips (SoCs) with automotive needs such as adjusting for challenging lighting and weather conditions in mind.

…