MPLR
TornadoVM is an open-source framework that enables Java applications to execute on heterogeneous hardware accelerators, such as GPUs and FPGAs. While TornadoVM simplifies programmability, understanding and optimizing the performance of applications on heterogeneous systems remains a significant challenge.
MPLR
The increasing prevalence of heterogeneous computing systems, incorporating accelerators like GPUs, has spurred the development of advanced frameworks to bring high performance capabilities to managed languages. TornadoVM is a state-of-the-art, open-source framework for accelerating Java programs. It enables Java applications to offload computation onto GPUs and other accelerators, thereby bridging the gap between the high-level abstractions of the Java Virtual Machine (JVM) and the low-level, performance-oriented world of parallel programming models, such as OpenCL and CUDA. However, this bridging comes with inherent tradeoffs. The semantic and operational mismatch between these two worlds - such as managed memory versus explicit memory control, or dynamic JIT compilation versus static kernel generation - TornadoVM to limit or exclude support for certain Java features. These limitations can hinder developer productivity and make it difficult to identify and resolve compatibility issues during development.
MPLR
With the increasing prevalence of machine learning and large language model (LLM) inference, heterogeneous computing has become essential. Modern JVMs are embracing this transition through projects such as TornadoVM and Babylon, which enable hardware acceleration on diverse hardware resources, including GPUs and FPGAs. However, while performance results are promising, developers currently face a significant tooling gap: traditional profilers excel at CPU-bound execution but become a “black box” when execution transitions to accelerators, providing no visibility into device memory management, execution patterns or cross-device data movement. This gap leaves developers without a unified view of how their Java applications behave across the heterogeneous computing stack.
CSR
Fully Homomorphic Encryption (FHE) enables secure computations on encrypted data, offering strong privacy guarantees for cloud computing, privacy-preserving machine learning, and confidential data processing. However, the computational overhead associated with FHE operations, due to the large size of ciphertext and the high arithmetic complexity, limits its practical applicability. In this work, we address this challenge by presenting an approach that is implemented within the OpenFHE library in order to offload the most dominant components of key switching for the BGV scheme on GPU hardware. In particular, the scope of this work is the performance improvement of the Approximate Modulus Downscaling (ApproxModDown) function. Our experimental evaluation shows that the proposed system can yield up to a 4.58× performance speedup against the vanilla OpenFHE ApproxModDown implementation, while also resulting in 1.16× performance mprovement per homomorphic multiplication and 1.08× improvement for end-to-end execution time.
EMCIS
The ENCRYPT project, an innovative collaborative project under the Hori-zon Europe Framework Programme, will integrate privacy-preserving tech-nologies across three key sectors, aiming to revolutionize the way sensitive data is processed and protected. This white paper outlines the project's core methodologies and applications, including the use of Fully Homomorphic Encryption, Trusted Execution Environments, Differential Privacy and advanced Hybrid Protec-tion Services. Through its platform, ENCRYPT seeks to address the chal-lenge of ensuring data privacy and utility, across federated data spaces within the EU. This is essential towards ENCRYPT’s compliance with GDPR standards while enabling secure and efficient data processing activities. ENCRYPT use cases stem from the domains of Fintech, Health and Cyber-security and through this, the versatility and impact of its results and activi-ties will be presented. Each use case leverages the project's cutting-edge pri-vacy-preserving technologies to safeguard data while still allowing for data-analytics to occur and enabling decision-making. From securing financial transactions, achieving patient data confidentiality and strengthening defenses against cyber threats, ENCRYPT's use cases showcase the project's innovations in privacy technology.
CSR
Fully homomorphic encryption (FHE) enables pro-cessing encrypted data without revealing sensitive information, making it applicable in fields like healthcare, finance, and legal. Despite its benefits, FHE has high computational complexity and performance overhead. To address this, researchers have explored hardware acceleration using Field-Programmable Gate Arrays (FPGAs) and Graphics Processing Units (GPUs). FPGAs are suitable for low-latency computations, while GPUs excel in parallel, high-throughput tasks. However, widespread FHE adoption remains elusive due to unresolved performance issues. This paper explores the challenges of offloading FHE computations to hardware accelerators, focusing on the OpenFHE library and the Brakerski-Gentry-Vaikuntanathan (BGV) scheme. It is the first study on adapting this scheme for GPU acceleration. We profile OpenFHE to identify computational bottlenecks and propose integrating parallelized CUDA computations within OpenFHE. Our solution, tested with varying numbers of mul-tiplicative depth, shows up to 26x performance improvement over non-accelerated implementations, proving the effectiveness of GPUs for FHE. However, the end-to-end performance is still up to 2x slower due to the overhead of marshaling and moving data between the CPU and GPU, accounting for over 97% of execution time.
SpringerBriefs in Computer Science
This book provides an introduction to both heterogeneous execution and managed runtime environments (MREs) by discussing the current trends in computing and the evolution of both hardware and software. To this end, it first details how heterogeneous hardware differs from traditional CPUs, what their key components are and what challenges they pose to heterogenous execution. The most ubiquitous ones are General Purpose Graphics Processing Units (GPGPUs) which are pervasive across a plethora of application domains ranging from graphics processing to training of AI and Machine Learning models. Subsequently, current solutions on programming heterogeneous MREs are described, highlighting for each current existing solution the associated advantages and disadvantages.
VMIL
The Standard Portable Intermediate Representation (SPIR- V) is a low-level binary format designed for representing shaders and compute kernels that can be consumed by OpenCL for computing kernels, and Vulkan for graphics rendering. As a binary representation, SPIR-V is meant to be used by compilers and runtime systems, and is usually performed by C/C++ programs and the LLVM software and compiler ecosystem. However, not all programming environments, runtime systems, and language implementations are C/C++ or based on LLVM.
MPLR
Java benchmarking suites like Dacapo and Renaissance are employed by the research community to evaluate the performance of novel features in managed runtime systems. These suites encompass various applications with diverse behaviors in order to stress test different subsystems of a managed runtime. Therefore, understanding and characterizing the behavior of these benchmarks is important when trying to interpret experimental results.
MPLR
Adopting heterogeneous execution on GPUs and FPGAs in managed runtime systems, such as Java, is a challenging task due to the complexities of the underlying virtual machine. The majority of current work has been focusing on compiler toolchains to solve the challenge of transparent just-in-time compilation of different code segments onto the accelerators. However, apart from providing automatic code generation, one of the challenges is also the seamless interoperability with the host memory manager and the Garbage Collector (GC). Currently, heterogeneous programming models on top of managed runtime systems, such as Aparapi and TornadoVM, need to block the GC when running native code (e.g, JNI code) in order to prevent the GC from moving data while the native code is still running on the hardware accelerator.
RISCV Summit Europe
In this talk, we will present the newly EU-funded project AERO (Accelerated EU Cloud) whose mission is to bring up and optimize the software stack of cloud deployments on top of the EU processor. After providing an overview of the AERO project, we will expand on two main components of the software stack to enable seamless acceleration of various programming languages on RISC-V architectures; namely, ComputeAorta which enables the generation of RISC-V vector instructions from SPIR-V binary modules, and TornadoVM which enables transparent hardware acceleration of managed applications. Finally, we will describe how the ongoing integration of ComputeAorta and TornadoVM will enable a plethora of applications from managed languages to harness RISC-V auto-vectorization completely transparently to developers.
ISMM
Scaling up the performance of managed applications on Non-Uniform Memory Access (NUMA) architectures has been a challenging task, as it requires a good understanding of the underlying architecture and managed runtime environments (MRE). Prior work has studied this problem from the scope of specific components of the managed runtimes, such as the Garbage Collectors, as a means to increase the NUMA awareness in MREs.
MoreVMs
In recent years, the Java Virtual Machine has evolved from a crossISA virtualization layer to a system that can also offer multilingual support. GraalVM paved the way to enable the interoperability of Java with other programming languages, such as Java, Python, R and even C++, that can run on top of the Truffle framework in a unified manner. Additionally, there have been numerous academic and industrial endeavors to bridge the gap between the JVM and modern heterogeneous hardware resources. All these efforts beacon the opportunity to use the JVM as a unified system that enables interoperability between multiple programming languages and multiple heterogeneous hardware resources.
VLDB
The ever-increasing demand for high performance Big Data analytics and data processing, has paved the way for heterogeneous hardware accelerators, such as Graphics Processing Units (GPUs) and Field Programmable Gate Arrays (FPGAs), to be integrated into modern Big Data platforms. Currently, this integration comes at the cost of programmability since the end-user Application Programming Interface (APIs) must be altered to access the underlying heterogeneous hardware. For example, current Big Data frameworks, such as Apache Spark, provide a new API that combines the existing Spark programming model with GPUs. For other Big Data frameworks, such as Flink, the integration of GPUs and FPGAs is achieved via external API calls that bypass their execution models completely.
VEE
During the last decade, managed runtime systems have been constantly evolving to become capable of exploiting underlying hardware accelerators, such as GPUs and FPGAs. Regardless of the programming language and their corresponding runtime systems, the majority of the work has been focusing on the compiler front trying to tackle the challenging task of how to enable just-in-time compilation and execution of arbitrary code segments on various accelerators. Besides this challenging task, another important aspect that defines both functional correctness and performance of managed runtime systems is that of automatic memory management. Although automatic memory management improves productivity by abstracting away memory allocation and maintenance, it hinders the capability of using specific memory regions, such as pinned memory, in order to perform data transfer times between the CPU and hardware accelerators.
VEE
Modern commodity devices are nowadays equipped with a plethora of heterogeneous devices serving different purposes. Being able to exploit such heterogeneous hardware accelerators to their full potential is of paramount importance in the pursuit of higher performance and energy efficiency. Towards these objectives, the reduction of idle time of each device as well as the concurrent program execution across different accelerators can lead to better scalability within the computing platform.
VEE
Although Graphics Processing Units (GPUs) have become pervasive for data-parallel workloads, the efficient exploitation of their tiered memory hierarchy requires explicit programming. The efficient utilization of different GPU memory tiers can yield higher performance at the expense of programmability since developers must have extended knowledge of the architectural details in order to utilize them.
MoreVMs, Workshop collocated with Programming
Since the early conception of managed runtime systems with tiered JIT compilation, several research attempts have been made to accelerate the bytecode execution. In this paper, we extend prior attempts by performing an initial analysis of whether heterogeneous hardware accelerators in the form of Graphics Processing Units (GPUs) and Field Programmable Gate Arrays (FPGAS) can help towards achieving higher performance during the bytecode interpreter mode.
ACM Transactions on Architecture and Code Optimization
In this article, we present FastPath_MP, a novel low-overhead and energy-efficient storage multi-path architecture that leverages FPGAs to operate transparently to the main processor and improve the performance and energy efficiency of accessing storage devices. We prototyped FastPath_MP on both Arm-FPGA Zynq 7000 SoC and Zynq UltraScale+ MPSoC and evaluated its performance against standard microbenchmarks as well as the real-world in-memory Redis database. Our results show that FastPath_MP achieves up to 82% lower latency, up to 12× higher throughput, and up to 10× more energy efficiency against the baseline storage path of the Linux kernel.
MPLR
The advent of modern cloud services, along with the huge volume of data produced on a daily basis, have increased the demand for fast and efficient data processing. This demand is common among numerous application domains, such as deep learning, data mining, and computer vision. In recent years, hardware accelerators have been employed as a means to meet this demand, due to the high parallelism that these applications exhibit. Although this approach can yield high performance, the development of new deep learning neural networks on heterogeneous hardware requires a steep learning curve. The main reason is that existing deep learning engines support the static compilation of the accelerated code, that can be accessed via wrapper calls from a wide range of managed programming languages (e.g., Java, Python, Scala). Therefore, the development of high-performance neural network architectures is fragmented between programming models, thereby forcing developers to manually specialize the code for heterogeneous execution. The specialization of the applications' code for heterogeneous execution is not a trivial task, as it requires developers to have hardware expertise and use a low-level programming language, such as OpenCL, CUDA or High Level Synthesis (HLS) tools. In this paper we showcase how we have employed TornadoVM, a state-of-the-art heterogeneous programming framework to transparently accelerate Deep Netts on heterogeneous hardware. Our work shows how a pure Java-based deep learning neural network engine can be dynamically compiled at runtime and specialized for particular hardware accelerators, without requiring developers to employ any low-level programming framework typically used for such devices. Our preliminary results show up to 6.45x end-to-end performance speedup and up to 88.5x kernel performance speedup, when executing the feed forward process of the network's training on the GPUs against the sequential execution of the original Deep Netts framework.
The Art, Science, and Engineering of Programming
In recent years, heterogeneous computing has emerged as the vital way to increase computers’ performance and energy efficiency by combining diverse hardware devices, such as Graphics Processing Units (GPUs) and Field Programmable Gate Arrays (FPGAs). The rationale behind this trend is that different parts of an application can be offloaded from the main CPU to diverse devices, which can efficiently execute these parts as co-processors. FPGAs are a subset of the most widely used co-processors, typically used for accelerating specific workloads due to their flexible hardware and energy-efficient characteristics. These characteristics have made them prevalent in a broad spectrum of computing systems ranging from low-power embedded systems to high-end data centers and cloud infrastructures.
DATE
This paper addresses the fundamental question of how modern Big Data frameworks can dynamically and transparently exploit heterogeneous hardware accelerators. After presenting the major challenges that have to be addressed towards this goal, we describe our proposed architecture for automatic and transparent hardware acceleration of Big Data frameworks and applications. Our vision is to retain the uniform programming model of Big Data frameworks and enable automatic, dynamic Just-In-Time compilation of the candidate code segments that benefit from hardware acceleration to the corresponding format. In conjunction with machine learning-based device selection, that respect user-defined constraints (e.g., cost, time, etc.), we enable dynamic code execution on GPUs and FPGAs transparently to the user. In addition, we dynamically re-steer execution at runtime based on the availability of resources. Our preliminary results demonstrate that our approach can accelerate an existing Apache Flink application by up to 16.5x.
PhD Forum - DATE
The rise of social networks along with the growth of big data have driven storage systems to cope with large scale of data volume. In recent years, the emergence of the Non-Volatile Memory Express (NVMe) standard has enabled SSD drives to deliver high I/O rates by leveraging the fastest available interconnect (i.e. PCIe) to the processing chip. Additionally, the appearance of FPGAs in data centres is creating opportunities to accelerate not only application functionality but also OS operations. Although the majority of the servers in data centres have been connected to the FPGAs via the PCIe interconnect, there are now also available heterogeneous System on Chips (SoCs) with multi-cores and FPGAs integrated on the same die, resulting in low latency and energy efficient communication. This work analyses the source of performance overhead in existing state-of-the-art storage devices and proposes a novel low overhead and energy efficient storage path called FastPath, that operates transparently to the processing cores. The experimental results showed that FastPath can achieve up to 82% lower latency, up to 12x higher performance, and up to 10x more energy efficiency for standard microbenchmark on an Arm-FPGA Zynq 7000 SoC. Further experiments were conducted on a state-of-the-art SoC, such as the Zynq UltraScale+ MPSoC, using a real application, such as the Redis in-memory database, which received requests by the Yahoo! Cloud Serving Benchmark (YCSB). The experimental evaluation showed that FastPath achieved up to 60% lower latency and 15% higher throughput than the baseline storage path in the Linux kernel.
FCCM
In this work, we propose an approach for transparent compilation and execution of Java programs onto Intel FPGA devices. In detail, we showcase how a managed runtime environment can leverage Intel OpenCL SDK to generate specialized FPGA code, enabling prototyping and acceleration of Java Programs onto FPGAs. Finally, we describe our implementation in the context of TornadoVM with a clear objective to ease FPGA programmability allowing integration with existing frameworks.
FPL
The constant growth of data and its importance to drive Machine Learning and Big Data is pushing storage systems towards ever increasing I/O bandwidth and lower latency requirements. In recent years, the Non Volatile Memory Express (NVMe) standard has enabled SSD drives to deliver high I/O rates by allowing the storage to be connected directly via the fastest available interconnect to the processing chip. In parallel, the adoption of FPGAs in data centers is creating opportunities to accelerate various applications and/or Operating System (OS) operations. While, FPGAs in data centers have been connected via PCIe to mostly x86 servers, we have now also available heterogeneous SoCs with multi-cores and FPGAs integrated on the same die and connected by an on-chip interconnect. In this paper, we present how to rethink and accelerate NVMe performance on heterogeneous SoC with integrated FPGAs providing a first research insight on the performance benefits of such an approach. We provide an analysis of the Linux block I/O layer and showcase the relationship between the system's performance and its I/O implementation. Consequently , we introduce an FPGA-based fast path which accelerates the access to the NVMe drive. Our comparative evaluation demonstrates that our FPGA-based FastPath achieves up to 71% lower latency and up to 5x higher I/O performance against the baseline system on a Zynq development board.
Arxiv cs.DC
The end of Dennard scaling combined with stagnation in architectural and compiler optimizations makes it challenging to achieve significant performance deltas. Solutions based solely in hardware or software are no longer sufficient to maintain the pace of improvements seen during the past few decades. In hardware, the end of single-core scaling resulted in the proliferation of multi-core system architectures, however this has forced complex parallel programming techniques into the mainstream. To further exploit physical resources, systems are becoming increasingly heterogeneous with specialized computing elements and accelerators. Programming across a range of disparate architectures requires a new level of abstraction that programming languages will have to adapt to. In software, emerging complex applications, from domains such as Big Data and computer vision, run on multi-layered software stacks targeting hardware with a variety of constraints and resources. Hence, optimizing for the power-performance (and resiliency) space requires experimentation platforms that offer quick and easy prototyping of hardware/software co-designed techniques. To that end, we present Project Beehive: A Hardware/Software co-designed stack for runtime and architectural research. Project Beehive utilizes various state-of-the-art software and hardware components along with novel and extensible co-design techniques. The objective of Project Beehive is to provide a modern platform for experimentation on emerging applications, programming languages, compilers, runtimes, and low-power heterogeneous many-core architectures in a full-system co-designed manner.
FPL
Graph mining is an important research area within the domain of data mining. One of the most challenging tasks of graph mining is frequent subgraph mining. This work presents the first FPGA-based implementation, to the best of our knowledge, of the most efficient and well-known algorithm for the Frequent Subgraph Mining (FSM) problem, i.e. gSpan. The proposed system, named High Performance Computing-gSpan (HPC-gSpan), achieves manyfold speedup vs. the official software solution of the gboost library when executed on a high-end CPU for various real-world datasets.
Nothing matches that filter.