Parallel Programming Environments and Domain-Specific Compilation

NHR@SW Method Lab

To the overview of the Method Labs →

Modern HPC systems combine increasingly heterogeneous processors and accelerators while continuing to grow in scale and complexity. Developing portable and scalable applications for these systems therefore requires suitable programming models, higher-level abstractions, and compiler technologies that can bridge the gap between scientific application logic and diverse hardware architectures.

The Method Lab Parallel Programming Environments and Domain-Specific Compilation supports the NHR community in selecting, applying, and developing programming approaches for modern HPC systems. Its expertise ranges from MPI, OpenMP, GASPI, one-sided communication, PGAS, and task-based programming to domain-specific languages, metaprogramming, DSL embedding, and compiler infrastructures such as LLVM.

A particular focus lies on domain-specific programming and high-performance derivative computations. The Method Lab adapts AnyDSL and related technologies to scientific use cases and supports differentiation workflows using open-source software such as CoDiPack, MeDiPack, and OpDiLib.

Through individual consulting, collaborative development, open-source software, and hands-on training, the Method Lab helps researchers improve the portability, scalability, and maintainability of scientific applications on current and emerging HPC architectures.

Parallel ProgrammingDomain-Specific LanguagesCompiler TechnologiesHeterogeneous SystemsAlgorithmic Differentiation
01

Competencies

Parallel Programming Models

Applying MPI, OpenMP, GASPI, one-sided communication, PGAS, and task-based programming to scalable scientific applications.

Domain-Specific Languages

Designing and implementing domain-specific abstractions that separate scientific application logic from hardware-specific execution strategies.

Compiler Technologies

Using metaprogramming, DSL embedding, intermediate representations, and compiler infrastructures such as LLVM.

Heterogeneous Programming

Adapting applications and programming environments to increasingly diverse processors and accelerators.

Algorithmic Differentiation

Supporting efficient derivative computations in sequential, MPI-parallel, and OpenMP-parallel scientific applications.

Scalable HPC Workflows

Developing and improving HPC workflows, including differentiation workflows and parallel visualization.

02

Services

  • Individual consulting on parallel programming models and suitable implementation strategies
  • Support for adopting MPI, OpenMP, GASPI, PGAS, and task-based programming
  • Development and integration of domain-specific languages using AnyDSL and related technologies
  • Support for metaprogramming, DSL embedding, LLVM, and compiler-based optimization
  • Integration of algorithmic differentiation into parallel scientific applications
  • Assistance with scalable HPC workflows and parallel visualization
03

Research

  • MPI, OpenMP, and GASPI
  • One-Sided Communication and PGAS
  • Task-Parallel Programming
  • Reproducible Communication Benchmarking
  • Cache-Aware MPI Benchmarking
  • Domain-Specific Languages
  • Metaprogramming and DSL Embedding
  • Compiler Infrastructures and Intermediate Representations
  • SPMD Kernels on CPUs and Accelerators
  • Heterogeneous Programming
  • Algorithmic Differentiation
  • Parallel Discrete Adjoints
04

Open-Source Software

Algorithmic Differentiation

CoDiPack

A C++ library for efficient forward- and reverse-mode algorithmic differentiation, including higher-order derivatives and different tape implementations.

View source → Differentiable MPI

MeDiPack

A message differentiation package that enables MPI communication for applications using algorithmic differentiation tools such as CoDiPack.

View source → Differentiable OpenMP

OpDiLib

An add-on for operator-overloading algorithmic differentiation tools that enables the differentiation of OpenMP-parallel programs.

View source → PGAS Benchmarking

GPI-2 Benchmarks

A collection of benchmarks for evaluating communication patterns and performance using the GPI-2 programming interface.

View source → MPI Benchmarking

Cold-Cache MPI

A benchmarking approach for studying MPI point-to-point performance while accounting for distortions introduced by processor caches.

View source → Portals 4 Benchmarking

PtlBench

A microbenchmark suite for evaluating Portals 4 communication operations and network behavior.

View source →
05

Training Activities

  • Efficient Parallel Programming with GASPIHands-on introduction to scalable one-sided communication and PGAS programming with GASPI
  • Introduction to CoDiPack and MeDiPackImplementing efficient algorithmic differentiation in C++ and MPI-parallel applications
  • Heterogeneous Programming with SYCLDeveloping portable parallel applications for heterogeneous processors and accelerators
  • Getting Started with LLVM – Build Your Own CompilerIntroduction to LLVM and the practical development of a simple compiler
  • Python for ScientistsUsing Python effectively for scientific computing, data processing, and HPC workflows
  • Parallel Programming with MPI and OpenMPFour-day workshop on distributed- and shared-memory parallel programming, offered in cooperation with HLRS
06

Community Activities

JuliaCon Global 2026

NHR@SW supported JuliaCon Global 2026 in Mainz as a local partner.

CERN DRD6 Collaboration

The Method Labs Parallel Programming and AI and ML contributed to the successful proposal establishing the CERN DRD6 “Collaboration on Calorimeter Design.” Nicolas R. Gauger serves on its Collaboration Board.

27th EuroAD Workshop 2025

The Method Lab hosted the 27th EuroAD Workshop at RPTU, supporting exchange within the algorithmic differentiation community.

International Research Exchange 2023

The Method Lab contributed to the ALICE-FSP meeting and the CERN SFT Group Meeting through presentations and discussions on algorithmic differentiation and differentiable detector simulations.

07

Team

Lead PIs

Prof. Dr. André BrinkmannSaarland University
Prof. Dr. Nicolas GaugerRPTU Kaiserslautern-Landau · Speaker
Dr. Daniel GrünewaldRPTU Kaiserslautern-Landau / Fraunhofer ITWM
Prof. Dr. Sebastian HackSaarland University
Prof. Dr. Sarah NeuwirthJGU Mainz
Prof. Dr. Philipp SlusallekSaarland University

Team Members

Niklas BartelheimerJGU Mainz
Dr. Valentin ChuravySaarland University
Dr. Jonas KorndörferSaarland University
Katharina RothRPTU Kaiserslautern-Landau
Max SagebaumRPTU Kaiserslautern-Landau
Christian SendlingerGoethe University Frankfurt
Dr. Felix SpanierGoethe University Frankfurt
Joachim TotzeckSaarland University
AVAmritanshu VermaJGU Mainz
08

Selected Publications

2026

  • H. W. Yew and N. R. GaugerAlgorithmic Differentiation in Large-Scale Scientific ComputingPoster at the NHR Conference 2026
  • J. Blühdorn and N. R. GaugerLocal Adjoints for Simultaneous Preaccumulations with Shared InputsProceedings of the 2024 International Conference on Algorithmic Differentiation, pp. 178–191, 2026
  • N. Bartelheimer, J. Domke, and S. NeuwirthHow Caching Distorts MPI Point-to-Point Performance33rd European MPI Users’ Group Meeting, 2026

2025

  • J. Blühdorn, P. Gomes, M. Aehle, and N. R. GaugerHybrid Parallel Discrete Adjoints in SU2Computers & Fluids, vol. 289, article 106528, 2025
  • J. Blühdorn and N. R. GaugerSimultaneous Preaccumulations in OpenMP-Parallel Automatic DifferentiationPoster at the NHR Conference 2025
  • J. H. Müller Korndörfer, A. Mohammed, A. Eleliemy, Q. Guilloteau, R. Krummenacher, and F. M. CiorbaA Comparative Study of OpenMP Scheduling Algorithm Selection StrategiesIEEE Access, vol. 13, pp. 151216–151234, 2025 · DOI
  • M. Ullrich, S. Hack, and R. LeißaMimIrADe: Automatic Differentiation in MimIRProceedings of the 34th ACM SIGPLAN International Conference on Compiler Construction, pp. 70–80, 2025
  • R. Leißa, M. Ullrich, J. Meyer, and S. HackMimIR: An Extensible and Type-Safe Intermediate Representation for the DSL AgeProceedings of the ACM on Programming Languages, vol. 9, POPL, pp. 95–125, 2025

2024

  • N. Bartelheimer and S. NeuwirthLeveraging Portals4 Microbenchmarks to Enhance GASPI Performance on BXI Networks2024 IEEE International Conference on Cluster Computing Workshops · DOI
  • J. Blühdorn and N. R. GaugerOpenMP-Parallel Discrete Adjoints in SU2 with OpDiLibPoster at the NHR Conference 2024
  • A. Tarraf, M. Schreiber, A. Cascajo, J.-B. Besnard, M.-A. Vef, D. Huber, S. Happ, A. Brinkmann, D. E. Singh, H.-C. Hoppe, A. Miranda, A. J. Peña, R. Machado, M. Garcia-Gasulla, M. Schulz, P. M. Carpenter, S. Pickartz, T. Rotaru, S. Iserte, V. López, J. Ejarque, H. Sirwani, J. Carretero, and F. WolfMalleability in Modern HPC Systems: Current Experiences, Challenges, and Future OpportunitiesIEEE Transactions on Parallel and Distributed Systems, vol. 35, no. 9, pp. 1551–1564, 2024 · DOI

2023

  • N. Bartelheimer and S. NeuwirthToward Reproducible Benchmarking of PGAS and MPI Communication Schemes2023 IEEE 29th International Conference on Parallel and Distributed Systems, pp. 1959–1967 · DOI
  • J. H. Müller Korndörfer, A. Eleliemy, O. S. Simsek, T. Ilsche, R. Schöne, and F. M. CiorbaHow Do OS and Application Schedulers Interact? An Investigation with Multithreaded ApplicationsEuro-Par 2023: Parallel Processing, pp. 214–228 · DOI
  • F. M. Ciorba, A. Mohammed, J. H. Müller Korndörfer, and A. EleliemyAutomated Scheduling Algorithm Selection in OpenMP22nd International Symposium on Parallel and Distributed Computing, ISPDC 2023, pp. 106–109 · DOI
  • J. Meyer, A. Alpay, S. Hack, H. Fröning, and V. HeuvelineImplementation Techniques for SPMD Kernels on CPUsInternational Workshop on OpenCL, 2023