HiRSE Seminar on Continuous Benchmarking, 2024-12-12, 14:00 CET

The Helmholtz Platform for Research Software Engineering has a regular seminar series that might be of interest to the CSDMS community.

This Thursday at 2 PM Central European Time there will be a seminar about continuous benchmarking. See below for details and a Zoom link.


You are warmly invited to our final seminar for 2024 on Thursday 12th December at 2pm when Dr Dirk Brömmel from Jülich Supercomputing Centre (Forschungszentrum Jülich) will join us at the HiRSE Seminar series to talk about Integrated Continuous Benchmarking.

Abstract: When developing software, it is often relevant to track its performance. As foundation for research where available time can be a limiting factor, this should hold even more true for research software. It is even vital when targeting high-performance computing (HPC). Since changes to the software itself, the used toolchains, or the system setup should not compromise how fast users obtain their results, benchmarking should be an integral part of testing, in particular for HPC codes.

Ideally, performance or scalability should only ever increase. At the same time, up-to-date benchmarks that are publicly available can advertise the code and inform users how to set-up the software in the most ideal way or whether they are achieving the expected performance. To limit the burden on developers, the aforementioned steps should be automated within continuous integration (CI) practices, introducing continuous benchmarking (CB) to it.

We show our approach that works standalone without additional services and combines results over time to enable further processing or for the generation of webpages. Via a fully authenticated runner on HPC systems, it also adds the possibility of benchmarks up to system scale.

Top tip: You can subscribe to the HiRSE Seminar Calendar using thislink: EUDAT B2DROP

The meeting will be held via Zoom: Launch Meeting - Zoom

Find the meeting ID and passcode in the calendar link

2 Likes

Thanks for sharing this. Will the seminar also be recorded or is it only available live?

They usually share them on the HiRSE YouTube channel: https://www.youtube.com/@Helmholtz_Platform_for_RSE

1 Like

Thanks, @wkearn. This sounds like it will be an interesting talk but might be a little early for me. I will definitely check it out if it’s posted to the HiRSE YouTube channel.

Along the lines of continuous benchmarking, we’ve been experimenting with Airspeed Velocity. Has anyone else tried this?

Here is a quick summary for those not wanting to wait for YouTube:

Most of the talk was about the benchmarking solution they have created at Forschungzentrum Jülich for their HPC applications based on running their JUBE tool on their HPC systems through a custom GitLab CI runner. There are some interesting concepts in their system architecture that I think would be useful for someone trying to build a similar system, but their tool is very much designed for their particular use case.

Bencher was mentioned as more ready-made solution.