Hello folks - I learned about the BMI ecosystem a few months ago, and I was inspired to put together and share the project I’ve worked on. Thanks to @mdpiper for a friendly conversation and encouragement!
I am calling it the bmi-operator (justin/bmi-operator, personal webhost link). It’s an orchestration tool that takes BMI models and runs them on Kubernetes, the distributed computing platform. Its main feature is queueing and durability: Kubernetes orchestrates the simulations and stores result data, so it’s easy to start many simulations with slight variations and check in on them, and access the results later.
It does also allow you to run multi-model scenarios: similar to how Landlab supports multiple components operating on a shared grid - but distributed! I did replicate a few Landlab tutorials using the tool to test it out, like the transport-length hillslope diffuser tutorial (my copy). I also included a version of the cem-grpc4bmi tutorial (my version). The interface looks roughly like this:
spec = bmi_operator.run.Spec(
name="tlhdiff-tutorial",
models=[flow_director, diffuser], # BMI model container parameters set up earlier
dt="1000yr",
end_time="2000000yr",
outputs_domain=outputs, # uses cloud storage for results
variables=[
bmi_operator.run.Variable( # set initial data and continuously save the variable through the simulation
"topographic__elevation",
seed=starting_data["topo.npy"],
every="100kyr",
)
],
)
run = client.runs.submit(spec)
run.wait(timeout=1800)
results = run.results()
times, elevations = results.topographic__elevation.every("100kyr")
And for the evidence, here’s a visualization you might make of the resulting timeseries data:
As for how it’s built: it uses the Kubernetes operator pattern, relies on the BMI models being containerized and integrated with grpc4bmi, and uses helm for installation. There’s a Python client library that is meant to be used in a Jupyter notebook to interact with it. If you have Docker, you can try it out! For someone developing a new model, I don’t think it would be useful. But it has some capabilities for someone who would be trying to use existing models, running many variations or long-running tasks.
I’ve enjoyed working on this and exploring some of the modeling work and research being done using BMI. I’m an infrastructure software engineer by trade, so, I thought an infrastructure project using tools I know would be a good way to take a first look at some of what CSDMS has built - and sometimes the best payoff of good API design, in my experience, is the platforms and tools you can easily build on top of it!
I’m happy to discuss the project, but even moreso, would love to contribute to the open-source BMI and CSDMS ecosystem where I can. Feel free to share your thoughts or email me at justin@palpant.us, I’m happy to chat, receive feedback, and learn!
Honest disclosure: I also used this project to gain some more experience with coding using LLMs - it’s taking off at my work, and I wanted to practice. I have kept a close eye and done a fair amount of coding the normal way, but certainly there are signs, styles, and likely, errors, from trying to develop in this way. It’s very alpha-stage work, and I’m also happy to share some of the pitfalls I ran into.