I’m not an energy modeller - I’m a software engineer and I’m trying to build some level of awareness of carbon intensity into tools for managing datacentres and working out where and when to schedule computing loads, and I have a couple of questions for areas outside my domain of expertise. I hope that’s ok.
Background
For those interested - here’s the issue where I’m listing notes to build some of these into a go software library, for consumption inside software for managing datacentre and servers like Kubernertes, Nomad and so on.
for those new to it, Nomad is a fairly well known open source piece of software that powers a significant chunk of the internet. Amongst others, it’s used by Cloudflare for example handle tonnes of traffic, and they detail their use here.
I’ve been chatting to the team there, and there’s now a prototype branch being worked on, that consumes the go library I’ve linked above.
That would mean anything that uses Nomad can also use carbon intensity information, to move computing loads through space with annual data, or assuming there is hourly data, move computing loads to time and space, to where the carbon intensity is lower.
You can see the readme below, which outlines at a high level how it’s planned to work:
For context, datacentres have surprising amount of flexibility for demand.
A typical rack can consume 10-20 kw of power, and well… datacentres can fit lots of racks into rooms, of which less than 20% are typically being used by compute jobs, with a signficiant majority waiting idle.
Sidenote: Tools like Nomad allow this idle capacity to be reduced, by moving reliability to another level - instead of having reliability through redundancy within the datacentre, you achieve redundancy across datacentres, and increasing the ‘density’ of compute on the servers that are running
Methodology qn 1 - global, annual marginal intensity of electricity:
I have a methodology question about marginal intensity figures, and I’d like to be able to derive sensible global or “europe” fallback figures, for cases when a machine doesn’t have access information about where in the world it’s running.
I know that publicly accessible marginal intensity figures exist on an annual basis for various countries. We’ve spoken about this a few times before, and the github issue linked above adds some more information.
The dataset listed above gives annual, per country based marginal intensity figures, based on the ‘operating margin’ and the ‘build margin’ numbers as documented in the issue.
If you wanted a global, annual figure for marginal intensity of electricity for say… 2021 I’m a little unsure of how you’d make this though.
Would you need a weighted average based on the assumed capacity or generation in each country? Or is there a more sensible approach than this?
If so, I know that Ember release some of this capacity / generation data, to make it possible to figure these numbers out, and they are licensed along very permissive lines. Is this a valid approach to take? If not, what would you suggest instead?
Qn 2 - Use cases beyond computing scheduling tools.
This project I’m working on art of this project where the goal is to find a way to annotate every single public address on the internet with some baseline carbon intensity data, so that other projects can consume this information for other interesting uses.
Once example I’ve seen is the idea of a carbon aware routing that ranks the routes for transmitting packets of data across the web, based on carbon intensity
Some fascinating ideas blogged. Let me add a couple of thoughts. Use of the instantaneous short‑run marginal carbon intensity of electricity at the point‑of‑supply is certainly conceptually attractive. And in some senses, mirrorsnodal pricing (aka locational marginal pricing or LMP) with the unit price replaced by the carbon intensity in this case. Hence marginal carbon shares some of the attributes of LMP clearance — while assuming away any strategic behavior here of course.
In LMP however, plant operators look forward and take stock, for instance, of their hydro reserves against their expected hydrology. But there is no equivalent foresight in marginal carbon intensity in the first instance. Notwithstanding future values could be estimated using system models and duly incorporated into the management of server farm loadings. But the raw marginal intensity does not provide especially useful information in this context, I don’t think.
Much the same problem is faced by those devising charging strategies and price incentives for electric vehicle users. Indeed with growing emobility, the low voltage network peak is increasingly being felt at midnight and not early evening as is historically the case.
So my suggestion is that some form of anticipatory marginal carbon intensity would be required to manage low carbon server farms. That said, the idea of routing internet traffic and shifting data crunching to better locations in time and space is very appealing across a range of scales from minutes to hours — but some prescience about the future evolution of the energy system is doubtless required too.
Once everything on the system is carbon neutral, this question naturally evaporates. HTH, R.
The libraries I’ve linked consume APIs from providers like Watttime - they provide marginal intensity APIs specifically to provide this kind of _ anticipatory marginal carbon intensity_ functionality.