Managing a distributed grid

In this episode, Astrid Atkinson, co-founder of Camus Energy,
talks about her company’s “grid orchestration” work of helping
utilities see, track, and coordinate the distributed energy
resources in their territories.


(PDF
transcript)


(Active
transcript)


Text transcript:


David Roberts


One of my favorite things I ever wrote was a 2018 piece for Vox
on grid architecture — the basic structure of the electricity
transmission and distribution networks. It was about how a
top-down system, with one-way power delivery from big power
plants to passive consumers, might evolve into a bottom-up
system, driven by local distributed energy resources.


Thanks to all-star illustrator Javier Zarracina, it even has
awesome animated illustrations.


One person who read that piece was Astrid Atkinson, who at the
time was a senior software engineer at Google. She had managed a
team that shifted Google search from a top-down system to a
massively distributed system, back before the term “the cloud”
existed and there was no template available. She and her team had
to develop the principles and best practices of getting reliable
performance out of millions of unreliable, loosely coordinated
machines. By doing so, they radically expanded the scale and
speed of what search could do.


She thought, wouldn’t it be cool if the power grid could make the
same shift? Unlike some people, though, she didn’t just blog
about it — in 2019, she left Google to co-found and run Camus
Energy, a software company that helps utilities see, track, and
coordinate the distributed energy resources in their territories.
The company calls what it does “grid orchestration.”


Atkinson has been a thought leader in pushing for a new grid
architecture. (See Camus’ white-paper series on “the rise of
local grid management.”) So I was super-excited to geek out with
her on this stuff. We talked about the conceptual shift from
centralized to distributed and the drivers making that shift
inevitable, plus getting more out of the grid we’ve already built
through coordination and efficiency, and how the utility sector
can evolve to better manage local resources. I really loved this
one.


Okay, then. Astrid Atkinson. Welcome to Volts. Thank you so much
for coming.


Astrid Atkinson


Thank you so much. I'm really excited to be here.


David Roberts


I'm so excited for this. Astrid, I have to tell you just by way
of preface that I had a weirdly difficult time preparing for
today's pod because I'm just so excited by this whole area, and
I'm so jazzed. I have so many things to ask you about, so many
things I want to say about all this stuff, and I'm kind of
overwhelmed and fried my circuits. But let's start here: Let me
describe for listeners what you did at Google and tell me if this
is an accurate description. So you were part of a team, I think,
leading a team that was shifting the way Google did things away
from a model where computing was done on a relatively limited set
of high-quality, extremely reliable data centers, tightly
centrally controlled, to a model where computing is done not on a
small set, but on thousands, millions of distributed computers
living all over the place, any one of which might be unreliably
connected or off periodically or weak or otherwise glitchy.


So basically, moving from a model of tightly coordinated, central
control, limited number of entities, to loosely coordinated
millions of entities, somehow getting aggregate reliability out
of massively distributed, individually unreliable machines. Is
that more or less accurate?


Astrid Atkinson


Yeah, that's about right. So my role was in the site reliability
engineering team at Google, which is a function that nobody's
ever heard of outside of the kind of tech industry. But you can
think of reliability engineering as being basically Google's
systems engineering function. It's the entity that's kind of
responsible for pulling all of the pieces together between sort
of software and operations and networking and hardware and
everything, and making sure that you can get them to kind of work
as a reliable system overall. And I was part of the original team
that kind of — it wasn't a function that existed in the industry
before Google made that transition.


I was part of that original team at Google and then led a lot of
Google's work around scaling out that model.


David Roberts


And so now the idea, more or less is to oversee or encourage a
parallel evolution of the electricity grid, basically, from a
limited number of tightly centrally controlled entities to a
loosely coordinated, massively distributed, huge number of
smaller entities, basically.


Astrid Atkinson


Yeah, I mean, that's definitely the hope. And that's partly
derived from the utility industry and the grid space's sense of
the changes that are needed and also partly derived from my sense
that there are a fair number of parallels between some of the
approaches that we took and kind of had to make up on the spot to
support massive growth and really significant changes in the way
that we managed systems for that work at Google. And there are
parallels with the changes that we need to go through on the grid
side. So it's less like "there's this one piece of technology
that we built at Google that will totally solve the problems" and
more like "we had to develop a set of approaches and a set of
kind of integrative and system level perspectives to figure out
how to make that change happen."


And I think a lot of those can be helpful in the grid space.


David Roberts


Yeah, I mean, I think one of the most intriguing things about
this is in doing that work, you extracted a set of principles for
how to design systems like this such that they are reliable, et
cetera. And it's those principles, I think it's like conceptually
those principles apply to the grid. Obviously, the individual
technologies might be different, circumstances are different, but
the principles of how to make it work, I think are a weirdly neat
fit. The reason I think we should maybe pull out, the reason that
this is not something that the utility really can choose to do or
not to do.


Utilities kind of have to do it for a couple of reasons. One is
all these distributed energy resources, Volts listeners are
familiar with these distributed — with these solar panels and hot
water heaters that can store energy and batteries, et cetera, et
cetera. All these sort of behind the meter distribution side,
distributed resources are coming online. Whether the utilities
like it or not, they are swarming —


Astrid Atkinson


It's happening.


David Roberts


online, it's happening. And right now utilities are just like
kind of hoping it works out, we'll get into that. That's one
reason. But the other reason is we're expecting, like you could
say of Google, like it couldn't have scaled to the size it got,
it couldn't do the amount of computing it's doing without going
through this transition.


You just can't at a certain point with a centrally controlled
system where you're tightly controlling a limited set of
entities, you just can't get big beyond a certain level. You run
into sort of computational limits of your computational resources
for a central controller. And we're expecting a lot more out of
the electricity system in coming years, as Volts listeners are
also very familiar with. We're going to two or three x the demand
that it has to satisfy in a much more complex way. So I think it
could be argued, and you have argued, and I think it's pretty
self-evident, that the electricity system cannot achieve the
scale we want out of it without going through this evolution.


There's just no way for the way it's currently run to get as big
as we want it to get.


Astrid Atkinson


Yeah, and if you want some simple examples of why that is,
utilities have spent a lot of money and a lot of time in the last
10 to 15 years installing smart meters. Right? They were supposed
to give us the kind of universal visibility into customer
activity that I think we can all intuitively think that we would
need to manage a rapidly changing grid with a lot more
complexity. But most utilities don't really have particularly
high-scaled data infrastructure. And so the idea of actually
being able to do something useful with all of that meter data,
SCADA data, kind of everything at scale on an ongoing basis in
real time, using that as a foundation for analysis and visibility
and those kinds of things.


It's really hard for them because usually the software systems
that gather and process that data, they're on premise within the
utility's own data center. They're typically not really using any
kind of modern scalability approaches beyond downsampling the
data, which means losing some of it. And they're usually running
on a single machine. So it gets really difficult to incorporate
very large amounts of data when you can only use one computer.


David Roberts


Yeah, if you talk to them, they're like, "Oh my God, we've got
all this data. It's overwhelming. The reason we're not doing more
with it is like, we can't do more with it, we're overwhelmed."
And then you look at the scale of data involved, and to a Google
it's a tiny amount of data. The scales are completely different.


Astrid Atkinson


Yeah. Just as a back-of-the-envelope calculation, I was trying to
figure out how many data points do Google's monitoring systems
collect on a daily basis? And it should be somewhere in the order
of 6 to 10 trillion.


David Roberts


Good God. That's like whatever atoms in the universe. I don't
know what the right comparison is.


Astrid Atkinson


Google is where I learned the term exabyte. They sort of just
came into this with a perspective on scale that was sort of
orders of magnitude larger. And you don't really think about the
amount of computing work that's required to get you the right
answer in under a second to any question that you might have
through search, but the short answer is hundreds of thousands of
computers all at once.


David Roberts


Right.


Astrid Atkinson


You can just do more.


David Roberts


Right. And so the current way utilities are running this with
sort of a single machine on premises chewing through this data
serially is just not going to get anywhere close to that scale.


Astrid Atkinson


Yeah, and to be fair, they'll scale up to five or ten machines.
But the systems that we had to build to kind of manage the Google
scale tech computing, they weren't five to ten machines, they
were 50 to 500,000 machines.


David Roberts


Right. Which you're not going to do on premises. You have to
basically move to the cloud to widely distributed resources. So,
Astrid, all this sort of preface, conceptual preface, is so my
jam. You probably know this because I wrote a big article about
grid architecture a few years ago.


Astrid Atkinson


Yes. And that article, if you're talking about the one from 2018,
was really influential in how we started thinking about the space
a little bit before starting the company.


David Roberts


I mean, that was literally my dream, writing that article, is
that some smart person who understands how this works with
computers will come over to the utility industry and bring it to
the electricity system. Literally. Like, you're the answer to my
prayer. But what you may not know is that this is my jam going
even way farther back, like back in the midst of time, back in
the midst of prehistory. I was getting a philosophy degree, I was
working on a PhD in philosophy, and my whole thing was studying
cognitive science and consciousness and sort of cognition and how
cognition works.


And one of the things I was deep into is this shift of people
describing cognition as distributed and describing the human
identity as distributed. Sort of this illusion of a unity that
actually comes out of massively distributed, relatively dumb
nodes coordinating to produce this emergent behavior that we call
consciousness. So the whole idea of moving from centralized to
distributed, that was in my bones even as far back as grad
school. So now it's like popping up all over the world. These are
really just thrilling times for me personally, I guess, is what
I'm saying. It's so cool to see all this stuff come together.


So one of the things you say in your presentations and such is
the first — step one, if you want to do this, if you want to
coordinate massive numbers of nodes to get a desired set of
behaviors as an outcome, the very first thing you need is
knowledge of the nodes. You need to know what's out there and
what it's doing and what its capacity is. And in the computer
world, we have the system on the Internet, we have the system of
IP addresses, an individual address for every single computer
that's connected to the Internet. And there are, as we say,
millions and millions of them.


So right there, it seems like right there, your attempt to pull
this over into electricity looks like it grounds out because we
just don't have that. Like when I hook a water heater up to the
grid, I don't think it gets an individual identifier, does it? Or
a solar panel, or an EV battery or EV charger, et cetera, et
cetera. This is the whole problem, right, is all these devices
are hooking up to the grid and utilities don't know where they
are or what they're doing or when they're going to do what
they're doing. And I can't — have trouble imagining a system that
would individually identify them.


So right there, right off the bat, how do you overcome this first
and most difficult problem? Because as you say in one of your
presentations, the pyramid, you show the pyramid of reliability.
And the base level, the bottom, the level upon which the rest of
the levels are built is monitoring. You have to know what's going
on out there on the edge of your network. And in electricity we
don't. So how the heck do we overcome that problem to even get to
the other problems?


Astrid Atkinson


Yeah, so there's really, from a technical perspective, kind of
two questions buried in the one that you just asked. One is about
addressing. And it's true that we don't really have an IP address
system for the grid. But the reason that we need that for the
Internet is that we ultimately need each individual piece of work
to end up at a specific location. Like an email needs to end up
at a certain address or a DNS query needs to go to a certain
server in order to get a response. Whereas in the electrical
system, we're really sort of thinking more generally about
managing need for work to ability to do work.


So, managing supply to demand, it doesn't really matter which
particular electron serves the need. One of the primary goals is
just that there are always exactly the right number, always
exactly the right amount. So the problem space is a little bit
different in that perspective. It doesn't mean we don't need to
know what's on the grid and where it is. But you don't have quite
the same need for an absolutely universal addressing system. Just
as an example: One of the ways that one could tackle this is
there's a bunch of really interesting work happening in the grid
management and grid architecture space in Australia, which I'd
love to talk about a little bit more when we get further through
this.


But as part of that, they basically extended a universal
identifier system that applies both to meters for one set of
identifiers and then to generators for another set. And that's
just kind of the country deciding to do things a certain way. And
it definitely, I think, can be a helpful enabler to have that. In
the absence of that, you can correlate it across multiple sources
because within any given utility, they do actually have a way to
identify specific assets. Typically that's something associated
with the meter number. So it's usually like a site identity or
like a site number or something like that that identifies a
particular customer location that associates with a billing
relationship and associates usually with a specific physical
meter and is like kind of the identity component that ties those
things together.


David Roberts


Right, but isn't the problem that behind any one of those meters
might be multiple devices that behave very differently and have
very different timing and etc.?


Astrid Atkinson


Yes, and should you want to be able to tie together the activity
of those devices with the utilities identification mechanism,
you're going to need to write some software. So one could do this
today by basically just like writing software to link together
the program identifier or the device asset identifier. Let's say
you're talking to ChargePoint chargers. You can get like a
ChargePoint charger ID, and with a modest amount of software, you
can link that together with a meter ID. It's all kind of a pain
in the ass, but it's doable.


David Roberts


Would it not be easier with something like an IP system, though?
I mean, is there any talk about is that a gleam in anyone's eye?


Astrid Atkinson


Yeah, I do like the Australian model, I think they call it. It's
like a distributed generation ID and then there's like a meter
ID. It's basically just an agreement that those things are going
to be unique across multiple systems. I think one of the sorts of
things that could be easily extracted as an example and it
certainly would help. I think there's a lot you can do without
it, but it would help. The other thing that's kind of buried in
your question, though, is "how can we see what's out there and
know what's happening?" And when we talk about monitoring in the
Internet space, as you mentioned from my diagram, which is
actually drawn by a former colleague of mine, Mikey Dickerson,
who went to go lead up — I don't know if you remember, but the
healthcare.gov rescue team under the Obama administration.


David Roberts


Heroes.


Astrid Atkinson


Mikey was a colleague of mine at Google, and he went to go lead
up that rescue effort. And so the diagram you're mentioning,
which is a lovely one. It's just like a little pyramid of — kind
of like Maslow's pyramid of needs, but for reliable computing —
it does have monitoring and system visibility as its foundation.
And what that looks like in the Internet computing space is that
for every systems component, whether you're talking about an
individual server or a network router or a piece of software or
whatever that's operating within the construct of a larger
system, you kind of want to know what it's doing.


That's why Google collects so many data points for their systems,
because they have quite a lot of them. But this is common across
the entire modern computing industry. If you're running a large
system, you would typically have a monitoring system that would
pick up ongoing telemetry, so like heartbeat information and
activity information from literally every component and then
assemble that. Firstly store that data and then assemble it into
some operator friendly dashboards and ways to keep an eye on what
the system is doing. We don't really have that, particularly the
distribution grid today. We do have it on the transmission side,
but that's not really how the distribution grid rolls at the
moment.


David Roberts


Yeah, and so absent a system like that being put into place with
unique identifiers for every little piece, ultimately it seems
like you're in the end modeling, basically estimating on some
level, are you not?


Astrid Atkinson


It's a mix usually. So in practice what that looks like is you do
end up with unique identifiers for those assets because if you're
bringing in a data stream from an EV charger or a battery or a
solar inverter or something like that, you want to know which one
it comes from. So in the absence of like a universal addressing
scheme, the way that we would do that from a technical
perspective is we'd say we've correlated this battery with this
site ID, so its identity is now site-id.battery or something like
that in our database. So you're actually creating that notion of
identity to do that.


And then we're pulling in data, or whoever is pulling in data
from ideally every one of those things. Now those things are all
exporting data all the time anyway. Like their vendors, their
installers are always installing them along with some kind of
remote monitoring capability.


David Roberts


Right. Just not to the utility.


Astrid Atkinson


No. And as mentioned, partly that's because the utility in
general is not very good at dealing with very large amounts of
data, but it's also because that's a lot of vendors to talk to
and a lot of devices to talk to.


David Roberts


Yes, and I would imagine a lot of different data formats and it
must be a mess of information.


Astrid Atkinson


This is where the modeling stuff comes in because in general,
there's lots of data available, but it's almost never universal,
it's almost never consistent across every utility or device type
or whatever. Sometimes you'll have utilities that have smart
meters with 20% of their customers, but not the others. Or you
might have ones where they have meters that give you a monthly
read for most customers and then a small subset that can give you
more frequent data. You might have utilities that know where all
their rooftop solar is and have even production meters that tell
you what it's doing, but most have an interconnection record
that's kind of separate from any other data set.


Maybe it's in their GIS. And most don't have a production meter
to tell you what's happening there. And so if you want to make
sense of all of that, you do need to use a fair bit of modeling
so that can be from everything. Like this meter data has a bunch
of drops where the network dropped out and I need to fill it in
so that the time series is complete to like the meter data has a
different frequency, like it's sampled every 15 minutes and SCADA
data is sampled every minute and I want to compare them.


So now, I need to fill in minutely data points using a model. Or
it might be I want to have some idea of what rooftop solar is up
to, but I don't have any production meters. I do know where there
are rooftop solar installations on my system though. And we do
have modeling tools that can do a forecast for rooftop solar. So
then, what you can do is forecast the likely output of a panel
that's yay big at location X, match that against the data that
you actually have from the meter, and then form more complex and
more complete machine learning models, which can then give you a
model-based view of what's probably happening.


David Roberts


Right. So this is where the machine learning and the AI comes in.


Astrid Atkinson


That's right.


David Roberts


Taking the data you do have and extrapolating to a more complete
—


Astrid Atkinson


Absolutely, and that's really valuable, not just for things like
forecasting, which is kind of the most common use case for it,
but it's also useful for understanding what might be happening
right now because most utilities don't really have great
visibility in real time. To anything that's sort of below the
substation or feeder circuit or maybe they have reclosers or
something on the line but they don't really have any visibility
at the distribution transformer or at the meter that is anything
like real time. Meter data usually trails by between 2 and 48
hours.


David Roberts


As I was reading about this and thinking about this, my mind kept
bumping up against one thing, which is that a model like this in,
I don't know, like global shipping of goods, where maybe you lack
certain information sets and you're deriving them from other
information and making a model. A sort of like extremely educated
guess. And you get close enough for government work, you get
close enough to make the thing work. But in electricity systems,
you really got to get it exactly right. Like the supply and
demand have to be exactly matched at every second. Can you get
from the limited data we have now, add a bunch of machine
learning and AI to derive a global picture of what's happening?
Is that global picture going to be accurate enough to meet the
sort of reliability standards we require of electricity
specifically? Like people are not very forgiving if you have
little gaps.


Astrid Atkinson


So the key question is "reliable enough." Is it going to be
perfectly reliable? Absolutely not. And for high fidelity low
latency applications, you need high fidelity low latency data.
But a lot of our applications are not high fidelity and low
latency. And so there's a lot to unpack in terms of how much data
you need for particular use cases. But broadly speaking, for
things like maintaining frequency of the bulk system, that's
something that needs to be done on like a subsecond basis.
Obviously, it's kind of a 60 Hz basis, but we have that
visibility on the transmission system, and we have transmission
operators whose job it is to maintain that frequency.


So if you don't have that kind of fidelity on the distribution
side, that's not necessarily going to impact stability from a
frequency maintenance perspective, especially if you do have fast
response assets like batteries and stuff like that, that the
transmission operator can call on to provide services. And there
the distribution operator, even if it's distribution connected,
doesn't really need to know about that as long as somebody knows
that they need it and can call on it. So when we think about what
we need in terms of data completeness on the distribution side,
it kind of depends on the use cases, right? If we want to be able
to just use flexible load to move our peak usage around so that
you're not running up huge demand charge bills on behalf of the
utility at peak usage times in the evening, you don't really need
real-time data for that.


You can just kind of move it around on a day ahead basis, right?
If you want to be able to manage, let's say you've got a house
with five powerwalls and you want to make sure that when you
charge those powerwalls, they don't blow up the transformer that
house is connected to. That's also something that benefits from
more real-time data, but doesn't strictly require it because you
can just kind of put a bit of a margin for error on your
calculation of transformer capacity and make sure you operate
within it. What you lose is a lot of additional efficiency and
the more you want to actually use that active management to
manage reliability and grid capacity conditions, the more you do
really need the actual real-time data.


But it can be a process. It doesn't have to be perfect from day
one.


David Roberts


Right. I mean, a lot of this is just about how to get started
with the crappy systems we have in front of us. I'm sort of like
—


Astrid Atkinson


I like to refer to it as real world data.


David Roberts


Right, which anyone who's dealt with the real world knows is
generally pretty crappy. So what's the sort of balance of efforts
of trying to organize people to produce more data on these
things, right. Or harmonize data, or come up with some sort of
harmonized transferable, mutually communicatable data systems
versus effort put into the machine learning and AI that can make
hay out of existing data? Does that make sense? Are you just
running with the data you got? Or are there also efforts underway
to produce better data or to harmonize data across all these
systems?


Astrid Atkinson


Yeah, there's really both. So there's a bunch of really good work
that's happening in the industry to try to get much higher
fidelity data collection and reporting at either the meter or an
equivalent kind of customer side component. So companies like
Sense do this, SPAN panels can do this. The meter manufacturers
are all working on it.


David Roberts


Are the smart meters that got installed in that first wave of
smart meter installs, are they useful for this? Like, are they
producing good data for this kind?


Astrid Atkinson


The real issue with smart meters, especially in the US, at least
from my perspective, is not actually the meters themselves: They
collect data every 15 minutes, which is pretty sparse in general
—


David Roberts


Smart meters do?


Astrid Atkinson


Yeah, sometimes. Often it's every hour and sometimes it's every
month.


David Roberts


That's not super smart.


Astrid Atkinson


It's not very fast. No. And a lot of times it really has to do
with the amount of data that the head end system, which is like
the software system that picks up the data, can bring back. It
also has to do with the bandwidth of the network that's available
to the meter to bring data back to the meter as well. So in
general, we collect and store a lot less data than we could. And
a lot of that has to do with limitations of the communications
network and of — not enough computers to bring in the data.


David Roberts


So you could theoretically get more out of those in the future if
you built up the infrastructure.


Astrid Atkinson


Yes. The big issue with the data collection from smart meters
today, from my perspective, is that most of them in the US
communicate via an RF mesh network, which basically uses kind of
a — I don't know, you could think of it as like a bucket brigade
for data. It's like kind of tossing from one meter to a repeater
to an upline repeater, to an upline repeater. And God knows what
it's doing out there on the network, but it takes like often 6
hours to get a message back. I like to picture the data, like,
having a little picnic out there on the line because it's very
slow.


David Roberts


It's like The Hobbit. It's like trudging through fields and over
mountains.


Astrid Atkinson


Yeah, and most of those systems will let you get a single point
read from a single meter more quickly. But if you wanted
universal, somewhat real-time visibility into what's happening at
every meter today, for anyone who has those systems, the only way
to get that is to forecast it. And so that's where you're really
sort of looking at the role of real-time forecasting to try to
fill that gap. Now, the issue with getting more instrumentation
out there or even getting those kinds of technologies in place
isn't really a technical one. It's more that most utilities have
never had it and they haven't really felt a need for it.


Right.


David Roberts


Like, moving around large amounts of data is a solved problem,
let's say technologically.


Astrid Atkinson


It is something that we have great technologies for.


David Roberts


But they're just not using it. And just as a general comment —
and this is something I'd say frequently — I feel like ordinary
people who are out there very familiar with tech and the Internet
and all that kind of stuff would be surprised if they knew how
comparatively low tech the grid is compared to the systems that
they're using day to day life. It's wild that there are still
people finding out about outages by getting a phone call from
Bob, who's walking his dog, and that causes Jerry to have to go
throw a physical switch somewhere. Like, the whole thing just
seems incredibly like the more you learn about it, you're like,
still in 2023?


I don't know if this was your impression when you started getting
into it.


Astrid Atkinson


I remember going to do a tour at San Diego Gas and Electric for
their operations center not that long after starting the company.
And the difference between data fidelity and kind of amount of
real-time operations on the transmission side versus the
distribution side is pretty shocking. The transmission side is
everything you'd expect. Right. I remember when we did that tour,
they walked us into a conference room, gave us a little advanced
talk, and then flipped a switch and the entire frosted glass wall
of the conference room went transparent. And you could see their
transmission operations room with their gigantic screens and all
these operators in their little consoles with each one has half a
dozen computers around them and it looks really high tech.


You go over to the distribution side and it's like a bunch of
desks people maybe have two monitors, and it's mostly people
taking calls and making calls to roll trucks to fix outages. We
have the technology. We even have it in the grid. We just need to
scale it.


David Roberts


Yeah. Distribution systems basically have been neglected and
badly need now to be beefed up in a number of different ways
because of the aforementioned flood of DERs coming.


Astrid Atkinson


Yeah, and if I can mention one of the parallels with my past life
doing operations at Google, because part of my job was actually
like on-call operations. I led the team that was responsible for
Google's homepage for about five years. So you went to Google.com
to see if your Internet was on between 2007 and 2012. That was my
team and I carried a pager myself for it. One of our big
strategic efforts as we were doing that work was really thinking
critically about building tools to help operators understand
system scale. So if you think about not only a very large system,
but also one that's undergoing constant change, where there might
be dozens or hundreds or thousands or tens of thousands of
individual systems interacting, changes are happening all the
time that you don't know about.


And that's true even in a Google type environment where you'd
think it would be all coordinated. But Google has like 100,000
software engineers. It might be 200,000 by now. And good God,
they can generate new things quickly. And so from the operations
side, one of your big challenges is making sure that what's
happening within that changing system is comprehensible to a
small group of people.


David Roberts


Yes. Can't change human bandwidth.


Astrid Atkinson


That's right. And so one of our kind of core philosophies about
that was we wanted to have tooling that would let us support
exponential growth in the system with sublinear growth of the
operations workforce required to support it.


David Roberts


Right.


Astrid Atkinson


And so that means you're making a lot of ongoing investments in
tooling and stuff like that. But it also means that that
technology investment is a fundamental part of how you scale the
system and that operations function is kind of intrinsically tied
to how and whether you can scale the system. So, you know, when I
saw that room at San Diego Gas and Electric of their distribution
operators working really hard to manage the complexity of the
system that they have, I kind of look at that from a systems
engineering perspective and kind of a large systems perspective.


And just think, like, wow, making sure that those folks have the
tools that they need to understand what's happening as we go
through all these changes. That's actually really close to the
heart of the problem.


David Roberts


Yeah. And is there an industry doing that? Do the utilities just
cook all that stuff in house? Where are the 100,000 engineers
that are helping utilities?


Astrid Atkinson


Well, I have a company of about 30, so we're short a few. No,
there are people, of course, across the industry that are working
on this. And that's typically a mix though, of the existing kind
of large scale vendors and folks within the utility trying to
kind of roll their own, which is good in the sense that they know
the problem space, but bad in the sense that very large scale,
real-time operations, cloud computing is sort of a specialty that
doesn't have a lot of overlap, necessarily with utility systems
training. So we could use a lot more person power on this
problem, for sure.


David Roberts


So the point of all this, and this is a point I thought you made
it really well in one of your talks is one way we could deal with
the growth in demand that we know is coming is just to build more
and more grid. But building grid is hard. There are NIMBY
problems and capacity problems, money problems. And our ability
to build out physical grid is rapidly going to be outstripped by
rising demand. So our only real alternative here is to make more
use of the grid we've got, is to use the grid we've got more
efficiently.


And basically what that means is coordinating the behavior of all
of these millions of distributed devices so that they work in
concert with generation and everything works in a big happy
system together. But right now, I think Volts listeners will be
familiar with crude proto versions of this, like demand response
systems where the aforementioned phone calls are like someone
calls you and says, "hey, will you not run your boiler on x hours
of x day?" And they're like, "okay," so that's a form of
coordination of those machines. But I think anyone can tell with
a little bit of thinking that that's not going to scale up to
millions of them.


So the key here, the heart of all this is beginning to automate
these things is to beginning to automate the behavior of these
thousands and millions of distributed devices. And so here again,
I think your experience in Google transfers pretty well since
automating tasks, automating work, work routines, and subroutines
is kind of at the heart of what you're doing. So talk a little
bit about the hierarchy of work and the ladder of automation.


Astrid Atkinson


Yeah, so there are a couple of things in that. First, I guess
it's maybe helpful to have a concrete example of what that looks
like at Google. So one of the things that we went through at
Google was reaching a point where the global load distribution
system, which manages all data center traffic, was needing to be
adapted to manage really rapid growth in demand for network
bandwidth. And so before that, we were really just kind of moving
search requests around across a set of global data centers. And
that has its own constraints. You need to be able to move that
traffic quickly.


You need to be able to respect capacity constraints. You don't
necessarily blow up substations if you get it wrong, but you
actually can overload and crash servers. And it has a lot of
analogs to the grid system in the sense that once you overload
servers, you can cause cascade failures that will take down
service globally. So there are consequences to getting that wrong
even when it's just search traffic.


David Roberts


To put it in electricity terms that the audience will be very
familiar with, you're just trying to avoid peaks in computing
load. Basically, you're trying to make the demand for computing
look like a nice smooth line, even though there's tons going on
beneath that line. If you can coordinate it all just right, you
get a nice smooth line. And spikes, because you have to build, as
you point out in your papers, any network is built to the size of
its tallest peak.


Astrid Atkinson


That's exactly right.


David Roberts


So the more you can smooth out those peaks, the more you can grow
the whole thing without demanding more infrastructure. And that
just transfers very straightforwardly to electricity. You're
trying to move load demand around and coordinate it, such as to
create a nice smooth demand line.


Astrid Atkinson


Yeah, that's exactly right. And that's kind of the fundamental
capacity planning constraint for any large-scale system like that
is really what's your peak demand and can you absolutely meet it,
or do you have a way to shed some of that demand if you're going
to cause system problems? Right, so when we transitioned to
needing to be able to scale the use of network bandwidth, it was
when Google bought YouTube and we needed to be able to serve lots
of cat videos. We started to think about not necessarily taking
every user request all the way from Azerbaijan to a data center
in California, but rather could you put capacity out in
Azerbaijan to serve the most popular Azerbaijani cat videos?


And so we went through this really big transition of how we
manage the network towards much more distributed capacity
management and also software-based network automation. So really
moving towards software-based management of where and how peaks
occur, because you're exactly right. And then also trying to move
as much work as possible to the edges of the network so that you
could get more work done on the network as a whole. So the
parallel in the energy system is moving as much work as possible,
as close to the user as possible. So that's moving it to
distribution, connected resources, and to DERs.


And what that gets you is a bunch of tools for optimizing your
network capacity, which, if we want to get our grid to do more
without building four times as much of it, is really what we need
to do now. So thinking about like, okay, what's the set of
technologies that makes that kind of transition possible? If you
need 10,000 times more capacity from an existing network, or
let's just say like ten times more, or even four times more,
which is probably about what we're going to need, you need some
tools to be able to handle that. So firstly, the more of that
work you can do close to the source, the more network you have to
work with.


Secondly, understanding what the network constraints are in a
really nuanced way, and also what the need for capacity and the
available supply to meet that is in a really nuanced way. The
monitoring piece, like what's out there? What's available not
just for the system as a whole, but on my street or your street
or my substation, is a really important part of that. And then
the analog to what we did at Google, which was to put caching,
which is basically computers that store information close to the
user. So we'd like preload the cat videos that are most popular
near your house is basically storage in the grid.


Right. And what that gave us was the ability to do some work
locally, which increases the reliability of the system as a
whole, because you don't have to have the whole network between
Azerbaijan and California working perfectly all the time. A lot
of times you can just do the work locally. But it also gave us a
lot more flexibility around being able to handle really extreme
variations in load because you could have some of that load
soaked up locally, you could potentially shed a little bit of it,
you could potentially move it around to other assets that could
soak it up somewhere else.


And so when I think about that from a grid perspective, the sorts
of changes that would support that, they're not huge. We need a
better sense of what's going on. We need more control over load
showing up, when and how it shows up. And ideally, the ability to
smooth out peaks and some reasonably large amount of storage
distributed broadly across the grid to like, substation or lower
would give us a lot of flexibility in how we manage that system.
So if we're not exactly right about exactly how much the planned
demand is going to show up, you can just kind of soak it up with
local storage and not have to worry about it too much.


David Roberts


Get you a little buffer there.


Astrid Atkinson


Yeah.


David Roberts


So the equivalent here is if I need an electron to run my toaster
and the grid can be coordinated, such as to provide that electron
from my neighbor's solar panel, let's say. You're using the
minimal amount of the network and leaving as much of the network
possible open for other things, basically, like you're minimizing
total network load work.


Astrid Atkinson


Absolutely. And to some degree, you do that already, right. Like,
let's say that your neighbor, like me, has an oversized solar
panel, I have 9 kW on my garage. During the day a lot of times
I'm back feeding into the system, so I'm powering my neighbor's
toaster. The reason that doesn't really get us very far in terms
of saving money on grid upgrades or really contributing to the
health of the system today is that all parts of the grid still
need to be sized as if that didn't exist because we don't have
any way to guarantee that the peaks don't show up.


David Roberts


Right. And this is the thing, is when you're building a network,
if you've eliminated 99% of the peaks, you still got to build the
network as big as the last remaining peak is.


Astrid Atkinson


Or you need a last-ditch alternative to be able to shed that one
remaining peak. There's a couple of different ways to do it, but
yes.


David Roberts


There's so much in here, but this gets to the grid architecture
question that I was writing about back in 2018 with my cool
diagrams. That my —


Astrid Atkinson


They were so cool.


David Roberts


I know. I've seen that you're using variations of them in your
presentations and it makes me so happy.


Astrid Atkinson


I'm glad it makes you happy because we thought that they were
amazing and really explanatory and found them very inspiring.


David Roberts


That's Javier at Vox. I'll put a link to that piece in the show
notes. But basically, the idea here is that you go to the lowest
level, the edge of the grid. Let's say the distribution node, a
node on the distribution grid. The idea is you satisfy as much of
the demands of that distribution node with resources within that
distribution node. Net that out. So the only power that the
distribution node is asking for from the transmission system is
whatever is left over once it has subtracted its own supply from
demand or demand from supply. And then you sort of move up
levels.


Right. So in a sense, everything is kind of a microgrid. It's
like microgrids within microgrids within microgrids. I don't even
know if that's the right terminology, but islands within islands
within islands.


Astrid Atkinson


Yeah. You ultimately be looking at a system with multiple
abstraction layers where some amount of management is done
locally, and then those nodes are connected to the broader
system, and then the broader system does its own coordination.
But the thing that's really nice about that model is that you
don't have to worry about your toaster from the California ISO.


David Roberts


Exactly. There's someone in between the California ISO and me
that's worrying about my toaster. Some entity or —


Astrid Atkinson


That's right, yeah. And technically, you could build a system
that did kind of coordinate from the ISO to the toaster. But what
you would lose in that kind of system that I think is really
important is some amount of local abstraction layers and local
management entities where you can optimize for different kinds of
work. So optimizing for the temperature in my house is actually
kind of different than optimizing for the load on my transformer
and kind of different than optimizing for overall supply and
demand for the state of California. It is reasonable to have
those things handled by different layers, different entities,
even.


David Roberts


Right. And there might be neighborhood goals or values or city
goals or values that are slightly different than the state goals
or values and so on. And also, just as a matter of computational
complexity, I mean, this gets to what I was trying to get out of
my article. It's one thing for these ISOs or RTOs — these sort of
entities that are coordinating transmission grids — it's one
thing when you're coordinating — call it like five big power
plants in a couple of dozen distribution nodes. Right. As we were
saying before, that's a finite number. You can control that more
or less centrally.


Astrid Atkinson


Model it deterministically.


David Roberts


Yeah, you can model it deterministically. But once you're getting
to thousands and millions of devices trying to coordinate all
that at every level from a single point, from a single operator
at the top. It just seems to me it's rapidly going to overwhelm
their ability, their ability to — and why would you want to? It
just doesn't make sense to have one giant regional organization
coordinating your toaster, as you say. All of this brings us to
the DSO model, which is I was delighted to find beloved by both
of us.


Astrid Atkinson


Absolutely.


David Roberts


Which is you need an entity — you know, we can discuss whether it
has to be a singular entity, but let's just call it an entity for
now. You need an entity at the distribution level, at the level
of that distribution node that is responsible for coordinating
the activities of all these DERs within that distribution node.
Matching supply and demand, netting it out, determining how much
power is needed by the distribution system. And then that node,
that DSO, the distribution system operator, does that work. And
then the only signal it needs to send up is a single signal, like
we need x amount for our distribution node.


Right. So it's taking all that complexity at the local level and
simplifying it before it sends it upward, basically. And you have
that at every level you're simplifying and sending upward. So
that once you're at the top level, the ISO, you're dealing with a
tractable number of signals rather than trying to talk to every
toaster.


Astrid Atkinson


Yeah, definitely. And even for very large scale load balancing
systems like the ones that we used at Google, they were still
built that way. Right? It was like local coordination across work
within a single data center that was then providing a simple
interface to global coordination systems, which basically were
like, "hey, how much load you got? How much load can you take?"


David Roberts


Right?


Astrid Atkinson


And then would readjust allocations every second or two.


David Roberts


Right. So from the transmission operator's perspective, a
distribution node will basically just be a single machine that
either has a set amount of demand or can offer a set amount of
supply.


Astrid Atkinson


It would ideally look a lot like battery does on the ISO system
today, I think. Where you've got a certain amount of power, need
a certain amount of flexibility available, perhaps a certain
amount of power available. But I think something like that would
be a small enough adjustment on top of how we think about
managing at the transmission level today to be practical.


David Roberts


Do you? There's a large amount of reform —


Astrid Atkinson


Don't get me wrong. There's definitely new software components,
new data components, new business model components, all the above
required to do this.


David Roberts


Well, it's the business part and the regulatory part that's
baffling to me because that's the hardest, in my experience, the
hardest part to get moving on. And right now, just to be clear
with everyone, there is no such entity in the US. Basically,
there's no entity responsible for handling the complexity at the
local level, simplifying it and then sending the signal up. So we
would have to create those out of parts. So I know Lorenzo
Kristov, who I drew on to write that original piece, who's kind
of the guru of all this stuff, I think has been beating his head
against the wall in California, trying to get them to run DSO
sort of test models or create test DSOs.


So who out there is doing that? And what does the DSO model look
like? Is it up and running and working anywhere or is it all
experiments?


Astrid Atkinson


Yeah. So this is where I get to talk about Australia, as
promised, because Australia has a really nice large scale
demonstration of this type of model that they're actually pretty
far into at this point. You know, like many regions, including
the UK, a couple years ago, Australia sort of took a look at the
oncoming challenges associated with DERs on the electrical
system.


David Roberts


Yeah. Swarming rooftop solar per some previous podcasts here,
tons of rooftop solar. Way ahead of anybody else.


Astrid Atkinson


Absolutely. And so they were a little bit more motivated, maybe,
than other places to try to solve this problem. And so there was
a bunch of really interesting work led up there, mostly out of
AEMO, but also out of the CSIRO and some of the universities and
stuff as well. And I know what this is because I'm from
Australia, but what they did was come up with a couple of kind of
trial models of coordination between AEMO, which is the
Australian market operator this is kind of the ISO, and the
distribution network operators and the aggregators and retailers
that were working within some of their key markets.


And one really relevant example of this work was something called
Project Edge, which you can Google, although you might need to, I
don't know, add AEMO or something to it, just AEMO. And what that
was doing was looking at a coordination model exactly like the
one that you're describing. So it was asking the distribution
network operator to take on more of a DSO role. It was asking the
aggregators to provide data about the location of their devices,
what they plan to do with them, those kinds of things. And AMO is
playing a coordinator role to kind of sponsor that and bring some
of the data together.


But ultimately, it's a sort of joint function between those three
parties. That's just one example of how this model could work.
The UK is taking a little bit of a different approach with the
central regulator and kind of central operator National Grid
taking a bit more of a central role in that. But still, it's kind
of looking at the idea of cutting up the system into multiple
localized components and then having some entity take on a
distribution system operator role to collect that data about the
state of the grid and the things that are participating in it.


Coordinate that with data from the aggregators and coordinate
that with data about market participation. And in Australia, they
liked the results of that so much that they are planning on
rolling it out nationwide.


David Roberts


Interesting.


Astrid Atkinson


Which is an advantage of being a smaller country.


David Roberts


You know, they've got so much rooftop solar now that they're
having duck curve problems and even some stability problems, I
think. So where they've tried this, they're solving that kind of
duck curve-ish problem.


Astrid Atkinson


Yeah. So some of the early applications in Australia particularly
focus on being able to curtail rooftop solar in places where they
have basically oversupply issues. So it's a duck curve problem.
Right. And just given the way that sort of social license and
kind of public sentiment works in Australia, if you're curtailing
somebody's solar, the Australian consumer citizen will kind of
expect you to pay for that just as much as they would expect you
to pay for the energy that they provided when it wasn't
curtailed. And so they needed a market construct to support that.
The other thing that is happening within that broader model that
I think is really interesting is also the ability to do basically
flexible interconnections.


So being able to basically say you can connect this new load, EV
bus charging is my favorite example, but only if you don't exceed
a certain capacity allocation. So you can connect it and it's
kind of up to you to manage that. And we will give you that
capacity allocation either as a fixed allocation or even better
as a dynamic allocation. And that idea of like a dynamic capacity
allocation that a user has to stay within is called dynamic
operating envelopes. And that's a sort of technical component of
that Australian work that helps the system operator to manage
basically the capacity allocation to every individual DER,
consumer or producer.


David Roberts


Right. And this is where again, automation comes in because
you're not going to get every bus charger operator to sit there
with their hand on the lever, no pulling it up and down as these
dynamic constraints change.


Astrid Atkinson


Yeah, this is a job for computers, this is not a job for humans.


David Roberts


So there are DSO models out there happening. In the US, let's
talk about the weird US situation. So in the US, recently, FERC
issued this order is it 2222?


Astrid Atkinson


Yeah.


David Roberts


2222, which says aggregators — which just in case people are not
familiar with these, that's just like a third party entity that
strikes contracts with dozens or hundreds or thousands of DER
owners to basically give them control over those DERs so that
they can treat them as an aggregate, treat them as a big, giant
generator, or treat them as a big, giant battery — and then
FERC's order 2222 says those aggregators can play in wholesale
energy markets, basically.


So you can take an aggregation of DERs and pretend to be a power
plant in the wholesale power market, basically, or act as though
you are a power plant in the power plant market. And that's a
response to the need for our system to make more of these DERs,
to use these DERs rather than just be a victim of these DERs, to
take some control of them and use them in such a way that they're
useful. But to me, that just feels cludgy, feels like a half-ass
solution. Because you have this weird thing where pretty soon,
like I said, these wholesale markets, depending on how many
aggregators you have, and I'm not sure totally how that market is
going to shake out, but there could be a lot of them eventually.
Again, you're just like, why have the one central coordination of
all those machines?


It just feels like that should be resolved, that the stuff among
individual DERs should be resolved at the local level and not
something for the ISO to have to be dealing with. And also, so
you have sort of aggregators kind of speaking directly with the
ISO, kind of bypassing any DSO. It's just a weird kind of hybrid
model of localness and centrality, I guess. To me this seems like
a temporary fix on the way to something better. How do you feel
about 2222?


Astrid Atkinson


I think it's definitely a key part of a solution. I actually
really like 2222 for a little bit of a roundabout reason. What it
does is mandate that aggregators should have access to wholesale
markets. And it is entirely silent on the interaction of the DERs
that they control with the network that they're actually located
on on the distribution side, which is kind of bad in the sense
that you really want to know probably what's happening with any
assets that are moving a lot of load around on the distribution
side.


David Roberts


Yeah, if you're running a distribution grid and a bunch of the
machines within it, their behavior is being coordinated on behalf
of the ISO. That might not be exactly the kind of behavior you
need for stability at the local level.


Astrid Atkinson


Absolutely. It might cause a lot of problems, and I think it's
actually likely to cause some problems. However, there's nothing
to stop a given distribution operator/utility from deciding that
they want to understand what's happening, that they want to be
able to play a coordinative role or at least get data from
aggregators about what they're doing and to say like, "look, I
want to take on a distribution system operator role within this
broader system." And there are some implementations of 2222 that
are kind of going that direction. The one I'm particularly
thinking of is PJM has a filing for their implementation that
does include coordinative function with the local utility.


And I think that's really interesting. Now it doesn't necessarily
100% meet the needs of aggregators because it basically says that
the local utility should have visibility into and perhaps
dispatch control over aggregate resources.


David Roberts


Kind of a veto, right? Because it's those local needs. That
ultimately are the that's the actual toaster coming on.


Astrid Atkinson


So, you don't really want to veto either. But one thing that I
think is really important to remember about those aggregators and
the resources that they're managing on the customer side is that
they want to get paid for the services they can provide. And one
way to get paid for that would be to bid them into the wholesale
market. But the value of the service that those local DERs could
provide is actually a lot higher to the distribution utility that
they're located on. If we look at kind of all up and down the
value stack, selling energy on the market is part of that value
stack.


But selling peak shaving at the local transformer or substation
level theoretically is very valuable in practice. No one's paying
today.


David Roberts


Right. If there were a market at the local level, this is the
thing. It almost seems like the DSO should be running a local
market.


Astrid Atkinson


Absolutely.


David Roberts


And the DSO should be sort of serving as the aggregator. Right.
So you let the local market do its thing, and then you bid for
whatever's left. You bid up into the wholesale market. Does that
make sense?


Astrid Atkinson


Yeah. And so I think what this does is provide an opening to
start having that conversation. FERC 2222 doesn't really provide
the structure for that kind of DSO/ISO aggregator coordination
function as we see in other countries. But it does provide kind
of a stick in the kind of carrot and stick sense, in the sense
that you could either choose to take a leadership role in
coordinating the behavior of assets on your network as a utility,
or you could just let that happen and kind of deal with the
stability and cost implications on the back end.


And don't get me wrong, there's probably a bunch of utilities
that will end up doing the latter, but there is an opportunity to
do the former, and some will go that way.


David Roberts


Yeah. So in your efforts to cajole utilities into doing things
differently, I feel like the landscape is littered with the
exhausted husks of people who have spent their lives trying to
get utilities to do things differently. You're focusing
specifically on co-ops and munis as kind of places to experiment
with this more local model, this DSO model. Why is that?


Astrid Atkinson


So, we do also work with investor-owned utilities, but primarily
with ones who have a vision for taking that leadership role and
really kind of want to move that direction. But one thing that's
really nice about working with co-ops and munis, which are both
nonprofit, typically local utility structures, is that they have
a very local set of motivations around serving the community,
keeping costs low within their specific community. And they're
also nonprofits, which makes things a lot simpler from a business
model perspective. They also are pretty sensitive to the cost of
energy, which for most investor and utilities is financially
speaking, it's a pass-through.


It's not really part of their profit model.


David Roberts


Right.


Astrid Atkinson


So for a coopera-muni to save a bunch of money on the cost of the
energy that they procure for their customers, it's like kind of a
direct benefit because that goes back to the customer in the form
of lowered bills or for co-op, they even sometimes send checks
back to their members, which I think is really cool, but they
care a lot about this and that's a business motivation for them.
Likewise, if they are looking at substantial system upgrades
because they're going to see a lot of load growth that might be
from electrification. But actually for these utilities today it's
more often because somebody is like planning on putting in a
factory or a data center or something like that.


They're open to looking at non-wireless alternatives and kind of
using smart management to avoid doing a very expensive substation
upgrade again because they would have to pass that cost back
along to a very limited number of members. And it's not like
investor and utilities don't have a similar broad motivation or
care about it. But if you're a co-op executive or staff member,
people come up to your front desk asking about like "hey, why is
my bill $15 higher?"


David Roberts


We'll run into you in the supermarket.


Astrid Atkinson


That's right. And it's just a very different relationship with
the community for those utilities.


David Roberts


So you think they're more open to this. Have you gotten movement?
Are there US co-ops and munis that are setting the standard here?


Astrid Atkinson


Yeah, so there are a bunch of utilities that have been really
interested in kind of moving down this path, towards a
distribution system operator model and taking on a more active
kind of local system operator role. And that's true across kind
of all of those segments from co-op to muni to some IOUs as well.
The thing that's nice about the co-ops is that they can move
faster because they're small, and sometimes also they will just
decide that something's working and decide to roll it out
broadly. Whereas if you're at an investor and utility and you had
the greatest thing in the world, you'd probably still be stuck
going through a kind of regulatory approval cycle for scaling it
up.


David Roberts


Process, process, process.


Astrid Atkinson


There's good and bad to that. But the co-ops are often
self-regulating or at least minimally regulated by the Public
Utility Commission because they are deemed to be acting in the
public interest so they can move quickly. And so there are a
subset of co-ops which are rural electric utilities that have
been experimenting with these models and are moving very quickly
in this direction and it looks different depending on the utility
as to what that looks like for them. Some of them have been
pushing really hard on generating energy locally and avoiding the
wheeling costs of shipping it across transmission and putting
that money back into the community.


So there's one that we work with in northern New Mexico that does
that, and they serve the area around Taos, and they decided that
they wanted to get to 100% of their daytime electricity load
served by local solar. They're actually at about 120% now.


David Roberts


Oh, wow.


Astrid Atkinson


And they're starting to look at models where they can export
power out of the community, which I think is a really interesting
economic growth opportunity for communities like that.


David Roberts


Yeah. Again, if you can imagine these local markets, you can also
imagine markets in between distribution nodes. Right. Like,
imagine if my distribution node, I'm handling all the complexity
within the DSO, handling all the complexity within the
distribution node, comes up with a certain amount of leftover
demand that it needs satisfied. It could procure that from the
next distribution node over. Right. And it's in a peer-to-peer
transaction and not have to involve the ISO at all,
theoretically.


Astrid Atkinson


Yeah. And there's probably a bunch of different ways that this
might play out, but you certainly could see a world where that's
the case. And it has a delightfully localized kind of quality to
it in the sense that the community is mostly self-supplying in
local electricity and including batteries and flexibility that
get them through the night. But they also have this opportunity
to potentially export a resource that they have a lot of, which
is pretty cool because it's sun.


David Roberts


Part of the delightful locality of it is that some communities
might value resilience more, so they'll want to, say, bank more
of their excess solar and batteries. Some communities might want
the money more so they'll be more likely to sell it across to a
different distribution. You can see communities will actually
have much more fine-grained control over their energy. And one of
the points Lorenzo makes a lot of times is this would make it
much easier for local electricity and energy policy to be
coordinated with local building policy and local transportation
policy, and basically, like, your local — becomes part of how you
want to run your local area.


Astrid Atkinson


Yep. And ideally, that would include members of the local
community getting paid for the flexibility that they provide,
too, because that's a big part of how we get this done. One other
example I'd want to give for that is there's another utility I
know pretty well. It's in Colorado. They're expecting to double
their load in the next five years.


David Roberts


Just population growth?


Astrid Atkinson


No, it's actually a mix of factories, clean energy production
facilities, delightful also, and data center loads, which are
really growing due to all those big cloud computing facilities
that we mentioned earlier. So they're looking at doubling their
current energy usage, and there's no way they're going to get
enough transmission additions built out to support that.


David Roberts


Right. Especially since it takes 10 to 15 years to build one
line.


Astrid Atkinson


Yes. Transmission is one of the few industries where you can pass
along a single project to your children, which is not a
compliment.


David Roberts


A legacy project.


Astrid Atkinson


Yeah. And so they're looking very closely at models where they
can trade generation and flexibility and storage amongst their
large users, particularly within their territory. Because the
usage profiles for a refrigeration facility are different from
that of a shipping facility. Maybe a lot of EVs charging and
different again from a factory facility. A lot of those would
also be inclined to put local generation into the commercial and
industrial site and getting all of that stuff signed up to become
part of the future overall supply profile for that territory, it
requires a DSO function. There's kind of no other way to do it.


David Roberts


Yeah. Somebody's got to be in charge of all that.


Astrid Atkinson


Yeah. And so that's a place where the utility taking that
leadership role really makes a lot of sense because I don't know
how else they would manage it.


David Roberts


Yeah. What do you think are the prospects of basically just local
distribution utilities right now which are just sort of running
the wires right now and billing customers growing into the DSO
role? Like when you bring it up to them, do they just blink at
you? It's such a sort of cosmic upgrade of their role and
importance and responsibilities, etc. Are any of them eager to do
that?


Astrid Atkinson


Yeah, I mean, there are a lot of utilities that see this as being
part of their future. I think the tricky part is it's a really
big change for an industry that hasn't had load growth in 20
years. Being able to suddenly adapt to all of the technology and
organizational and business model changes required to support
that is going to be a lift.


David Roberts


Yeah. It's wild. Like 20 years without load growth and without
really substantial change in the industry and all of a sudden now
it's like new tech, new models, new regulatory models, new legal
models like boom boom boom boom boom. It's a lot.


Astrid Atkinson


Yeah. And so there are utilities that are definitely thinking
about this. Like SCE has a public roadmap that's really nice that
covers a transition to a DSO model. The only thing that's a bit
of an issue with the normal utility process for this kind of
change is that it tends to be very slow and in many cases we're
going to need to see rapid adaptation in the next like three to
five years. And for most utilities, they're used to kind of
planning on a five to ten-year time horizon.


David Roberts


Yeah. Utilities and PUCs, I mean, I'm not sure that PUCs are
exactly legendary for being agile either.


Astrid Atkinson


They do compare notes. So if you get something that works well in
one place, a lot of times they will try to spread that more
broadly. And so, from my perspective, one of the best things that
we can do to make this happen is just show it operating at scale
in as many places as possible. Not all of those will be right the
first time, but it's not the kind of problem that you can really
sit down for five years, come up with a solution to and then
implement.


David Roberts


Right, which is the utility way, right. You make this point in
one of your talks too. It's like the whole software world model,
which is that you sort of build, iterate, test, learn, rebuild,
reiterate, test, learn that's foreign to the utilities.


Astrid Atkinson


And it's not that, in the software world, it's not like we don't
plan ahead or design things. Of course we do. But you design with
the idea that rapid iteration is going to get you closer to the
goal more quickly.


David Roberts


Right.


Astrid Atkinson


And we've seen really good success with this in adjacent
industries. Right. Like one of my old bosses at Google was really
interested in skydiving and decided that he wanted to jump from
space from a weather balloon.


David Roberts


Such a Google guy.


Astrid Atkinson


This is definitely a Google problem. And so in order to do this,
he acquired — this is kind of post-Google for him, but he got
really interested in spacesuit design, and then worked with the
company in the US, that's a premier designer of spacesuits, who
had not designed a new one since, like, 1973, to rapidly develop,
prototype, test and deploy new spacesuits so that he could jump
from space in a weather balloon. Which by the way is a total
badass move. He's an engineer from the ground up. But so they
worked in a really iterative kind of rapid development and
testing cycle and they got that done within a couple of years and
now the same company is developing the spacesuits that are like
the next generation ones that are used for SpaceX and Blue Origin
and all of those.


David Roberts


Yeah. I mean, if nothing else, this is like a chance to be a
hero, right? This is a wide open field and there's just so many
opportunities here for innovation, for people to try new things
and show successes.


Astrid Atkinson


Yeah, it's a really exciting time in the industry broadly and for
the subset of folks at utilities that are really actually
interested in thinking about what the future of the industry is
going to look like and kind of working towards building that.
Boy, it's an exciting time to be in that industry too.


David Roberts


I know. I've been writing about this stuff for 20 years now and
it is still somewhat head spinning how all of a sudden it's just
all happening.


Astrid Atkinson


It's changing really fast on the ground.


David Roberts


It's just waiting and waiting and then boom, all of a sudden it's
all happening. So I would feel bad if we did this whole thing and
I didn't give you a chance to sort of say what your company does.
Your company is called Camus. Why Camus, by the way? I was
thinking existentialists. This must be about the existential
despair that you experience when you contemplate trying to reform
utilities?


Astrid Atkinson


Not exactly, but related. So we are named for this philosopher,
and the reason for that is actually really specific. So Camus
wrote this essay called The Myth of Sisyphus.


David Roberts


Yes.


Astrid Atkinson


There's a short version in a longer book by the same name.


David Roberts


"One must imagine Sisyphus happy." One of my all-time favorite
philosophy quotes.


Astrid Atkinson


Yeah. So the short version is only about three pages long, and I
totally recommend it. But Camus is basically asking, "how do you
create a sense of meaning in the face of a large, uncontrolled,
potentially godless universe?" Right? Like, where do you derive a
sense of purpose? And his answer for this was basically, you pick
something, you work on it, you find joy in the process. Not in
the notion that you're going to win, but in the everyday act of
pushing a rock up a hill and following it back down.


David Roberts


I love it. At the base level, I think there are sort of like
levels of your service. At the base level, you're just helping
utilities be more aware of DERs and then sort of laddering up
from there, like coordinating them, et cetera, et cetera. So, how
high up do you go on that ladder?


Astrid Atkinson


Well, since we've gone to the trouble of describing a DSO and
what it does, our goal is really to create a software platform
that will enable a utility to take on that role. So a much more
real time and local operations model that can include local
resources as part of the supply and demand landscape and
ultimately include them into capacity management and network
management for the grid and let them get paid for it.


David Roberts


Got it. So you're creating the tools for the DSOs for whenever
they show up?


Astrid Atkinson


We are creating the tools for the utilities that want to take on
that role and working with them to figure out what that looks
like in practice.


David Roberts


Right. And we should emphasize for the utilities out there
listening, this is not a binary thing, it's not like —


Astrid Atkinson


It's a process.


David Roberts


jumping off the high dive into the whatever. There are pieces of
this you can adopt, one at a time.


Astrid Atkinson


Yes, that's right. It's a process. Right. The question is not
like, what's it take to get to the grid of tomorrow. The question
is more like what is the set of reasonable steps that you can
take with the data and control capabilities of today to add more
sophistication, get better visibility, add coordination, talk to
aggregators, coordinate with FERC 2022 deployment and 2222
deployments, all of those kinds of things. So it's going to be a
process, but it doesn't have to be impossible.


David Roberts


But it's just great that this idea of making the electricity
system more like the Internet goes way back. As I'm sure you're
aware, Al Gore was talking about the Internet. Of course, he's
trying to coin a term for it back in 2007. And I went through a
period of hype for it, and being very excited about it, and DERs,
and all that. And then I sort of ran up against utility and
transigence and had my life force drained. But now it seems like
at long last the hype cycle has come back around and it's
actually happening now.


Like there's actual things happening, actual movement in that
direction happening.


Astrid Atkinson


It's all happening.


David Roberts


It's very exciting. So final question then, and this is kind of a
bit of a curveball, but I'm curious. So say we imagine our
glorious future here in the US, where we have revolutionized the
system. And we now have all local electricity being administered
and run by DSOs, who, as we say, resolve the complexity of the
local area before passing on a signal upward, maybe nested a
couple of levels. Maybe like the level above them has three or
four distribution nodes and the level above them, et cetera, et
cetera. But by the time it gets to the transmission level, you've
already maximized the use of local resources.


Basically, the goal here is to maximize the use of local
resources before calling on large scale distant resources. So
imagining that glorious future where we've made that happen, how
much of net US energy do you think will come from local resources
versus still coming from big utility scale power plants on the
transmission system? Do you have either a predicted or desired
balance of those two in your kind of perfect world?


Astrid Atkinson


So I've seen modeled estimates that put that somewhere between
30% to 50%. And you talk to other folks in the space who do
modeling on this. There's some really good work from Vibrant
Clean Energy that did a bunch of work on the potential role of
local resources a couple of years ago. But I think somewhere in
that kind of 40% ish space is likely and practical. In Australia
right now, by the way, it's sometimes 50 and sometimes 70% local.
So it's just like do you mean instantaneous or overall? Because
if it's instantaneous, sometimes it could be 100% or even more if
we're storing.


David Roberts


Right. I mean overall, like on a net yearly basis —


Astrid Atkinson


But on an overall basis, because the sun does go down and the
wind doesn't always blow, as you covered very well. I feel like
40% is pretty reasonable.


David Roberts


Interesting. That'd be a good, like something to go to one of
those betting sites, start a pool on, say, 2040. What's the net
balance? Well, Astrid, this has been an absolute delight, as I
knew it would be. I love this whole subject. I love what you're
doing. As I say, it feels like you are someone that I willed into
being by writing my 2018 piece, which is I'm sure you existed
before that, but —


Astrid Atkinson


I certainly did. But the piece made a huge difference in how I
was thinking about the space. So I'm pretty grateful for that.


David Roberts


Awesome. Well, it's a delight to have you on, and I'd love to
have you back on again sometime once this stuff evolves some,
because this is an endlessly, inexhaustibly fascinating topic. So
thank you again.


Astrid Atkinson


Thank you so much.


David Roberts


Thank you for listening to the Volts podcast. It is ad-free,
powered entirely by listeners like you. If you value
conversations like this, please consider becoming a paid Volts
subscriber at volts.wtf. Yes, that's volts.wtf. So that I can
continue doing this work. Thank you so much and I'll see you next
time.


This is a public episode. If you'd like to discuss this with other
subscribers or get access to bonus episodes, visit
www.volts.wtf/subscribe