Source: EIA Short-Term Energy Outlook Data Browser

As always, you can read this post on my Substack/vanity project, Low Energy.

After a decade-plus of stagnant growth, electricity demand in the US is rising, driven by the increasing electrification of sectors like transportation and AI data centers. Electricity prices are rising too, up 5.5% from last year. Despite the early sunset of tax credits for wind and solar, and an effort to halt solar and wind farm construction on federal lands, solar and wind with storage¹ will be vital to meeting growing demand, absent government interference.²

Planning and operating solar and wind farms require public datasets and open-source software. The former is at risk due to budget cuts at government agencies, while the latter is under-supported. Both the public and private sectors need to ensure the continued provision of these public goods, so power sellers and utilities can operate solar and wind resources as efficiently as possible and ensure grid reliability.³

Long-term Modeling and Public Datasets

Before building a solar or wind site, a power seller/utility needs to have a reasonable idea of how much power the site will generate throughout the year. This is necessary information to make capital investment decisions.

The weather exhibits seasonal patterns. Unless you expect those patterns to break down over the life of the project, it is reasonable to use historical weather to assess the generation potential of a proposed solar or wind site. The companies making these decisions have not collected weather data across wide areas in sufficient quantities to make these investment decisions. They need to rely on public data sets.

As an example, I pulled total irradiance and wind speed at 10 meters for Lisbon, Portugal.⁴ The data came from the PVGIS SARAH3 dataset, which is maintained, updated, and made publicly available by the European Commission’s Joint Research Centre. The seasonality is clear in the plot of irradiance; irradiance peaks in summer, and the variance drops in summer, indicating more consistently sunny days.

Daily Mean Irradiance for Lisbon (2014-2024)

There are, of course, changes in hourly irradiance throughout the year as the days grow longer into summer and then get shorter again.

Average Hourly Irradiance for Lisbon each day of the year (2005-2024)

The seasonality exists for wind, but less obviously.

Daily Mean Wind Speed at 10m for Lisbon (2014-2024)

The pattern is clearer when plotting average hourly wind speed throughout the year.

Average Hourly Wind Speed at 10 m for Lisbon each day of the year (2005-2024)

In the US, sources such as the National Solar Radiation Database and the Wind Resource Database are used as bases for simulating generation from proposed solar and wind sites. Both of those databases are maintained by the National Renewable Energy Laboratory (NREL), which is reportedly targeted for a staff reduction of 33% due to budget cuts in the One Big Beautiful Bill Act. It is possible that any layoffs at NREL will not affect their ability to maintain and update accurate databases and other critical functions. However, the recent experiences of the National Oceanic and Atmospheric Administration and the National Weather Service raise concerns.⁵

The ability to make long-term investment decisions in solar and wind is predicated on the availability of up-to-date, high-quality weather data. Suppose that some areas of the US are experiencing droughts more often. This would increase solar generation in those areas. Perhaps some places are getting more precipitation.⁶ This would reduce the average capacity factor in the affected areas. Both scenarios are conditions that need to be factored into the decision to build a solar plant, and up-to-date accurate public datasets will capture those conditions.

Once a solar or wind site is built, it needs to be operated, requiring short-term forecasting of generation. While public data sets are important in this domain, open-source software plays a vital role.

Short-term Planning and Open-Source Tools

Accurately forecasting solar and wind generation is important when making unit commitment decisions. You need a mean forecast and range to determine what other generators are required to meet demand and provide sufficient reserves. Publicly available forecasts, alone, are insufficient for this task.

Solar and wind farms have small areas compared to the grid sizes used in public weather forecasts. For that reason, many power sellers/utilities augment publicly available weather measurements with measurements from their own weather stations. Their own meteorology teams then feed those measurements into weather models with finer grids over the areas important to the company.⁷ Next, these forecasts are ensembled with machine learning models and forecasts available through open-source software.

Open-source libraries do more than serve as APIs to publicly available models. Packages like Solar Forecast Arbiter, PyPSA, and windpowerlib are used to translate forecasted irradiance and wind to power and optimize unit dispatch to meet load while minimizing costs. They’re also under-supported.

Message from the pvlib developers regarding the forecast module.

Take pvlib, for example. It is a Python translation of a MATLAB package from Sandia National Laboratory that is widely used in the private sector. It currently has 211 open issues on GitHub and has had to deprecate functionality to focus on what its team can support. It has received funding from Department of Energy grants and non-profits like NumFOCUS. But that support is insufficient, and private-sector support is lacking. Although some companies contribute, most do not. A company’s contribution needn’t be financial (directly at least); they could allow employees to contribute by closing issues and adding features during work hours.

Although free-riding is natural, more development hours are ultimately wasted across the sector when in-house packages are developed to extend the functionality of open-source libraries. I’m not sure how to solve the collective action problem in this case. Perhaps EPRI can serve as an organizing force in the sector to encourage companies to contribute to widely used packages.

Unfortunately, lack of private support isn’t the only threat to renewable open-source software. Many important tools are supported by government entities like NREL. Funding and personnel cuts will likely affect support for tools like reV. That makes private-sector support for these resources more important.

Conclusion

Planning and operating solar and wind farms, like many problems in the modern world, are solved with public data and open-source software. But the provision of these public goods is impeded by personnel and budget cuts in the public sector, and insufficient support from the private sector.

Hopefully, cuts won’t affect the scheduled updates and availability of public datasets, but resources allocated to these projects will come at the expense of research into areas like improved inverters. There isn’t much to be done aside from persuading policy-makers that they are important to making the decisions necessary to supply the US with affordable and reliable electricity and supporting candidates who recognize their importance.

Fortunately, more can be done to support the open-source ecosystem for applications in renewable energy and energy more broadly. Either direct financial support to development teams or allowing employees to spend a fraction of their work hours on development can improve stability and extend functionality of open-source tools. Overcoming free-riding is an issue, but it has been accomplished in the numerical computing and machine-learning fields.

While public funding may be constrained by political realities, private companies—who benefit directly from these resources—can step up. Supporting open-source projects would strengthen the entire sector and ensure the reliability of the electric grid.

1 A really great post on batteries.

2 President Trump has threatened to halt permits for solar and wind farms.

3 Power sellers denote electricity generators in competitive markets, while utility denotes electricity generators that are natural monopolies in their areas of operation.

4 I pulled data for Lisbon because I didn’t want to make an API key to access the NSRDB and I like Lisbon.

5 NOAA and the NWS had a sledgehammer taken to them. And sattellites are getting cut back on.

7 The public sector played an important role in the development of numerical weather forecasting.