Although commodities markets are experiencing significant levels of uncertainty, commodity trading generated more than $100 billion EBIT in 2023, which translates to more than $150 billion in gross margin. Given this dynamic, success in the years to come will increasingly hinge on the ability to manage and respond to unpredictable market circumstances. Players with a comprehensive, nuanced understanding of the markets will be best positioned to extract value, while the rest will need to invest in new skills and capabilities.
The rapid growth of commodities markets has drawn a wave of new entrants—such as tech-focused trading players, hedge funds, and banks, as well as players involved in mining and processing—creating a need for additional liquid and risk management offerings. In our view, this is a positive development. Such vibrant markets can both increase value pools and help facilitate the energy transition through investments in new and emerging technologies and products. Doing so could have broader benefits in terms of meeting global climate goals and building and scaling a greener, cleaner future.1
This article examines the state of the industry across commodities and explores two trends that will likely shape markets for years to come: increasingly interconnected commodities markets and the growing importance of power. These developments will have far-reaching implications for commodity traders and for players along the value chain. Portfolio optimization and data-driven trading models can help industry players keep pace with shifting markets and rising competition.
Taking stock of the industry
Commodity trading value pools have shown resilience despite increased stability in the market environment, decreased volatility2 in commodity prices—from 2022 to 2023, price volatility decreased by 30 percent for West Texas Intermediate (WTI), 58 percent for spot Dutch TTF Natural Gas, 38 percent for Henry Hub, and 27 percent for copper—and ongoing worries about an impending recession. After growing rapidly from 2021 to 2022, total trading values overall remained relatively stable in 2023 (Exhibit 1).
Despite the decrease in market prices, commodity markets remain tight, and changes in demand and supply have become harder to predict (Exhibit 2). Further, uncertainty around the security of the energy supply contributes to price volatility, which is amplified by higher, shifting interest rates. High interest rates affect the cost of debt required for investing in new capacities and could result in changing or even abandoning some business opportunities. This dynamic particularly affects large projects with substantial capital expenditure commitments.
Supply chain disruption has also constrained global trade activity. Both large-scale geopolitical developments (the COVID-19 pandemic and Russia’s invasion of Ukraine) and local events (such as the slowing of cargo traffic in the Panama Canal because of drought and Houthi attacks on cargo ships in the Red Sea), have affected product availability and prices. The invasion of Ukraine has had wider implications than those on trade and supply chains, but they are not the subject of this article.3
In addition, commodity prices fluctuate based on the economic performance of specific countries: energy scarcity in a single large economy, such as China or the United States, can propagate all over the world, which will likely affect all other commodities and food prices as well as rates of inflation.
An examination of different commodities reinforces the varying impact of these factors.
Oil and oil-based products
Development: Oil and oil-based products are still the largest value pool, but their profitability decreased in 2023. Although there was more flat-price volatility in 2022 versus 2023, there was still a significant amount of physical volatility in 2023, particularly due to trade flow reorders in the wake of the Russian invasion of Ukraine and the Houthi missile attacks in the Red Sea. The end of 2022 and the beginning of 2023 also saw the introduction of WTI as part of the Brent assessment. Ship utilization remained up and volatility remained in the freight market. Finally, Urals oil was diverted to India before being sent back to Europe in the form of products.
Competition outlook: According to the 2023 McKinsey Global Energy Perspective, total demand for oil is projected to continue growing for much of this decade and then fall after 2030.4 And in the Achieved Commitments scenario, oil demand is reduced by nearly 50 percent by 2050. Until then, competition is likely to intensify as increasing numbers of large players become active in trading oil and oil-based products. For example, national oil companies and legacy oil marketers are already ramping up trading capabilities and leaning on their large balance sheets when seeking their share of the value pool. At the same time, subscale intermediate players could be at risk of “shrinking,” given constraints on working capital and risk capital as well as competition from integrated players. Still, risk tends to create opportunities for niche plays. This is particularly true in specialty petrochemicals, for which large traders are likely to focus more on liquefied natural gas (LNG) and other plays.
Power and gas
Development: Overall, the trading value pool for power and gas increased in 2023. Volatility in power and gas in Europe has remained above average (when compared with 2017–20), although there has been some reconciliation of losses that occurred in 2022, when disrupted gas deliveries from Russia required countries to procure gas on squeezed short-term markets.
Competition outlook: Markets could see increased competition. Utilities and renewables players could ramp up trading capabilities, and tech-focused trading players (formerly start-ups) could scale up and reinvest profits to broaden their activities. At the same time, large trading players focused on oil and gas or metals could increase their investments in power, and hedge funds and banks could be attracted by growing value pools. New opportunities in power and gas (but mostly in power) will likely emerge around three topics: entering new markets, data-driven trading, and new assets.
Entering new markets. Although trading markets in power and gas are still developing, a number of industry players are using synergies to enter new markets by leveraging existing capabilities. For instance, European players have entered the US power market and are now making moves toward India, Japan, and China, where markets are being reformed.
Data-driven trading. A rapidly changing market environment, combined with easier access to market data, has created opportunities for new players to enter the energy sector,5 and many of these new entrants are tech savvy and enthusiastic about data analysis.6 This trend can be further accelerated by the new generation of digital natives, many of whom are seeking to capture new digital opportunities.7 In turn, power and gas exchanges are reforming short-term markets to allow trades to be increasingly automated, which, in combination with advances in technology or market analysis, can be leveraged to translate data into trade signals more quickly and with better accuracy.
New assets. Changing energy markets require investments in new technologies and grid infrastructure. Investments in battery energy storage systems (BESS) are already growing rapidly, with more than $5 billion invested in 2022 and expectations to reach as much as $150 billion by 2030.8 In addition, the connection with other commodities, such as hydrogen and its derivatives, as well as power grid-related limitations, could create opportunities for new asset optimization.
LNG
Development: The LNG market continued to grow in 2023. In addition, LNG remained crucial to maintain energy security in Europe, with import capacity growing by 40 billion cubic meters (bcm) in 2023, with additional capacity of 30 bcm expected to be added in 2024.9 Despite the decrease in natural-gas prices and volatility (compared with record highs in 2021–22), driven by stronger nuclear production, reduced demand from industrials, and mild winter conditions in countries that rely on gas for heating, traders were able to capitalize on the opportunities between Europe and East Asia.
Competition outlook: Given the increasingly prominent role of LNG in global gas markets, particularly Europe, we expect market competition will also intensify. Oil and gas traders as well as large merchant players will thus have an opportunity to ramp up their trading capabilities in a commodity for which a global scale and large balance sheet provide a competitive advantage. As an example, utilities that relied on long-term pipeline gas contracts, such as those in Europe, could seek to expand their LNG trading capabilities on the strength of their existing customer footprints.10
Agriculture
Development: In 2023, the agricultural market faced robust demand for food and animal feed; global supply disruptions such as the Russian invasion of Ukraine, which is a major grain exporter; and increasing production costs (for example, fertilizer prices were affected by the surge of natural gas prices). Given limited feedstocks and processing capacity, demand for sustainable fuels derived from oilseeds has increased the margins of oilseed-processing facilities, with oilseed traders capturing value by trading Soybean crush spreads.
Competition outlook: Large agricultural players are expected to retain control over the production side, effectively driving global trade while adopting techniques from traders of oil and oil-based products. This trend will likely be strengthened by increased connectivity between prices for food and agriculture and prices for oil and oil-based products. At the same time, new start-ups with strong technological capabilities have the potential to disrupt the market. So too do existing traders of oil and oil-based products with downstream portfolios that require switching to green fuels. Expanding business into new locations, however, requires local, specialized knowledge because regulations and export restrictions differ across markets. In addition, geopolitical events can drastically alter export and import activities. For example, Ukraine used to export its agricultural products primarily via ports in the Black Sea but has recently started using alternative routes.
Metals and mining
Development: Metals and mining also experienced a decrease in trading profitability in 2023. Elevated energy prices increased mining and processing costs, while commodity prices came down from the highs of 2022. Nickel production increased significantly in 2023—largely because of Indonesia—while lithium saw a weak growth rate. More liquidity also entered the sector, with players increasingly setting up metals trading desks for commodities related to the energy transition.
Competition outlook: Metals and mining is expected to become more important over the next few years, with demand increasing alongside the proliferation of energy-transition technologies. In turn, this demand will likely give rise to new geopolitical powers and could change trading dynamics as we know them. Large mining players, similar to players in oil and oil-based products, are expected to engage in more trading activities, resulting in further competition. That said, falling margins are causing producers of metals and minerals to structure offtake contracts differently. New contracts are increasingly moving away from simple price-based formulas and instead focusing on securing margin shares that manufacturers can achieve by processing raw materials—for example, processing lithium into the batteries required for BESS or electric vehicles (EVs). Opportunities for new entrants will likely be constrained to so-called minor metals, such as battery-grade manganese, or the secondary market. The latter is becoming increasingly important because scrap and recycled materials are integral to a circular economy.11 At the same time, the minor metals market, which also includes cobalt, was previously characterized by low trading volumes and limited price transparency but is now maturing and gaining momentum, creating new opportunities for data-driven traders. For example, demand for uranium is expected to rise over the next few years as more nuclear energy capacity is needed to fill gaps in the clean-energy power mix.12
Two major trends shaping commodity markets in the coming years
Looking ahead, commodity markets will be influenced by greater interconnection and the growing role of power in the energy transition.
Commodity markets are becoming more interconnected
As energy markets tightened, they also became more intertwined. The average correlation between commodities core to the energy transition doubled, reaching 56 percent in 2022–23 compared with 27 percent in 2015–19. In addition, increased diversification of supply has resulted in fewer point-to-point long-term relationships and greater exposure to short-term contracts. The LNG market, in particular, has been affected by the change. Of 635 active LNG tankers operating worldwide, approximately 100 were launched in the past three years, with the overall number expected to surpass oil carriers by 2028.13 On this point, a higher number of ships strengthens connections among global markets, such as power and gas.
This trend appears at odds with news reports highlighting rising geopolitical fragmentation and countries’ efforts to become more independent. However, some regionalization has been observed, mostly among scarce commodities such as agriculture decoupling from the global price dynamics and midstream products in transition metals value chains.
Flexible contracts are on the rise. Several of the previously mentioned geopolitical events are pushing buyers to favor flexibility in long-term contracts to manage demand risk. Doing so typically entails higher exposure to global prices as residual volumes tend to be priced at the current market levels. The best example of this dynamic today is competition between Asia and Europe over additional LNG volumes. Gas prices for these markets are highly correlated and mostly affected by global environment factors such as global supply of LNG (mainly the availability of regasification capacity in the United States) and local gas demand (Exhibit 3).14
Not all markets follow this pattern. Commodities for which supply chain security is critical, such as agriculture and some metals, are often protected by local authorities. In such instances, interconnection could translate into additional imbalances across markets and thus lead to enhanced trading opportunities.
Interdependencies across commodities are creating new opportunities. New cross-commodity trading opportunities are emerging. In hydrogen, for example, evolving regulations involving clean energy are creating new linkages between oil and oil-based products and food and agriculture (Exhibit 4). Similarly, LNG and gas are being used as inputs for the production of ammonia or hydrogen for fertilizers.
The following examples illustrate additional cross-commodity opportunities:
- Evolving regulation. The European Union has adopted mandatory fuel targets requiring refineries to use green hydrogen in the refining process or to ensure that higher percentages of aviation fuel are derived from sustainable fuels. In such cross-commodity opportunities, power could command a higher share of the cost and volume in the fuel end product. Therefore, refinery and fuel station players in particular are motivated to become more active in renewable power, as evidenced by Germany’s recent offshore wind site auction and the recent introduction of the European Union’s Carbon Border Adjustment Mechanism (CBAM).15
- Supply chain links. Renewables require vast amounts of raw materials. For example, the generation of one terawatt-hour of electricity from solar and wind can consume, respectively, 300 and 200 percent more metals than the same number of terawatt-hours from a gas-fired power plant, on a copper-equivalent basis.16 Consequently, some players are setting up metals trading desks for hedging or procurement. Moreover, metal prices are a fundamental component of levelized cost of electricity (LCOE) calculations and are thus becoming increasingly important to understand long-term power forecasts and origination opportunities.
- Similarity in capabilities and ‘natural hedge’ strategies. Lithium mining and refining are in many ways similar to oil production and refining in risk profile and in the core capabilities needed. Therefore, it’s natural that some oil and gas players have become active in lithium mining, refining, and trading not only to capture expected growth but also to hedge long-term oil prices as EVs scale up. This is even leading some oil and gas players to move into minerals and metals extraction, refining, and trading, albeit over the long term.
- Using connectivity to boost market share. Given the above interdependencies, commodity trading companies are seeking to achieve global scale and branch out into additional commodities.
As markets become more interconnected and geopolitical events cause occasional transportation challenges, players with a global presence can better predict market dislocations. This reach allows players to better reorder global flows, given the increased connections across markets. For commodities that are not traded globally, such as power, global presence and scale can bring other benefits. For instance, in trading that relies heavily on data and analytics, global players can extend these capabilities across multiple regional markets.
In cross-commodity opportunities, balance-sheet strength becomes even more crucial to cover margin calls and establish a counterparty’s creditworthiness. Players that are active across markets and commodities can rapidly deploy and focus capital where major dislocations are happening.
Players that can observe existing correlations between commodities and develop strategies to monetize market volatility across the full spectrum of available commodities will have greater flexibility to respond to recent or anticipated macroeconomic events. In addition, a cross-commodity presence can help players diversify their trading portfolios and maximize the risk/return expectations of invested capital. This entails an information advantage as well as the capability to build predictive analytics based on proprietary insights.
The growing role of power in the energy transition
To meet the goals of the Paris Agreement and help enable an affordable, reliable, competitive path to net zero, power will play a central role in the energy transition.17 Higher value pools and competition across the value chain provide clear opportunities for players to benefit. Power’s fundamental role in the energy transition is also reflected in projected growth of up to 5 percent per annum, reaching $1.3 trillion to $2.4 trillion by 2040.18 Therefore, it will be crucial in replacing hydrocarbons through electrification and the production of synthetic gases and fuels.
As a commodity, power faces unique challenges. Power is a singular commodity because it needs to be generated around the same time and in close proximity to where it is consumed.
The initial stages of the energy transition have so far relied on solar and wind, but achieving the next 50 percent will be far more complex. There is no one-size-fits-all solution to clean distribution and renewables generation, with options such as nuclear, hydrogen, and carbon capture requiring significant investments. And current grids, already operating at capacity, need expansion for further decarbonization as well as accommodating EVs, offshore wind generation, and hydrogen production. In Germany, the annual build-out of the transmission grid is expected to grow by a factor of five—approximately 1,900 kilometers (km) per annum by 2035 versus approximately 400 km. Other EU countries would need to double their investments in transmission lines and other infrastructure to €550 billion per year by 2030.19
Meeting increasingly bold decarbonization goals will require significant amounts of energy from wind and solar. In fact, renewables are expected to make up the bulk of the power mix from 2030 to 2050, with wind and solar seeing the largest shares.20 As a result, power will be heavily dependent on other commodities—for example, wind turbines are made up of 40 to 50 percent steel as well as copper and aluminum.21
Traders can capture value by supporting the energy transition. Since affordable decarbonization will require efficient use of resources, trading can play a crucial role in delivering power in a more reliable, affordable, and clean manner. The following actions could better position traders to stay on top of changing markets.
1. Buy clean power to help build assets. Traders can work alongside private investors and governments to provide commercially sound projections for power offtake agreements or asset leases. On this point, long-term power purchase agreements (PPAs) for clean power have become increasingly popular, reaching more than eight gigawatts (GW) in 2022 in Europe, the Middle East, and Africa (up from approximately two GW in 2018).22 Such agreements can expand interconnections and storage, helping industry players build and operate battery energy storage systems. Another example of potential synergy between traders and asset investors and power off-takers is evidenced by the Guarantees of Origin (GoOs) market. GoOs represent an increasingly large and rapidly evolving market closely related to renewable power. The GoOs market showed significant volume growth over the past few years followed by price growth of less than €1 per megawatt-hour (MWh) to more than €8 per MWh and above in 2022.23 For investors, GoOs represent an indispensable enabler for renewables projects to take investment decisions. For customers, GoOs could be a critical route to achieve decarbonization targets, especially over the medium term.
2. Lower risks for customers. Traders can help end consumers by structuring products such as baseload and 24/7 PPAs that in turn enable investment in clean dispatchable power alongside renewables. In addition, traders can offer fixed-price contracts, relieving consumers of undesired risks and helping shift demand to cheaper periods. Finally, traders can provide end users with time- or location-specific green certificates, which can promote new builds and decarbonization support for specific sectors.24
3. Balance the power system. The growth of renewable capacity in the power system and decommissioning of legacy conventional power plants create a need for new flexible assets to ensure system reliability. By 2037, for example, Germany could have up to 80 GW of flexible energy assets, which can react to the wholesale system price.25 Traders have incentives to predict and translate weather-related supply and demand impact into price signals and concrete reactions to supply and demand. They can help balance the system by optimizing both conventional flex assets such as combined-cycle gas turbine (CCGT) power plants and new technologies, such as BESS.
4. Derisk the power supply chain. Even today, traders don’t always practice price hedging when purchasing raw materials (the offshore wind industry in particular has recently felt the squeeze, with high volatility affecting the prices of raw materials26), but decoupling investment from price swings can support a more reliable pathway to decarbonization. On this point, five low-carbon technologies are projected to be critical for the energy transition: solar, wind, EVs, heat pumps, and green hydrogen.27 For this to happen, however, massive amounts of minerals and metals will be needed (Exhibit 5). As an example, the supply of many minerals and materials required by key lower-carbon technologies could face a shortage by 2030, unless action is taken.28 Traders can therefore support the energy transition by securing and derisking supplies of materials—and by providing materials cost-effectively and in the right quantity and location.
5. Enable cost-effective green molecules and manage cross-commodity exposures. Trading organizations are well placed to promote hydrogen trading hubs and logistics and help ensure that the most cost-effective type of fuel enters the market, considering both commodity and transportation costs. Hydrogen shipping could then be expedited by converting hydrogen to synthetic fuels (such as ammonia or methanol) at export hubs, a method that will likely become increasingly important to global flows after 2030. For example, some estimates show Europe’s hydrogen demand growing from 58 to 71 million tons (MT) per year, and its local production capacity ranging from 28 to 60 MT per year, with the remaining amount covered by imports.29 However, it is far from certain that global hydrogen markets or derivatives will reach economic viability. The share of cross-commodity transactions for which traders help source renewable power and offtake agreements for hydrogen or sustainable fuels (power-to-X) is likely to increase. Traders can also contribute to the economic viability of hydrogen assets to help further market development by supporting the creation of a liquid market for risk management.
How commodity traders can differentiate
The shifting dynamics in commodity markets call for trading organizations to invest in new capabilities to capture value. In some cases, lower barriers to market entry will give rise to niche players powered by data and AI. Stakeholders may want to consider the following options:
Prioritize portfolio optimization to manage illiquid risks
Trading companies and asset players alike have become quite adept at managing liquid risks (for which there is a market to hedge against). However, they must also consider illiquid risk, such as when the lifetime of an asset extends beyond the liquid market horizon. Many of these assets are left unmanaged because of their difficulty and the high costs of externalization. This is particularly the case during periods of high and sustained volatility, when the constantly changing risk landscape requires rethinking risk management strategies.
Illiquid risks are often assigned high premiums, and negative correlation can support diversification. For example, electricity consumers typically rely on a stable power supply and are increasingly aware of the risk involved in off-taking power from intermittent assets. Similarly, renewable-asset owners that hedge production in the wholesale market may struggle with capture prices diverging significantly from baseload prices—a growing concern as renewable penetration rates rise. Hence, demand is increasing for products such as baseload PPAs for consumers or profile swaps for players in renewables.
Why portfolio optimization is critical. Today, there is a fair amount of uncertainty around which new technologies will prevail as winners to support the energy transition and what new regulation and initiatives could emerge. And of course, currently unforeseen external challenges can also transpire.
Opportunities are emerging to structure new offers for players that do not want the risk, including baseload and 24/7 clean-power PPAs (Exhibit 6).30 On this point, some energy end users that want to decarbonize are not large enough to set up economically viable in-house trading teams to manage market risk and energy needs and are thus looking for partners to provide these services. The concentration of exposure in a single geographic location can also be considered, and potential access to multiple markets could create additional diversification.
In addition, the value placed on flexible capacity is often underestimated. Many companies calculate asset value based on a limited number of market scenarios (sometimes just one) and on a stand-alone basis. For instance, a refinery has the potential to offer a range of options because operators can adjust inputs by feedstock quality and output by volume and product type or quality.
Finally, investors are assessing not only the level of returns (preferably high) but also the likelihood that returns will meet or exceed targets. Thus, companies adopting a portfolio approach can expect to provide returns that are more predictable.
Implementing portfolio optimization. Companies can consider risk/return trade-offs by combining the following three parts:
- A simulation engine that provides a view on interdependent distributions of liquid and illiquid risks. Players can go beyond direct and anchor price simulations to a fundamental model that uses uncertainties about supply and demand as inputs. From there, these advanced simulations can be combined with stress testing to understand the impact of abnormal or difficult-to-predict scenarios on the portfolio.
- An engine that converts risk-factor scenarios into gross margin, earnings, and cash flows for each asset or position. For each risk-factor scenario, a view on gross margin, earnings, and cash flow can be generated for each asset. These views will depend on the type and location of an asset as well as on the specific characteristics of each asset or position. For example, instruments with margin calls will generate a different cash profile than those without, and their impact on earnings will differ under hedge accounting or trading accounting.
- Adapted operating model. To capture the benefits of sophisticated portfolio optimization capabilities, companies need more than just tools; they can adapt investment and commercial decisions to incorporate the portfolio point of view. For instance, a higher return should be expected from an asset that increases the overall portfolio risk. In contrast, a lower return could be expected from investing in an asset that reduces the overall portfolio risk.
Overall, our research shows that players with sophisticated portfolio optimization capabilities, teams, and operating models outperform the industry by at least 1 to 2 percent risk adjusted return on capital employed.
Embrace data-driven trading models
The increased market volatility of the past few years has been accompanied by rising profits for data-driven traders (Exhibit 7). In fact, 87 percent of data-driven trading companies in Europe made more than €100 million EBIT in 2022 (versus 100 percent making less than €100 million pre-2019), and almost 30 percent created profits of more than €1 billion in 2022.
AI and generative AI are also shaping all industries—the new generation of industry players will likely be company leaders in the next five to ten years. On this point, some players, including banks, hedge funds, and utilities, have already started investing in the AI capabilities necessary to participate in changing markets. This is especially the case in power and gas, which has highly dynamic short-term traded markets largely shaped by the weather.
Players that embrace these changes are likely to capture additional value. In 2022, data-driven players captured almost a quarter of the power and gas value pool, jumping from less than 5 percent in the previous year. This growth has come about in a short time and with relatively small teams. For example, a number of players that employ fewer than 50 people have entered the space as recently as 2020.
Our analysis highlights how heightened levels of uncertainty are driving large value pools in trading. Although increased profits provide a positive outlook on the industry, they are also attracting more competition. To expand capabilities and agility, players will need to think through the macrotrends to determine which cross-commodity opportunities are the best fit, what role traders can play in power, and how to differentiate across managing illiquid risks, data-driven trading, and having deep capabilities in niche commodities.
