Introduction to Radial Basis Functions for Adaptive Control Problems
Published:
Radial basis functions (RBFs) have been used for adaptive system control in multiple water resource systems publications. They show up regularly in blog posts, such as here, and here, and others. However, they have never gotten their own detailed post.
Radial Basis Functions
Radial basis functions (RBFs) are real-valued functions whose output depends only on the distance from a central point, known as the centroid. Mathematically, an RBF depends on the norm of the difference between the input vector (x) and a vector defining the centroid (c).
They are a form of “Universal Approximator” meaning that they are flexible enough to describe any function if you combine enough of them. For example, neural networks are also universal approximators.
As described below, there are multiple different RBF formulations. So, you might ask, what makes an RBF an RBF?
The core characteristic that makes a function an RBF is the fact that the function depends on the distance between an input vector and a fixed “center” point.
Let’s use the following notation to define the “radius”, or the Euclidean distance:
\[r = ||x-c|| = \sqrt{\sum(x_i - c_i)^2}\]Where $x$ is a vector of inputs and $c$ is a vector of centers.
Now, we can define an arbitrary RBF as the function:
\[\phi(x) = \phi(||x-c||)\]This use of Euclidean distance results in RBFs having radial symmetry, meaning that the output value only depends on the distance and not the directionality of the input vector from the center point. In other words, radial symmetry suggests that the function value is constant on a hypersphere centered at C.
The real power of RBFs for function approximation comes from the aggregate combination of multiple RBFs. This is done using a weighted aggregation scheme such as:
\[f(x) = \sum w_i \phi_i(x)\]Where $w_i$ are weights applied to each individual RBF function.
In order to make the function useful, you then need to fit the function parameters so that they are useful/optimal for your modeling or control problem.
Alternative RBF Formulations
As mentioned above, an RBF function is simply a function which is dependent upon distance from a centroid. With that in mind, it makes sense that there are multiple alternative RBF formulations. In this section I share some common formulations, however it is worth highlighting that custom variates appear frequently across the literature.
Also, the formulations shown below are aggregate approximation functions which technically include multiple RBFs. However, the RBFs are really most powerful when used in this aggregated form, so I’ll present them as such.
For each formulation I adopt the following notation, where:
- $\mathbf{x}$: Is a vector containing input data.
- $\mathbf{c}$: Is a vector containing the values of the centers. This should be the same length as the $\mathbf{x}$ vector.
- $|\mathbf{x}-\mathbf{c}|$: The Euclidean distance between the input vector and the center vector.
- $m$: The number of RBFs in the aggregation function.
- $\sigma$: A scale parameter controlling the width of the RBF.
Cubic RBFs
Cubic RBFs are characterized by their polynomial growth, where the output scales with the cube of the distance between the input and the center.
\[f(\mathbf{x}) = \sum_{i=1}^m w_i \left|\frac{\mathbf{x}-\mathbf{c}_i}{\sigma_i}\right|^3\]Gaussian RBFs
Gaussian RBFs, perhaps the most widely used variant, are given by:
\[f(\mathbf{x}) = \sum_{i=1}^m w_i \exp\left(-\frac{||\mathbf{x}-\mathbf{c}_i||^2}{2\sigma_i^2}\right)\]The Gaussian RBF’s output decays exponentially with the squared distance from the center, resulting in a localized, smooth approximation of the function around the centroid.
Multi-quadratic RBFs
Multi-quadratic RBFs blend linear and quadratic terms under a square root, balancing the growth of cubic RBFs and the sharp decay of Gaussian RBFs. This gives them a more gradual fall-off, making them a solid choice for situations that need smooth transitions with broader coverage.
\[f(\mathbf{x}) = \sum_{i=1}^m w_i \sqrt{||\mathbf{x}-\mathbf{c}_i||^2 + \sigma_i^2}\]Notably, I’ve not seen any applications of the multi-quadratic RBF in water resources systems literature. If you have, please leave a reference in the comments.
RBF Applications in Water Resources Systems
RBFs have become popular for their use in Direct Policy Search (DPS) applications for adaptive system control.
DPS is an approach to control problems that seeks to optimize a parameterized policy directly. DPS approaches were first popularized for applications in robotic control problems (Deisenroth et al., 2013). In DPS, the control policy is represented as a function that maps the system state to a control action.
The key advantage of using RBFs in DPS is their ability to approximate complex, nonlinear relationships between the system state and the optimal control action. The DPS thus seeks policies that are adaptive and state-aware. This means that the control action taken by the policy can vary depending on the current state of the system, allowing for more effective and efficient management.
It’s worth noting that any flexible function can be used in a DPS context. For example, artificial neural networks (ANNs) have also been used to define the policies used in DPS. One cool use of ANN is shown in Zaniolo et al. (2021), where they use evolutionary algorithms to search for the best ANN structure during the optimization search.
However, RBFs have unique properties that make them appealing. Their radial symmetry and local behavior result in smoother and more interpretable policies compared to the “black box” representations such as deep ANNs.
This appeal is demonstrated by the fact that RBFs show up in multiple different case studies:
Lake Pollution Control (Quinn et al., 2017): This study used cubic RBFs to design policies that map pollution concentration states to release decisions, balancing ecological and economic objectives.
Fishery Management (Hadjimichael et al., 2020): Gaussian RBFs were utilized to model policies that adaptively regulate harvest efforts based on prey population states, optimizing for sustainable yields.
Reservoir Operations (Giuliani et al., 2016): RBFs provided a means to control reservoir releases by mapping inputs like storage levels and inflow forecasts to decision variables, allowing for adaptive water allocation.
Hydropower Financial Risk Management (Hamilton et al., 2021): Gaussian RBFs were applied to model decisions involving financial management strategies, using complex state information such as fund balances and market indices to determine hedging actions.
Table: A few uses of RBFs in the water resources literature.
| Citation | Control Problem | RBF Type | Inputs (State Information) | Outputs (Decisions) |
|---|---|---|---|---|
| Quinn et al. (2017) | Lake Pollution | Cubic | Pollution Concentration | Pollution Release |
| Hadjimichael et al. (2020) | Fishery Harvesting | Gaussian | Prey Population | Harvest Effort |
| Giuliani et al (2016) | Reservoir Operations | Gaussian | Volume, Prior Release, Next days inflow | Reservoir Release |
| Hamilton et al. (2021) | Hydropower Financial Risk | Gaussian | Fund balances, debt, power price index, cash flow | Withdrawal & Hedging |
Ultimately, RBFs are one small piece of the much larger adaptive control space. They become valuable when combined with DPS and multi-objective evolutionary algorithms.
Regardless, I think it is valuable building a strong understanding of these core concepts before approaching a large system control problem. Especially since the choice of RBF (or approximation function more generally) can influence the characteristics and tradeoffs of your final “optimal” control policy (Salazar et al., 2024).
References
- Deisenroth, M. P., Neumann, G., & Peters, J. (2013). A survey on policy search for robotics. Foundations and Trends® in Robotics, 2(1–2), 1-142.
- Giuliani, M., Castelletti, A., Pianosi, F., Mason, E., & Reed, P. M. (2016). Curses, tradeoffs, and scalable management: Advancing evolutionary multiobjective direct policy search to improve water reservoir operations. Journal of Water Resources Planning and Management, 142(2), 04015050.
- Quinn, J. D., Reed, P. M., & Keller, K. (2017). Direct policy search for robust multi-objective management of deeply uncertain socio-ecological tipping points. Environmental modelling & software, 92, 125-141.
- Hadjimichael, A., Reed, P. M., & Quinn, J. D. (2020). Navigating deeply uncertain tradeoffs in harvested predator‐prey systems. Complexity, 2020(1), 4170453.
- Zaniolo, M., Giuliani, M., & Castelletti, A. (2021). Neuro-evolutionary direct policy search for multiobjective optimal control. IEEE Transactions on Neural Networks and Learning Systems, 33(10), 5926-5938.
- Hamilton, A. L., Characklis, G. W., & Reed, P. M. (2022). From Stream Flows to Cash Flows: Leveraging Evolutionary Multi‐Objective Direct Policy Search to Manage Hydrologic Financial Risks. Water Resources Research, 58(1), e2021WR029747.
- Salazar, J. Z., Kwakkel, J. H., & Witvliet, M. (2024). Evaluating the choice of radial basis functions in multiobjective optimal control applications. Environmental Modelling & Software, 171, 105889.