## Abstract

Controlling active colloidal particle swarms could enable useful microscopic functions in emerging applications at the interface of nanotechnology and robotics. Here, we present a computational study of controlling self-propelled colloidal particle propulsion speeds to cooperatively capture and transport cargo particles, which otherwise produce random dispersions. By sensing swarm and cargo coordinates, each particle’s speed is actuated according to a control policy based on multiagent assignment and path planning strategies that navigate stochastic particle trajectories to targets around cargo. Colloidal swarms are shown to dynamically cage cargo at their center via inward radial forces while simultaneously translating via directional forces. Speed, power, and efficiency of swarm tasks display emergent coupled dependences on swarm size and pair interactions and approach asymptotic limits indicating near-optimal performance. This scheme exploits unique interactions and stochastic dynamics in colloidal swarms to capture and transport microscopic cargo in a robust, stable, error-tolerant, and dynamic manner.

## INTRODUCTION

The relatively recent ability to synthesize self-propelled colloidal particles has generated considerable interest in understanding and manipulating these particles based on their similarities to microorganisms (*1*, *2*) and their potential use in microscopic devices (*3*, *4*). Collections of these particles are often referred to as active matter and are scientifically interesting because of their nonequilibrium nature and unique emergent behaviors and properties, including their effective thermodynamics (*5*–*7*), collective motions and stresses (*8*–*10*), and connections to living systems (*11*). From a technological standpoint, self-propelled particles have been proposed for use in nano/microrobotics (*4*, *12*), biomedical devices (*13*), environmental remediation (*14*), sensing (*15*), separations (*3*), assembly (*16*), and micromachines (*17*, *18*). Many approaches to harnessing self-propelled particles are based on single particles, although cases involving assembly and micromachines begin to consider how self-propelled particle ensembles can be exploited to perform useful functions. Without control, self-propelled particle random walks (*19*) often produce by default randomly fluctuating fluid structures (*20*), effective phase separation (*5*), or small clusters (*16*) but do not spontaneously produce useful work characteristic of machines.

One potential route for exploiting self-propelled particles in applications is to formally control their individual and collective trajectories to perform useful microscopic functions. Examples of feedback-controlled self-propelled particles include navigation in free-space (*21*) and mazes (*22*). These examples involve noninteracting particles, which simplifies control schemes but also limits the complexity of emergent behaviors and functions. External fields have also been used for feedback control of interacting particles in stochastic self-assembly (*23*) of perfect crystals (*24*) as well as deterministic cargo transport within clusters (*18*), but without the ability to address individual particles. We recently learned of a study, while finalizing this manuscript (*25*), where feedback was used to specify individually addressable self-propelled particle speeds (*26*); however, the objective was mimicking group formation in living systems. We are unaware of any precedent for feedback control of interacting and independently addressable self-propelled colloidal particles to perform precision microscopic functions.

Desirable features of animal and robot swarms provide motivation to exploit useful properties of interacting and independently addressable self-propelled colloidal particles to perform useful functions, although with consideration of additional unique aspects such as stochastic motion and pair potentials. For example, ants collaborate to carry cargo (e.g., food) beyond any individual’s capabilities through both instantaneous local observations (distributed control) and additional nonlocal information based on memory, communication, social structure, etc. (effective central control) (*27*). Such an approach is necessary to efficiently perform coordinated work in a collaborative manner, which has been mimicked in robot swarms (*28*). In a contrasting example on microscopic length scales, bacterial swarm motion is generally not sufficiently organized to perform complex tasks in a stable efficient manner. Although a bacterial swarm has been shown in a simple demonstration to deliver colloidal cargo (*29*), this example illustrates intelligent human intervention to coordinate bacterial motion on a pattern, which is effectively introducing central control. The common feature in these examples is the performance of tasks by autonomous agents interacting via nontrivial rules, which is difficult to program into colloidal interactions. Although light-activated self-propelled particles have been assembled into unsteady (*16*) and steady-state (*30*) structures via controlled illumination, the particles in these examples are not independently addressable and have not performed any obvious measurable work.

Here, we aim to understand whether individually addressable self-propelled colloidal particles can be controlled collectively as a swarm to perform useful machine-like functions. In particular, we report computer simulations of a scheme to enable feedback control of an ensemble of self-propelled colloidal particles to operate as a reconfigurable swarm capable of carrying out tasks including capturing and transporting microscopic Brownian cargo. Capture is achieved via caging of cargo within the microstructure of a self-propelled colloidal swarm, which has repulsive interactions with the cargo, so cargo remains unattached to swarm particles. This strategy is adaptive and flexible as it avoids covalent or noncovalent anchoring of cargo to self-propelled particles (*3*, *4*). Feedback-controlled swarm reconfigurability provides the ability to adapt and respond to stochastic disturbances via corrective actions. Our approach is based on self-propelled colloids with individually actuatable speeds mediated by an underlying array, which could be realized by a number of externally but locally actuated transport mechanisms (Fig. 1 and movie S1).

There are a number of challenges with this scheme from a control perspective based on the unique behaviors of colloids. Because each particle’s propulsion direction is determined by uncontrolled Brownian rotation, not all degrees of freedom can be controlled deterministically (i.e., the swarm is underactuated and stochastic). An essential aspect to enable swarm functions is the determination of a control policy to achieve the necessary coordinated motion. In short, swarm capture and transport of cargo require multiagent stochastic control to direct particles to cage cargo and simultaneously generate directional forces in a cooperative manner. In addition to executing these functions over minimum times and distances, we also consider the optimization of energy efficiency based on task power output relative to propulsion power input, which enables the use of scarce resources. By mimicking intelligent swarms (e.g., animals and robots), this approach demonstrates centralized multiagent stochastic control over self-propelled colloidal swarms to perform cargo capture and transport in a robust, stable, and efficient manner.

## RESULTS AND DISCUSSION

### Controlling colloidal swarms

A particle-scale dynamic model of self-propelled colloidal swarm particles and the cargo particle is implemented to capture the dominant contributions of Brownian rotation and translation and colloidal interactions commonly observed in colloidal feedback control experiments (see Methods and fig. S1) (*23*, *24*, *31*). Although more rigorous and complex models of hydrodynamic interactions [e.g., translation-rotation coupling (*32*), concentration and configuration dependence (*33*)] and propulsion mechanisms (*34*) could introduce quantitative differences, the conceptual problem and general multiagent control algorithms presented in the following are not expected to change. Practically, for an equation of motion with different or additional terms, only the details of planning paths of swarm particles to surround cargo would need to be updated in the general approach demonstrated in this work.

Feedback control of self-propelled particle swarms requires the following elements (Fig. 1A): (i) sensing the system state via particle tracking, i.e., the self-propelled particle positions and orientations and cargo position; (ii) calculating self-propulsion speeds of each particle based on the current state and desired future state (based on a control objective); and (iii) actuating each particle’s speed by considering light-activated self-propelled colloids on a light array [e.g., a liquid-crystal display (*35*) screen surface]. Actuation could also be achieved on spatially addressable electrodes or other arrays that mediate locally actuated transport mechanisms (*36*–*39*), which could modify some terms of the dynamic model, but the control problem would be conceptually similar.

The nontrivial element in this control scheme is the determination of the policy to close the loop between sensing the system state and actuating each particle’s speed to achieve the specified control objectives. To achieve both cargo capture and transport, at each control update time, Δ*t*_{C}, the control policy must assign swarm particles to track target locations around cargo and plan paths to navigate particles toward multiple dynamic targets (Fig. 1B). Throughout the process of self-propelled particles surrounding and capturing the cargo within a cage, a lattice of target sites is dynamically constructed to track the Brownian cargo particle (Fig. 1C). In the reference frame of the cargo particle, the lattice target sites are fixed. Each swarm particle is assigned to a unique target to minimize the sum of distances between all particles and their targets, which can be expressed by a time-dependent assignment function, *g**, given by**r**_{S} and **r**_{T} are swarm particle and target position vectors indexed by the set *I*_{S} = [1, 2, …, *N*], *g*(*i*|*t*) denotes the assignment for particle *i* such that **r**_{T,g(i|t)} is the target location particle *i* is assigned to track at time step *t*, and *G* is the set of all possible assignments (see Methods). The solution of Eq. 1 is obtained using the Hungarian algorithm for combinatorial optimization (*40*). The assignment decision captured by *g** can be interpreted as maximizing the coordination and cooperation of the self-propelled particles making up the swarm. Each swarm particle is assigned to track target positions around the cargo to minimize the total traveled distance of all particles instead of its own traveled distance (e.g., self-propelled particles do not necessarily track their nearest target; see fig. S2). The assignment solution also naturally avoids crossing paths that might result in swarm particle collisions (fig. S2).

After assigning self-propelled particles to targets around cargo, the propulsion speed for each particle is determined as part of path planning. Given particle assignment to targets, *g**, the self-propulsion speed for each particle, *v*** _{S,i}*, can be formulated as

**r**

_{S,i}(

*t*+ Δ

*t*

_{C}), and their assigned targets,

**r**

_{T,g*(i|t)}(

*t*+ Δ

*t*

_{C}), in the future at the next control update period given a maximum allowed propulsion speed,

*v*

_{max}. Because the probability distribution of future swarm particle and target positions depends on particle interactions and stochastic dynamics, the solution of Eq. 2 is not trivial [e.g., deterministic solutions remain the subject of deep learning methods (

*41*)].

We make several simplifying assumptions to yield a solution to Eq. 2, which can be practically tested on the basis of the performance of the resulting control policy. By considering relatively short control update times, we assume the following: (i) Particle positions are primarily determined by propulsion rather than interactions with nearby particles, which allows independent optimization of each particle’s speed; (ii) rotational diffusion has limited time to reorient particles, which allows consideration of current orientations (instead of future probabilistic orientations); and (iii) target locations are relatively unchanged on the basis of their reference frame connected to the relatively slow cargo translational diffusion, which allows consideration of current target positions (instead of future probabilistic positions). Note that because feedback control generally corrects errors, these assumptions do not have to be strictly satisfied. In any case, by considering these assumptions, an approximate solution of Eq. 2 can be written as (see Methods and fig. S3)*d _{i}* = (

**r**

_{T,g*(i|t) − rS,i})·

**n**

*is the projected distance of the swarm particle-target vector onto the swarm particle orientation vector. The policy in Eq. 3 optimizes each particle’s speed to minimize the expected distance between each self-propelled particle and its assigned target position at each control update time. When targets are in front of self-propelled particles, propulsion is adjusted in proportion to the projected distance (up to a maximum). If targets are behind self-propelled particles (*

_{i}*d*≤ 0), then no propulsion is applied to avoid increasing distance to targets. The assignment and path planning strategies together avoid lattice vacancies and dynamic arrest since cooperative motion is automatically programmed into this policy. The policy in Eq. 3 and its performance is similar to our previous findings (

_{i}*22*) in the limit of single particles navigating to single targets (fig. S3), but the current policy is amenable to continuous propulsion and real-time computation with multiple particles and targets (compared with the infinite horizon Markov decision process framework used in our prior work).

Using the assignment and planning strategies in Eqs. 1 and 3, we designed an algorithm to capture Brownian cargo by navigating self-propelled particles to sites surrounding cargo (see Methods, Algorithm 1). To study different swarm sizes, lattice sites around cargo consist of one to five complete hexagonal non–close-packed coordination shells (6, 18, 36, 60, and 90 particles; fig. S3). We chose lattice spacing as the pair potential minimum separation to minimize swarm potential energy (fig. S1), which prevents swarm particles from propelling against pair repulsion. A control update time, Δ*t*_{C} = 0.1 s, was empirically optimized and satisfies the assumptions underlying Eq. 3(<1/*D*_{R}, <*a*^{2}/*D*_{T}). By measuring the system state at each control update time, new target lattice coordinates are constructed, and target assignments and propulsion speeds are determined using Eqs. 1 and 3. Practically, updating assignments based on updated system states is critical to avoid swarm particles blocking each other from reaching targets and is essential to enabling the simple navigation policy in Eq. 3. In contrast, if particle-target assignments were fixed, then a much more sophisticated and computationally expensive navigation policy would be required for each swarm particle to dynamically navigate a reconfigurable maze of its self-propelled neighbors (*22*).

The cargo transport algorithm (see Methods, Algorithm 2) has many of the same elements as the capture algorithm with several new aspects. Cargo transport requires the swarm to continue caging cargo but also assigns a subset of swarm particles to use propulsion to simultaneously generate a net directional force (Fig. 1D). Particles are assigned to the transport task based on the following: (i) proximity to their targets, which indicates particles that can afford to generate propulsion in another direction for a control update period; (ii) their coordination number, where a greater number indicates that they are likely to remain on the lattice to maintain cargo capture; and (iii) their orientation relative to the desired transport direction, where the projected component of their velocity determines their contribution available for transport. Particles selected for transport are set to maximum propulsion for one control update period to produce translation. Particles not selected for transport continue to have target assignments and speeds optimized via the capture algorithm.

As previewed in Introduction, it can be noted that the centralized multiagent control scheme developed in this work is more similar to animal and robot swarms than to microorganism swarms. Centralized control based on global information is necessary to gain sufficient control authority and effective coordination to overcome challenges due to underactuation (no control over rotation) and stochastic positions and orientations (Brownian motion). Coordination is achieved in the central control design by (i) specifying a target hexagonal lattice that is energetically favorable and minimizes energy usage once targets are reached and (ii) assigning targets and path planning in a manner that minimizes travel distances, crossing paths, and target competition. In the following, this degree of coordination via central control of multiple agents is shown to be an effective strategy that established a benchmark for robust, stable, and near-optimal control based on task performances approaching asymptotic theoretical limits.

### Swarm cargo capture and transport demonstration

We now demonstrate cargo capture and transport for transient and steady-state processes using the feedback control algorithms for colloidal swarms of varying sizes and pair interactions. For swarms starting from initial configurations encircling cargo with a radius of ~300*a*, the average transient time to capture cargo takes ~300 to 450 s, which increases slightly for larger swarm sizes presumably due to cooperative motion (fig. S3). For different combinations of swarm size and pair attractions, representative images (fig. S3) and movies (movies S2 to S5) depict transient cargo capture processes starting from a radius of ~20*a* and steady-state captured configurations.

To better illustrate the transient cargo capture and its evolution into steady-state cargo transport, we track a number of quantitative metrics for the *N* = 90 swarms and three levels of pair attractions (mechanism are most pronounced in this largest swarm studied) (Fig. 2). Representative configurations and trajectories of all swarm particles and the cargo are plotted for the course of an entire capture and transport process for a ~1300-s total time period (Fig. 2, A and B, movies S7 to S9 for *N* = 90, and movie S10 for other cases). Trajectories show pair attraction improves the process effectiveness including maintaining cargo capture and transporting cargo over longer distances.

In all cases, self-propelled particles rapidly approach targets from a starting radius of ~20*a* in ~50 s based on particle distances from targets, Δ*d*_{S} (Fig. 2C). The cargo is captured in a similar or slightly shorter time than the swarm particles reaching their targets, as can be seen from small fluctuations of the cargo position about the swarm centroid, Δ*d*_{C} (Fig. 2D). The degree to which self-propelled particles maintain target lattice positions at steady state, after the transient capture process but before translation is commenced, is enhanced for increased attraction with Δ*d*_{S} ≈ 2*a*, 1*a*, and 0.2*a* for pair potential energy minima of *U*_{M} = 0*kT*, 3.2*kT*, and 5.3*kT*. The degree of cargo capture at steady state is also correlated with particles maintaining target positions and, thus, an effective cage, which is apparent from steady-state values of Δ*d*_{C} ≈ 0.8*a*, 0.2*a*, and 0.1*a* for increasing pair attraction.

After cargo capture, transport is initiated and quantified by the directed translational motion of the cargo along the transport direction of *x* axis (Fig. 2B). The ability of the feedback control algorithms to maintain target positions and cargo capture while generating directional translational forces is enhanced by particle pair attraction. Two-dimensional swarm and cargo trajectories (Fig. 2B) visually show a correlation between swarm attraction and transport effectiveness. No pair attraction produces the shortest directed translation (*x*_{C} ≈ 200*a*), the greatest stochastic orthogonal motion (*y*_{C} ≈ 40*a*), and the greatest fluctuations of swarm and cargo particles about targets (〈Δ*d*_{S}〉 ≈ 4*a*, 〈Δ*d*_{C}〉 ≈ 2*a*). These results are consistent with the tendency for uncontrolled active particle ensembles to display large density fluctuations (*2*, *20*, *42*). In contrast, swarms with 5.3*kT* pair attraction yield the greatest translated distance (*x*_{C} ≈ 700*a*) with minimal diffusion in the orthogonal direction (*y*_{C} ≈ 5*a*) while maintaining close adherence to targets (〈Δ*d*_{S}〉 ≈ *a*, 〈Δ*d*_{C}〉 ≈ *a*). Feedback-controlled swarms with more pair attraction appear to better maintain cargo capture and simultaneously transport steadier, faster, and further (~50 to 200%).

Cargo capture and transport effectiveness versus pair attraction (Fig. 2) illustrate how feedback control together with swarm potential energy resists the natural tendency for the swarm to disperse via Brownian motion (for no propulsion and full propulsion). On the basis of this consideration, we quantify how self-propulsion energy is injected into the swarm at each control update time [in contrast to a uniform energy addition that simply raises the effective temperature (*7*)]. The instantaneous input power to the swarm at each time step is a sum of the power to each self-propelled particle given by*v*_{S,i}**n*** _{i}*, and propulsion force, 6πμ

*av*

_{S,i}

**n**

*(Stokes drag times velocity is force balancing propulsion force at low Reynold’s number). Because the assignment of particles to capture and transport processes is defined in our algorithms, it is also possible to track the energy input to each subpopulation of the swarm performing either capture (*

_{i}*W*

_{C}) or transport (

*W*

_{T}) where

*W*

_{S}(

*t*) =

*W*

_{C}(

*t*) +

*W*

_{T}(

*t*).

For the *N* = 90 swarms at different pair attraction levels, trajectories of total swarm power, *W*_{S}, as well as capture and transport subpopulation powers, *W*_{C} and *W*_{T}, are shown during capture and transport processes (Fig. 2E). For each case, power is initially high as all particles are assigned to the capture process, and many have maximum propulsion values (Fig. 2A, I); the initial power is ~50% of the maximum power for all particles with *v*_{max}. This initially high-power input decays markedly over ~50 s as particles start reaching assigned targets (Fig. 2A, II) and continue performing minor adjustments to retain captured cargo at steady state at >100 s (Fig. 2A, III). The steady-state capture power, *W*_{C}, plateaus at decreasing values with increasing pair attraction. This reflects the role of decreased swarm potential energy via pair attraction competing against entropically driven swarm dispersion, so less power is required via feedback-controlled propulsion.

After cargo transport is initiated at 300 s, *W*_{C} and *W*_{T} quickly reach time-averaged plateaus with different means, fluctuations, and relative levels for each pair attraction. During cargo transport, less power is required to maintain cargo capture as pair attraction increases, which allows more resources to be directed toward transport. At the highest pair attraction, transport power exceeds that for cargo capture; this results from swarm pair attraction maintaining a crystalline cage around cargo (movie S9), and a resulting substantial drop in the power requirement for the feedback-controlled capture processes. These results indicate an opportunity to manage swarm resources as part of optimizing its functionality subject to different objectives (e.g., fastest transport, greatest distance, efficiency). In the following sections, we first analyze swarm configurations and forces to provide mechanistic insights into swarm functions and then return to a comprehensive analysis of energy management as a design consideration.

### Swarm structures and forces: Cargo capture

With successful demonstration of controlled colloidal swarm capture and transport of cargo, we now examine emergent microstructures and directional forces generated during steady-state processes. Continuing with the *N* = 90 swarms as a benchmark, the measured steady-state time-averaged two-dimensional density profile and inward radial force components are reported versus swarm pair attraction (Fig. 3A). In the case of no attraction (movie S2), the steady-state swarm density profile has no obvious angular dependence, and the radial dependence indicates a crystalline cage around the core cargo that is surrounded by a nonuniform fluid structure decays exponentially with a long tail to ~20*a*. As swarm pair attraction increases to 3.2*kT* and 5.3*kT* (movies S3 and S4), the proportion of particles in the crystalline core increases, and the fluid diminishes. At the highest pair attraction, the swarm fluid periphery all but vanishes, with a few particles infrequently escaping via stochastic disturbances, as a sort of gas phase, but feedback control always returns particles to the swarm. Swarm pair attraction together with feedback control mediates different swarm configurations caging cargo.

Because pair attraction <6*kT* is insufficient to crystallize passive colloids in two dimensions (e.g., movie S6) (*43*), crystalline cages around cargo are clearly nonequilibrium structures formed via feedback control. Because entropy disperses swarms with either no propulsion or uncontrolled full propulsion, the feedback scheme must generate in a controlled manner a net inward radial force that overcomes dispersion. The inward radial force toward the central cargo, as well as swarm center of mass, due to self-propulsion can be obtained from the radial component of each particle’s instantaneous velocity vector as **F*** _{a}* = 6πμ

*av*

_{S}

**n**. For comparison, we also compute each particle’s instantaneous force component in the tangential direction (i.e., orthogonal to the radial direction). Results show that feedback control produces a net positive inward radial component due to self-propulsion (Fig. 3A). Tangential forces are ~10× smaller and randomly oriented, which shows that the control algorithm generates no other undesirable net directional stresses. Microstructures and forces for smaller swarms are qualitatively similar to the

*N*= 90 swarms (e.g., see fig. S3 for varying swarm sizes, and figs. S5 and S6 for

*N*= 60 swarms).

Swarm radial forces decrease as pair attraction increases, which can be understood based on swarm potential energies and concentration profiles. We can show there exists a balance between controlled self-propelled particle inward pressure and the two-dimensional swarm osmotic pressure. The swarm osmotic pressure is determined by sedimentation equilibrium profiles (*44*), which agree with the hard disk equation of state (*45*) in the absence of attraction or propulsion (see the Supplementary Materials and fig. S4). As higher pair attractions decrease swarm osmotic pressure, the net inward pressure due to controlled self-propelled swarm particles also correspondingly decreases. Although the control policy is designed to steer independently addressable particles to targets around cargo, this analysis shows from another viewpoint that, collectively, a delicate swarm-scale force balance emerges. Furthermore, these results demonstrate a consistent interpretation of the swarm concentration, microstructure, and force profiles including the presence of crystalline cores coexisting with peripheral nonuniform fluids.

To characterize steady-state cargo capture performance versus swarm size and pair attraction, we report metrics for cargo location relative to the swarm center (Fig. 3B), cargo diffusivity relative to its free space value (Fig. 3C), and swarm particle locations relative to targets (Fig. 3D). All metrics are obtained from mean squared displacement (MSD) analyses (fig. S7). Cargo localization relative to the swarm center and swarm particle localization relative to target sites are both finite in all cases, indicating successful capture. Cargo localization less than the particle radius is achieved for greater swarm sizes and pair attraction due to strong confinement via a locally crystalline core (as a sort of freezing criterion) (*46*). Relative cargo diffusivities greater than unity indicate apparent cargo heating (*7*) via buffeting by swarm particles for small swarms and pair attraction. In contrast, vanishing diffusivities for large swarms and pair attraction again indicate crystallization in the swarm core. Swarm particle localization on targets indicates the degree of control fidelity, which is also aided by increased pair attraction and swarm size, which both favor crystallization on target sites [as a sort of critical nucleus size (*47*)]. Briefly, increasing swarm size and pair attraction together with controlled inward propulsion forces overcome entropy to maintain a dense crystalline cage around cargo with a coexisting lower density fluid periphery.

To demonstrate that this feedback control scheme without modification is robust and stable in the presence of disturbances (besides Brownian motion), we measured cargo and swarm particle MSDs relative to targets for steady-state cargo capture processes in the presence of added noise. For all swarm sizes and pair potentials reported in Fig. 3, Gaussian uncertainty with a relative SD of 5% was simultaneously added to particle positions and orientations, swarm particle speeds, and swarm particle attraction. Slight increase in steady-state MSD plateau values relative to the noise free cases (fig. S7) shows that the control algorithm is robust in the presence of disturbances to sensors, actuators, and system state data. Mechanisms to ensure robust stable control include (i) a controller design for stochastic Brownian motion already suited to correct for uncertainties, (ii) actuation of swarm particles in proportion to distance and alignment with targets and with a maximum speed to avoid overshooting and input energy spikes, and (iii) pair attraction to maintain swarm structure integrity. Practically, the level of disturbance investigated in fig. S7 is beyond anything encountered in prior feedback-controlled colloidal assembly experiments (*23*, *24*, *31*), but implementation of these control approaches in complex media (e.g., tissue, soil) could encounter more severe disturbances.

### Swarm structure and forces: Cargo transport

In comparison to steady-state cargo capture, unique changes arise in swarm structures and forces structure during steady-state cargo transport. Cargo transport by the *N* = 90 swarms for different pair attraction levels produces anisotropic concentration profiles and directional force polarization in the transport direction (Figs. 2A, IV, and 4A and movies S7 to S10). The concentration profile during transport is compressed on the swarm’s leading edge and expanded on the trailing side. Distortion to the swarm concentration profile and structure decreases with increasing pair attraction. A low-concentration longtail is observed in all cases corresponding to detachment of individual particles from the swarm, which then undergo free navigation to rejoin the swarm (*22*).

The swarm force distribution in the translation direction, based on the projection of self-propelled velocity vectors (Fig. 4A and see Methods), has a bipolar structure that is most evident in the cases with less pair attraction. These plots indicate the swarm’s trailing side has particles with positive forces, indicating they are directed toward the cargo and in the transport direction; in contrast, the swarm’s leading side has many particles with predominantly negative forces, indicating they are also directed toward the cargo for caging purposes and pointed in the opposite direction to the transport. The force profile in the direction of transport results from a compromise of simultaneously translating the cargo and maintaining the caging structure, which can be aided by increasing pair attraction. After subtracting directional forces, the remaining component of the radial forces during steady-state transport is positive, indicating a net inward pressure on the cargo similar to steady-state capture (Fig. 3). In all cases, the inward self-propelled force via feedback control is essential to avoid the tendency for swarm dispersion during transport (movie S11).

Within the bipolar force distribution in Fig. 4A, as expected, there is a net force in the positive transport direction that produces translation of the swarm (i.e., the red lobe is greater than the blue lobe). As swarm pair attraction is increased, the proportion of propulsion vectors with greater magnitudes in the positive direction increases, which allows the transport speed to be increased. Because pair attraction maintains cargo capture more easily with less assignment of particles to the capture process, more self-propulsion resources can be assigned to generate forces in the transport direction. This finding is consistent with the analyzed energy distribution between capture and transport functions in Fig. 2 but now can be understood in terms of particle-scale directional forces that simultaneously maintain capture while generating translational forces.

As in the capture process, steady-state transport performance versus swarm size and pair attraction is characterized by cargo localization (Fig. 4B), cargo diffusivities (Fig. 4C), and swarm localization (Fig. 4D) (from MSD analyses in fig. S8). Similar to steady-state capture, during steady-state transport, trends indicate increased swarm size and pair attraction both produce better cargo and swarm particle localization and decreased diffusivities. However, during transport, cargo and swarm particles show less localization at targets compared with the capture process for the same swarm size and interactions. This finding is perhaps consistent with expectations since swarm particles must split resources between two functions and maintain condensed caging structures with lower fidelity. Results in fig. S8 for addition of extra Gaussian noise to simulations of steady-state cargo transport (same noise introduced in capture process in fig. S7) again show the control scheme is robust in the presence of simultaneous disturbances to sensors, actuators, and system state variables. In addition to mechanisms already noted for stabilizing control, limiting the number of particles that contribute directional forces for transport enables the majority of swarm particles to maintain cargo capture and the swarm structural integrity.

### Swarm power management

We now investigate how swarm parameters determine energy management and efficiencies to optimize functions based on different objectives (e.g., speed and distance) and constraints (e.g., limited resources). We first discuss mechanisms using rendered configurations to illustrate the intricate interplay of pair attraction, swarm size, and swarm particle assignments to capture and transport processes (Fig. 5A). During transport, maintaining cargo capture via the caging structure integrity is inherently a priority over transport (Algorithm 2); this is coded into criteria for whether particles are eligible to produce transport based on their orientation, positions relative to targets, and coordination number. For smaller swarms and weaker particle attraction, a substantial portion of propulsion energy is necessarily directed toward capture since condensed structures enclosing cargo disassemble more easily. As swarm size and pair attraction increase, crystalline cages around cargo require significantly less feedback-controlled propulsion input, thus freeing up resources for a greater proportion of propulsion to be directed toward transport.

Because transport speed and energy consumption are inherently linked and because maximizing speed is an interesting design criterion in its own right, we first analyze how speed depends on swarm parameters. We summarize cargo transport speeds versus swarm size and pair attraction (Fig. 5B and movies S7 to S10). Transport speeds are negligible for small swarms (*N* < 60) with low attraction (<3*kT*) due to their inability to simultaneously maintain cargo capture and produce propulsion for transport (<4 particles contribute to transport; Fig. 5A). While all swarm size and pair attraction combinations result in successful cargo capture, the locus of points indicating finite transport speeds in Fig. 5B indicates a threshold for successful cargo transport. Low attraction swarms (<3*kT*) can have relatively large speed fluctuations (Fig. 5B), consistent with large swarm particle position fluctuations (Fig. 4A and fig. S6). At attraction ~15*kT*, the large fluctuations in the *N* = 6 swarms occur as the result of structures that cannot easily reconfigure to maximize transport speed.

As more swarm particles contribute to transport, faster transport speeds are obtained for larger swarms and more pair attraction, up to a limit (Fig. 5, A and B). For each swarm size, a marked speed increase occurs upon swarm crystallization around cargo, which occurs at different attraction levels for each swarm size. This appears as a sort of critical crystal size similar to heterogeneous nucleation, with the critical difference that feedback control is involved in determining nonequilibrium steady-state microstructures. Swarm crystallization aids cargo capture and requires less feedback control, which allows more propulsion to be committed to transport. Maximum transport speeds with increasing swarm size and pair attraction approach a theoretical limit (see the Supplementary Materials), based on assuming all swarm particles have propulsion assigned to transport, as given by^{1}/_{6} of the time based on Brownian rotation. On the basis of these findings, swarm size and potential energy can be chosen to design the cargo transport speed.

Instead of maximizing transport speed, power use in the presence of limited resources could be another design criterion. Our results can be analyzed to find how swarm parameters influence energy consumption including total energy, distribution between functions, and transport efficiency. Similar to the analysis of instantaneous power for the *N* = 90 swarms (Fig. 2), we summarize steady-state power during capture and transport processes versus swarm size and attraction (Fig. 5C and fig. S9). Larger swarms require more energy for both capture and transport. For example, for cargo capture in the absence of attraction, more power is required to make more self-propelled particles maintaining steady-state targets (Fig. 5C). In this limiting case, power per particle is ~15*kT*/s for all swarm sizes, which is close to one-sixth of constant full-propulsion power per particle (i.e., 6πμ*av*_{max}^{2}/6 = 116.4/6 = 19.42*kT*/s; particles oriented within ±30° of targets use near full propulsion or one-sixth of the time for Brownian rotation). This shows that in the absence of attraction, particles consume energy near the maximum rate to maintain targets.

As pair attraction is increased, energy consumption for steady-state capture decreases (Fig. 5C). This results from swarm attraction maintaining targets with less propulsion (~1*kT*/s per particle at highest attraction investigated). The lowest capture power is obtained for the smallest swarm size at the highest and lowest attraction levels. At intermediate attraction levels, slightly larger swarms have the lowest power, presumably as the result of the increased size aiding formation of condensed swarm states around cargo. In contrast to steady-state capture, steady-state transport consumes energy in proportion to swarm size but is mostly independent of attraction (fig. S9). The independence from attraction arises from the fact that during transport, all swarm particles experience self-propulsion the whole time. However, the proportion of propulsion assigned to maintaining capture and producing transport changes continuously as pair attraction changes.

In addition to the qualitative mechanistic interpretation already provided in Fig. 5A, the proportion of self-propelled particle power dedicated to transport relative to the total swarm power is quantified by the ratio, *W*_{T}/*W*_{S} = *W*_{T}/(*W*_{C} + *W*_{T}), as a function of swarm size and pair attraction (Fig. 5D). Trends in percent swarm power directed toward transport mimic the maximum velocity results (Fig. 5B), which occurs for similar reasoning (i.e., velocity is directly related to transport power). At low pair attraction, little to no transport occurs since nearly all resources are assigned to maintaining cargo capture. As pair attraction increases, a sharp increase in transport power occurs since pair attraction assists cargo caging. These trends are most obvious in the *N* = 90 swarms, where only 15% of swarm power is assigned to transport with no pair attraction but jumps to 80% at ~5*kT* attraction and asymptotically approaches ~100% with >20*kT* attraction. One consideration is that the capture algorithm in combination with limited propulsion rates cannot overcome swarm particle aggregation for >8*kT* pair attraction; however, this is overcome easily by assembling swarm particles around cargo at lower pair attraction and then increasing attraction after swarm particles are on lattice target positions. These findings show finite resources must be assigned to capture power for all practical conditions, since activating transport alone will disperse the swarms at all attraction levels investigated (movie S11).

One way to quantify the effectiveness of swarm functions is to define an efficiency based on work performed relative to input energy [e.g., uncontrolled active colloids (*48*)]. However, in the context of a colloidal swarm acting as a transport device, efficiency may also be considered on the basis of distance covered divided by energy input (e.g., miles per gallon for cars). Transport efficiency can be defined for colloidal swarms based on average cargo transport speed and average swarm power during transport as

Using this definition, for each swarm size, higher pair attraction leads to monotonically increasing higher energy efficiency (Fig. 5E). The dependence on pair attraction emerges for similar reasons as the *v*_{C} and *W*_{T} dependence on attraction (Fig. 5, B and D); more power is available for transport as pair attraction assists the maintenance of cargo capture.

To understand how transport efficiency depends on swarm size, we first consider the high pair attraction limit where small swarms are most efficient. This can be understood based on the fact that the max velocity at high attractions is weakly dependent on swarm size (Eq. 5 and Fig. 5B), whereas swarm transport power at high attraction is nearly proportional to swarm size (fig. S9). These observations can be captured by an asymptotic analysis of the high pair attraction limit of Eq. 6 to give (see the Supplementary Materials)

In contrast to the high attraction limit, larger swarms are found to be most efficient as swarm pair attraction vanishes. This arises from capture being more efficient in large swarms in the absence of attraction; this allows a greater proportion of power to be directed toward transport functions, which ultimately also makes transport more efficient in large swarms in the absence of attraction. A continuous transition between low and high attraction limits leads to a uniquely efficient swarm size for each pair attraction level. In summary, (i) increasing pair attraction for a given swarm size increases efficiency, (ii) small swarms are most efficient at high attraction, (iii) large swarms are most efficient at low attraction, and (iv) intermediate swarms sizes are most efficient at intermediate attraction levels. Swarm design for cargo transport energy efficiency has a nontrivial solution.

Analysis of swarm functions and parameters illustrates an opportunity for quantitative optimization based on design constraints such as swarm size, energy budget, feasible pair attraction, minimum transport speeds, etc. Intermediate-sized swarms of *N* = 36 to 60 and ~5*kT* of pair attraction are perhaps most practical in terms of energy efficiency and avoiding conditions that cause swarm aggregation. Smaller swarms might more easily navigate in confined spaces to transport cargo through porous media, whereas larger swarms produce more stable caging of cargo via multiple coordination shells. Ultimately, we have demonstrated a scheme for capturing and transporting cargo based on individually addressable colloidal particles on an array and show how physics associated with the colloidal domain such as Brownian motion and particle interactions provide unique challenges and opportunities compared with macroscopic insects or robot swarms.

## CONCLUSIONS AND OUTLOOK

A multiagent stochastic control algorithm was developed to enable a swarm of individually addressable self-propelled colloidal particles to perform nontrivial machine-like tasks of capturing and transporting microscopic cargo. By sensing cargo and swarm particle positions, assignment and path planning strategies enable feedback control over swarms by simply specifying each particle’s propulsion speed (whereas direction is stochastic via Brownian rotation). The control policy enables swarm functions in a minimally complex model that cannot be reduced to a simpler heuristic, and without control, swarms randomly disperse. This scheme exploits unique interactions and stochastic dynamics in colloidal swarms to capture and transport microscopic cargo in a robust, stable, error-tolerant, and dynamic manner that is unconventional compared with macroscopic swarms.

Swarm size, interactions, entropy, and control together influence the ability of swarms to simultaneously capture and transport cargo. The control policy navigates independently addressable swarm particles to targets around cargo, which collectively results in a number of nontrivial emergent swarm behaviors including controlled condensation around cargo, generation of net caging and translational forces, and the ability to tune task efficacy, efficiency, and speed. Forces within swarms show feedback control generates a net inward radial force that balances swarm osmotic pressure, which determines average swarm radial microstructures. Microstructures are polarized as some swarm propulsion resources are redirected toward cargo transport via translational forces. The extent of swarm capabilities for cargo localization, transport speed, power management, and fuel efficiency displays nontrivial dependencies on swarm size and interactions. Careful design of swarm characteristics is shown to optimize functions subject to application constraints and enables robust stable control in the presence of substantial disturbances to sensors, actuators, and system state variables. The approach of cargo transport speeds and efficiency toward expected theoretical asymptotic limits suggests the central controller scheme developed in this work provides a benchmark for near-optimal performance.

Our scheme for controlling colloidal swarms provides a general framework to incorporate additional physics, different control objectives, and strategies to address more severe disturbances to control variables. The algorithm as reported could be implemented in a number of currently available experimental systems involving light- or field-mediated local transport mechanisms, but continued advances in particle synthesis, externally triggered transport mechanisms, and imaging could enable implementation of related approaches in other materials and applications. Depending on system details, our algorithms could be modified to include, for example, different hydrodynamic or propulsion models or different particle shapes that influence local cargo corralling. In addition, the central control scheme in this work can be adapted to include, for example, distributed control elements suitable for larger swarms with more sophisticated capabilities (e.g., memory and communication), filtering to deal with noisy sensor data, and data-driven approaches (e.g., reinforcement learning) to achieve simultaneous dynamic modeling and path planning. Increasingly ambitious control objectives for colloidal swarms could include navigating cargo through mazes (fig. S10), herding many cargo particles, assembling passive colloids, or performing as microscopic machines (fig. S10). Ultimately, the control and analysis framework for colloidal swarms provide a basis to develop robotic colloidal systems that could enable unique technologies in nano-, micro-, bio-, and environmental systems and applications.

## METHODS

### Brownian dynamic simulations

An equation of motion for the positions, **r**_{S}, and orientations, ϕ, of *N* self-propelled swarm particles and a single cargo particle is given by**D**_{t} and *D*_{r} are the translational and rotational diffusivities for spheres, *v*_{S,i} is propulsion speed (i.e., the light modulated control input), **n*** _{i}* = (cos(ϕ

*), sin(ϕ*

_{i}*)) is the particle orientation,*

_{i}*kT*is thermal energy, ∆

*t*is the time step,

*i*is the particle index, and superscript B indicates Brownian displacements (with zero mean and variances of 2

**D**

_{t}Δ

*t*and 2

*D*

_{r}Δ

*t*). Forces due to particle pair interactions,

**F**, are computed from the gradients of scalar potentials as

*43*)

*B*contains material property constants, κ

^{−1}is Debye length,

*a*is particle radius,

*r*= ||

_{ij}**r**

_{S,i}−

**r**

_{S,j}|| is particle pair separation,

*V*

_{ex}is excluded volume between spheres (

*43*), given by

Simulations were performed using an integration time step of 0.1 ms and using other parameters listed in Table 1. The maximum propulsion speed, *v*_{max}, is based on typical experiments (*3*, *4*). The control update time of 0.1 s is based on prior feedback control experiments (*22*, *24*), although it is validated in practice as well.

### Multiagent assignment and control policy

An assignment function, *g*(·|*t*), was used to represent the target assignment decision at time, *t*. For example, let *I*_{S} = [1, 2, 3] be set of three swarm particles, and an assignment function *g*(· | *t* = 0) at time 0, denoted by *g*_{0}, be set as *g*_{0}(1) = 2, *g*_{0}(2) = 3, and *g*_{0}(3) = 1 to represent assigning swarm particles 1, 2, and 3 to targets 2, 3, and 1, respectively. Mathematically, the function *g*(·|*t*) is a permutation from the set of swarm particles, *I*_{S}, to the set of targets, *I*_{S} (targets have the same index set). After obtaining the assignment, *g**, using Eq. 1 as described in the Results and Discussion, the self-propulsion speed for each swarm particle, *v*_{i}*, can be formulated using Eq. 2. Solving Eq. 2 is difficult because the probability of future swarm and target positions **r**_{S,i}(*t* + Δ*t*_{C}) and **r**_{T,i}(*t* + Δ*t*_{C}) are complicated because of particle interactions and stochastic dynamics. By assuming on short time scales that swarm particle positions are primarily determined by propulsion and not influenced by interactions with nearby particles, the optimization of Eq. 2 can be decomposed into the following optimization problem for each swarm particle, *g**(*i|t*) ∈ *I*_{S}, as*i*, **r**_{S}(*t* + Δ*t*_{C}) under velocity, *v*_{S,i}**n*** _{i}*, is given by integration of Eq. 8 from

*t*to

*t*+ Δ

*t*

_{C}as

*t*

_{C}< < 1/

*D*

_{r}, the probability densities of

**r**

_{S,i}(

*t*+ Δ

*t*

_{C}) and

**r**

_{T,g*(i|t)}(

*t*+ Δ

*t*

_{C}) are concentrated around their mean values, and an approximate solution to Eq. 12 is given as

**r**

_{S,i}(

*t*+ Δ

*t*

_{C})〉 ≈

**r**

_{S,i}(

*t*) + v

_{S,i}

**n**

*Δ*

_{i}*t*

_{C}for translation with stochastic orientations (

*49*) and 〈

**r**

_{T,g*(i|t)}(

*t*+ Δ

*t*

_{C})〉 =

**r**

_{T,g*(i|t)}(

*t*) since the target is undergoing driftless Brownian translation. With the mean position formulated, the solution to Eq. 14 is given by Eq. 3.

### Swarm cargo capture algorithm

The cargo capture algorithm (Algorithm 1) was repeated after every control update time interval, Δ*t*_{C}. Initially, the target sites, **r**_{T}, were constructed on the basis of the current cargo position. Target sites were positioned as hexagonal lattices with a minimum spacing corresponding to a minimum pair potential energy. Target assignments and self-propulsion speeds were obtained via Eqs. 1 and 3 based on the current system state (positions and orientations of swarm particles and target sites). The system state was updated after a control update time of Δ*t*_{C} = 0.1 s.

**Algorithm 1.**

Cargo capture algorithm

1 Loop with frequency Δ*t*_{C}:

2 Reconstruct the target sites around the cargo.

3 Calculate target assignments and self-propulsion speeds for swarm particles from Eqs. 1 to 3.

4 Update the position of swarm particles and cargo within Δ*t*_{C}.

5 End Loop

### Swarm cargo transport algorithm

The cargo transport algorithm (Algorithm 2) was repeated after every control update time interval, Δ*t*_{C}. Initially, the target sites, **r**_{T}, were constructed on the basis of the current cargo position. Target sites were positioned as hexagonal lattices with a minimum spacing corresponding to a minimum pair potential energy. A subset of swarm particles was selected by satisfying the following: (i) an orientation condition (|ϕ* _{i}* − θ | < ϕ

_{c}, i.e., swarm particle

*i*is oriented within a tolerance of the transport direction), (ii) a position condition (||

**r**

_{S,i}−

**r**

_{T,g*(i)}|| <

*a*, i.e., swarm particle

*i*is positioned within a tolerance of its assigned target), and (iii) a coordination number condition [coordination number (

*i*)

*>**n*

_{C}, i.e., swarm particle

*i*has sufficient neighbors within a tolerance to promote swarm structural integrity]. The selected subset of swarm particles was assigned maximum self-propulsion to generate swarm transport. Remaining swarm particles were assigned targets and speeds according to Algorithm 1. The system state was updated after a control update time of Δ

*t*

_{C}= 0.1 s. Parameters used in the cargo transport algorithm include ϕ

_{c}= 30°,

*n*

_{C}= 3 for

*N*= 90,

*n*

_{C}= 4 for

*N*= 60,

*n*

_{C}= 5 for

*N*= 36 and 18, and

*n*

_{C}= 2 for

*N*= 6. For pair attraction >8

*kT*for all swarm sizes,

*n*

_{C}= 0 to investigate the asymptotic speeds and efficiencies when all swarm particles are engaged in transporting.

**Algorithm 2.**

Cargo transport in θ direction algorithm

1 Loop with frequency Δ*t*_{C}:

2 Construct the target sites around the cargo

3 For each swarm particle *i*:

4 If |ϕ* _{i}* − θ | < ϕ

_{c}and ||

**r**

*−*

_{S,i}**r**

_{T,g*(i|t)}|| <

*a*and neighbor(

*i*)

*>**n*

_{C}

5 Swarm particle *i* is labeled as transporter.

6 Set the self-propulsion speed for swarm particle *i* to be maximum speed.

7 Else

**8** Swarm particle *i* is labeled as capturer.

9 Calculate self-propulsion speed for swarm particle *i* from Eqs. 1 to 3.

10 End For

11 Update the position of swarm particles and cargo within Δ*t*_{C}.

12 End Loop

## SUPPLEMENTARY MATERIALS

Supplementary material for this article is available at http://advances.sciencemag.org/cgi/content/full/6/4/eaay7679/DC1

Section S1. Nomenclature

Section S2. Supplemental Methods and Results

Fig. S1. Swarm particle pair potentials with different amounts of attraction.

Fig. S2. Examples illustrating the optimal assignment solution.

Fig. S3. Optimal control policy for single particles–single targets, multiple particles–multiple targets, and cargo capture.

Fig. S4. Osmotic pressure from sedimentation equilibrium and swarm configurations.

Fig. S5. Steady-state density and force distribution during the capture process for *N* = 60.

Fig. S6. Steady-state density and force distribution during the transport process for *N* = 60.

Fig. S7. MSD analysis for cargo and swarm particles during steady-state capture.

Fig. S8. MSD analysis for cargo and swarm particles during steady-state transport.

Fig. S9. Steady-state energy consumption rate for swarm particles during cargo transport.

Fig. S10. Example extensions of colloidal swarm functions.

Movie S1. High-quality three-dimensional rendering of cargo capture by *N* = 60 swarms for *U*_{M} = 5.3*kT*.

Movie S2. Cargo capture by *N* = 90 swarms for *U*_{M} = 0*kT*.

Movie S3. Cargo capture by *N* = 90 swarms for *U*_{M} = 3.2*kT*.

Movie S4. Cargo capture by *N* = 90 swarms for *U*_{M} = 5.3*kT*.

Movie S5. Cargo capture by *N* = 6 (0*kT*), 18 (5.3*kT*), 36 (5.3*kT*), and 60 (3.2*kT*) swarms.

Movie S6. Crystalline swarm melting without capture control for *N* = 18, 36, 60, and 90 and *U*_{M} = 5.3*kT*.

Movie S7. Cargo transport by *N* = 90 swarms for *U*_{M} = 0*kT*.

Movie S8. Cargo transport by *N* = 90 swarms for *U*_{M} = 3.2*kT*.

Movie S9. Cargo transport by *N* = 90 swarms for *U*_{M} = 5.3*kT*.

Movie S10. Cargo capture and transport by *N* = 36 (3.2*kT*), 36 (5.3*kT*), 60 (3.2*kT*), and 60 (5.3*kT*) swarms.

Movie S11. Cargo transport without capture control for *N* = 18 (8.7*kT*), 36 (5.3*kT*), 60 (5.3*kT*), and 90 (5.3*kT*) swarms.

This is an open-access article distributed under the terms of the Creative Commons Attribution-NonCommercial license, which permits use, distribution, and reproduction in any medium, so long as the resultant use is **not** for commercial advantage and provided the original work is properly cited.

## REFERENCES AND NOTES

**Acknowledgments:**

**Funding:**We acknowledge financial support from the NSF Cyber Enabled Discovery and Innovation grant (CMMI-1124648).

**Author contributions:**Y.Y. performed and analyzed the computer experiments. Y.Y. and M.A.B. designed the computer experiments and their analysis, interpreted the data, and wrote the manuscript.

**Competing interests:**The authors declare that they have no competing interests.

**Data and materials availability:**All data needed to evaluate the conclusions in the paper are present in the paper and/or the Supplementary Materials. Additional data related to this paper may be requested from the authors.

- Copyright © 2020 The Authors, some rights reserved; exclusive licensee American Association for the Advancement of Science. No claim to original U.S. Government Works. Distributed under a Creative Commons Attribution NonCommercial License 4.0 (CC BY-NC).