Smart Swarm: Using Animal Behaviour to Organise Our World

To some people, this may come as a surprise. How could ants, bees, or termites know something we don’t? How could such insignificant creatures solve complicated problems better than an $11-billion-a-year airline like Southwest? If ants are so smart, why aren’t they flying the 737s? The fact is, these and other creatures have been dealing with the most difficult kinds of problems for millions of years: Will there be enough food for the colony this week? Where will it be found? How many workers will the hive need to build a nest? How will the weather affect the herd’s migration this year? The way they’ve responded to these challenges has been to evolve a special form of group behavior that is flexible, adaptive, and reliable.

Translated into mathematical formulas, the principles of a smart swarm have given businesses powerful tools to untangle some of the knottiest problems they face. Manufacturing companies have experimented with them to optimize production, for example. Telephone companies have tested them to speed up calls. Aircraft mechanics and engineers have applied them to identify problems in new airplanes. And intelligence agents have used them to keep track of a dangerous world.

How does a smart swarm work? We’ll find out in the first three chapters by following biologists into the field to unlock the secrets of collective behavior. As these researchers have discovered, social insects such as ants, bees, and termites distribute problem solving among many individuals, each of which is following simple instructions but none of which sees the big picture. Nobody’s in charge. Nobody’s telling anybody else what to do. Instead, individuals in such groups interact with one another in countless ways until a pattern emerges—a tipping point of motion or meaning—that enables a colony of ants to find the nearest pile of seeds, or a school of herring to dodge a hungry seal.

In the fourth chapter we’ll look at the subtle role that individuals play in keeping a group on course. For groups such as flocks of birds, schools of fish, or herds of caribou, which are made up of individuals largely unrelated to one another, the key to survival demands a set of skills that balances group behavior with self-interest. As humans, we share many common problems with such groups, since we’re often torn by the same impulses—to cooperate but also to profit, to do what’s right for the community but also to look out for ourselves and our families.

Not every swarm is smart, of course. Group behavior also has a dark side. In the fifth chapter we’ll find out what scientists have discovered about locusts to explain why peaceful groups of grasshoppers suddenly explode into voracious plagues. To learn how human instincts can go haywire, we’ll also follow the work of researchers who have studied fatal crowd disasters among religious pilgrims in Saudi Arabia—and what’s been done to prevent such accidents in the future. What separates a smart swarm from its stupid cousin? Why does a happy crowd suddenly turn into a rampaging mob? The reason, simply put, is that a smart swarm uses its collective power to sort through countless possible solutions while the mob unleashes its chaotic energy against itself. And that makes it so important to understand how a smart swarm works—and how to harness its power.

As everyday life grows more complicated, we increasingly find ourselves facing the same problems of uncertainty, complexity, and change, drowning in too much information, bombarded with too much instant feedback, facing too many interconnected decisions. Whether we realize it or not, we too are caught up in worlds of collective phenomena that make it more difficult than ever to guide our companies, communities, and families with confidence. These challenges are already upon us, so we need to be prepared. The best way to do that, as you’ll see in the pages ahead, is to turn to the experts—not the ones on cable TV but those in the grass, in the air, in the lakes, and in the woods.

1 ANTS Who’s in Charge Here? (#ulink_dd3fa605-fda6-57af-a9c6-befbe0602bdc)

Just off Route 533 in southwestern New Mexico, a barbed-wire fence surrounds sixty acres of what used to be a sprawling cattle ranch at the foot of the Chiricahua Mountains. Some years ago, at the request of biologist Deborah Gordon, Stanford University bought the property to keep out of the hands of developers a small research site she’d established. But the subdivisions and convenience stores never materialized. In fact, not much at all has happened on this little patch of the Sonoran Desert to disturb the current residents of the site, including several hundred colonies of red harvester ants (Pogonomyrmex barbatus). For more than two decades now, Gordon has documented the life histories of these colonies, where, day in and day out, season after season, ants go about their business with a curious mix of efficiency and utter chaos.

The workday starts early at Colony 550, an older nest of some ten thousand ants near the eastern border of the site. From dawn to midmorning, one group after another emerges from the nest to carry out various tasks. The first on the job are patrollers, who poke their heads out of the entrance hole just before sunrise. Appearing to be in no hurry, they mill around on the circular nest mound, inspecting the pebbly surface like groundskeepers at a golf course assessing the health of a green. If something has happened during the night, patrollers will be the first ants to know. Has the rain left a pile of debris on a foraging trail? Has the wind redistributed the seeds the ants collect for food? What are the neighbors up to this morning? As they wander farther and farther from the nest entrance, patrollers may bump into scouts from nearby colonies doing exactly the same thing, and, if they do, forager ants from both sides might later fight. “Last week, for some reason, we noticed quite a few foragers walking around with the heads of other ants attached to their bodies,” says Mike Greene, a biologist from the University of Colorado–Denver who was doing research at the site. “They’d clearly been having little ant wars.”

The patrollers are soon joined by a crew of nest maintenance workers, each carrying a bit of dirt, seed husk, or other trash up from below ground. In contrast to the patrollers, they seem narrowly focused on their tasks, searching for a suitable place to deposit their loads. The moment they find one, they drop what they’re carrying, turn around, and head back down into the nest.

Next come a handful of midden workers, who tidy up what the maintenance workers have left behind. Not that they do this in any sensible way. If you watch one working for a while, Greene says, you’ll probably find it puzzling. “Midden workers remind me of my fifteen-month-old daughter. They take an object from Point A and drop it at Point B. Then they pick something else up and go to Point C. It all seems very random.” A time-lapse movie of the morning’s activity, though, would show a pile of dirt and ant trash steadily growing along one edge of the nest mound. “So it turns out they’re organized, after all,” he says.

The last to appear are the foragers, who greatly outnumber the other workers. Streaming out of the entrance hole, they charge directly for the tall grass that rings the nest mound and disappear into a sea of Mormon tea, acacia, and snakeweed. Following ant highways through the underbrush, the foragers may venture as far as sixty feet from the nest in search of seeds. Because these seeds, for the most part, have ridden the winds from other parts of the desert, rather than coming from plants on the site, they tend to be scattered in unpredictable ways. So it could take a forager as long as twenty minutes to find one. As soon as it does, it picks up the seed and carries it straight back to the nest.

By nine a.m., the nest hole has taken on the appearance of a frantic subway entrance, with ants rushing in and out. In a colony like 550, which is nearly twenty years old, the nest may be six feet deep. Down below, in an elaborate network of tunnels and chambers, as Gordon describes in her book Ants at Work, other groups of ants are busily stacking seeds in storage chambers, according to size and shape; removing dead ants, grasshopper legs, and other unwanted objects from the nest; tending brood; caring for the queen; or simply standing ready in reserve.

From top to bottom, Colony 550 seems to be a model of efficiency, with each group performing its task in an orderly sequence—an impression strengthened by each ant’s habit of constantly touching its antennae with every other ant it meets, as if to make sure that everybody’s on the same page. From patrollers and maintenance workers to midden workers and foragers, every member of the colony seems to be following a master plan, like tiny cogs in a machine or the employees of a successful factory.

But that’s not what’s happening here at all.

Despite its well-managed appearance, Colony 550 does not function like any organization you are ever likely to encounter. It has no bosses, managers, or supervisors of any kind. The queen, despite her lofty title, wields no authority. Her sole function is to lay eggs, not to give commands. When patrollers venture out into the grass, they’re not taking orders from a squad leader. When nest maintenance workers repair a tunnel, they’re not following any blueprints. Young ants entering the work force don’t have to sit through an orientation meeting or memorize a mission statement, because they never need to see the big picture. No ant ever understands the purpose of its own labor, why it needs to complete the job, or how it fits in.

Yet the colony does just fine. Consider the way it responds quickly and effectively to changes in its environment. If patrollers this morning discover a tasty pile of seeds, additional ants will head out to look for more within minutes, and these additional ants will become foragers. Did last night’s storm damage the nest? More maintenance workers will show up to repair it, even if that requires younger nurse ants to pitch in. Depending on the challenge or the opportunity, the colony as a whole calculates quickly and precisely how many workers are needed to take care of a job, then adjusts its resources accordingly.

This flexible system, evolved during 140 million years of ant history, is one of the main reasons that the world’s fourteen thousand or so known species of ants have flourished in a bewildering variety of ecosystems, from tropical rain forests to city sidewalks. Their way of doing things may look messy, but it enables them to accomplish amazing feats, such as organize highways, build elaborate nests, and stage epic raids—all without any leadership, game plan, or the least sense of mission.

How do they do it?

Ants Aren’t Smart

Every morning in August, Deborah Gordon sets out from the Southwestern Research Station near Portal, Arizona, and drives just across the border into New Mexico to observe red harvester ants. Every afternoon, once the ants have retreated underground to escape from the blazing heat, the biologist returns to the station with a renewed sense of wonder—not that the ants are so skillful at what they do, but that they appear to be such little dummies.

“If you watch an ant try to do something, you’ll be impressed by how inept it is,” she says. “Often, it doesn’t go about things the way you think would be best, it doesn’t remember anything for very long, and it doesn’t seem to care if it succeeds.” Only one in five ants actually accomplishes what it sets out to do. “The longer you watch an ant the more you end up wanting to help it.”

Gordon doesn’t study ants as individuals, though. Her research focuses on the behavior of ant colonies. As colonies, she says, ants are capable of solving problems far beyond the abilities of individuals, such as how to find food, allocate resources, or respond to competition from neighbors.

“Ants aren’t smart,” she clarifies. “Ant colonies are.”

The central focus of Gordon’s research has been the ants’ system of task allocation, which is how a colony decides which jobs need to be done on any particular day. Given all the uncertainties that red harvesters face—from the iffy availability of food to competition from neighbors—a colony must calculate as a group how many workers to send out foraging, how many to keep on patrol, how many to hold back to tend brood, and so on.

“One of my favorite moments in the movie Antz is a scene I call the Bureau of Task Allocation,” she says of the 1998 DreamWorks animated film. “The ants are brought to some bureaucrats—they’ve got clipboards—behind a counter, and each ant is just stamped, and given its task. This, of course, is the way we organize our work, where certain individuals have the job of assigning work to other individuals. So it’s easy for us to imagine that there’s somebody in there with a clipboard, telling somebody else what to do.” But that’s not how the ants do it.

To understand the real process of task allocation, Gordon and fellow biologist Mike Greene conducted a series of experiments a few years ago with foragers. They knew that a colony, depending on circumstances, doesn’t forage every day. It might be too cold or windy to go outside, or there might be a hungry lizard waiting at the edge of the nest mound. Patrollers seem to be the key to this decision. As they return from their early-morning scouts of the neighborhood, they’re greeted near the nest entrance by a crowd of foragers. The foragers touch antennae with the patrollers, and if they bump into the right number of patrollers, the foragers are more inclined to go out. The behavior of the patrollers, in other words, informs the decisions of the foragers.

It doesn’t happen in the way you might expect, though. “The patrollers aren’t passing along anything elaborate,” Gordon says. “They’re not coming back and giving instructions to the foragers, saying go here and do this. The message is merely in the contact. And that’s what’s hardest for us to understand, because we keep falling into the temptation to think that they’re doing it the way that we would.”

To get to the bottom of this group-oriented behavior, she and Greene conducted an experiment using fake patrollers. First they captured real patrollers leaving several colonies one morning. Then, after waiting thirty minutes, they dropped tiny glass beads coated with the smell of patrollers into each nest entrance. Red harvesters, like most ants, are covered with a layer of grease that keeps them from drying out. This grease, made of hydrocarbons, carries an odor specific not only to their colony but also to their task group. “For the ants, you might say, chemicals are what vision is for us,” Greene says. When foragers inside the nest encountered the glass beads coated with patroller hydrocarbons, they took them for real patrollers.

What Gordon and Greene wanted to know was whether the rate at which foragers encountered patrollers made any difference. If it did, that might represent an important mechanism in the colony’s decision-making process. So they varied the speed at which they dropped patroller beads into each nest. In the first of four trials, they added one bead every three minutes. In the second, one bead every forty-five seconds. In the third, one bead every ten seconds. In the last, one bead every second. The results were dramatic.

In the first two trials, the relatively slow rates prompted few foragers to go out. The same was true of the fourth trial with the fastest rate. But in the third trial, when foragers encountered glass beads at just the right rate—one bead every ten seconds—they left the nest in a big rush with four times as many foragers.

“The rate needs to be about ten seconds because that must be how long an ant can remember what happened to it,” Gordon says. “If an ant has to wait forty-five seconds to meet another ant, it forgets the previous one. It’s as if the encounter never happened.” Red harvesters, it seems, have a very short attention span. If the rate is too fast, meanwhile, that may mean that something has driven foragers back to the nest, such as a predator. The rate has to be just right.

A forager’s decision, that is, doesn’t depend on it receiving instructions from a patroller or figuring out on its own what’s needed. It depends instead on the ants following a simple rule of thumb: If it meets the right number of patrollers returning at the right rate, it goes out looking for seeds. If it doesn’t, it stays put. “Nobody’s deciding whether it’s a good day or not to forage,” Gordon says. “The collective is, but no particular ant is.”

Once the first foragers leave the nest, a separate mechanism kicks in to regulate the total number of foragers that go out that day. The key encounters this time take place between foragers only. As successful foragers return to the nest with seeds, they’re met at the nest entrance by foragers waiting in reserve. This contact stimulates the inactive ants to go out. Foragers normally don’t come back until they find something. So the faster the foragers return, the faster other ants go out, enabling the colony to tune its work force to the probability of finding food.

This simple rule, applied by one forager after another in the crowded space near the entrance hole, functions like a simple calculator for the colony. The sum of all the decisions by all the ants gives the colony the answer to the question “How many foragers do we need searching for food today?”

The ants aren’t smart. The colony is.

THIS INTRIGUING BEHAVIOR, of course, isn’t unique to ants. Many groups of animals, from honeybees to herring, tackle difficult problems without direction from leaders. They do it through a phenomenon that scientists call self-organization—the first principle of a smart swarm. Although examples of self-organization appear all around us in nature, scientists have studied it intensively only during the past few decades. First described by chemists and physicists, the term originally referred to the spontaneous appearance of patterns in physical systems, such as the rippling of sand dunes or the hypnotic spirals that form when certain chemical reactants are combined. Later it was adopted by biologists to explain the intricate structure of wasp nests, the synchronized flashing of some species of fireflies, and the way that swarms of bees, flocks of birds, and schools of fish instinctively coordinate their actions.

What these phenomena all have in common is that none of them is imposed from the top by a master plan. The patterns, shapes, and behaviors we see in such systems don’t come from preexisting blueprints or designs, but emerge on their own, from the bottom up, as a result of interactions among their many parts. We call an ant colony self-organizing because nobody’s in charge, nobody knows what needs to be done, and nobody tells anybody else what to do. Each ant goes through its day responding to whatever happens to it, to the other ants it bumps into, and to changes in the environment—what scientists call “local” knowledge. When an ant does something, it affects other ants, and what they do affects still others, and that impact ripples through the colony. “No ant understands its own decisions,” Gordon says. “But each ant’s decision is linked to another ant’s decision and the whole colony changes.”

Although the ultimate origins of self-organization remain something of a mystery, researchers have identified three basic mechanisms by which it works: decentralized control, distributed problem-solving, and multiple interactions. Taken together, these mechanisms explain how the members of a group, without being told to, can transform simple rules of thumb into meaningful patterns of collective behavior.

To get a feel for how these mechanisms work, consider a day at the beach with your family or friends. When you first arrive, you don’t stand around waiting for someone to give you instructions. Apart from certain restrictions imposed by the community (no nudity, no pets, no alcohol, for example) you’re on your own. Nobody tells you where to sit, what to do, whether to go into the water or not (unless the lifeguard gets bossy). Everybody can do pretty much what they want to, which is one way of describing decentralized control.

If it’s a beautiful day and the beach is crowded, of course, it might take some time to find the perfect place to sit down. You don’t want to choose a spot too close to the water, or your beach chairs and blanket could get soaked by a big wave. Nor do you want to sit far away from the water, where you can’t feel the ocean breeze. If you plan to go swimming, it might be convenient to choose a location near the lifeguard, as every family with little children has already figured out (which is why all those umbrellas are clustered around the guard’s stand). In the end, you choose a space with just enough room to spread your blanket yet maintain the proper distance in all directions from your neighbors’ blankets, which is the unspoken rule of thumb at the beach. If you could look down from a helicopter, you’d see a mosaic of blankets evenly spaced from one another, reflecting the success of the crowd’s distributed problem-solving.

Then something curious happens. Just as you’re settling into your beach chair with Stephen King’s latest novel, you notice that a few people have stood up to look at the water. Then a few more do the same thing. And a few more. Suddenly it seems like everybody’s standing and looking at the water, so you do too. You don’t have any idea why, but you’re suddenly alert, full of questions. What’s going on? Is somebody drowning? Is there a shark? What’s everybody looking at? What began, perhaps, as a simple act of curiosity by a few individuals—staring at the water—spreads from person to person down the beach, snowballing into a collective state of alarm. That’s how infectious multiple interactions can be. And the impressive thing is, if there had been a shark, everybody would have found out about it almost as quickly as if someone had shouted “Jaws” with a bullhorn.

“If we each respond to little pieces of information, and we follow certain rules, the whole crowd will organize in a certain way,” Mike Greene says, “just like when we’re looking down on an ant colony, we can actually see its behavior change, even though none of the ants is aware of it.”

Day in and day out, that is, self-organization provides an ant colony like 550 with a reliable way to manage an unpredictable environment. Wouldn’t it be useful if we could do the same thing?

The Traveling Salesman Problem

One afternoon in the summer of 1990, an Italian graduate student named Marco Dorigo was attending a workshop at the German National Research Center for Computer Science near Bonn. At the time, Dorigo was working on a doctoral thesis in Milan about ways to solve difficult computational problems. The talk he’d come to hear was by Jean-Louis Deneubourg, a professor from the Free University of Brussels, about his research with ants. “I was already interested in ways that natural systems could be used as inspiration for information science,” Dorigo says. “But this was the first time anybody had made a connection between ant behavior and computer science.”

In his presentation, Deneubourg described a series of experiments that he and his colleagues had done with common black ants known as Argentine ants (Iridomyrmex humilis). Like many ants, this species leaves a trail of chemical secretions when foraging. Such chemicals, called pheromones, come from glands near the tip of the ant’s abdomen, and they act as powerful signals, telling other ants to follow their trails. Foragers normally lay down such trails after they have found a promising source of food. As they return to the nest, they mark their paths so that other ants can retrace them to the food. But Argentine ants are different. They lay down pheromone trails during the search phase as well. That appealed to Deneubourg, who was curious about how foragers decided where to explore.

In one experiment in his lab, Deneubourg and his colleagues placed a bridge between a large tub containing a colony of Argentine ants and another tub containing food. The bridge had a special design. About a fourth of the way across, it split into two branches, both of which led to the food, but one of which was twice as long as the other. How would the little explorers deal with this?

As you might expect, the ants quickly determined which branch was best (this is the same species, after all, that demonstrates such a knack for locating maple syrup spilled on your kitchen floor). In most trials of the experiment, after an initial period of wandering, all of the ants chose the shorter branch.

The pheromone trail was the key. As more and more ants picked the shorter branch, it accumulated more and more of their pheromone, increasing the likelihood that other ants would choose it. Here’s how it works: Let’s say two ants set out across the bridge at the same time. The first ant takes the shorter branch, and the second the longer one. By the time the first ant reaches the food, the second is only halfway across the bridge. By the time the first ant returns all the way to the colony, the second ant has just arrived at the food. To a third ant standing at the split in the bridge at this point, the pheromone trail left by the first ant would be twice as strong as that left by the second (since the first ant went out and returned), making it more likely to take the shorter branch. The more this happens, the stronger the pheromone trail grows, and the more ants follow it.