Reproductive success increases with degree of kinship in cooperative coalitions of female red howler monkeys (Alouatta seniculus)

Evaluation of evolutionary mechanisms proposed to promote cooperative behavior depends on the relative influence of the behavior on the reproductive success of individuals, the reproductive success of the group in which they interact behaviorally, and the degree of gene correlation among cooperators. The genetic relationship within cooperative coalitions of female red howler monkeys was examined for three populations with different densities and growth rates. Patterns of gene correlation change within coalitions is documented using data from the mitochondrial and nuclear genomes, and long-term census monitoring. Differences in fecundity and infant survivorship within and between groups of unrelated (rˉ=0) and related (rˉ≥ 0.25) females are compared. Females that emigrate from their natal groups form coalitions with other migrant females. These coalitions attempt to establish a territory and, once successful in producing offspring, exclude other females from feeding resources. Females in these coalitions had different mtDNA haplotypes and a genetically estimated mean r of 0, supporting demographic data on emigration patterns indicating that these females rarely have the opportunity to form coalitions with kin. Patterns of recruitment and rate of matriline development within social groups supported behavioral data indicating that females actively attempt to promote their own matriline as breeders over that of other females, and that some matrilines are more successful at this than others. Mean r among females was significantly higher in coalitions established as social groups for several generations (rˉ=0.44). In these groups, females all shared the same mtDNA haplotype, and mtDNA haplotype divergence was significantly higher between than within groups. Females in coalitions with kin had significantly higher reproductive success than females in unrelated coalitions in all populations. This difference was not a function of coalition size, number of males, socionomic sex ratio, or primiparity, although anecdotal evidence suggests that allomothering may compensate for inept new mothers in related coalitions more often than in unrelated ones. Differences in territory quality could not be ruled out as a potential causal factor in the saturated populations, but were unlikely in the low-density, growing population. There were substantial differences among long-established coalitions in overall reproductive output in all three populations, and this was significantly correlated with the number of breeding females. Increase in coalition size was a function of both group age and the behavioral tolerance among females. Regardless of the underlying reasons for the patterns observed, reproductive success clearly increases with degree of gene correlation among females within cooperative coalitions, and coalitions that recruit more daughters produce more offspring. The nature of the cooperative relationship among group females directly influences both of these outcomes. This is associated with substantial genetic differentiation among social groups within populations, creating conditions in which genetic tendencies towards cooperative behavior can become tightly associated with group reproductive success. Received: 15 September 1999 / Revised: 27 April 2000 / Accepted: 27 May 2000