Genetic divergence and a hybrid zone between Baltic and North Sea Mytilus populations (Mytilidae: Mollusca)

Suppose that selection acts at one or more loci to maintain genetic differences between hybridising populations. Then, the flow of alleles at a neutral marker locus which is linked to these selected loci will be impeded. We define and calculate measures of the barrier to gene flow between two distinct demes, and across a continuous habitat. In both cases, we find that in order for gene flow to be significantly reduced over much of the genome, hybrids must be substantially less fit, and the number of genes involved in building the barrier must be so large that the majority of other genes become closely linked to some locus which is under selection. This conclusion is not greatly affected by the pattern of epistasis, or the position of the marker locus along the chromosome.

It is show that Wright's F-statistics can be defined as ratios of gene diversities of heterozygosities rather than as the correlations of uniting gametes. This definition is applicable irrespective of the number of alleles involved or whether there is selection or not. The relationship between F-statistics and Nei's gene diversity analysis is discussed.

Existing theory for inferences about linkage disequilibrium is restricted to a measure defined on gametic frequencies. Unless gametic frequencies are directly observable, they are inferred from genotypic frequencies under the assumption of random union of gametes. Primary emphasis in this paper is given to genotypic data, and disequilibrium coefficients are defined for all subsets of two or more of the four genes, two at each of two loci, carried by an individual. Linkage disequilibrium coefficients are defined for genes within and between gametes, and methods of estimating and testing these coefficients are given for gametic data. For genotypic data, when coupling and repulsion double heterozygotes cannot be distinguished. Burrows' composite measure of linkage disequilibrium is discussed. In particular, the estimate for this measure and hypothesis tests based on it are compared to the usual maximum likelihood estimate of gametic linkage disequilibrium, and corresponding likelihood ratio or contingency chi-square tests. General use of the composite measure, whether or not random union of gametes is an appropriate assumption, is recommended. Attention is given to small samples, where the non-normality of gene frequencies will have greatest effect on methods of inference based on normal theory. Even tools such as Fisher's z-transformation for the correlation of gene frequencies are found to perform quite satisfactorily.