Outdated Archival Version

These pages are not updated anymore. They reflect the state of 20 August 2005. For the current production of this journal, please refer to http://www.math.psu.edu/era/.

%_ **************************************************************************
%_ * The TeX source for AMS journal articles is the publishers TeX code     *
%_ * which may contain special commands defined for the AMS production      *
%_ * environment.  Therefore, it may not be possible to process these files *
%_ * through TeX without errors.  To display a typeset version of a journal *
%_ * article easily, we suggest that you retrieve the article in DVI,       *
%_ * PostScript, or PDF format.                                             *
%_ **************************************************************************
% Author Package file for use with AMS-LaTeX 1.2
\controldates{4-SEP-2002,4-SEP-2002,4-SEP-2002,4-SEP-2002}
 
\RequirePackage[warning,log]{snapshot}
\documentclass{era-l}
\issueinfo{8}{03}{}{2002}
\dateposted{September 6, 2002}
\pagespan{20}{28}
\PII{S 1079-6762(02)00102-6}

\copyrightinfo{2002}{American Mathematical Society}

\newtheorem{theorem}{Theorem}
\newtheorem{acknowledgement}[theorem]{Acknowledgement}
\newtheorem{algorithm}[theorem]{Algorithm}
\newtheorem{axiom}[theorem]{Axiom}
\newtheorem{corollary}[theorem]{Corollary}
\newtheorem{criterion}[theorem]{Criterion}
\newtheorem{conjecture}[theorem]{Conjecture}
\newtheorem{lemma}[theorem]{Lemma}
\newtheorem{proposition}[theorem]{Proposition}


\theoremstyle{remark}
\newtheorem{case}[theorem]{Case}
\newtheorem{claim}[theorem]{Claim}
\newtheorem{conclusion}[theorem]{Conclusion}
\newtheorem{notation}[theorem]{Notation}
\newtheorem{remark}[theorem]{Remark}

\theoremstyle{definition}
\newtheorem{condition}[theorem]{Condition}
\newtheorem{definition}[theorem]{Definition}
\newtheorem{example}[theorem]{Example}
\newtheorem{exercise}[theorem]{Exercise}
\newtheorem{problem}[theorem]{Problem}
\newtheorem{solution}[theorem]{Solution}
\newtheorem{summary}[theorem]{Summary}


\begin{document}

\title{Regular neighbourhoods and canonical decompositions for groups}

\author{Peter Scott}
\address{Mathematics Department, University of Michigan, Ann Arbor, MI
48109, USA}
\email{pscott@umich.edu}
\thanks{First author partially supported by NSF grants DMS 034681 and 9626537}

\author{Gadde A. Swarup}
\address{Department of Mathematics and Statistics, University
of Melbourne, Victoria 3010, Australia}
\email{gadde@ms.unimelb.edu.au}

\date{May 1, 2002, and, in revised form, July 23, 2002}

\subjclass[2000]{Primary 20E34; Secondary 57N10, 57M07}

\keywords{Graph of groups, almost invariant set, characteristic
submanifold}

\commby{Walter Neumann}

\begin{abstract}
We find canonical decompositions for finitely presented groups which
essentially specialise to the classical JSJ-decomposition when restricted to
the fundamental groups of Haken manifolds. The decompositions that we obtain
are invariant under automorphisms of the group. A crucial new ingredient is
the concept of a regular neighbourhood of a family of almost invariant subsets
of a group. An almost invariant set is an analogue of an immersion.
\end{abstract}

\maketitle

This article is an announcement of work whose details will appear
later. In this work, we study analogues for groups of the classical
JSJ-decomposition (see Jaco and Shalen \cite{JS}, Johannson \cite{JO} and
Waldhausen \cite{Waldhausen}) for orientable Haken $3$-manifolds. The
orientability restriction is not essential but it will simplify our
discussions. Previous work in this field has been carried out by Kropholler
\cite{K}, Sela \cite{S1}, Rips and Sela \cite{RS}, Bowditch \cite{B1}, 
\cite{B2}, Dunwoody and Sageev \cite{D-Sageev}, and Fujiwara and Papasoglu
\cite{FP}, but none of these results yields the classical JSJ-decomposition
when restricted to the fundamental group of an orientable Haken manifold. In
our work, we give a new approach to this subject, and we give decompositions
for finitely presented groups which essentially specialise to the classical
JSJ-decomposition when restricted to the fundamental groups of Haken
manifolds. An important feature of our approach is that the decompositions we
obtain are unique and are invariant under automorphisms of the group. In
previous work such strong uniqueness results were only found for
decompositions of word hyperbolic groups. Most of the results of the
previous authors for virtually polycyclic groups can be deduced from our work.
But our arguments use some of the results of these authors, particularly those
of Bowditch. In addition, we use the important work of Dunwoody and Roller in
\cite{D-Roller}. Our work also yields some extensions of the results on the
Algebraic Annulus and Torus Theorems in \cite{SS2}, \cite{B1}, and
\cite{D-Swenson}. It should be remarked that even though we obtain canonical
decompositions for all finitely presented groups, these decompositions are
often trivial. This is analogous to the fact that any finitely generated group
possesses a free product decomposition, but this decomposition is trivial
whenever the given group is freely indecomposable.

Our ideas are based on an algebraic generalisation of the Enclosing Property
of the classical JSJ-decomposition. This property can be described briefly as
follows. For an orientable Haken $3$-manifold $M$, Jaco and Shalen \cite{JS}
and Johannson \cite{JO} proved that there is a family $\mathcal{T}$ of
disjoint essential annuli and tori embedded in $M$, unique up to isotopy, and
with the following properties. The manifolds obtained by cutting $M$ along
$\mathcal{T}$ are simple or are Seifert fibre spaces or $I$-bundles over
surfaces. The Seifert and $I$-bundle pieces of $M$ are said to be
\textit{characteristic}, and any essential map of the annulus or torus into
$M$ can be properly homotoped to lie in a characteristic piece. This is called
the Enclosing Property. The characteristic submanifold $V(M)$ of $M$ consists
essentially of the union of the characteristic pieces of the manifold obtained
from $M$ by cutting along $\mathcal{T}$. The fundamental group $G$ of $M$ is
the fundamental group of a graph $\Gamma$ of groups, whose underlying graph is
dual to the frontier of $V(M)$. Thus the edge groups of $\Gamma$ are all
isomorphic to $\mathbb{Z}$ or $\mathbb{Z}\times\mathbb{Z}$, and the vertex
groups are the fundamental groups of simple manifolds or of Seifert fibre
spaces or of surfaces. The uniqueness up to isotopy of the splitting family
$\mathcal{T}$ implies that $\Gamma$ is unique. Further, the Enclosing Property
implies that any subgroup of $G$ which is represented by an essential annulus
or torus in $M$ is conjugate into a characteristic vertex group.

All the previous algebraic analogues of the topological JSJ-decomposition
consist of producing a graph of groups structure $\Gamma$ for a given group
$G$ with some ``characteristic'' vertices. The algebraic analogue of the
topological Enclosing Property which was used is the property that certain
``essential'' subgroups of $G$ must be conjugate into one of the
characteristic vertex groups of $\Gamma$, but the precise meaning of the word
``essential'' varies depending on the authors. For example, when trying to
describe all splittings of a group $G$ over infinite cyclic subgroups,
previous authors produced a decomposition with infinite cyclic edge groups,
such that if $G$ splits over an infinite cyclic subgroup $H$, then $H$ is
conjugate into a characteristic vertex group. These results are often referred
to vaguely but collectively as the JSJ-decomposition of a finitely presented
group. While they are commonly regarded as being an algebraic analogue of the
topological JSJ theory, none of them recovers the topological result when
applied to the fundamental group of an orientable Haken $3$-manifold. One
reason for this is the fact that the Enclosing Property of the characteristic
submanifold $V(M)$ is stronger than the algebraic analogue discussed above.

Our results also yield a graph of groups structure $\Gamma$ for a given group
$G$ with some ``characteristic'' vertices, but our algebraic generalisation of
the topological Enclosing Property corresponds more closely to the topological
situation. Also when trying to describe all splittings of a group $G$ over
infinite cyclic subgroups, the decomposition we produce may have edge groups
which are not infinite cyclic. This reflects the topological situation better.

Here is an introduction to our ideas. As mentioned before, the Enclosing
Property of the characteristic submanifold $V(M)$ of a Haken $3$-manifold $M$
means that $V(M)$ contains a representative of every homotopy class of an
essential annulus or torus in $M$. We will say that it \textit{encloses} every
essential annulus and torus in $M$. In our work, we introduce a natural
algebraic analogue of enclosing. If we restrict attention to embedded
surfaces, this analogue is simple to explain. First recall the graph of groups
structure $\Gamma$ for $G=\pi_{1}(M)$, whose underlying graph is dual to the
frontier $\operatorname{fr}(V(M))$ of $V(M)$. Let $F$ be an embedded essential
annulus or torus in $V(M)$, so that $F$ determines a splitting of $G$, and let
$\Gamma_{F}$ denote the graph of groups structure for $G$, whose underlying
graph is dual to $\operatorname{fr}(V(M))\cup F$. Thus $\Gamma_{F}$ has one
more edge than $\Gamma$, and this extra edge corresponds to $F$. Collapsing
this extra edge of $\Gamma_{F}$ yields $\Gamma$ again. Now let $G$ be any
group, let $\Gamma$ be a graph of groups structure for $G$, and let $\sigma$
be a splitting of $G$. We say that $\sigma$ is \textit{enclosed} by a vertex
$v$ of $\Gamma$, if there is a graph of groups structure $\Gamma_{\sigma}$ for
$G$, with an edge $e$ which determines the splitting $\sigma$, such that
collapsing the edge $e$ yields $\Gamma$, and $v$ is the image of $e$. We
emphasise that the condition that $\sigma$ is enclosed by the vertex $v$ is in
general stronger than the condition that the edge group of $\sigma$ is
conjugate into the vertex group of $v$. This is particularly clear if $\sigma$
is a free product decomposition of $G$, as then the edge group of $\sigma$
is trivial.

An important observation is that $V(M)$ is closely related to a regular
neighbourhood of some (finite) union of essential annuli and tori in $M$. In
some cases, one can choose a finite family of essential annuli and tori in
$V(M)$ so that $V(M)$ is a regular neighbourhood of their union. More usually,
$V(M)$ can be obtained from such a regular neighbourhood by adding solid tori
to compressible torus boundary components. In particular, except for a few
special cases, $V(M)$ is minimal, up to isotopy, among incompressible
submanifolds of $M$ which enclose every essential annulus and torus in $M$.
Thus it seems natural to think of $V(M)$ as a regular neighbourhood of all the
essential annuli and tori in $M$. The peripheral pieces of the characteristic
submanifold can be thought of as a regular neighbourhood of all the essential
annuli only. Our main results can be thought of as algebraic versions of these
statements. In order to explain our ideas further, we need to discuss the
algebraic analogues of immersed annuli and tori and the algebraic analogue of
a regular neighbourhood.

An analogue of a $\pi_{1}$-injective immersion in codimension $1$ has been
studied by group theorists for some time. If $H$ denotes the image in $G$ of
the fundamental group of the codimension-$1$ manifold, this analogue is a
subset of $G$ called a $H$-almost invariant set or an almost invariant set
over $H$. Any $\pi_{1}$-injective immersion in codimension $1$ has a
$H$-almost invariant set associated to it in a natural way. In particular,
this applies to any splitting of $G$. Further, there is a natural idea of what
it means for an almost invariant subset of a group $G$ to be enclosed by a
vertex of a graph of groups decomposition for $G$, which generalises the idea
of enclosing a splitting.

We briefly recall the basic properties of almost invariant sets. We say that
two sets are \textit{almost equal} if their symmetric difference is finite. A
subset $X$ of $G$ is $H$\textit{-almost invariant} if $HX=X$, and for every
$g\in G$, $H\backslash X$ and $(H\backslash X)g$ are almost equal. A
$H$-almost invariant subset $X$ of $G$ is \textit{nontrivial} if $H\backslash
X$ is infinite and has infinite complement in $H\backslash G$. A $H$-almost
invariant subset $X$ of $G$ is \textit{equivalent} to a $K$-almost invariant
subset $Y$ if the images of $X\cap Y^{\ast}$ and $X^{\ast}\cap Y$ in
$H\backslash G$ are both finite. This relation \emph{is} an equivalence
relation, and is very natural. For example, the $H$-almost invariant sets
associated to homotopic $\pi_{1}$-injective immersions in codimension $1$ are
equivalent. These ideas are closely related to the theory of ends of spaces
and groups. In particular, $G$ has a nontrivial $H$-almost invariant subset if
and only if the pair $(G,H)$ has more than one end. The appropriate notions of
intersection and disjointness for such sets were introduced by Scott in
\cite{Scott:Intersectionnumbers}, and were further developed in \cite{SS}. In
\cite{Scott:Intersectionnumbers}, Scott defined the intersection number of two
nontrivial almost invariant subsets of a group and showed it was symmetric.
Further his definition generalises the natural idea of intersection number of
curves on a surface, and the intersection number of surfaces in a $3$-manifold
introduced in \cite{FHS}. In \cite{SS}, the main results, Theorems 2.5 and
2.8, were algebraic analogues of the facts that curves on a surface with
intersection number zero can be homotoped to be disjoint, and that a curve
with self-intersection number zero can be homotoped to cover an embedding.
These results do strongly suggest that an almost invariant set is the
appropriate analogue of an immersion.

The key new idea of our work is an algebraic version of regular neighbourhood
theory. This part of our work applies to arbitrary finitely generated groups.
We define an algebraic regular neighbourhood of a family of nontrivial almost
invariant subsets of a group $G$. The idea of a regular neighbourhood of two
splittings was developed by Fujiwara and Papasoglu \cite{FP} in special cases
(it can be seen that their enclosing technique yields the same result as our
regular neighbourhood construction in these cases), but from our point of view
enclosing almost invariant sets is more basic. In our algebraic construction
of regular neighbourhoods, as well as several other techniques, we have
greatly benefited from the two papers of Bowditch \cite{B1} \cite{B2}.
Bowditch's use of pretrees showed us how to enclose almost invariant sets
under very general conditions. In the case of word hyperbolic groups, Bowditch
\cite{B1} was effectively the first to enclose such sets although he does not
use this terminology.

In order to explain the idea of an algebraic regular neighbourhood, we return
to the characteristic submanifold $V(M)$ of a $3$-manifold $M$ and the graph
of groups decomposition $\Gamma$ of $G=\pi_{1}(M)$, whose underlying graph is
dual to the frontier $\operatorname{fr}(V(M))$ of $V(M)$. Note that $\Gamma$
is naturally a bipartite graph, because its vertices correspond to components
of $V(M)$ or of $M-V(M)$, and each edge of $\Gamma$ joins vertices of distinct
types. The vertices which correspond to $V(M)$ will be called $V_{0}$-vertices
and the vertices which correspond to $M-V(M)$ will be called $V_{1}$-vertices.
Here are two properties of $V(M)$, which have algebraic analogues. The first
is the Enclosing Property, which says that any essential annulus or torus in
$M$ is enclosed by $V(M)$. The second is that if $F$ is any embedded essential
surface in $M$, not necessarily an annulus or torus, and if $F$ has
intersection number zero with every essential annulus and torus in $M$, then
$F$ is homotopic into $M-V(M)$. These conditions are not sufficient to
characterise $V(M)$ up to isotopy, but they do contain much of the information
needed for such a characterisation. The algebraic analogue of the Enclosing
Property is that the almost invariant subsets of $G$ which correspond to
essential annuli or tori are enclosed by the $V_{0}$-vertices of $\Gamma$. The
algebraic analogue of the second property is that the splitting associated to
$F$ is enclosed by a $V_{1}$-vertex of $\Gamma$.

Now let $G$ be a finitely generated group with a family of subgroups
$\{H_{\lambda}\}_{\lambda\in\Lambda}$. For each $\lambda\in\Lambda$, let
$X_{\lambda}$ denote a nontrivial $H_{\lambda}$-almost invariant subset of
$G$. Then our algebraic regular neighbourhood of the $X_{\lambda}$'s in $G$
will be a bipartite graph of groups structure $\Gamma$ for $G$ such that the
$V_{0}$-vertices of $\Gamma$ enclose the $X_{\lambda}$'s, and splittings of
$G$ which have intersection number zero with each $X_{\lambda}$ are enclosed
by the $V_{1}$-vertices of $\Gamma$. In addition, we need to insist that
$\Gamma$ is minimal in order to have any uniqueness results. There is one
further technical condition which we need to impose, but we will not describe
it here.

Our main result on regular neighbourhoods is the following.

\begin{theorem}
\label{regnbhdsexist} $\!$Let $G$ be a finitely generated group with a family of subgroups $\{H_{\!\lambda}\}_{\lambda\in\Lambda}$. For each $\lambda\in\Lambda$,
let $X_{\lambda}$ denote a nontrivial $H_{\lambda}$-almost invariant subset of
$G$.

\begin{enumerate}
\item If an algebraic regular neighbourhood of the $X_{\lambda}$'s in $G$
exists, then it is unique.

\item If $\Lambda$ is finite, and each $H_{\lambda}$ is finitely generated,
then there is an algebraic regular neighbourhood of the $X_{\lambda}$'s in
$G$.
\end{enumerate}
\end{theorem}

In case (2), every $V_{0}$-vertex group of the regular neighbourhood is
finitely generated. However, the $V_{1}$-vertex groups and the edge groups
need not be finitely generated. This is why we allow subgroups $H_{\lambda}$
which are not finitely generated in our definition of an algebraic regular
neighbourhood, and this turns out to be an important aspect of the theory.
Surprisingly, case (2) of the above theorem remains true if we allow the
$H_{\lambda}$'s not to be finitely generated, so long as we insist that
whenever this occurs, $X_{\lambda}$ is associated to a splitting.

Now we discuss how we apply this theory to produce our new decompositions of
finitely presented groups. It is simpler to restrict attention to one-ended
groups. In addition we need to restrict our attention to finitely presented
groups because we use certain accessibility results, for example the main
result of Bestvina and Feighn in \cite{B-F}. As Fuchsian groups play an
important role in the results, we should explain that we use the term to
include not only discrete groups of isometries of the hyperbolic plane, but
also to include discrete groups of isometries of the Euclidean plane. The
extra groups included are all virtually $\mathbb{Z}\times\mathbb{Z}$.

Before stating our results, we need a little terminology to describe special
types of vertex in a graph of groups $\Gamma$. A vertex $v$ of $\Gamma$ is of
\textit{isolated} type if it has valence $2$, is not the sole vertex of a
loop, and the two edges incident to $v$ carry groups which map to the vertex
group $G(v)$ of $v$ by an isomorphism. A vertex $v$ of $\Gamma$ is of
\textit{finite-by-Fuchsian} type if $G(v)$ is a finite-by-Fuchsian group,
where the Fuchsian group is not finite nor two-ended, and there is exactly one
edge of $\Gamma$ which is incident to $v$ for each peripheral subgroup $K$ of
$G(v)$ and this edge carries $K$. Thus a vertex cannot be both isolated and of
finite-by-Fuchsian type.

Let $E_{1}$ denote the family of equivalence classes of all nontrivial almost
invariant subsets of $G$ which are over a two-ended subgroup. It would seem
simpler to directly consider the family of all such almost invariant subsets
of $G$, rather than their equivalence classes. But there are situations where
this causes problems. This is a minor point which the reader should ignore at
a first reading. Our first canonical decomposition result is the following.

\begin{theorem}
If $G$ is a one-ended, finitely presented group, then there is a regular
neighbourhood $\Gamma(E_{1}:G)$ of $E_{1}$ in $G$. Each $V_{0}$-vertex $v$ of
$\Gamma(E_{1}:G)$ satisfies one of the following conditions:

\begin{enumerate}
\item $v$ is isolated.

\item $v$ is of finite-by-Fuchsian type.

\item $G(v)$ is the full commensuriser $Comm_{G}(H)$ for some two-ended
subgroup $H$, such that $e(G,H)\geq2$.
\end{enumerate}

$\Gamma(E_{1}:G)$ consists of a single vertex if and only if $E_{1}$ is empty,
or $G$ itself satisfies one of conditions (2) or (3) above.
\end{theorem}

We will denote $\Gamma(E_{1}:G)$ by $\Gamma_{1}$. Thus $\Gamma_{1}$ is a
finite bipartite graph of groups decomposition of $G$ with the following
properties. Any element of $E_{1}$ is enclosed by some $V_{0}$-vertex of
$\Gamma_{1}$, and each $V_{0}$-vertex of $\Gamma_{1}$ encloses some element of
$E_{1}$. In particular, any splitting of $G$ over a two-ended subgroup is
enclosed by some $V_{0}$-vertex of $\Gamma_{1}$. If $\sigma$ is a splitting of
$G$, and if $\sigma$ has intersection number zero with every element of
$E_{1}$, then $\sigma$ is enclosed by a $V_{1}$-vertex of $\Gamma_{1}$. Our
uniqueness result for regular neighbourhoods implies that this decomposition
is unique.

The uniqueness of $\Gamma_{1}$ has further implications. As $E_{1}$ is
obviously invariant under automorphisms of $G$, it follows that the same holds
for $\Gamma_{1}$. This is one of the main features of our work. Previously the
only such result was for the special case when $G$ is word hyperbolic.

A remarkable feature of the decomposition $\Gamma_{1}$ is that $V_{0}$-vertex
groups of type (3) need not be finitely generated. This does not contradict the
comments after Theorem \ref{regnbhdsexist}, as the family $E_{1}$ may be
infinite.

It is possible to obtain the decompositions of previous authors from
$\Gamma_{1}$ by splitting $\Gamma_{1}$ at the $V_{0}$-vertices of
commensuriser type. Doing this yields decompositions in which all the vertex
groups are finitely generated, but these decompositions are no longer unique
in general.

When $G$ is the fundamental group of a Haken manifold $M$, the $V_{0}%
$-vertices of $\Gamma_{1}$ essentially correspond to the peripheral components
of the characteristic submanifold $V(M)$ of $M$. In particular, the
decomposition of $G$ given by $\Gamma_{1}$ essentially corresponds to the full
characteristic submanifold when $M$ is atoroidal. The isolated $V_{0}%
$-vertices correspond to components of $V(M)$ of the form $A\times I$, where
$A$ denotes an annulus, the $V_{0}$-vertices of finite-by-Fuchsian type
correspond to $I$-bundle components of $V(M)$, and the $V_{0}$-vertices of
commensuriser type correspond to peripheral Seifert fibre space components of
$V(M)$.

Now we discuss generalisations of the above result. Let $E_{k}$ denote the
collection of equivalence classes of all the nontrivial almost invariant
subsets of $G$ over virtually polycyclic (VPC) subgroups of $G$ of Hirsch
length $k$. (For brevity, we will refer to the length rather than the Hirsch
length of a VPC group throughout the rest of this paper.) Note that $E_{0}$ is
empty if $G$ is one-ended. We will need the following generalisation of the
idea of a vertex of Fuchsian type. A vertex $v$ of a graph of groups $\Gamma$
is of \textsl{VPC-by-Fuchsian type} if $G(v)$ is a VPC-by-Fuchsian group,
where the Fuchsian group is not finite nor two-ended, and there is exactly one
edge of $\Gamma$ which is incident to $v$ for each peripheral subgroup $K$ of
$G(v)$ and this edge carries $K$. If the length of the normal VPC subgroup of
$G(v)$ is $k$, we will say that $G(v)$ is of \textsl{length} $k$. Note that if
$G=G(v)$, then the Fuchsian quotient group corresponds to a closed orbifold.

Our first generalisation is the following result.

\begin{theorem}
If $G$ is a one-ended, finitely presented group which does not split over VPC
groups of length $