Title of Paper: The cohomology of the Steenrod algebra and representations of the general linear groups Author: Nguy\^{e}n H. V. Hung 2000 Mathematics Subject Classification: Primary 55P47, 55Q45, 55S10, 55T15. Address of Author: Current Address: Department of Mathematics, Wayne State University 656 W. Kirby Street, Detroit, MI 48202 (USA) E-mail address: nhvhung@@math.wayne.edu Permanent Address: Department of Mathematics, Vietnam National University, Hanoi 334 Nguyen Trai Street, Hanoi, Vietnam E-mail address: nhvhung@@vnu.edu.vn Abstract: Let $Tr_k$ be the algebraic transfer that maps from the coinvariants of certain $GL_k$-representation to the cohomology of the Steenrod algebra. This transfer was defined by W. Singer as an algebraic version of the geometrical transfer $tr_k: \pi_*^S((B\V _k)_+) \to \pi_*^S(S^0)$. It has been shown that the algebraic transfer is highly nontrivial, more precisely, that $Tr_k$ is an isomorphism for $k=1, 2, 3$ and that $Tr= \oplus_k Tr_k$ is a homomorphism of algebras. In this paper, we first recognize the phenomenon that if we start from any degree $d$, and apply $Sq^0$ repeatedly at most $(k-2)$ times, then we get into the region, in which all the iterated squaring operations are isomorphisms on the coinvariants of the $GL_k$-representation. As a consequence, every finite $Sq^0$-family in the coinvariants has at most $(k-2)$ non zero elements. Two applications are exploited. The first main theorem is that $Tr_k$ is not an isomorphism for $k\geq 5$. Furthermore, $Tr_k$ is not an isomorphism in infinitely many degrees for each $k > 5$. We also show that if $Tr_{\ell}$ detects a nonzero element in certain degrees of $\text{Ker}(Sq^0)$, then it is not a monomorphism and further, $Tr_k$ is not a monomorphism in infinitely many degrees for each $k>\ell$. The second main theorem is that the elements of any $Sq^0$-family in the cohomology of the Steenrod algebra, except at most its first $(k-2)$ elements, are either all detected or all not detected by $Tr_k$, for every $k$. Applications of this study to the cases $k=4$ and $5$ show that $Tr_4$ does not detect the three families $g$, $D_3$, $p'$ and $Tr_5$ does not detect the family $\{h_{n+1}g_n |\; n\geq 1\}$.