THE HIT PROBLEM FOR THE MODULAR INVARIANTS OF LINEAR GROUPS NGUY^E~N H. V. HU_.NG AND TR^A`N NGO.C NAM Abstract.Let the mod 2 Steenrod algebra, A, and the general linear group, GLk := GL(k, F2), act on Pk := F2[x1, ..., xk] with deg(xi) = 1 in the u* *sual manner. We prove that, for a family of some rather small subgroups G of * *GLk, every element of positive degree in the invariant algebra PGkis hit by A* * in Pk. In other words, (PGk)+ A+ . Pk, where (PGk)+ and A+ denote respectively the submodules of PGkand A consisting of all elements of positive degree. This family contains most of the parabolic subgroups of GLk. It should be noted that the smaller the group G is the harder the problem turns out t* *o be. Remarkably, when G is the smallest group of the family, the invariant al* *gebra PGkis a polynomial algebra in k variables, whose degrees are 8 and fix* *ed while k increases. It has been shown in [3] that, for G = GLk, the inclusion (PGLkk)+ A+ . Pk is equivalent to a week algebraic version of the long-standing c* *onjec- ture stating that the only spherical classes in Q0S0 are the elements of* * Hopf invariant one and those of Kervaire invariant one. 1.Introduction Let Pk := F2[x1, . .,.xk] be the polynomial algebra over the field of two ele* *ments, F2, in k variables x1, ..., xk, each of degree 1. It is equipped with the usual* * structure of module over GLk := GL(k, F2) by means of substitutions of variables. The mod* * 2 Steenrod algebra, A, acts upon Pk by use of the formula 8 < xi, j = 0, Sqj(xi) = : x2i, j = 1, 0, otherwise, and subject to the Cartan formula Xn Sqn(fg) = Sqj(f)Sqn-j(g), j=0 for f, g 2 F2[x1, . .,.xk]. Let G be a subgroup of GLk. Then Pk possesses the induced structure of G- module. Denote by PkG the subalgebra of all G-invariants in Pk. Since the action of GLk and that of A on Pk commute with each other, PkGis also an A-module. In [3], the first named author is interested in the homomorphism jG : F2 (PkG) ! (F2 Pk)G A A ____________ 1The work was supported in part by the National Research Program, N01.4.2. 22000 Mathematics Subject Classification. Primary 55S10, Secondary 55Q45 3Key words and phrases. Steenrod algebra, Invariant theory, Dickson invarian* *t, M`ui invariant. 1 2 NGUY^E~N H. V. HU_.NG AND TR^A`N NGO.C NAM induced by the identity map on Pk. He also sets up the following conjecture for G = GLk and shows that it is equivalent to a weak algebraic version of the long- standing conjecture stating that the only spherical classes in Q0S0 are the ele* *ments of Hopf invariant one and those of Kervaire invariant one. Conjecture 1.1. ([3]) jGLk = 0 in positive degrees for k > 2. This has been established for k = 3 in [3] and then for arbitrary k > 2 in [6* *]. That the conjecture is no longer valid for k = 1 and k = 2 is respectively show* *n in [3] to be an exposition of the existence of the Hopf invariant one and the Kerv* *aire invariant one classes. In the present paper, we are interested in the following problem: Which subgr* *oup G of GLk possesses jG = 0 in positive degrees? It should be noted that, as obse* *rved in the introduction of [3], jG = 0 in positive degrees() (PkG)+ A+ . Pk, where (PkG)+ and A+ denote respectively the submodules of PkGand A consisting of all elements of positive degree. Therefore, the smaller the group G is the h* *arder the problem turns out to be. For instance, we have understood that jG 6= 0 for G = {1}, G = GL1 or G = GL2. Furthermore, let Tk be the Sylow 2-subgroup of GLk consisting of all upper triangular matrices with entries 1 on the main diag* *onal. Then jTk 6= 0, indeed V1 = x1 is a Tk-invariant, however x1 62 A+ . Pk. The problem we are interested in is closely related to the hit problem of de- termination of F2 Pk. This problem has first been studied by F. Peterson [11], A R. Wood [16], W. Singer [14], and S. Priddy [12], who show its relationships to* * sev- eral classical problems in cobordism theory, modular representation theory, Ada* *ms spectral sequence for the stable homotopy of spheres, stable homotopy type of c* *las- sifying spaces of finite groups. The tensor product F2 Pk has explicitly been * *com- A puted for k 3 (see [9]). It seems unlikely that an explicit description of F2* * Pk A for general k will appear in the near future. There is also another approach, t* *he qualitative one, to the problem. By this we mean giving conditions on elements * *of Pk to show that they go to zero in F2 Pk, i. e. belong to A+ . Pk. Peterson* *'s A conjecture [11], which has been established by Wood [16], claims that F2 Pk = 0 A in certain degrees. Recently, W. Singer, K. Monks, and J. Silverman have refined Wood's method to show that many more monomials in Pk are in A+ . Pk. (See Silverman [13] and references therein.) In this paper, we prove that jG = 0 in positive degrees, or equivalently (PkG* *)+ A+ . Pk, for a family of some rather small groups G. This family contains most * *of the parabolic subgroups of GLk. Observing the obstructions of the Hopf invariant one and the Kervaire invaria* *nt one classes, it seems necessary to make the hypothesis that G GL3 in order to get jG = 0 in positive degrees. Let us consider the subgroup n` A * ' o G1 o G2 := 0 B | A 2 G1, B 2 G2 GLk, where G1 is a subgroup of GLn and G2 is a subgroup of GLk-n for n k. We are especially interested in the case G1 = GLn and G2 = 1k-n, the unit subgroup of GLk-n. We suppose n > 2 so that GL3 GLn o 1k-n. Here is an interpretation of THE HIT PROBLEM FOR THE MODULAR INVARIANTS OF LINEAR GROUPS 3 this group, which does not depend on coordinates. Let V be an F2-vector space of dimension k and W a vector subspace of dimension n. Then, the subgroup GLn o 1k-n can be interpreted as the subgroup of GL(V ) consisting of all isomorphisms ' : V ! V with '(W ) = W and __'= idV=W , where __'denotes the induced homomorphism of ' on V=W . We compute the algebra of GLn o 1k-n-invariants by combining the works of L. E. Dickson [1] and H. M`ui [10]. M`ui's invariant of degree 2n-1 is defined* * as follows Y Vn = (~1x1 + . .+.~n-1xn-1 + xn). ~j2F2 Dickson's invariant of degree 2n - 2s is defined by the inductive formula Qn,s= Q2n-1,s-1+ VnQn-1,s, where, by convention, Qn,n= 1, Qn,s= 0 for s < 0. Then, Dickson proves in [1] that F2[x1, ..., xn]GLn = F2[Qn,0, ..., Qn,n-1], while M`ui shows in [10] that F2[x1, ..., xk]Tk = F2[V1, ..., Vk]. To generalize these works, we set Y Vn+1(xi) = (~1x1 + . .+.~nxn + xi), ~j2F2 for n < i k. Then, we get Theorem 1.2. For k n, F2[x1, ..., xk]GLno1k-n= F2[Qn,0, ..., Qn,n-1, Vn+1(xn+1), ..., Vn+1(xk)]. The purpose of this paper is to prove Theorem 1.3. (Main theorem) jGLno1k-n = 0 in positive degrees if and only if n > 2. Obviously, GL3 o 1k-3 is the smallest group among all the ones of the form GLn o 1k-n for n > 2. Being applied to this group, the main theorem shows that F2[Q3,0, Q3,1, Q3,2, V4(x4), ..., V4(xk)]+ A+ . Pk, where degQ3,0= 7, degQ3,1= 6, degQ3,2= 4, degV4(xi) = 8 for 3 < i k. This gives a large family of elements, which are hit by A in Pk. Remarkably, the deg* *rees of all the generators of this polynomial algebra are small and do not depend on* * k. Let us now study the parabolic subgroup of GLk: 0 1 A1 * nB A2 C o GLk1,...,km= BB . CC| Ai2 GLkiwithk1 + . .+.km = k . @ .. A 0 Am It is easily seen that GLk1o 1k-k1 is a subgroup of GLk1,...,km. Therefore, we * *have Corollary 1.4. jGLk1,...,km= 0 in positive degrees if and only if k1 > 2. 4 NGUY^E~N H. V. HU_.NG AND TR^A`N NGO.C NAM Let G be a subgroup of GLk and ! 2 GLk. It is easily seen that Pk!G!-1= !PkG. As the action of GLk on Pk commutes with that of A, Theorem 1.3 and Corollary 1* *.4 also claim that jG = 0 for any subgroup G, which is conjugate either to GLn o1k* *-n with n > 2 or to GLk1,...,kmwith k1 > 2. Note that GLk is a special case of the parabolic subgroup GLk1,...,kmwith k =* * k1 and m = 1. Hence we obtain an alternative proof for Conjecture 1.1: Corollary 1.5. ([6]) jGLk = 0 in positive degrees if and only if k > 2. The readers are referred to [4] and [5] for some problems, which are related * *to the main theorem and Corollary 1.5. Additionally, the problem of determination of F2 (PkGLk) and its applications have been studied by Hu_.ng - Peterson [7],* * [8]. A The paper contains 5 sections and is organized as follows. We determine the algebra of GLn o 1k-n-invariants in Section 2 and study the action of A on this algebra for n = 3 in Section 3. The main theorem and its corollaries are proved in Section 4 assuming the truth of Lemma 4.2 as a key tool. Finally, we show th* *is lemma in Section 5 and then complete the proof of the main theorem. 2. The invariant algebra of GLn o 1k-n The action of GLk on Pk = F2[x1, . .,.xk] is precisely discribed as follows. * *For every ! = (!ij)kxk 2 GLk and any f 2 Pk, one defines (!f)(x1, . .,.xk) = f(!x1, . .,.!xk), where !x1, . .,.!xk are given by X !xj = !ijxi (1 j k). 1 i k Then, each subgroup G of GLk possesses the induced action on Pk. Using the notations given in the introduction, we get the following theorem, which is also numbered as Theorem 1.2. Theorem 2.1. For k n, F2[x1, . .,.xk]GLno1k-n= F2[Qn,0, . .,.Qn,n-1, Vn+1(xn+1), . .,.Vn+1(xk)]. We prove this theorem by three lemmata. Lemma 2.2. The polynomials Qn,0, . .,.Qn,n-1, Vn+1(xn+1), . .,.Vn+1(xk) are GLn o 1k-n-invariants. Proof. The polynomials Qn,0, . .,.Qn,n-1 depend only`on x1,'. .,.xn but not on xn+1, . .,.xk. On the other hand, the action of A0 E * 2 GLn o 1k-n on k-n x1, . .,.xn, where Ek-n denotes the unit (k - n) x (k - n)-matrix, is exactly t* *he same as that of A 2 GLn. Therefore, according to Dickson [1], Qn,0, . .,.Qn,n-1 are GLn o 1k-n-invariants. Note that Vn+1(xi) can be re-written as follows Y Vn+1(xi) = (x + xi) (n < i k), x2Vn THE HIT PROBLEM FOR THE MODULAR INVARIANTS OF LINEAR GROUPS 5 where Vn denotes the F2-vector space spanned by x1, . .,.xn. For a given matrix ! 2 GLn o 1k-n, setting a := !1ix1 + . .+.!nixn and we get !xi= (!1ix1 + . .+.!nixn) + xi= a + xi (n < i k). Obviously, the map ! |Vn: Vn ! Vn is bijective. Then, so is the map Vn ! Vn, which brings x to y = !x + a. Thus, we obtain Y Y !Vn+1(xi) = (!x + !xi) = (!x + a + xi) x2Vn x2Vn Y = (y + xi) = Vn+1(xi), y2Vn for n < i k. This means Vn+1(xn+1), . .,.Vn+1(xk) are GLn o 1k-n-invariants. The lemma is proved. 2 Lemma 2.3. The polynomials Qn,0, . .,.Qn,n-1, Vn+1(xn+1), . .,.Vn+1(xk) are algebraically independent. Proof. The lemma is shown by induction on k n. For k = n, from Dickson [1], Qn,0, . .,.Qn,n-1 are algebraically independent. Suppose inductively that the lemma holds for k - 1 n. Assume that we are given an algebraic identity f(Qn,0, . .,.Qn,n-1, Vn+1(xn+1), . .,.Vn+1(xk)) = 0, where f is a polynomial in the indicated variables. We think of f as a polynomi* *al in the variable Vn+1(xk): Xq f = fj(Qn,0, . .,.Qn,n-1, Vn+1(xn+1), . .,.Vn+1(xk-1))Vnj+1(xk). j=0 Recall that Qn,0, . .,.Qn,n-1, Vn+1(xn+1), . .,.Vn+1(xk-1) do not depend on xk, while Vn+1(xk) is a polynomial of degree 2n in xk. Now we consider f to be a polynomialnin xk. Its leading coefficient, which corresponds to the monomial x2kq, is nothing but fq(Qn,0, . .,.Qn,n-1, Vn+1(xn+1), . .,.Vn+1(xk-1)). Thus,* * by the algebraic independence of x1, . .,.xk, we get fq(Qn,0, . .,.Qn,n-1, Vn+1(xn+1), . .,.Vn+1(xk-1)) = 0. n(q-1) n Iteratedly, consider the coefficients of the monomials x2k , . .,.x2k.0and we have fj(Qn,0, . .,.Qn,n-1, Vn+1(xn+1), . .,.Vn+1(xk-1)) = 0 (0 j q). Hence, applying the inductive hypothesis to f0, . .,.fq, we conclude that they * *all are the zero polynomial. Therefore, so is f. The lemma is proved. 2 Lemma 2.4. Every GLno1k-n-invariant polynomial g(x1, . .,.xk) is a polynomial in the variables Qn,0, . .,.Qn,n-1, Vn+1(xn+1), . .,.Vn+1(xk). Proof. The lemma is also proved by induction on k n. For k = n, it is due to Dickson [1]. Suppose inductively that the lemma is tr* *ue for k - 1 n. We start by an observation, which is actually due to H. M`ui [10* *], 6 NGUY^E~N H. V. HU_.NG AND TR^A`N NGO.C NAM claiming that if a GLn o 1k-n-invariant polynomial admits xk as a factor, then * *it also admits Vn+1(xk) as a factor. Let g0 be the sum of all monomials in g which are not divisible by xk. Then g - g0 has xk as a factor and therefore admits Vn+1(xk) as a factor. Suppose g - g0 = g0Vnp+1(xk), where g0 is not divisible by xk. Since g0 does not depend on xk, it is a GLn o 1k-1-n-invariant in F2[x1, . .,.xk-1]. By means of the inductive hypothesis, g0* * is a polynomial in Qn,0, . .,.Qn,n-1, Vn+1(xn+1), . .,.Vn+1(xk-1). Denote by degxkg the degree of g regarded as a polynomial in xk. It is clear * *that degxkg0 i1(R), then X X Sqi(R) = S + T, where each S is an H-monomial with i2(S) < i2(R), and each T is an H-monomial with h(R) < h(T ) h(R) + i=4. Proof. (i) According to Proposition 3.1, if X is one of the fundamental H-invar* *iants Q0, Q1, Q2, W4, . .,.Wk, then Sq4X = XQ2. Hence, using the Cartan formula, we get ` ' X Sq4i(R) = h(R)i RQi2+ S, where each term S is an H-monomial with i2(S) < i2(R) + i. (ii) We write R = R1. .R.h, where h = h(R) and Rp is one of the fundamental H-invariants Q0, Q1, Q2, W4, . .,.Wk, for 1 p h. Using again the Cartan formula and Proposition 3.1, we have X Sqi(R) = Sqj1(R1) . .S.qjh(Rh) j1+...+jh=i X X = S + T. h(S)=h(R) h(T)>h(R) As deg Q2 = 4 is the smallest number of the degrees of Q0, Q1, Q2, W4, . .,. Wk, the degree information shows that h(R) < h(T ) h(R) + i=4, for every T in the sum. Consider an arbitrary term S = Sqj1(R1) . .S.qjh(Rh) in the sum. As h(S) = h(R), we can see that jp = 0 for every p with Rp being one of the invariants Q0, W4, . .,.Wk. Suppose the contrary that i2(S) i2(R). (Then, we have actually i2(S) = i2(R) because of h(S) = h(R).) By Proposition 3.1, jp = 0 for every p w* *ith Rp = Q2. So, jp could be non-zero just only in the case Rp = Q1. Furthermore, as h(S) = h(R) and by Proposition 3.1, if jp 6= 0 then jp = 1. Therefore, i = j1 + . .+.jh i1(R). This contradicts to the hypothesis that i > i1(R). The lemma is proved. 2 Lemma 3.4. Suppose R is an H-monomial in`PkH and'n is a non-negative integer such that i2(R) 2n - 1 (mod 2n) and h(R)2n-1= 0. Then n+1 __ i2(R)-2n-1 X R = Sq2 (R Q2 ) + S, __ i2(R) where R := R=Q2 and each term S in the sum is an H-monomial with s(S) < n. 8 NGUY^E~N H. V. HU_.NG AND TR^A`N NGO.C NAM Proof. We have __ i2(R) __ h(R) = h(R Q2 ) = h(R ) + i2(R) __ n n h(R ) + 2 - 1 (mod 2 ). __ ` h(R) ' Hence h(R ) + 2n-1 - 1 h(R) - 2n-1 (mod 2n). As 2n-1 = 0, the term 2n-1 occurs in the 2-adic expansion of h(R) - 2n-1. Thus ` __ n-1 ' ` __ ' ` n-1 ' h(R Q22 -1) = h(R ) + 2n-1 - 1 = h(R) - 2 = 1. 2n-1 2n-1 2n-1 __2n-1-1 Applying Lemma 3.3 (i) to RQ2 and i = 2n-1, we get n+1 __ 2n-1-1 __2n-1 X 0 Sq2 (R Q2 ) = RQ2 + S , where each S0 is an H-monomial in PkH satisfying __ 2n-1-1 n-1 n i2(S0) < i2(R Q2 ) + 2 = 2 - 1. This inequality implies s(S0) < n. Put a := i2(R) - (2n - 1) 0 (mod 2n). By the Cartan formula and Proposi- tion 3.1, we have n+1 __ i2(R)-2n-1 2n+1 __ 2n-1-1 a Sq2 (R Q2 ) = Sq (R Q2 Q2) n+1 __ 2n-1-1 a __2n-1-1 2n+1 a = Sq2 (R Q2 )Q2 + RQ2 Sq (Q2) __ 2n-1 X 0 a __2n-1-1 2n+1 a = (R Q2 + S )Q2 + RQ2 Sq (Q2) X __ n-1 n+1 = R + S0Qa2+ RQ22 -1Sq2 (Qa2), where each term S0Qa2in the sum satisfies s(S0Qa2) < n, because s(S0) < n and a 0 (mod 2n). On the other hand, from Proposition 3.1, if Sq2n+1(Qa2) 6= 0 t* *hen it is not divisible by Q2. Therefore __ 2n-1-1 2n+1 a 2n-1-1 s(R Q2 Sq (Q2)) = s(Q2 ) = n - 1. To sum up, we can write n+1 __ i2(R)-2n-1 X R = Sq2 (R Q2 ) + S, where each term S satisfies s(S) < n. The lemma is proved. 2 Lemma 3.5. Suppose R is an H-monomial in PkH, which is not divisible by Q2, while n and i are positive integers satisfying h(R) 0 (mod 2n), i1(R) 2n - 1, 2n i 2n+1. Then n X X Sqi(RQ22-1) = S + T, where each term S is an H-monomial in PkH with s(S) < n,`while each'term T is an H-monomial in PkH with i2(T ) 2n - 1 (mod 2n) and h(T2)n-1= 0. THE HIT PROBLEM FOR THE MODULAR INVARIANTS OF LINEAR GROUPS 9 n-1 Proof.nNote that i 2n > 2n - 1 i1(R) = i1(RQ22 ). Applying Lemma 3.3 (ii) to RQ22-1 and i, we get n-1 X X Sqi(RQ22 ) = S + T, n-1 where each S is an H-monomial with i2(S) < i2(RQ22 ) = 2n - 1, while each T is an H-monomial with n-1 2n-1 h(RQ22 ) < h(T ) h(RQ2 ) + i=4. For each S in the sum, as i2(S) < 2n - 1, it implies s(S) < n. For each T in the sum, we have n-1 n n h(RQ22 ) = h(R) + 2 - 1 < h(T ) h(R) + 2 - 1 + i=4 h(R) + 2n + (2n-1 - 1). Hence h(R) + 2n h(T ) h(R) + 2n + (2n-1 - 1). Combining`these'inequalities with the hypothesis h(R) 0 (mod 2n), we obtain h(T2)n-1= 0. Finally, suppose i2(T ) = (2n - 1) + b, where b is an integer (that can be po* *sitive, negative or zero). If b 0 (mod 2n) then i2(T ) 2n - 1 (mod 2n). Otherwise* *, if b 6 P0 (mod 2n), then s(T ) < n and such a T can be considered as a term in the sum S. The lemma is proved. 2 4. Proofs of the main theorem and its corollaries The following two lemmata will play a key role in the proof of the main theor* *em. Lemma 4.1. Let R 6= 1 be a product of some distinct elements in the set {Q0, Q1, Q2, W4, . .,.Wk}. Then R 2 Sq1Pk + Sq2Pk. __ __ Proof. We write_R = RS with R | Q1Q2 and S | Q0W4. .W.k.__ If Q1 6 | R , then from Proposition 3.1, Sq1(R) = Sq1(R S) = 0. Hence, by [6, Lemma_2.5],_R 2 Sq1Pk. If R = Q1, then by Proposition 3.1, R = Q1S = Sq2(Q2)S = Sq2(Q2S) 2 Sq2Pk. __ Finally, if R = Q1Q2, then by [6, Lemma B], we have Q1Q2 = Sq1u1 + Sq2u2 for some elements u1, u2 2 Pk. Then R = Q1Q2S = (Sq1u1 + Sq2u2)S = Sq1(u1S) + Sq2(u2S) 2 Sq1Pk + Sq2Pk. The lemma follows. 2 We postpone the proof of the next lemma until the last section. Lemma 4.2. Suppose R is an H-monomial in PkH , u 6= 1 is an arbitrary element in Pk and n is a positive integer. (i)If s(R) < n, then Ru2n 2 A+ . Pk.` ' (ii)If i2(R) 2n - 1 (mod 2n) and h(R)2n-1= 0, then Ru2n 2 A+ . Pk. (iii)If i2(R)n= 2n-1 i1(R), h(R) 2n-1 (mod 2n) and u 2 Sq1Pk+Sq2Pk, then Ru2 2 A+ . Pk. 10 NGUY^E~N H. V. HU_.NG AND TR^A`N NGO.C NAM Proof of the main theorem. It suffices to show the theorem for the group H = GL3 o 1k-3, as this is the smallest one of the groups GLn o 1k-n for n > 2. Moreover, using Theorem 1.2, we need only to prove that (PkH)+ = F2[Q0, Q1, Q2, W4, . .,.Wk]+ A+ . Pk for every k > 2. Suppose R is an H-monomial of positive degree in PkH. We need to show that R 2 A+ . Pk. Set n := s(S). Then, by definition, i2(R) 2n - 1 (mod 2n+1). Let us consider the following four cases. Case 1: Q2n2divides R. Combining this with the_hypothesis i2(R) 2n -1_(mod 2n+1), it implies Q2n+* *12 dividing R. Denoting R__:= R=Q2n+12, we have i2(R ) = i2(R) - 2n+1 2n - 1_(mod_ 2n+1). Thus s(R )_=_n < n + 1. Applying Lemma 4.2 (i) to the triple (R , Q2, n +,1)we get R = RQ2n+122 A+ . Pk. Case 2: There_exists u 2 {Q0, Q1, W4,_. .,.Wk} such that u2n+1divides R. Setting R := R=u2n+1,_we have s(R ) = s(R)_= n < n+1. Applying Lemma 4.2 (i) to the triple (R , u, n + 1), we get R = Ru2n+12 A+ . Pk. Case 3: i0(R), i1(R), i2(R), i4(R), . .,.ik(R)nall are 2n+1 - 1 and there exi* *sts u 2 {Q0, Q1, Q2, W4, . .,.Wk} such that u2 divides R. By Case 1, u 6= Q2. Furthermore, since i2(R) 2n+1 - 1 and i2(R) 2n - 1 (mod 2n+1), it implies i2(R) = 2n - 1. We investigate the following three sub-cases. Case 3a: n = 0. Then, by Lemma 4.1, R 2 Sq1Pk + Sq2Pk.` ' Case 3b: n 1 and there exists m with 0 < m n and h(R)2m-1= 0. Ob- __ m ` h(__R)' viously i2(R) 2m - 1 (mod 2m ). Put R := R=u2 and we have 2m-1 = ` ' ` ' h(R) - 2m h(R) __ 2m-1 __ = 2m-1 = 0. Applying Lemma 4.2 (ii) to the triple (R , u, m), we get R = Ru2m 2 A+`. Pk. ' Case 3c: n 1 and h(R)2m-1= 1 for every m with 0 < m n. It implies __ n h(R) 2n - 1 (mod 2n). We write uniquely R in the form R = R v2 , where v 6= 1 is a certain product of distinct_elements in the set {Q0, Q1,_W4, ._.,.W* *k}__ (consequently,_i2(v)_= 0), and R is a certain H-monomial with i0(R ), i1(R ), i* *2(R ), i4(R ), . .,.ik(R ) all 2n - 1. Observe that __ __ i2(R_) = i2(R) - 2ni2(v)= 2n - 1 i1(R ), h(R ) = h(R) - 2nh(v) 2n - 1 (mod 2n). __ By Lemma 4.1,_vn2 Sq1Pk+Sq2Pk. Applying Lemma 4.2 (iii) to the triple (R , v, n* *), we get R = Rv2 2 A+ . Pk. Case 4: i0(R), i1(R), i2(R), i4(R), . .,.ik(R) all are 2n - 1. THE HIT PROBLEM FOR THE MODULAR INVARIANTS OF LINEAR GROUPS 11 In particular, i2(R) = 2n - 1, since i2(R) 2n - 1 (mod 2n+1). It should be noted that n > 0, otherwise R = 1 with degree 0. We also examine the following three sub-cases. Case 4a: n = 1. Then, by Lemma 4.1, R 2 Sq1Pk + Sq2Pk.` ' Case 4b: n 2 and there exists m with 0 < m < n and h(R)2m-1= 0. __ m It`is_obvious'that`i2(R)' 2m`- 1 (mod' 2m ). Put R := R=Q22 and we have h(R ) = h(R) - 2m = h(R) = 0. Applying Lemma 4.2 (ii) to the 2m-1 __ 2m-1 2m-1_ triple (R , Q2, m), we`get R'= RQ2m22 A+ . Pk. Case 4c: n 2 and h(R)2m-1= 1 for every m with 0 < m < n. It implies __ n-1 h(R) 2n-1 - 1 (mod 2n-1). We write uniquely R in the form R = Ru2 , where u 6= 1 is a certain product_of distinct elements in the_set {Q0,_Q1, Q2,_W4, ._* *.,.Wk}_ with_i2(u) = 1, and R is a certain H-monomial with i0(R ), i1(R ), i2(R ), i4(R* * ), . .,. ik(R ) all 2n-1 - 1. Note that __ __ i2(R_)= i2(R) - 2n-1i2(u)= 2n - 1 - 2n-1 = 2n-1 - 1 i1(R ), h(R ) = h(R) - 2n-1h(u) 2n-1 - 1 (mod 2n-1). By_Lemma 4.1, we have u 2 Sq1Pk+Sq2Pk._Applying Lemma 4.2 (iii) to the triple (R , u, n -,1)we obtain R = Ru2n-12 A+ . Pk. The main theorem is completely proved. 2 Proof of Corollary 1.4. Note that GLk1o 1k-k1 is a subgroup of GLk1,...,km. So,* * by the main theorem, we have (PkGLk1,...,km)+ (PkGLk1o1k-k1)+ A+ . Pk, for k1 > 2. If k1 = 1, then it is easily seen that Q1,02 (F2[x1]GL1)+ (PkGLk1,...,km)+ . However, Q1,0= x1 62 A+ . Pk. Finally, if k1 = 2, then we observe that Q2,12 (F2[x1, x2]GL2)+ (PkGLk1,...,km)+ , while Q2,1= x21+ x22+ x1x2 62 A+ . Pk. The corollary is proved. 2 Since the general linear group GLk is a special case of the parabolic subgroup GLk1,...,kmwith k = k1 and m = 1, Corollary 1.5 follows. 5. Proof of Lemma 4.2 The lemma is proved by induction. Its starting case is handled by the followi* *ng lemma. Lemma 5.1. Suppose R is an H-monomial in PkH with s(R) = 0, and u 6= 1 is an arbitrary element in Pk. Then Ru2 2 Sq1Pk + Sq2Pk. 12 NGUY^E~N H. V. HU_.NG AND TR^A`N NGO.C NAM Proof. We consider the following two cases. Case 1: i1(R) 0 (mod 2). By Proposition 3.1, we have Sq1(Ru2) = Sq1(R)u2 = 0. So, using [6, Lemma 2.5], we get Ru2 2 Sq1Pk. Case 2: i1(R) 1 (mod 2). Put S = R=Q1Qi22with i2 = i2(R). Since s(R) = 0, the number i2 is even. Then we have Ru2 = (SQi22u2)Q1 = (SQi22u2)Sq2(Q2) = Sq2(SQi22u2Q2) + Sq2(SQi22u2)Q2 = Sq2(SQi2+12u2) + Sq2(Su2)Qi2+12. It is easy to see that Sq2(Su2) = Sq2(S)u2 + S(Sq1u)2. Combining this with the fact i1(S) = i1(Sq2S) 0 (mod 2), we obtain Sq1(Sq2(Su2)) = 0. Then, by [6, Lemma 2.5], this gives Sq2(Su2) = Sq1v for some v 2 Pk. Therefore Sq2(Su2)Qi2+12= Sq1vQi2+12= Sq1(vQi2+12). So, in any case, we have Ru2 2 Sq1Pk + Sq2Pk. The lemma is proved. 2 Proof of Lemma 4.2. The proof is divided into three steps. Step 1: If 4.2 (i) and 4.2 (ii) are valid for every n N, then so is 4.2 (iii)* * for every n N. Suppose u = Sq1v1 + Sq2v2 for some v1, v2 2 Pk. We have n 1 2 2n Ru2 = R(Sq v1 + Sq v2) n 2 2n = R(Sq1v1)2 + R(Sq v2) n 2n 2n 2n = [Sq2 (Rv1 ) + Sq (R)v1 ] n+1 2n 2n 1 2n 2n+1 2n + [Sq2 (Rv2 ) + Sq (R)(Sq v2) + Sq (R)v2 ] n 2n 2n+1 2n 2n 1 2n = [Sq2 (Rv1 ) + Sq (Rv2 ) + Sq (R)(Sq v2) ] n 2n 2n+1 2n + [Sq2 (R)v1 + Sq (R)v2 ]. Note that n 1 2n 2n 1 2n 1 1 2n Sq2 (R)(Sq v2) = Sq [R(Sq v2) ] + R(Sq Sq v2) n 1 2n 1 1 = Sq2 [R(Sq v2) ] (as Sq Sq = 0). Thus n n n n+1 n Ru2 + Sq2 (R)v21 + Sq2 (R)v22 2 A+ . Pk. __ 2n-1 __ Set R := R=Q2_ . Obviously, R is an H-monomial in PkH_that_is not divisible by Q2 with h(R ) = h(R) - (2n - 1) 0 (mod 2n) and i1(R ) = i1(R) 2n - 1. Using Lemma 3.5, we get __ 2n-1 P P Sq2n(R) = Sq2n(R_Q2_n ) = P S1 + P T1, Sq2n+1(R) = Sq2n+1(R Q22-1) = S2 + T2, THE HIT PROBLEM FOR THE MODULAR INVARIANTS OF LINEAR GROUPS 13 where each term S1 or S2 is an H-monomial with s(S1) < n and s(S2) < n, while each`term'T1`or T2 is'an H-monomial with i2(T1) i2(T2) 2n - 1 (mod 2n) and h(T1) h(T2) 2n-1 = 2n-1 = 0. Hence n X 2n X 2n X 2n X 2n + Ru2 + S1v1 + S2v2 + T1v1 + T2v2 2 A . Pk. From the hypothesis,nLemma 4.2n(i) is valid for the triples (S1, v1, n)and (S2,* * v2, n); that means that S1v21 and S2v22 are in A+ . Pk for every S1, S2. Also by the hy* *- n pothesis,nLemma 4.2 (ii) holds for the triples (T1, v1, n) and (T2,nv2, n), so * *T1v21 and T2v22 both belong to A+ . Pk for every T1, T2. Therefore Ru2 2 A+ . Pk. Step 1 is proved. Step 2: If 4.2 (i) holds for every n N, then so does 4.2 (ii) for every n N. Applying Lemma 3.4, we obtain n+1 __ i2(R)-2n-1 X R = Sq2 (R Q2 ) + S, __ i2(R) where R := R=Q2 and each term S in the sum is an H-monomial with s(S) < n. So n n+1 __ n-1 n X n Ru2 = Sq2 (R Qi2(R)-22 )u2 + Su2 . Since s(S) < n,nby the hypothesis, Lemma 4.2 (i) holds for the triple (S, u, n), that means Su2_ 2 A+n.-Pk1for every S in the sum. Set ~R:= RQi2(R)-22 . By the Cartan formula, we get n+1 2n 2n+1 2n 2n 1 2n 2 2n Sq2 (R~)u = Sq (R~u ) + Sq (R~)(Sq u) + ~R(Sq u) n+1 2n 2n 1 2n 1 1 2n = Sq2 (R~u ) + Sq [R~(Sq u) ] + ~R(Sq Sq u) n + ~R(Sq2u)2 n+1 2n 2n 1 2n 2 2n = Sq2 (R~u ) + Sq [R~(Sq u) ] + ~R(Sq u) . Thus n+1 n n Sq2 (R~)u2 + ~R(Sq2u)2 2 A+ . Pk. n-1 It is easy to see that s(R~) = s(Qi2(R)-22 ) = n - 1 < n. So, from the hypothes* *is,n Lemma 4.2 (i) holds for the triple (R~, (Sq2u)2, n - 1); that means R~(Sq2u)2 2 A+ . Pk. Hence Sq2n+1(R~)u2n 2 A+ . Pk. Finally, we have n 2n+1 2n X 2n + Ru2 = Sq (R~)u + Su 2 A . Pk. Step 2 is proved. Step 3: 4.2 (i) is valid for every n. This is proved by induction on n. For n = 1, from the hypothesis s(R) < 1 it yields s(R) = 0. By Lemma 5.1, Ru2 2 Sq1Pk + Sq2Pk. So Lemma 4.2 (i) holds for n = 1. Now let n > 1 and suppose inductively that 4.2 (i) has been proved for every smaller value of n. By Steps 1 and 2 above, 4.2 (ii) and 4.2 (iii) are also val* *id for every smaller value of n. We consider the following three cases. Case 1: s(R) = 0. 14 NGUY^E~N H. V. HU_.NG AND TR^A`N NGO.C NAM Then, by Lemma 5.1, n 2n-1 2 + Ru2 = R(u ) 2 A . Pk. ` ' Case 2: There exists an integer m with 0 m < s(R) and h(R)2m = 0. Combining the facts m < s(R) < n and i2(R) 2s(R)- 1 (mod 2s(R)+1), we get m + 1 < n and i2(R) 2m+1 - 1 (mod 2m+1 ). Since m + 1 < n and by the inductive hypothesis, we can apply Lemma 4.2 (ii) to the triple (R, u2n-m-1, m+* *1) and have Ru2n = R(u2n-m-1)2m+1 2 A+ . Pk. ` ' Case 3: s(R) > 0 and h(R)2m = 1 for every m with 0 m < s(R). It implies h(R) 2s(R)- 1_(modp 2s(R))._ Set p := s(R) > 0. We write uniquely_R in_the form_R = RS2_, where R_and_S are certain H-monomials with i0(R ), i1(R ),_i2(R_), i4(R ), . .,.ik(R ) all 2p - 1. Note that i2(R ) i2(R)_(mod 2p) 2p - 1 (mod_ 2p), as p = s(R). Combining this with the fact i2(R ) 2p - 1, we obtain i2(R ) =_2p - 1. Since p = s(R), * *so 2p does not occur in the dyadic expansion of i2(R) = i2(R )+2pi2(S) = 2p-1+2pi2(S). Hence, it implies s(S) = 0. Applying Lemma 5.1 to the H-monomial S and v := u2n-p-16= 1, we get Sv2 2 Sq1Pk + Sq2Pk. On the other hand, we observe that __ p p p p h(R ) = h(R) - 2 h(S) h(R) (mod 2 ) 2 - 1 (mod 2 ), __ p __ i2(R ) = 2 - 1 i1(R ). Using the inductive hypothesis together_with the assumption p = s(R) < n, we can apply Lemma 4.2 (iii) to the triple (R , Sv2, p) and get n __ 2 2p + Ru2 = R(Sv ) 2 A . Pk. Step 3 is proved. Therefore, Lemma 4.2 follows. 2 References [1]L. E. Dickson, A fundamental system of invariants of the general modular li* *near group with a solution of the form problem, Trans. Amer. Math. Soc. 12 (1911), 75-98. [2]Nguy^e~n H. V. Hu_.ng, The action of the Steenrod squares on the modular in* *variants of linear groups, Proc. Amer. Math. Soc. 113 (1991), 1097-1104. [3]Nguy^e~n H. V. Hu_.ng, Spherical classes and the algebraic transfer, Trans.* * Amer. Math. Soc. 349 (1997), 3893-3910. [4]Nguy^e~n H. V. Hu_.ng, The weak conjecture on spherical classes, Math. Zeit* *. 231 (1999), 727-743. [5]Nguy^e~n H. V. Hu_.ng, Spherical classes and the Lambda algebra, Trans. Ame* *r. Math. Soc. 353 (2001), 4447-4460. [6]Nguy^e~n H. V. Hu_.ng and Tr^a`n Ngo.c Nam, The hit problem for the Dickson* * algebra, Trans. Amer. Math. Soc. 353 (2001), 5029-5040. [7]Nguy^e~n H. V. Hu_.ng and F. P. Peterson, A-generators for the Dickson alge* *bra, Trans. Amer. Math. Soc. 347 (1995), 4687-4728. [8]Nguy^e~n H. V. Hu_.ng and F. P. Peterson, Spherical classes and the Dickson* * algebra, Math. Proc. Camb. Phil. Soc. 124 (1998), 253-264. [9]M. Kameko, Products of projective spaces as Steenrod modules, Thesis, Johns* * Hopkins Uni- versity 1990. THE HIT PROBLEM FOR THE MODULAR INVARIANTS OF LINEAR GROUPS 15 [10]H. M`ui, Modular invariant theory and cohomology algebras of symmetric grou* *ps, Jour. Fac. Sci. Univ. Tokyo, 22 (1975), 310-369. [11]F. P. Peterson, Generators of H*(RP1 ^ RP1 ) as a module over the Steenrod * *algebra, Abstracts Amer. Math. Soc., No 833, April 1987. [12]S. Priddy, On characterizing summands in the classifying space of a group, * *I, Amer. Jour. Math. 112 (1990), 737-748. [13]J. H. Silverman, Hit polynomials and the canonical antiautomorphism of the * *Steenrod algabra, Proc. Amer. Math. Soc. 123 (1995), 627-637. [14]W. M. Singer, The transfer in homological algebra, Math. Zeit. 202 (1989), * *493-523. [15]N. E. Steenrod and D. B. A. Epstein, Cohomology operations, Ann. of Math. S* *tudies, N0 50, Princeton Univ. Press 1962. [16]R. M. W. Wood, Modular representations of GL(n, Fp) and homotopy theory, Le* *cture Notes in Math. 1172, Springer Verlag (1985), 188-203. Department of Mathematics Vietnam National University, Hanoi 334 Nguy^e~n Tr~ai Street, Hanoi, Vietnam Email address: Nguy^e~n H. V. Hu_.ng: nhvhung@hotmail.com Tr^a`n Ngo.c Nam: trngnam@hotmail.com