CHARACTERIZING SEPARATING INVARIANTS DEDICATED TO LUCHO AVRAMOV ON THE OCCASION OF HIS 60TH BIRTHDAY MARA D. NEUSEL AND M"UF`IT SEZER Abstract.We study separating algebras for rings of invariants of finite * *groups. We give an algebraic characterization for these. Furthermore, we describ* *e a particularly nice separating subalgebra for rings of invariants of p-gro* *ups in characteristic p. This leads to a characterization of subalgebras such t* *hat their p-root and integral closure is equal to the ring of invariants (see Prop* *erty (?) below). Finally, we present separating sets for invariants rings of nonm* *odular representations of abelian groups whose size depends only on the degree * *of the representation. Let F be an algebraically closed field and let G be a finite group. Consider* * a faithful representation ae : G ,! GL(n, F) of degree n. It induces an action of the group G on the symmetric algebra on the dual space V *, which we denote by F[V ]. The subring of G-invariants is denoted by F[V ]G . We note that the vector space V decomposes into disjoint G-orbits. * *We denote the orbit space by V=G = {[v] = {gv|g 2 G}|v 2 V }. Any invariant f 2 F[V ]G is constant on the G-orbits [v]. Indeed, F[V ]G F[V * *] is the largest subalgebra with this property. A finitely generated graded subalgeb* *ra A F[V ]G (or more generally a subset in F[V ]G ) is called separating if for * *any two distinct G-orbits [v] 6= [w] there exists a function f 2 A separating the two, * *i.e., f(v) 6= f(w). __ This notion was introducted in Definition 2.3.8 in [2]. Denotepby_A the integr* *al closure of the algebra A (in its field of fractions) and by A its p-root clos* *ure in F[V ], where p is the characteristic of F. In Theorem 2.3.12 ibid. it is shown * *that p ___ (?) A = F[V ]G , provided that A F[V ]G is a finitely generated separating graded subalgebra. * *The converse is not valid, see Example 2.3.14 ibid. Remark 1. We note that for fields that are not algebraically closed, the notion "separating" does not make sense. For example, consider the finite field F2 with two elements. The general linear group GL(2, F2) is a finite group of order 6. * *Its ring of invariants F2[x, y]GL(2,F2)is a polynomial ring generated by d2,0= x2y + xy2 and d2,1= x2 + xy + y2, see, e.g., Theorem 6.1.4 in [11]. The vector space ____________ Date: April 10, 2008, 14 h 29 min. 1 2 M. D. NEUSEL AND M. SEZER V = spanF2{e1, e2} decomposes into the two orbits V \ 0 and {0}. Note that the subalgebra F2[d2,1] F2[x, y]GL(2,F2) is separating, but the extension~is~neither finite nor integral. Even worse, t* *he subgroup Z=3 generated by 01 11 has the same orbits on V . However, we could consider the invariants over the algebraic closure of F2. We obtain __ GL(2,F ) __ GL(2,F ) F[x, y] 2= F F2F2[x, y] 2. __ Taking into account the orbits of the group action on V = span_F{e1, e2} we see that the subalgebra generated by the degree two invariant is, as expected, no l* *onger separating: d2,1vanishes on the orbit [(1, !)] for a primitive 3rd root of unit* *y !. Separating invariants have been studied by several people, see, e.g., [7], [3* *], [4], [2] and the references there. All of these studies show that separating invari* *ants are often better behaved than the ring of invariants itself, e.g., there are al* *ways separating algebras that satisfy Noether's bound, see Corollary 3.9.14 in [2], * *or, separating invariants of vector invariants can be obtained by polarizations, se* *e [4]. In this paper we continue the study of separating invariants. In Section 1 we will describe a separating subalgebra for the ring of invaria* *nts of a finite p-group P over a field of characteristic p. We note that generating in* *variants of p-groups are usually difficult to describe. Indeed, apart from individual ca* *ses, the only large families of modular representations of finite p-groups for which com* *plete (but maybe not minimal) generating sets for the invariants are known are the (a* *ll of them) representations of cyclic groups of order p, see [6], and the indecomp* *osable representations of cyclic groups of order p2, see [10]. In both cases, the rin* *gs of invariants are generated by norms, transfers, and invariants up to a certain de* *gree. The reason for including all invariants up to some degree is that norms and tra* *nsfers can be employed to decompose invariants usually only after some degree and not all invariants at small degrees are norms or (relative) transfers. We want to s* *how that in contrast norms and transfers suffice to separate orbits for all represe* *ntations of any p-group. This study, moreover, will enable us to characterize subalgebr* *as A F[V ]P satisfying Property (?). In Section 2 we turn to the other extreme: We will consider nonmodular rep- resentations of abelian groups. With Noether's bound, generating sets of the ri* *ng of invariants can be easily described leading to an upper estimate on the number of generators depending on the group order and the degree of the representation. However, again separating invariants have yet a simpler structure: We will desc* *ribe separating sets whose size depends only on the degree of the representation. We close the introduction with a characterization of separating subalgebras in algebraic terms. Theorem 2. Let ae : G ,! GL(n, F) be a faithful representation of a finite group G over some algebraically closed field F. Then a finitely generated subalgebra A F[V ]G is separating if and only if the following two conditions are satisfied: (1) The extension OE : A ,! F[V ]G is integral. (2) The induced map OE* : Max (F[V ]G ) -! Max (A) SEPARATING INVARIANTS 3 between the spectra of maximal ideals is bijective. Proof.Choose coordinates for V *and write F[V ] = F[x1, . .,.xn]. For any point v = (v1, . .,.vn) 2 V we obtain a maximal ideal mv = (x1 - v1, . .,.xn - vn) ( F[V ]. Two maximal ideals mv and mw coincide upon contraction to F[V ]G , mv \ F[V ]G = mw \ F[V ]G if and only if v and w lie in the same G-orbit. Thus the spectrum of maximal id* *eal in F[V ]G is given by Max (F[V ]G ) = {mv \ F[V ]G |[v] 2 V=G}. If A is separating, then the extension A ,! F[V ]G is integral by Theorem 2.3* *.12 in [2]. Thus, the induced map on the spectra of maximal ideals is surjective. Furthermore, for any two points v, w 2 V lying in different G-orbits, there is a function f 2 A such that f(v) 6= f(w). The function f - f(w) separates the two points as well and moreover vanishes at w. Thus we may assume that f 2 J (w) = mw F[V ] is contained in the vanishing ideal of the point w, but f 62 J (v) = mv. Thus f 2 mw \ A but f =2mv \ A. Therefore, the induced map on the spectra of maximal ideals is also injective. Conversely, if A satisfies the two conditions mentioned above, then for any t* *wo distinct G-orbits [v] and [w] we find mv \ A 6= mw \ A. Therefore, there is a function f 2 A vanishing on v but not on w (and vice vers* *a). Thus A is separating. Since p-root extensions form a relatively nice subset of integral extensions * *we have the following characterization. Proposition 3. Let ae : G ,! GL(n, F) be a faithful representation of a finite * *group G over some algebraically closed field F. Then a finitely generatedpintegrally_* *closed graded subalgebra A F[V ]G is separating if and only if A = F[V ]G . Further* *more, in that case the induced map on the prime ideal spectra __ Spec(F[V ]G ) -! Spec(A ) is bijective. Proof.If F has characteristic zero, then an integrally closed separating subalg* *ebra A is equal to the ring of invariants, since there are no nontrivial purely inse* *parable extensions. Thus let F have finite characteristic. If A is separating, then p ___ p__ A = A = F[V ]G p __ by Theorem 2.3.12 in [2]. Conversely, if A = F[V ]G , then there exists an in* *teger s 2 N0 such that p __ ps p__ G A A A = F[V ] . 4 M. D. NEUSEL AND M. SEZER p__ p__ps Since with A also A is separating, so is A. The last statement follows by Exercise 15 in Chapter 5 of [1]. Corollary 4. Let ae : G ,! GL(n, F) be a faithful representation of a finite gr* *oup G over some algebraicallypclosed_field F. Let A F[V ]G be a finitely generated * *graded subalgebra. If A = F[V ]G then A is separating. Proof.Immediate from the preceding result. Example. Let ae : G ,! GL(n, F) be a representation of a finite group G. Denote* * by FG the group algebra and let V (G) = FG V be the induced module. The group G acts on V (G) by left multiplication on the first component. We obtain a surjective G-equivariant map between the rings of polynomial functions jG : F[V (G)] -! F[V ]. By restriction to the induced ring of invariants, we obtain the classical Noeth* *er map, see Section 4.2 in [11], jGG: F[V (G)]G -! F[V ]G . We note that V (G) is the n-fold regular representation of G. Thus F[V (G)]G are the n-fold vector invariants of the regular representation of G. In the classi* *cal nonmodular case the map jGGis surjective, see Proposition 4.2.2 in [11]. This d* *oes not remain true in the modular case. However, as shown in Proposition 2.2 of [9] the p-root closure of the image of the Noether map is equal to F[V ]G . Thus, b* *y the preceding result, the image of the Noether map is separating. 1. Separating subalgebras for modular p-Groups In this section we want to present a new construction for separating subalge- bras of rings of invariants of finite p-groups over an algebraically closed fie* *ld F of characteristic p. This result is particularly interesting because it gives ris* *e to a characterization of subalgebras satisfying Property (?). We start with a recollection of two methods to construct invariants. For f 2 * *F[V ], we define the norm of f, denoted N(f), by Y g(f) 2 F[V ]G . g2G Furthermore, the transfer is defined by X TrG : F[V ] -! F[V ]G , f 7! g(f). g2G We obtain a relative version in the following way: Let H be a subgroup of G. Th* *en the relative transfer (from H to G) is given by X TrGH: F[V ]H ! F[V ]G , TrGH(f) = ~g(f), ~g2G=H where the sum runs over a set of coset representatives of H in G. We set X I = Im(TrGH) F[V ]G , H b, we have that a - b 2 A as can be seen as follows. By construction there are two invariants xeii. .x.ennand xfii. .x.fnn2 M(S) and thus we obtain two equations eiOi+ . .+.enOn = 0 and fiOi+ . .+.fnOn = 0 such that ei= a, fi= b, and ej = fj = 0 for j =2S. Taking the difference of the* *se equations yields (ei- fi)Oi+ . .+.(en - fn)On = 0, with ei- fi = a - b. The coefficients of this equation are not necessarily non- negative. However, since G is finite, we can choose for each 1 j n, a posit* *ive integer (namely, the order of Oj) oj such that ojOj = 0. Therefore by adding enough positive multiples of ojOj for j 2 S \ {i}, we get an equation hiOi+ . .+.hnOn = 0 with hi= a - b, hj 0 for j 2 S and hj = 0 for j =2S. It follows that A [ {0} * *is a lattice in N0 and hence generated by its smallest positive member, say amin. Let mS = xeii. .x.enn2 M(S) be the smallest monomial in M(S) w.r.t. lexicographic order with x1 > x2 > . .>. xn such that ei = amin. We show that the collection of these monomials mS, for every S {1, . .,.n} is separating. Proposition 10. The set T = {mS|S {1, 2, . .,.n}} is separating. In particula* *r, the minimal size of a separating set is at most 2n - 1. Proof.We assume to the contrary that the monomials in T do not separate the distinct orbits [v], [w] 2 V=G. We will show that this implies that m(v) = m(w) for any invariant monomial m, and hence for any invariant, which is the desired contradiction. Let m = xe11xe22. .x.enn2 F[V ]G . Denote by S the complement in {1, 2, . .,.n} of {j | ej = 0}. Thus m 2 M(S). We proceed by induction on the order of S. If |S| = 1, say S = {j}, then m = xt.ojjfor some positive integer t, since m * *is invariant. Furthermore, mS = xojj2 T . Since we are assuming the monomials in T do not separate v = (v1, . .,.vn) and w = (w1, . .,.wn) we find that m(v) = vt.ojj= wt.ojj= m(w). Since this is true for any choice of j we are done. Next, we assume that |S| > 1, and the result has been proven for sets of smal* *ler size. SEPARATING INVARIANTS 9 Let i denote the smallest integer in S. By construction there exists a positi* *ve integer r such that the monomial mrS= xfii. .x.fnn satisfies ei= fi. Hence, _m_ = xei+1i+1.x.e.nn_2 F(V )G . mrS xfi+1i+1.x.f.nn is a rational invariant. Let J denote the set of indices j such that xj appears in the denominator of _m_ oj mrS. Since xj is an invariant for all j 2 J , it follows that for some suitabl* *y large t 2 N m Y m0:= ___mr xtojj2 F[V ]G Sj2J is an invariant monomial. Moreover, since xi does not appear in m0 and all the indices of the variables that appear in m0 come from S, we have m02 M(S0) for some S0( S. Consider 0. mr m = _m____S_Qoj. j2J xj Since mS 2 T , the monomial mrSdoesQnot separate v and w. Moreover, by our induction hypothesis m02 M(S0) and j2Txojj2 M(J ) do not separate v and w either, becauseQS0, J ( S.QBut the value of m at a point is uniquely determined by m0, mS and j2JxojjifQ j2Jxojjis non-zero at that point. Therefore m does separate v and w if j2JxojjisQnon-zero at one (hence both) of v and w. On the other hand if j2Jxojjvanishes at a point, thenQm also vanishes at th* *at point because J S, namely if a variable appears in j2Jxojj, it also appears* * in m. Finally, the monomial m; corresponding to empty set is just 1, hence it is not needed in a separating set. This completes the proof. We demonstrate in the following example that the bound of the previous propo- sition is sharp. Example. Let G = Z3 be the cyclic group of order 3 acting diagonally on the pol* *y- nomial ring C[x1, x2] with complex coefficients by oe(xi) = ~xifor 1 i 1, w* *here ~ is a primitive 3rd root of unity and oe is a generator of G. The invariant ri* *ng is minimally generated by {x31, x21x2, x1x22, x32}. As the previous proposition pr* *edicts, the set {x31, x1x22, x32} is separating. If there were a separating set consis* *ting of two elements, say f1, f2, then C[f1, f2] C[x1, x2]G is a finite extension and* * more- over C[x1, x2]G is the normalization of C[f1, f2], because there are no nontriv* *ial purely inseparable extensions in characteristic zero. But this is impossible be* *cause C[f1, f2] is a regular ring and hence is integrally closed. Meanwhile the separating set in Proposition 10 can be refined substantially f* *or cyclic groups of prime order as we show next. Proposition 11. Let G be a cyclic group of prime order. Furthermore assume that n 2 and Oj 6= 0 for 1 j n. Then T = {mS|S {1, 2, . .,.n} and |S| = 1, 2} 10 M. D. NEUSEL AND M. SEZER 2+n is separating. Note that the order of T does not exceed n___2. Proof.Let |S| = 1, say S = {j}, then mS = xojj. Assume next that |S| = 2 with S = {i, j} and i < j. Since ~(G) is cyclic of prime order and Oi, Oj 6= 0 there* * exists a unique positive integer ai,j< oj such that Oi+ ai,jOj = 0 2 ~(G). Hence xixai,jjis an invariant monomial. Since ai,jis the smallest among the pos* *itive integers k such that xixkjis invariants it follows that mS = xixai,jj. Thus we * *have obtained T = {xojj}1 j n [ {xixai,jj}1 i i. Then xi does not appear in ___m_____= xei+1i+1.x.e.nn_. (xixai,jj)ei xai,jeij It follows that for sufficiently large t 2 N m0(xixai,jj)ei m = ___________toj, xj for some m0 that lies in M(S0) for some proper subset S0 in S. The value of m at a point is uniquely determined by m0, (xixai,jj)eiand xojj, if j-th coordina* *te of that point is non-zero. In this case m does not separate v and w by induction since m0 2 M(S0) and xojj, xixai,jj2 T . On the other hand if xojj(v) = 0 (hence xojj(w) = 0), then m(v) = m(w) = 0 as well since j 2 S, i.e., xj appears in m. References [1]M.F. Atiyah and I.G. Macdonald. Introduction to Commutative Algebra. Addiso* *n-Wesley Publishing Company, Reading, MA, 1969. [2]H. Derksen and G. Kemper. Computational invariant theory. Invariant Theory * *and Algebraic Transformation Groups, I. Springer-Verlag, Berlin, 2002. Encyclopaedia of Ma* *thematical Sciences, 130. [3]M. Domokos. Typical separating invariants. Transform. Groups, 12(1):49-63, * *2007. [4]J. Draisma, G. Kemper, and D. Wehlau. Polarization of separating invariants* *. to appear in Canad. J. Math., 2006. [5]P. Fleischmann. Relative trace ideals and Cohen-Macaulay quotients of modul* *ar invariant rings. In Computational methods for representations of groups and algebras (* *Essen, 1997), volume 173 of Progr. Math., pages 211-233. Birkh"auser, Basel, 1999. [6]P. Fleischmann, M. Sezer, R. J. Shank, and C. F. Woodcock. The Noether numb* *ers for cyclic groups of prime order. Adv. Math., 207(1):149-155, 2006. [7]G. Kemper. Separating invariants. preprint available at http://www-m11.ma.t* *um.de/ kem- per/publications.html, 2007. [8]M. D. Neusel. The transfer in the invariant theory of modular permutation r* *epresentations. Pacific J. of Math., 199(1):121-135, 2001. [9]M. D. Neusel and M. Sezer. The Noether map I. to appear in Forum Math., 200* *7. SEPARATING INVARIANTS 11 [10]M. D. Neusel and M. Sezer. Invariants of modular indecomposable representat* *ions of Zp2. to appear in Math. Ann., 2008. [11]M.D. Neusel and L. Smith. Invariant theory of finite groups, volume 94 of M* *athematical Surveys and Monographs. American Mathematical Society, Providence, RI, 2002. [12]P. E. Newstead. Introduction to moduli problems and orbit spaces, volume 51* * of Tata In- stitute of Fundamental Research Lectures on Mathematics and Physics. Tata In* *stitute of Fundamental Research, Bombay, 1978. Department of Math. and Stats., Texas Tech University, MS 1042 Lubbock, TX 79409, USA E-mail address: Mara.D.Neusel@ttu.edu Department of Mathematics, Bilkent University, Ankara 06800, Turkey E-mail address: sezer@fen.bilkent.edu.tr