Invariant theory for singular α-determinants

From the irreducible decompositions' point of view, the structure of the cyclic GL_n (C)-module generated by the α-determinant degenerates when α = ± frac(1, k)(1 ≤ k ≤ n - 1) (see [S. Matsumoto, M. Wakayama, Alpha-determinant cyclic modules of gl_n (C), J. Lie Theory 16 (2006) 393-405]). In this paper, we show that - frac(1, k)-determinant shares similar properties which the ordinary determinant possesses. From this fact, one can define a new (relative) invariant called a wreath determinant. Using (GL_m, GL_n)-duality in the sense of Howe, we obtain an expression of a wreath determinant by a certain linear combination of the corresponding ordinary minor determinants labeled by suitable rectangular shape tableaux. Also we study a wreath determinant analogue of the Vandermonde determinant, and then, investigate symmetric functions such as Schur functions in the framework of wreath determinants. Moreover, we examine coefficients which we call (n, k)-sign appeared at the linear expression of the wreath determinant in relation with a zonal spherical function of a Young subgroup of the symmetric group S_{n k}.