Abstract: Waveguide trapped modes (or bound states of quantum billiards, or guided waves for diffraction gratings) are examples of non-uniqueness for the Laplacian or other elliptic operators. Both theoretical and numerical study of them has a long history. We suggest a new approach to detect trapped modes numerically. It is based on an existence criterion of trapped modes connected with the spectrum of a so-called augmented scattering matrix (ASM). This unitary matrix takes into account not only oscillating modes but also finitely many of those which grow (attenuate) in amplitude at infinity. Dealing with exponentially growing solutions causes difficulties for numerical analysis. We introduce a method which reduces computation of ASM to minimization of a quadratic functional. To get the functional one has to solve an auxiliary boundary value problems in the domain truncated at a finite distance R; the minimizer of the functional exponentially tends to the actual ASM as R goes to infinity. The numerical scheme is not sensitive to the geometry of the problem and, in particular, can be applied to a computation of the scattering as well. In the talk we present a wide range of new examples concerning trapped modes and guided grating waves. This is common work with B.A. Plamenevskii and P. Neittaanmaki.