Gravitational waves (GWs) are inevitably induced at second order in cosmological perturbations through nonlinear couplings with first-order scalar perturbations, the existence of which is well established by recent cosmological observations. So far, the evolution and the spectrum of the secondary induced GWs have been derived by taking into account the sources of GWs only from the product of first-order scalar perturbations. Here we newly investigate the effects of purely second-order anisotropic stresses of photons and neutrinos on the evolution of GWs, which have been omitted in the literature. We present a full treatment of the Einstein–Boltzmann system to calculate the spectrum of GWs with anisotropic stress based on the formalism of the cosmological perturbation theory. We find that photon anisotropic stress amplifies the amplitude of GWs by about 150%, whereas neutrino anisotropic stress suppresses that of GWs by about 30% on small scales k≳1.0  hMpc-1 compared to the case without anisotropic stress. The second-order anisotropic stress does not affect GWs with wave numbers k≲1.0  hMpc-1. The result is in marked contrast with the case at linear order, where the effect of anisotropic stress is damping in the amplitude of GWs.