\begin{abstract} A novel parallel technique for Fourier-Galerkin pseudo-spectral methods with applications to two-dimensional Navier-Stokes equations and inviscid Boussinesq approximation equations is presented. It takes advantage of the programming structure of the phase-shift de-aliased scheme for pseudo-spectral codes, and combines the task-distribution strategy [Z. Yin, H.J.H. Clercx, and D.C. Montgomery, An easily implemented task-based parallel scheme for the Fourier pseudospectral solver applied to 2D Navier-Stokes turbulence, Comput. Fluids, Vol.33, 509 (2004).] and parallelized Fast Fourier Transform scheme. The performances of the resulting MPI Fortran90 codes with the new procedure on SGI 3800 are reported. For fixed resolution of the same problem, the peak speed of the new scheme can be twice as fast as the old parallel methods. The parallelized codes are used to solve some challenging numerical problems governed by the Navier-Stokes equations and Boussinesq equations. Some interesting physical phenomena (the double-valued $\omega - \psi $ structure in two-dimensional decaying turbulence and the collapse of the bubble cap in the Boussinesq simulation), are solved by using the proposed parallel algorithms. \end{abstract}