The best thing that could happen for scientific computing with free software on GNU/Linux is the GNU Scientific Library GSL. It however has source code only in C and people who use FORTRAN will find that a let down. Pouncing on this opportunity it is recommended that GSL is another reason (in addition to the GCC C compiler, coupled with the advantages of C programming) for starting to learn to use C. In addition to this, the two best source code repositories for Numerical Methods and libraries are Netlib and GAMS. There are new numerical packages being developed outside the usual "write a FORTRAN program, get a numerical subroutine from INTERNET for solving the numerics" concepts. The merits and demerits of this approach are debatable, but there exist more options like Object Oriented Numerics GSL and GiNaC which are exciting developments.
Netlib: An amazing amount of free source code for Numerical Methods. Netlib is THE source code repository which contains an innumerable amount of source code for Numerical Methods. It also has an active discussion forum wherein you can submit your queries and stay posted for help. Netlib also has a Parallel Tools Library and a search by subject.
GAMS: Guide to Available Mathematical Software GAMS has a very useful search using which one can search for keywords (example: ``diffusion'' to search for a diffusion equation solver). However the browse by package at GAMS reveals that a lot of the software they provide is a link to the netlib repository.
Object Oriented Numerics A site devoted to object oriented numerics. It has a Mailing list, Extensive Links to freely available libraries (OO of course) and freely available tools for object oriented scientific computing.
GNU Scientific Library The GNU Scientific Library (GSL) is a collection of numerical routines written from scratch in C. It provides an Applications Programming Interface (API) for C programmers and also allows wrappers to be written for very high level languages. It covers a wide range of numerical computing topics, has a good manual, is widely portable and is distributed under the GNU General Public License.
GiNaC GiNaC is designed to allow the creation of software which need symbolic manipulations embedded in them. It extends C++ by a set of algebraic capabilities and is recursively named for GiNaC is not a Computer Algebra system. It is distributed under the terms and conditions of the GNU general public license (GPL).
FFTW FFTW is a collection of fast C routines for computing the Discrete Fourier Transform in one or more dimensions. It includes complex, real, and parallel transforms, and can handle arbitrary array sizes efficiently. This package includes both the double- and single-precision FFTW uniprocessors and the threads libraries.
LAPACK LAPACK (Linear Algebra PACKage) is a standard library for numerical linear algebra. LAPACK provides routines for solving systems of simultaneous linear equations, least-squares solutions of linear systems of equations, eigenvalue problems, and singular value problems. LAPACK is coded in Fortran77 and is built with egcs. It is well documented and widely used (and therefore widely tested).
SuperLU SuperLU is a general purpose library which performs an LU decomposition for the direct solution of large, sparse, non-symmetric systems of linear equations on high performance machines. Its written in C and is callable from either C or Fortran.
ARPACK ARPACK is a set of Fortran77 subroutines designed to solve large scale eigenvalue problems. A Users Guide is available. The above link also gives information about a parallel version of ARPACK - PARPACK and a object oriented version ARPACK++.
Computational Fluid Dynamics codes This link contains a comprehensive listing of public domain, shareware and freeware Computational Fluid Dynamics codes links with a description of each CFD code.