Memory/Swap issue

I have a Compaq DL380 with dual 1.3 GHz processors and 2.5 GB's of Memory, running Redhat 7.3. The main applications are Apache 1.3 and Samba 2.2.6 with about 200 users. My issue is the server does not seem to be using the large amount of RAM properly. When the server is first booted the users are very happy, as the go about their work the amount of Memory being used keeps increasing and the preformance starts going down. The system does not seem to be freeing up the cached memory. If I do not re-boot and wait long enough (several days) the system will finally freeze up and have to be re-booted. How can I make this preform better?