My steps are:

1. install the 2.6.28.11 server;
2. without install gnome, "sudo apt-get install build-essential linux-headers-`uname -r`"
3. wget the cuda 2.3 driver for ubuntu 9.04
4. at this moment, since no gnome installed, no X-server is on, "sudo sh NVIDIA_LONG_NAME_64_BIT_DRIVER.run"
5. It asked me if I would like to install 32 bit compatible OPEN GL library, i said yes
6. Installed Nvidia driver sucessfully
7. install gnome
8. reboot
9. in gnome, run "sudo nvidia-xconfig"

After done with above, go to System -> Preferences -> Display, it would say something about some feature is not there, it asks whether you want to use alternative driver from your graphics card. Choose "YES", then you can see the Nvidia X driver Utilities Panel

ONE MORE THING: after changing all the display settings, you would like to save that xorg.config, but the system would tell you it could not remove/create xorg.config.backup. What I did was, clicked on the preview settings, and copied the whole thing, manually changed xorg.config by editor.

And now you can reboot, and you will see the display setting is still there!

Add /usr/local/cuda/bin to $PATH, /usr/local/cuda/lib64 to $LD_LIBRARY_PATH, ~/NVIDIA_GPU_Computing_SDK/C to  $NVSDKCUDA_ROOT.You should put them into ~/.bashrc to make it permernant!