If it is a normal cat 5 ethernet cable then you should have no problem. Also look at the modem on the back of your ps2 and check that the lights are working when you try to connect or maybe try and stick in your network disc again.
I've always had this problem. Works fine plugged into the back of the XBOX 360, plug it into the back of the PS2 and it can't see the connection. Tried absolutely everything, I've even borrowed my network engineer brother-in-law for an hour and he couldn't get it going.
I gave up getting the PS2 online a while ago I'm afraid.
If you're using the Ethernet cable that came with the Xbox with ICS on the PS2 then it wont work as the cable that comes with the xbox is a straight through cable but the xbox can still use it as a cross-over. Don't ask how but it somehow can =\
Just purchase yourself a crossover cable and you'll be sorted mate Plus you'll be able to use the cross-over on both xbox and PS2.