net: mscc: serval: Remove delay when serdes is configured

When serdes configuration was written in hardware there was a delay
of 100ms to be sure that configuration was written. But the delay is not
needed because already the function serdes_write it is checking that the
operation finished.

Therefore remove the mdelay. This improves the speed of configuring the
network driver.

Signed-off-by: Horatiu Vultur <horatiu.vultur@microchip.com>
Reviewed-by: Daniel Schwierzeck <daniel.schwierzeck@gmail.com>
Acked-by: Joe Hershberger <joe.hershberger@ni.com>